kernel/git/next/linux-next.git/mm/percpu.c, branch master

percpu: remove unnecessary return in pcpu_populate_pte()

2026-09-11T03:20:47+00:00

Since commit c6f239796b55 ("mm/memblock: add memblock_alloc_or_panic interface"), pcpu_populate_pte() no longer has an error label after the return statement, so the return statement has no effect. Remove unnecessary return. No functional change. Link: https://lore.kernel.org/20260901165307.1026248-5-ekffu200098@gmail.com Signed-off-by: Sang-Heon Jeon Cc: Christoph Lameter Cc: Dennis Zhou Cc: Tejun Heo Signed-off-by: Andrew Morton

percpu: remove unnecessary cpumask_clear() in pcpu_build_alloc_info()

2026-09-11T03:20:46+00:00

pcpu_build_alloc_info() clears mask and unconditionally sets it to cpu_possible_mask. Nothing uses it in between, so cpumask_clear() has no effect. Remove unnecessary cpumask_clear(). No functional change. Link: https://lore.kernel.org/20260901165307.1026248-4-ekffu200098@gmail.com Signed-off-by: Sang-Heon Jeon Cc: Christoph Lameter Cc: Dennis Zhou Cc: Tejun Heo Signed-off-by: Andrew Morton

percpu: remove unnecessary initialization in pcpu_build_alloc_info()

2026-09-11T03:20:46+00:00

pcpu_build_alloc_info() initializes nr_groups to 1 and unconditionally sets it to the number of groups. Nothing uses it in between, so the initialization has no effect. Remove unnecessary initialization. No functional change. Link: https://lore.kernel.org/20260901165307.1026248-3-ekffu200098@gmail.com Signed-off-by: Sang-Heon Jeon Cc: Christoph Lameter Cc: Dennis Zhou Cc: Tejun Heo Signed-off-by: Andrew Morton

percpu: remove redundant assignments to bits

2026-09-11T03:20:46+00:00

Patch series "percpu: remove code with no effect". While reading the percpu initialization path, I found some minor cleanups for unnecessary initializations and an obsolete return statement. No functional change. This patch (of 4): pcpu_chunk_refresh_hint() and pcpu_find_block_fit() set bits to 0 and later call pcpu_next_md_free_region() or pcpu_next_fit_region(), which unconditionally set *bits to 0. Nothing uses it in between, so the assignments have no effect. Remove redundant assignments. No functional change. Link: https://lore.kernel.org/20260901165307.1026248-1-ekffu200098@gmail.com Link: https://lore.kernel.org/20260901165307.1026248-2-ekffu200098@gmail.com Signed-off-by: Sang-Heon Jeon Cc: Dennis Zhou Cc: Tejun Heo Cc: Christoph Lameter Signed-off-by: Andrew Morton

percpu: fix the comment about which sizes share a slot

2026-09-11T03:20:20+00:00

A percpu allocation is at least PCPU_MIN_ALLOC_SIZE bytes, and __pcpu_size_to_slot() returns 1 for sizes below 16 bytes and 2 for sizes from 16 to 31 bytes. So fix the wrong comment. Link: https://lore.kernel.org/20260818164831.3138490-1-ekffu200098@gmail.com Signed-off-by: Sang-Heon Jeon Cc: Dennis Zhou Cc: Tejun Heo Cc: Christoph Lameter Signed-off-by: Andrew Morton

mm/percpu: avoid IO/FS reclaim in backing allocations

2026-07-29T04:11:47+00:00

Commit 9a5b183941b5 ("mm, percpu: do not consider sleepable allocations atomic") allows sleepable GFP_NOIO and GFP_NOFS percpu allocations to take pcpu_alloc_mutex. This avoids premature allocation failures, but it also makes the mutex visible to callers from constrained IO/FS contexts. Thread A calls pcpu_alloc_noprof() with GFP_KERNEL and takes pcpu_alloc_mutex. Since the internal allocation is not constrained by NOFS, it may enter FS reclaim while still holding pcpu_alloc_mutex, creating a dependency like: pcpu_alloc_mutex -> fs_reclaim -> FS lock At the same time, Thread B may already hold an FS lock and then call pcpu_alloc_noprof() with GFP_NOFS. It will try to acquire pcpu_alloc_mutex and block, creating the reverse dependency: FS lock -> pcpu_alloc_mutex This can still form a potential deadlock cycle. Avoid the dependency by restricting percpu backing allocations to GFP_NOIO. The public allocation still uses the caller's GFP context to decide whether it may block, but the internal memory allocations performed while pcpu_alloc_mutex is held cannot recurse into IO or FS reclaim. Link: https://lore.kernel.org/20260618130414.96383-5-kaitao.cheng@linux.dev Fixes: 9a5b183941b5 ("mm, percpu: do not consider sleepable allocations atomic") Signed-off-by: Kaitao Cheng Cc: Christoph Lameter Cc: Dennis Zhou Cc: Michal Hocko Cc: Pedro Falcato Cc: Shivam Kalra Cc: Tejun Heo Cc: Uladzislau Rezki (Sony) Cc: Vlastimil Babka Signed-off-by: Andrew Morton

mm/percpu: honor GFP constraints when populating chunks

2026-07-29T04:11:46+00:00

pcpu_alloc_noprof() derives pcpu_gfp from the caller supplied GFP mask and passes it down to pcpu_populate_chunk(). pcpu_alloc_pages() already uses that mask for backing page allocation. However, the populate slow path still has internal allocations and page table allocations which can lose the caller's allocation context. The temporary pages array is allocated by pcpu_get_pages() with GFP_KERNEL, and pcpu_map_pages() maps the backing pages through vmap_pages_range_noflush() using GFP_KERNEL. The latter can allocate vmalloc page tables implicitly, so a caller which deliberately uses GFP_NOFS or GFP_NOIO can still enter FS or IO reclaim while populating a percpu chunk. This has the same concern as chunk creation: callers such as blk-cgroup may use GFP_NOIO because they hold locks which can be involved in queue freeze or IO reclaim dependencies. If an allocation reaches the percpu slow path and needs to populate previously unbacked pages, the internal GFP_KERNEL allocations can defeat that context. One possible case is blk-cgroup after commit 5d726c4dbeed ("blk-cgroup: fix possible deadlock while configuring policy"). blkg_conf_prep() now serializes against blkcg_deactivate_policy() with q->blkcg_mutex, and blkg_alloc() was changed to GFP_NOIO for that reason: CPU0: blkg_conf_prep() mutex_lock(q->blkcg_mutex) blkg_alloc(..., GFP_NOIO) alloc_percpu_gfp(..., GFP_NOIO) pcpu_alloc_noprof(..., GFP_NOIO) pcpu_populate_chunk(GFP_NOIO) pcpu_get_pages() pcpu_map_pages() -> if the selected percpu chunk has unpopulated pages, chunk population may do internal GFP_KERNEL allocations -> direct reclaim / writeback can issue IO to this queue -> IO waits because the queue is frozen CPU1: blkcg_deactivate_policy() blk_mq_freeze_queue(q) mutex_lock(q->blkcg_mutex) -> waits for CPU0 ... unfreeze only happens after q->blkcg_mutex is acquired/released So the concern is that the caller deliberately uses GFP_NOIO because it may hold a lock which can be acquired after queue freeze, but the percpu slow path can temporarily lose that allocation context. Pass pcpu_gfp through pcpu_get_pages(), pcpu_map_pages() and __pcpu_map_pages(). Apply the corresponding memalloc scope around vmap_pages_range_noflush(), because vmalloc page table allocation does not pass the GFP mask down explicitly. Keep the first chunk setup path using GFP_KERNEL, matching the previous early-init behavior. Link: https://lore.kernel.org/20260618130414.96383-3-kaitao.cheng@linux.dev Fixes: 9a5b183941b5 ("mm, percpu: do not consider sleepable allocations atomic") Signed-off-by: Kaitao Cheng Acked-by: Dennis Zhou Acked-by: Michal Hocko Cc: Christoph Lameter Cc: Pedro Falcato Cc: Shivam Kalra Cc: Tejun Heo Cc: Uladzislau Rezki (Sony) Cc: Vlastimil Babka Signed-off-by: Andrew Morton

mm: memcontrol: return root object cgroup for root memory cgroup

2026-04-18T07:10:44+00:00

Memory cgroup functions such as get_mem_cgroup_from_folio() and get_mem_cgroup_from_mm() return a valid memory cgroup pointer, even for the root memory cgroup. In contrast, the situation for object cgroups has been different. Previously, the root object cgroup couldn't be returned because it didn't exist. Now that a valid root object cgroup exists, for the sake of consistency, it's necessary to align the behavior of object-cgroup-related operations with that of memory cgroup APIs. Link: https://lore.kernel.org/e9c3f40ba7681d9753372d4ee2ac7a0216848b95.1772711148.git.zhengqi.arch@bytedance.com Signed-off-by: Muchun Song Signed-off-by: Qi Zheng Acked-by: Johannes Weiner Acked-by: Shakeel Butt Reviewed-by: Harry Yoo Cc: Allen Pais Cc: Axel Rasmussen Cc: Baoquan He Cc: Chengming Zhou Cc: Chen Ridong Cc: David Hildenbrand Cc: Hamza Mahfooz Cc: Hugh Dickins Cc: Imran Khan Cc: Kamalesh Babulal Cc: Lance Yang Cc: Liam Howlett Cc: Lorenzo Stoakes (Oracle) Cc: Michal Hocko Cc: Michal Koutný Cc: Mike Rapoport Cc: Muchun Song Cc: Nhat Pham Cc: Roman Gushchin Cc: Suren Baghdasaryan Cc: Usama Arif Cc: Vlastimil Babka Cc: Wei Xu Cc: Yosry Ahmed Cc: Yuanchu Xie Cc: Zi Yan Signed-off-by: Andrew Morton

percpu: add double free check to pcpu_free_area()

2026-01-31T22:22:52+00:00

Percpu memory provides access via offsets into the percpu address space. Offsets are essentially fixed for the lifetime of a chunk and therefore require all users be good samaritans. If a user improperly handles the lifetime of the percpu object, it can result in corruption in a couple of ways: - immediate double free - breaks percpu metadata accounting - free after subsequent allocation - corruption due to multiple owner problem (either prior owner still writes or future allocation happens) - potential for oops if the percpu pages are reclaimed as the subsequent allocation isn't pinning the pages down - can lead to page->private pointers pointing to freed chunks Sebastian noticed that if this happens, none of the memory debugging facilities add additional information [1]. This patch aims to catch invalid free scenarios within valid chunks. To better guard free_percpu(), we can either add a magic number or some tracking facility to the percpu subsystem in a separate patch. The invalid free check in pcpu_free_area() validates that the allocation's starting bit is set in both alloc_map and bound_map. The alloc_map bit test ensures the area is allocated while the bound_map bit test checks we are freeing from the beginning of an allocation. We choose not to check the validity of the offset as that is encoded in page->private being a valid chunk. pcpu_stats_area_dealloc() is moved later to only be on the happy path so stats are only updated on valid frees. Link: https://lkml.kernel.org/r/20260123205535.35267-1-dennis@kernel.org Link: https://lore.kernel.org/lkml/20260119074813.ecAFsGaT@linutronix.de/ [1] Signed-off-by: Dennis Zhou Cc: Sebastian Andrzej Siewior Cc: Chistoph Lameter Cc: Christoph Lameter Cc: Dennis Zhou Cc: Tejun Heo Signed-off-by: Andrew Morton

percpu: fix race on alloc failed warning limit

2025-09-09T06:45:10+00:00

The 'allocation failed, ...' warning messages can cause unlimited log spam, contrary to the implementation's intent. The warn_limit variable is accessed without synchronization. If more than threads enter the warning path at the same time, the variable will get decremented past 0. Once it becomes negative, the non-zero check will always return true leading to unlimited log spam. Use atomic operation to access warn_limit and change condition to test for non-negative (>= 0) - atomic_dec_if_positive will return -1 once warn_limit becomes 0. Continue to print disable message alongside the last warning. While the change cited in Fixes is only adjacent, the warning limit implementation was correct before it. Only non-atomic allocations were considered for warnings, and those happened to hold pcpu_alloc_mutex while accessing warn_limit. [vdumitrescu@nvidia.com: prevent warn_limit from going negative, per Christoph Lameter] Link: https://lkml.kernel.org/r/ee87cc59-2717-4dbb-8052-1d2692c5aaaa@nvidia.com Link: https://lkml.kernel.org/r/ab22061a-a62f-4429-945b-744e5cc4ba35@nvidia.com Fixes: f7d77dfc91f7 ("mm/percpu.c: print error message too if atomic alloc failed") Signed-off-by: Vlad Dumitrescu Reviewed-by: Baoquan He Cc: Christoph Lameter (Ampere) Cc: Dennis Zhou Cc: Tejun Heo Signed-off-by: Andrew Morton