kernel/git/stable/linux.git/mm/huge_memory.c, branch master

mm/huge_memory: transfer the pmd dirty bit to the folio on zap

2026-09-02T03:26:22+00:00

zap_huge_pmd_folio() propagates the pmd young bit to the folio for the file case, but not the dirty bit. The pte path does propagate it, in zap_present_folio_ptes() and so does the pmd split path, in __split_huge_pmd_locked(). For most file mappings the omission is harmless, because writing to a shared file mapping goes through page_mkwrite(), which dirties the folio. tmpfs is different: it has no page_mkwrite(), and vma_wants_writenotify() is false for it, so a *read* fault on a MAP_SHARED tmpfs mapping installs a writable pmd via do_read_fault(). do_read_fault() does not call fault_dirty_shared_page(), so subsequent stores through that mapping set only the hardware dirty bit in the pmd and never call folio_mark_dirty(). A shmem folio allocated by a fault is marked uptodate but not dirty (see the clear: block in shmem_get_folio_gfp()), so PG_dirty is never set at all. Unmapping such a folio - munmap(), or exit_mmap() when the process dies - then loses the only record that it was written, because zap_huge_pmd() drops the pmd without transferring the dirty bit. Reclaim afterwards sees a clean shmem folio: the whole swap-out block in shrink_folio_list() is inside "if (folio_test_dirty(folio))", so pageout() is skipped and the folio falls into __remove_mapping(). There, folio_is_file_lru() is false for a swapbacked folio, so no shadow entry is created and __filemap_remove_folio(folio, NULL) simply empties the i_pages slot. The data is freed without ever being written to swap, and the next fault on that index returns a freshly zeroed folio. This is silent data loss for any process that keeps state in a MAP_SHARED tmpfs segment across an unmap - for example a cache handed from one process generation to the next through /dev/shm. It requires the folio to be PMD-mapped, so it only shows up once shmem THP is enabled (which is what we did in Meta fleet and started noticing crashes); with THP off the pte path transfers the dirty bit correctly. It also only becomes visible when swap is enabled, because with no swap device shmem folios (which are on the anon LRU) are not scanned by reclaim at all, so the clean folio is never dropped. Reproduced on x86_64 with a tmpfs mounted huge=within_size: read-fault a 2MB-backed region, write a known pattern through the resulting mapping, munmap, force reclaim of the cgroup, then re-map and read back. Without this patch the region reads back as zeros and vmstat shows zswpout 0 - the data was discarded rather than swapped. With this patch the region reads back correctly and the pages are swapped out as expected. With huge=never, or when the first touch is a write, the test passes either way. Link: https://lore.kernel.org/20260819101222.3732660-1-usama.arif@linux.dev Fixes: b5072380eb61 ("thp: support file pages in zap_huge_pmd()") Signed-off-by: Usama Arif Acked-by: David Hildenbrand (Arm) Reviewed-by: Kiryl Shutsemau Acked-by: Hugh Dickins Tested-by: Lance Yang Reviewed-by: Zi Yan Reviewed-by: Lorenzo Stoakes (ARM) Reviewed-by: Baolin Wang Cc: Barry Song Cc: Dev Jain Cc: Johannes Weiner Cc: Liam R. Howlett Cc: Nhat Pham Cc: Rik van Riel Cc: Ryan Roberts Cc: Shakeel Butt Cc: Signed-off-by: Andrew Morton

mm/huge_memory: use folio's memcg inside __folio_split()

2026-08-25T01:42:59+00:00

Patch series "Honor XA_FLAGS_ACCOUNT in xas_split_alloc() and charge to folio's memcg", v3. __GFP_ACCOUNT is needed for xarray node allocation accounting when XA_FLAGS_ACCOUNT is set. Commit 7b785645e8f13 ("mm: fix page cache convergence regression") fixed a workingset regression with it. xas_split_alloc() does not have it and needs to be fixed. In addition, based on Sashiko's review[1] and Johannes' confirmation[2], to charge the right memcg, folio's memcg needs to be active during folio split. Add that before adding __GFP_ACCOUNT. There is no workingset convergence regression related to missing __GFP_ACCOUNT in xas_split_alloc() and the impact to userspace should be minor. This patch (of 2): During a pagecache folio split, an xarray node allocation can happen and needs to charge at folio's memcg instead of folio split invoker's memcg, because for example folio split can happen during reclaim and reclaim's active memcg might not be folio's memcg. Switch to folio's memcg at the beginning and switch back afterwards. Link: https://lore.kernel.org/20260804-add-gfp_account-to-xas_split_alloc-v3-0-38cb3ff325c5@nvidia.com Link: https://lore.kernel.org/20260804-add-gfp_account-to-xas_split_alloc-v3-1-38cb3ff325c5@nvidia.com Link: https://sashiko.dev/#/patchset/20260727-add-gfp_account-to-xas_split_alloc-v1-1-9fae6bf64838%40nvidia.com?part=1 [1] Link: https://lore.kernel.org/all/amtcBZ-_QVRgCd6b@cmpxchg.org/ [2] Fixes: 6b24ca4a1a8d ("mm: Use multi-index entries in the page cache") Signed-off-by: Zi Yan Suggested-by: Johannes Weiner Reviewed-by: Baolin Wang Acked-by: Lorenzo Stoakes (ARM) Acked-by: Johannes Weiner Cc: Barry Song Cc: David Hildenbrand Cc: Dev Jain Cc: Lance Yang Cc: Liam R. Howlett Cc: Matthew Wilcox (Oracle) Cc: Ryan Roberts Cc: William Kucharski Cc: Signed-off-by: Andrew Morton

mm/rmap: use anon pgoff to track MAP_PRIVATE file-backed anon folios

2026-08-25T01:42:53+00:00

Currently anonymous folios belonging to CoW'd MAP_PRIVATE file-backed mappings are indexed by their page offset within the file in which they were originally mapped. This differs from anonymous folios belonging to pure anon mappings which are indexed by their anonymous page offset (the address at which they'd belong in the VMA when first faulted). This change fixes this inconsistency, always indexing anonymous folios by their anonymous page offset regardless of the VMA to which they belong. The foundations have been laid such that we need only switch this functionality on such by: * Using linear_anon_page_index() in __folio_set_anon() to assign the folio's index to the anonymous linear index rather than the file-backed one. * Otherwise using linear_anon_page_index() in all instances where anonymous folios are being referenced or manipulated. * Replacing vma_address() with vma_filebacked_address() or vma_anon_address() as appropriate. * Updating the merging logic to check that anonymous page offsets are aligned as well as filebacked ones for MAP_PRIVATE file-backed VMAs, introducing needs_adjacent_anon_pgoff() to figure out when this is required. * Updating linear_folio_page_index() to invoke linear_anon_page_index() if the folio is anonymous. * Updating vma_address_end() to use the VMA's anonymous page offset when pvmw->pgoff is anonymous. * Correcting folio_within_range() to use anonymous page offset for anonymous folios. This will have no impact on merging of anonymous VMAs, whose page offset and anonymous page offset are identical, nor will it impact shared file-backed VMAs, which will continue to be merged based on the file-backed page offset. However, MAP_PRIVATE file-backed mappings must now be aligned on anonymous page offset as well. In most instances this should have no impact on merging of file-backed mappings, which are usually not merged all that often, let alone MAP_PRIVATE mapped ones, and rarely remapped and faulted before being moved back in place (the case in which a merge may now fail). One subtle impact of this change is in NUMA interleaving - since commit 88c91dc58582 ("mempolicy: migration attempt to match interleave nodes"), migration heuristically tries to maintain interleaving behaviour matching the policy using folio indices. When doing migration of CoW'd MAP_PRIVATE-file backed ranges, the 'base' upon which the interleaving behaviour is performed will vary for these ranges. However the commit notes that ranges spanning multiple VMAs will already cause varying bases, and that this is an acceptable approximation. It is very unlikely real world use-cases will be impacted by this (MAP_PRIVATE file-backed mappings are already an edge case), and all that will happen is that such ranges will cause interleaving to be rotated over the CoW'd range, with little to no impact. This commit lays the foundations for future scalable CoW work which needs to track some remaps, meaning that most remap tracking can be avoided, and in nearly all cases the anonymous page offset will be able to be used to quickly find the VMA in an mm. Note that the need_rmap_locks check doesn't need to be updated, as any remapping will offset both the anonymous and file-backed page offset, so it suffices to check only one. Link: https://lore.kernel.org/20260813-b4-scalable-cow-virt-pgoff-v5-14-c21581c0c3c8@kernel.org Signed-off-by: Lorenzo Stoakes (ARM) Acked-by: David Hildenbrand (Arm) Cc: Adrian Hunter Cc: Alexander Deucher Cc: Alexander Gordeev Cc: Alexander Shishkin Cc: Alistair Popple Cc: Arnaldo Carvalho de Melo Cc: Arnd Bergmann Cc: Baolin Wang Cc: Baoquan He Cc: Barry Song Cc: Boris Brezillon Cc: Byungchul Park Cc: Chengming Zhou Cc: Chris Li Cc: Christan König Cc: Christian Borntraeger Cc: Claudio Imbrenda Cc: Dave Airlie Cc: Dev Jain Cc: Gerald Schaefer Cc: Greg Kroah-Hartman Cc: Gregory Price (Meta) Cc: Harry Yoo Cc: Heiko Carstens Cc: Huang Ray Cc: "Huang, Ying" Cc: Ian Rogers Cc: Ingo Molnar Cc: James Clark Cc: Jan Kara Cc: Jann Horn Cc: Janosch Frank Cc: Jason Gunthorpe Cc: Jiri Olsa Cc: John Hubbard Cc: Joshua Hahn Cc: Kairui Song Cc: Kees Cook Cc: Kemeng Shi Cc: Lance Yang Cc: Liam R. Howlett Cc: Liviu Dudau Cc: Maarten Lankhorst Cc: Marc Rutland Cc: "Masami Hiramatsu (Google)" Cc: Matthew Auld Cc: Matthew Brost Cc: Matthew Wilcox (Oracle) Cc: Maxime Ripard Cc: Miaohe Lin Cc: Michal Hocko Cc: Mike Rapoport Cc: Muchun Song Cc: Namhyung kim Cc: Naoya Horiguchi Cc: Nhat Pham Cc: Nico Pache Cc: Oleg Nesterov Cc: Oscar Salvador Cc: Pedro Falcato Cc: Peter Xu Cc: Peter Zijlstra Cc: Rakie Kim Cc: Rik van Riel Cc: Rodrigo Vivi Cc: Ryan Roberts Cc: Steven Price Cc: Suren Baghdasaryan Cc: Sven Schnelle Cc: Thomas Hellström Cc: Thomas Zimemrmann Cc: Vasily Gorbik Cc: Vlastimil Babka Cc: xu xin Cc: Zi Yan Signed-off-by: Andrew Morton

mm/huge_memory: update remove_migration_pmd() to accept a folio

2026-08-25T01:42:52+00:00

This function does not need to accept a page and requiring it to is unnecessary and misleading. make_[writable, readable]_device_private_entry() must be passed a PMD-aligned PFN as they immediately used to obtain a softleaf PMD entry and the same argument applies to folio_add_[anon, file]_rmap_pmd(). While we are here, update a VM_BUG_ON() to a VM_WARN_ON_ONCE(). No functional change intended. Link: https://lore.kernel.org/20260813-b4-scalable-cow-virt-pgoff-v5-12-c21581c0c3c8@kernel.org Signed-off-by: Lorenzo Stoakes (ARM) Acked-by: David Hildenbrand (Arm) Cc: Adrian Hunter Cc: Alexander Deucher Cc: Alexander Gordeev Cc: Alexander Shishkin Cc: Alistair Popple Cc: Arnaldo Carvalho de Melo Cc: Arnd Bergmann Cc: Baolin Wang Cc: Baoquan He Cc: Barry Song Cc: Boris Brezillon Cc: Byungchul Park Cc: Chengming Zhou Cc: Chris Li Cc: Christan König Cc: Christian Borntraeger Cc: Claudio Imbrenda Cc: Dave Airlie Cc: Dev Jain Cc: Gerald Schaefer Cc: Greg Kroah-Hartman Cc: Gregory Price (Meta) Cc: Harry Yoo Cc: Heiko Carstens Cc: Huang Ray Cc: "Huang, Ying" Cc: Ian Rogers Cc: Ingo Molnar Cc: James Clark Cc: Jan Kara Cc: Jann Horn Cc: Janosch Frank Cc: Jason Gunthorpe Cc: Jiri Olsa Cc: John Hubbard Cc: Joshua Hahn Cc: Kairui Song Cc: Kees Cook Cc: Kemeng Shi Cc: Lance Yang Cc: Liam R. Howlett Cc: Liviu Dudau Cc: Maarten Lankhorst Cc: Marc Rutland Cc: "Masami Hiramatsu (Google)" Cc: Matthew Auld Cc: Matthew Brost Cc: Matthew Wilcox (Oracle) Cc: Maxime Ripard Cc: Miaohe Lin Cc: Michal Hocko Cc: Mike Rapoport Cc: Muchun Song Cc: Namhyung kim Cc: Naoya Horiguchi Cc: Nhat Pham Cc: Nico Pache Cc: Oleg Nesterov Cc: Oscar Salvador Cc: Pedro Falcato Cc: Peter Xu Cc: Peter Zijlstra Cc: Rakie Kim Cc: Rik van Riel Cc: Rodrigo Vivi Cc: Ryan Roberts Cc: Steven Price Cc: Suren Baghdasaryan Cc: Sven Schnelle Cc: Thomas Hellström Cc: Thomas Zimemrmann Cc: Vasily Gorbik Cc: Vlastimil Babka Cc: xu xin Cc: Zi Yan Signed-off-by: Andrew Morton

mm: provide vma_[flags_]is_cow_mapping() and remove is_cow_mapping()

2026-08-25T01:42:50+00:00

All remaining callers of is_cow_mapping() are invoking it in the form of is_cow_mapping(vma->vm_flags) or an indirected version of this. Therefore, provide a helper - vma_is_cow_mapping() to directly test the VMA. Additionally provide a new helper vma_flags_is_cow_mapping() which performs the check using the new vma_flags_t type, and share this logic between vma_is_cow_mapping() and vma_desc_is_cow_mapping(). With these changes, no callers of is_cow_mapping() remain, so remove it. Also update the userland VMA tests to reflect the change. No functional change intended. [akpm@linux-foundation.org: fix kerneldoc comment typo, per Lorenzo] Link: https://lore.kernel.org/aob1goSSPH6sTN9y@gremlin Link: https://lore.kernel.org/20260813-b4-scalable-cow-virt-pgoff-v5-2-c21581c0c3c8@kernel.org Signed-off-by: Lorenzo Stoakes (ARM) Acked-by: David Hildenbrand (Arm) Cc: Adrian Hunter Cc: Alexander Deucher Cc: Alexander Gordeev Cc: Alexander Shishkin Cc: Alistair Popple Cc: Arnaldo Carvalho de Melo Cc: Arnd Bergmann Cc: Baolin Wang Cc: Baoquan He Cc: Barry Song Cc: Boris Brezillon Cc: Byungchul Park Cc: Chengming Zhou Cc: Chris Li Cc: Christan König Cc: Christian Borntraeger Cc: Claudio Imbrenda Cc: Dave Airlie Cc: Dev Jain Cc: Gerald Schaefer Cc: Greg Kroah-Hartman Cc: Gregory Price (Meta) Cc: Harry Yoo Cc: Heiko Carstens Cc: Huang Ray Cc: "Huang, Ying" Cc: Ian Rogers Cc: Ingo Molnar Cc: James Clark Cc: Jan Kara Cc: Jann Horn Cc: Janosch Frank Cc: Jason Gunthorpe Cc: Jiri Olsa Cc: John Hubbard Cc: Joshua Hahn Cc: Kairui Song Cc: Kees Cook Cc: Kemeng Shi Cc: Lance Yang Cc: Liam R. Howlett Cc: Liviu Dudau Cc: Maarten Lankhorst Cc: Marc Rutland Cc: "Masami Hiramatsu (Google)" Cc: Matthew Auld Cc: Matthew Brost Cc: Matthew Wilcox (Oracle) Cc: Maxime Ripard Cc: Miaohe Lin Cc: Michal Hocko Cc: Mike Rapoport Cc: Muchun Song Cc: Namhyung kim Cc: Naoya Horiguchi Cc: Nhat Pham Cc: Nico Pache Cc: Oleg Nesterov Cc: Oscar Salvador Cc: Pedro Falcato Cc: Peter Xu Cc: Peter Zijlstra Cc: Rakie Kim Cc: Rik van Riel Cc: Rodrigo Vivi Cc: Ryan Roberts Cc: Steven Price Cc: Suren Baghdasaryan Cc: Sven Schnelle Cc: Thomas Hellström Cc: Thomas Zimemrmann Cc: Vasily Gorbik Cc: Vlastimil Babka Cc: xu xin Cc: Zi Yan Signed-off-by: Andrew Morton

Merge branch 'mm-hotfixes-stable' into mm-stable to pick up

2026-08-25T01:40:27+00:00

already-upstream changes to memcontrol.c, needed by "memcg: move mem_cgroup_swappiness and vm_swappiness to mm/swap.h".

mm: convert __get_unmapped_area() to use vma_flags_t

2026-08-07T01:57:01+00:00

Update __get_unmapped_area() to be parameterised by vma_flags_t rather than vm_flags_t as part of the effort to move VMA flags from a system word to a bitmap. We cascade the changes up to arch_get_unmapped_area_topdown() and arch_get_unmapped_area(), where, for now, we use vma_flags_to_legacy() in order to propagate the VMA flags. No functional change intended. Link: https://lore.kernel.org/20260711-b4-vma-flags-mm-v2-3-0fa2357d5431@kernel.org Signed-off-by: Lorenzo Stoakes Reviewed-by: Lance Yang Reviewed-by: Zi Yan Reviewed-by: Vlastimil Babka (SUSE) Cc: Baolin Wang Cc: Barry Song Cc: Christian Brauner Cc: Dave Airlie Cc: David Hildenbrand Cc: Dev Jain Cc: Jani Nikula Cc: Jan Kara Cc: Jann Horn Cc: Mike Rapoport Cc: Muchun Song Cc: Nico Pache Cc: Oscar Salvador Cc: Pedro Falcato Cc: Suren Baghdasaryan Cc: Thomas Zimmermann Signed-off-by: Andrew Morton

mm: huge_memory: fix kobject cleanup in thpsize_create error

2026-08-07T01:56:56+00:00

When kobject_init_and_add() fails, the kobject API requires calling kobject_put() to properly clean up the memory, not direct kfree(). According to the kobject API documentation, kobject_init_and_add() calls kobject_init() internally. If the subsequent kobject_add() fails, the kobject has still been initialized and must be cleaned up via the reference count mechanism (kobject_put), not direct kfree(). Direct kfree() leaves the kobject's internal state (including the reference count and kset membership) uncleaned, which can cause: - Memory leaks of kobject internal structures - Potential use-after-free if there are pending references - Inconsistent state with the rest of the error handling code This fix matches the pattern used elsewhere in the kernel and in the same function (err_put label) which correctly uses kobject_put(). Link: https://lore.kernel.org/20260713054154.120915-1-zenghongling@kylinos.cn Fixes: 3485b88390b0 ("mm: thp: introduce multi-size THP sysfs interface") Signed-off-by: Hongling Zeng Suggested-by: Baolin Wang Reviewed-by: Lorenzo Stoakes (ARM) Acked-by: Zi Yan Reviewed-by: Baolin Wang Cc: Barry Song Cc: David Hildenbrand Cc: Dev Jain Cc: Hongling Zeng Cc: Lance Yang Cc: Liam R. Howlett Cc: Nico Pache Cc: Ryan Roberts Signed-off-by: Andrew Morton

mm/huge_memory: initialise workingset state before folio split

2026-08-05T03:02:03+00:00

xas_try_split() adds __GFP_ACCOUNT for page-cache xa_nodes, but __folio_split() leaves the xa_state's xa_lru unset. That lets a live, memcg-charged xa_node exist without being linked into the mapping's shadow_nodes list_lru; when reclaim later walks the list_lru it trips VM_WARN_ON(!css_is_dying()). Use mapping_set_update() to install both the workingset update callback and the shadow_nodes list_lru on the xa_state. Link: https://lore.kernel.org/20260725101419.3938406-1-matt@readmodwrite.com Fixes: 58729c04cf10 ("mm/huge_memory: add buddy allocator like (non-uniform) folio_split()") Signed-off-by: Matt Fleming Reported-by: syzbot+c5b060ce82921a2fd500@syzkaller.appspotmail.com Closes: https://syzkaller.appspot.com/bug?extid=c5b060ce82921a2fd500 Reviewed-by: Zi Yan Acked-by: David Hildenbrand (Arm) Cc: Baolin Wang Cc: Barry Song Cc: Dave Chinner Cc: Dev Jain Cc: Kairui Song Cc: Lance Yang Cc: Liam Howlett Cc: Lorenzo Stoakes Cc: Matthew Wilcox (Oracle) Cc: Muchun Song Cc: Nico Pache Cc: Roman Gushchin Cc: Ryan Roberts Cc: Shakeel Butt Cc: Signed-off-by: Andrew Morton

mm/huge_memory: separate out CONFIG_PERSISTENT_HUGE_ZERO_FOLIO logic

2026-08-05T03:02:02+00:00

Rather than mixing the refcounted and non-refcounted CONFIG_PERSISTENT_HUGE_ZERO_FOLIO logic, separate the two out cleanly so it is clear what happens when this configuration option is set and what happens when it is not. Introduce HUGE_ZERO_UNSET_PFN to abstract the ~0UL assignment, only introduce the refcount, lock and shrinker if !CONFIG_PERSISTENT_HUGE_ZERO_FOLIO, abstract initialisation and teardown, abstract the huge zero folio allocation from refcounting. Also change a BUG_ON() to WARN_ON_ONCE() while we're at it. No functional change intended. Link: https://lore.kernel.org/20260730-fix-refcounted-huge-zero-v2-2-c5d8a41b317f@kernel.org Signed-off-by: Lorenzo Stoakes (ARM) Fixes: 3b77e8c8cde5 ("mm/thp: make is_huge_zero_pmd() safe and quicker") Cc: Baolin Wang Cc: Barry Song Cc: David Hildenbrand (Arm) Cc: Dev Jain Cc: Hannes Reinecke Cc: Hengbin Zhang Cc: Hugh Dickins Cc: Kiryl Shutsemau Cc: Lance Yang Cc: Liam R. Howlett Cc: Nico Pache Cc: Pankaj Raghav Cc: Ryan Roberts Cc: Yang Shi Cc: Zi Yan Cc: Signed-off-by: Andrew Morton