kernel/git/stable/linux-stable.git/mm/rmap.c, branch linux-rolling-stable

mm/rmap: use huge_ptep_get() in try_to_migrate_one()

2026-09-07T15:36:34+00:00

commit 9be12ffa78cd087d41086f937a156435d4e588e9 upstream. try_to_migrate_one() is used by folio migration to replace a present mapping with a migration entry. For hugetlb folios, page_vma_mapped_walk() returns the pte pointer to the hugetlb folio in pvmw.pte, but the code reads the huge pte entry with ptep_get(). On arches which provide their own huge_ptep_get() to dereference a huge pte pointer, accessing via ptep_get() would cause pte_pfn(), pte_present() etc to misbehave. It is not clear whether this has a trivially visible effect to userspace. Use huge_ptep_get() to dereference a huge pte pointer. Commit a98a2f0c8ce1 copied the bug from try_to_unmap_one into try_to_migrate_one. [akpm@linux-foundation.org: coding-style cleanups] Link: https://lore.kernel.org/20260703114202.365553-4-dev.jain@arm.com Fixes: a98a2f0c8ce1 ("mm/rmap: split migration into its own function") Signed-off-by: Dev Jain Acked-by: Muchun Song Acked-by: David Hildenbrand (Arm) Cc: Alistair Popple Cc: Andi Kleen Cc: Anshuman Khandual Cc: Byungchul Park Cc: Catalin Marinas Cc: Dave Hansen Cc: Gregory Price Cc: Harry Yoo Cc: "Huang, Ying" Cc: Jann Horn Cc: Josh Poimboeuf Cc: Joshua Hahn Cc: Jun'ichi "Nick" Nomura Cc: Kiryl Shutsemau Cc: Lance Yang Cc: Liam R. Howlett Cc: Lorenzo Stoakes Cc: Matthew Brost Cc: Mel Gorman Cc: Naoya Horiguchi Cc: Oscar Salvador Cc: Pedro Falcato Cc: Rakie Kim Cc: Ralph Campbell Cc: Rik van Riel Cc: Ryan Roberts Cc: Vlastimil Babka Cc: Will Deacon Cc: Zi Yan Cc: Signed-off-by: Andrew Morton Signed-off-by: Greg Kroah-Hartman

mm/rmap: use huge_ptep_get() in try_to_unmap_one()

2026-09-07T15:36:34+00:00

commit f5407e9b697c940e78b27ff63c6e14d8d171adc3 upstream. Patch series "Fix incorrect access of hugetlb pte entries", v3. There are various places which use ptep_get() to get the pte entry corresponding to a hugetlb folio. Some arches (like s390) have special handling to compute the pteval, so they provide huge_ptep_get(). Use this helper consistently. Additionally, some code paths may provide huge_ptep_get with an unaligned address. This is a problem on arm64 (I checked other arches and it looks fine for them), which is fixed in patch 1. The fix is made to be backport-friendly: the cleaner fix would be to perhaps pass the hstate to huge_ptep_get() - that is wider churn and we can do that later. This patch (of 5): try_to_unmap_one() handles hugetlb folios when memory failure needs to replace a poisoned hugetlb mapping with a hwpoison entry. In that case page_vma_mapped_walk() returns the pte pointer to the hugetlb folio in pvmw.pte, but the code reads it with ptep_get(). On arches which provide their own huge_ptep_get() to dereference a huge pte pointer, accessing via ptep_get() would cause pte_pfn(), pte_present() etc to misbehave. It is not clear whether this has a trivially visible effect to userspace. Just use huge_ptep_get() for dereferencing a huge pte pointer. Link: https://lore.kernel.org/20260703114202.365553-1-dev.jain@arm.com Link: https://lore.kernel.org/20260703114202.365553-3-dev.jain@arm.com Fixes: c7ab0d2fdc84 ("mm: convert try_to_unmap_one() to use page_vma_mapped_walk()") Signed-off-by: Dev Jain Reported-by: David Hildenbrand Reviewed-by: Muchun Song Acked-by: David Hildenbrand (Arm) Cc: Alistair Popple Cc: Andi Kleen Cc: Anshuman Khandual Cc: Byungchul Park Cc: Catalin Marinas Cc: Dave Hansen Cc: Gregory Price Cc: Harry Yoo Cc: "Huang, Ying" Cc: Jann Horn Cc: Josh Poimboeuf Cc: Joshua Hahn Cc: Jun'ichi "Nick" Nomura Cc: Kiryl Shutsemau Cc: Lance Yang Cc: Liam R. Howlett Cc: Lorenzo Stoakes Cc: Matthew Brost Cc: Mel Gorman Cc: Naoya Horiguchi Cc: Oscar Salvador Cc: Pedro Falcato Cc: Rakie Kim Cc: Ralph Campbell Cc: Rik van Riel Cc: Ryan Roberts Cc: Vlastimil Babka Cc: Will Deacon Cc: Zi Yan Cc: Signed-off-by: Andrew Morton Signed-off-by: Greg Kroah-Hartman

mm: remove page_mapped()

2026-05-29T04:04:57+00:00

Let's replace the last user of page_mapped() by folio_mapped() so we can get rid of page_mapped(). Replace the remaining occurrences of page_mapped() in rmap documentation by folio_mapped(). Link: https://lore.kernel.org/20260427-page_mapped-v1-3-e89c3592c74c@kernel.org Signed-off-by: David Hildenbrand (Arm) Reviewed-by: Matthew Wilcox (Oracle) Cc: Alexei Starovoitov Cc: Andrii Nakryiko Cc: Eduard Zingerman Cc: Harry Yoo Cc: Jann Horn Cc: Jiri Olsa Cc: John Paul Adrian Glaubitz Cc: Kumar Kartikeya Dwivedi Cc: Liam R. Howlett Cc: Lorenzo Stoakes Cc: Martin KaFai Lau Cc: Michal Hocko Cc: Mike Rapoport Cc: Rich Felker Cc: Rik van Riel Cc: Song Liu Cc: Suren Baghdasaryan Cc: Vlastimil Babka Cc: Yonghong Song Cc: Yoshinori Sato Signed-off-by: Andrew Morton

mm/rmap: initialize nr_pages to 1 at loop start in try_to_unmap_one

2026-05-22T02:06:12+00:00

Initialize nr_pages to 1 at the start of each loop iteration, like folio_referenced_one() does. Without this, nr_pages computed by a previous folio_unmap_pte_batch() call can be reused on a later iteration that does not run folio_unmap_pte_batch() again. mmap a 64K large folio with MAP_ANONYMOUS | MAP_DROPPABLE, then call madvise(MADV_FREE), then make the last page device-exclusive via HMM_DMIRROR_EXCLUSIVE. Trigger node reclaim through sysfs. Now, in try_to_unmap_one(), we will first clear the first 15 out of 16 entries mapping the lazyfree folio. This will set nr_pages to 15. In the next pvmw walk, this nr_pages gets reused on a device-exclusive pte, thus potentially corrupting folio refcount/mapcount. At the moment, I have a userspace program which can make the kernel spit out a trace, but the blow up is in folio_referenced_one(), because there are existing bugs in the interaction between device-private and rmap (which too I am investigating). I did a one liner kernel change to avoid going into folio_referenced_one(), and the kernel blows up at folio_remove_rmap_ptes in try_to_unmap_one which is what I wanted. Note that the bug is there not since file folio batching but lazyfree folio batching, since device-exclusive only works for anonymous folios. Userspace visible effect is simply kernel crashing somewhere due to refcount/mapcount corruption. Link: https://lore.kernel.org/20260518063656.3721056-1-dev.jain@arm.com Fixes: 354dffd29575 ("mm: support batched unmap for lazyfree large folios during reclamation") Signed-off-by: Dev Jain Acked-by: Barry Song Acked-by: David Hildenbrand (Arm) Reviewed-by: Lorenzo Stoakes Cc: Anshuman Khandual Cc: Barry Song Cc: Dev Jain Cc: Harry Yoo Cc: Jann Horn Cc: Liam R. Howlett Cc: Rik van Riel Cc: Ryan Roberts Cc: Vlastimil Babka Cc: Signed-off-by: Andrew Morton

mm/mglru: fix cgroup OOM during MGLRU state switching

2026-04-05T20:53:33+00:00

When the Multi-Gen LRU (MGLRU) state is toggled dynamically, a race condition exists between the state switching and the memory reclaim path. This can lead to unexpected cgroup OOM kills, even when plenty of reclaimable memory is available. Problem Description ================== The issue arises from a "reclaim vacuum" during the transition. 1. When disabling MGLRU, lru_gen_change_state() sets lrugen->enabled to false before the pages are drained from MGLRU lists back to traditional LRU lists. 2. Concurrent reclaimers in shrink_lruvec() see lrugen->enabled as false and skip the MGLRU path. 3. However, these pages might not have reached the traditional LRU lists yet, or the changes are not yet visible to all CPUs due to a lack of synchronization. 4. get_scan_count() subsequently finds traditional LRU lists empty, concludes there is no reclaimable memory, and triggers an OOM kill. A similar race can occur during enablement, where the reclaimer sees the new state but the MGLRU lists haven't been populated via fill_evictable() yet. Solution ======== Introduce a 'switching' state (`lru_switch`) to bridge the transition. When transitioning, the system enters this intermediate state where the reclaimer is forced to attempt both MGLRU and traditional reclaim paths sequentially. This ensures that folios remain visible to at least one reclaim mechanism until the transition is fully materialized across all CPUs. Race & Mitigation ================ A race window exists between checking the 'draining' state and performing the actual list operations. For instance, a reclaimer might observe the draining state as false just before it changes, leading to a suboptimal reclaim path decision. However, this impact is effectively mitigated by the kernel's reclaim retry mechanism (e.g., in do_try_to_free_pages). If a reclaimer pass fails to find eligible folios due to a state transition race, subsequent retries in the loop will observe the updated state and correctly direct the scan to the appropriate LRU lists. This ensures the transient inconsistency does not escalate into a terminal OOM kill. This effectively reduce the race window that previously triggered OOMs under high memory pressure. This fix has been verified on v7.0.0-rc1; dynamic toggling of MGLRU functions correctly without triggering unexpected OOM kills. Link: https://lkml.kernel.org/r/20260319-b4-switch-mglru-v2-v5-1-8898491e5f17@gmail.com Signed-off-by: Leno Hou Acked-by: Yafang Shao Reviewed-by: Barry Song Reviewed-by: Axel Rasmussen Cc: Yuanchu Xie Cc: Wei Xu Cc: Jialing Wang Cc: Yu Zhao Cc: Kairui Song Cc: Bingfang Guo Signed-off-by: Andrew Morton

mm: support batched checking of the young flag for MGLRU

2026-04-05T20:53:16+00:00

Use the batched helper test_and_clear_young_ptes_notify() to check and clear the young flag to improve the performance during large folio reclamation when MGLRU is enabled. Meanwhile, we can also support batched checking the young and dirty flag when MGLRU walks the mm's pagetable to update the folios' generation counter. Since MGLRU also checks the PTE dirty bit, use folio_pte_batch_flags() with FPB_MERGE_YOUNG_DIRTY set to detect batches of PTEs for a large folio. Then we can remove the ptep_test_and_clear_young_notify() since it has no users now. Note that we also update the 'young' counter and 'mm_stats[MM_LEAF_YOUNG]' counter with the batched count in the lru_gen_look_around() and walk_pte_range(). However, the batched operations may inflate these two counters, because in a large folio not all PTEs may have been accessed. (Additionally, tracking how many PTEs have been accessed within a large folio is not very meaningful, since the mm core actually tracks access/dirty on a per-folio basis, not per page). The impact analysis is as follows: 1. The 'mm_stats[MM_LEAF_YOUNG]' counter has no functional impact and is mainly for debugging. 2. The 'young' counter is used to decide whether to place the current PMD entry into the bloom filters by suitable_to_scan() (so that next time we can check whether it has been accessed again), which may set the hash bit in the bloom filters for a PMD entry that hasn't seen much access. However, bloom filters inherently allow some error, so this effect appears negligible. Link: https://lkml.kernel.org/r/378f4acf7d07410aa7c2e4b49d56bb165918eb34.1772778858.git.baolin.wang@linux.alibaba.com Signed-off-by: Baolin Wang Reviewed-by: Rik van Riel Acked-by: David Hildenbrand (Arm) Cc: Alistair Popple Cc: Axel Rasmussen Cc: Barry Song Cc: Catalin Marinas Cc: Dev Jain Cc: Jann Horn Cc: Johannes Weiner Cc: Liam Howlett Cc: Lorenzo Stoakes (Oracle) Cc: Matthew Wilcox (Oracle) Cc: Michal Hocko Cc: Mike Rapoport Cc: Qi Zheng Cc: Ryan Roberts Cc: Shakeel Butt Cc: Suren Baghdasaryan Cc: Vlastimil Babka Cc: Wei Xu Cc: Will Deacon Cc: Yuanchu Xie Signed-off-by: Andrew Morton

mm: rmap: add a ZONE_DEVICE folio warning in folio_referenced()

2026-04-05T20:53:15+00:00

The folio_referenced() is used to test whether a folio was referenced during reclaim. Moreover, ZONE_DEVICE folios are controlled by their device driver, have a lifetime tied to that driver, and are never placed on the LRU list. That means we should never try to reclaim ZONE_DEVICE folios, so add a warning to catch this unexpected behavior in folio_referenced() to avoid confusion, as discussed in the previous thread[1]. [1] https://lore.kernel.org/all/16fb7985-ec0f-4b56-91e7-404c5114f899@kernel.org/ Link: https://lkml.kernel.org/r/64d6fb2a33f7101e1d4aca2c9052e0758b76d492.1772778858.git.baolin.wang@linux.alibaba.com Signed-off-by: Baolin Wang Reviewed-by: Alistair Popple Acked-by: David Hildenbrand (Arm) Cc: Axel Rasmussen Cc: Barry Song Cc: Catalin Marinas Cc: Dev Jain Cc: Jann Horn Cc: Johannes Weiner Cc: Liam Howlett Cc: Lorenzo Stoakes (Oracle) Cc: Matthew Wilcox (Oracle) Cc: Michal Hocko Cc: Mike Rapoport Cc: Qi Zheng Cc: Rik van Riel Cc: Ryan Roberts Cc: Shakeel Butt Cc: Suren Baghdasaryan Cc: Vlastimil Babka Cc: Wei Xu Cc: Will Deacon Cc: Yuanchu Xie Signed-off-by: Andrew Morton

mm: add folio_test_lazyfree helper

2026-04-05T20:53:03+00:00

Add folio_test_lazyfree() function to identify lazy-free folios to improve code readability. Link: https://lkml.kernel.org/r/20260221093918.1456187-4-vernon2gm@gmail.com Signed-off-by: Vernon Yang Acked-by: David Hildenbrand (Red Hat) Reviewed-by: Lance Yang Reviewed-by: Dev Jain Reviewed-by: Barry Song Cc: Baolin Wang Cc: Liam Howlett Cc: Lorenzo Stoakes Cc: Masami Hiramatsu Cc: Mathieu Desnoyers Cc: Nico Pache Cc: Ryan Roberts Cc: Steven Rostedt Cc: Zi Yan Signed-off-by: Andrew Morton

mm/rmap: clear vma->anon_vma on error

2026-03-22T00:36:33+00:00

Commit 542eda1a8329 ("mm/rmap: improve anon_vma_clone(), unlink_anon_vmas() comments, add asserts") alters the way errors are handled, but overlooked one important aspect of clean up. When a VMA encounters an error state in anon_vma_clone() (that is, on attempted allocation of anon_vma_chain objects), it cleans up partially established state in cleanup_partial_anon_vmas(), before returning an error. However, this occurs prior to anon_vma->num_active_vmas being incremented, and it also fails to clear the VMA's vma->anon_vma field, which remains in place. This is immediately an inconsistent state, because anon_vma->num_active_vmas is supposed to track the number of VMAs whose vma->anon_vma field references that anon_vma, and now that count is off-by-negative-1 for each VMA for which this error state has occurred. When VMAs are unlinked from this anon_vma, unlink_anon_vmas() will eventually underflow anon_vma->num_active_vmas, which will trigger a warning. This will always eventually happen, as we unlink anon_vma's at process teardown. It could also cause maybe_reuse_anon_vma() to incorrectly permit the reuse of an anon_vma which has active VMAs attached, which will lead to a persistently invalid state. The solution is to clear the VMA's anon_vma field when we clean up partial state, as the fact we are doing so indicates clearly that the VMA is not correctly integrated into the anon_vma tree and thus this field is invalid. Link: https://lkml.kernel.org/r/20260318122632.63404-1-ljs@kernel.org Fixes: 542eda1a8329 ("mm/rmap: improve anon_vma_clone(), unlink_anon_vmas() comments, add asserts") Signed-off-by: Lorenzo Stoakes (Oracle) Reported-by: Sasha Levin Closes: https://lore.kernel.org/linux-mm/20260302151547.2389070-1-sashal@kernel.org/ Reported-by: Jiakai Xu Closes: https://lore.kernel.org/linux-mm/CAFb8wJvRhatRD-9DVmr5v5pixTMPEr3UKjYBJjCd09OfH55CKg@mail.gmail.com/ Acked-by: David Hildenbrand (Arm) Acked-by: Vlastimil Babka (SUSE) Tested-by: Jiakai Xu Acked-by: Harry Yoo Cc: Jann Horn Cc: Liam Howlett Cc: Rik van Riel Cc: Sasha Levin (Microsoft) Signed-off-by: Andrew Morton

mm/huge_memory: fix early failure try_to_migrate() when split huge pmd for shared THP

2026-03-10T23:01:49+00:00

Commit 60fbb14396d5 ("mm/huge_memory: adjust try_to_migrate_one() and split_huge_pmd_locked()") return false unconditionally after split_huge_pmd_locked(). This may fail try_to_migrate() early when TTU_SPLIT_HUGE_PMD is specified. The reason is the above commit adjusted try_to_migrate_one() to, when a PMD-mapped THP entry is found, and TTU_SPLIT_HUGE_PMD is specified (for example, via unmap_folio()), return false unconditionally. This breaks the rmap walk and fail try_to_migrate() early, if this PMD-mapped THP is mapped in multiple processes. The user sensible impact of this bug could be: * On memory pressure, shrink_folio_list() may split partially mapped folio with split_folio_to_list(). Then free unmapped pages without IO. If failed, it may not be reclaimed. * On memory failure, memory_failure() would call try_to_split_thp_page() to split folio contains the bad page. If succeed, the PG_has_hwpoisoned bit is only set in the after-split folio contains @split_at. By doing so, we limit bad memory. If failed to split, the whole folios is not usable. One way to reproduce: Create an anonymous THP range and fork 512 children, so we have a THP shared mapped in 513 processes. Then trigger folio split with /sys/kernel/debug/split_huge_pages debugfs to split the THP folio to order 0. Without the above commit, we can successfully split to order 0. With the above commit, the folio is still a large folio. And currently there are two core users of TTU_SPLIT_HUGE_PMD: * try_to_unmap_one() * try_to_migrate_one() try_to_unmap_one() would restart the rmap walk, so only try_to_migrate_one() is affected. We can't simply revert commit 60fbb14396d5 ("mm/huge_memory: adjust try_to_migrate_one() and split_huge_pmd_locked()"), since it removed some duplicated check covered by page_vma_mapped_walk(). This patch fixes this by restart page_vma_mapped_walk() after split_huge_pmd_locked(). Since we cannot simply return "true" to fix the problem, as that would affect another case: When invoking folio_try_share_anon_rmap_pmd() from split_huge_pmd_locked(), the latter can fail and leave a large folio mapped through PTEs, in which case we ought to return true from try_to_migrate_one(). This might result in unnecessary walking of the rmap but is relatively harmless. Link: https://lkml.kernel.org/r/20260305015006.27343-1-richard.weiyang@gmail.com Fixes: 60fbb14396d5 ("mm/huge_memory: adjust try_to_migrate_one() and split_huge_pmd_locked()") Signed-off-by: Wei Yang Reviewed-by: Baolin Wang Reviewed-by: Zi Yan Tested-by: Lance Yang Reviewed-by: Lance Yang Reviewed-by: Gavin Guo Acked-by: David Hildenbrand (arm) Reviewed-by: Lorenzo Stoakes (Oracle) Cc: Signed-off-by: Andrew Morton