kernel/git/stable/linux.git/mm/memory-failure.c, branch master

mm: introduce and use vma_filebacked_address()

2026-08-25T01:42:51+00:00

In cases where we know that the VMA is file-backed, use vma_filebacked_address() rather than vma_address(). This lays the foundation for using the anonymous page offset via vma_anon_address(). Also add an assert to ensure that the VMA whose address is required is not anonymous. No functional change intended. Link: https://lore.kernel.org/20260813-b4-scalable-cow-virt-pgoff-v5-6-c21581c0c3c8@kernel.org Signed-off-by: Lorenzo Stoakes (ARM) Tested-by: syzbot@syzkaller.appspotmail.com Acked-by: David Hildenbrand (Arm) Cc: Adrian Hunter Cc: Alexander Deucher Cc: Alexander Gordeev Cc: Alexander Shishkin Cc: Alistair Popple Cc: Arnaldo Carvalho de Melo Cc: Arnd Bergmann Cc: Baolin Wang Cc: Baoquan He Cc: Barry Song Cc: Boris Brezillon Cc: Byungchul Park Cc: Chengming Zhou Cc: Chris Li Cc: Christan König Cc: Christian Borntraeger Cc: Claudio Imbrenda Cc: Dave Airlie Cc: Dev Jain Cc: Gerald Schaefer Cc: Greg Kroah-Hartman Cc: Gregory Price (Meta) Cc: Harry Yoo Cc: Heiko Carstens Cc: Huang Ray Cc: "Huang, Ying" Cc: Ian Rogers Cc: Ingo Molnar Cc: James Clark Cc: Jan Kara Cc: Jann Horn Cc: Janosch Frank Cc: Jason Gunthorpe Cc: Jiri Olsa Cc: John Hubbard Cc: Joshua Hahn Cc: Kairui Song Cc: Kees Cook Cc: Kemeng Shi Cc: Lance Yang Cc: Liam R. Howlett Cc: Liviu Dudau Cc: Maarten Lankhorst Cc: Marc Rutland Cc: "Masami Hiramatsu (Google)" Cc: Matthew Auld Cc: Matthew Brost Cc: Matthew Wilcox (Oracle) Cc: Maxime Ripard Cc: Miaohe Lin Cc: Michal Hocko Cc: Mike Rapoport Cc: Muchun Song Cc: Namhyung kim Cc: Naoya Horiguchi Cc: Nhat Pham Cc: Nico Pache Cc: Oleg Nesterov Cc: Oscar Salvador Cc: Pedro Falcato Cc: Peter Xu Cc: Peter Zijlstra Cc: Rakie Kim Cc: Rik van Riel Cc: Rodrigo Vivi Cc: Ryan Roberts Cc: Steven Price Cc: Suren Baghdasaryan Cc: Sven Schnelle Cc: Thomas Hellström Cc: Thomas Zimemrmann Cc: Vasily Gorbik Cc: Vlastimil Babka Cc: xu xin Cc: Zi Yan Signed-off-by: Andrew Morton

mm/rmap: rename anon_vma_interval_tree_() to anon_rmap_tree_()

2026-08-05T02:18:58+00:00

To be consistent with the newly renamed mapping_rmap_tree_*(), rename the anon_vma_interval_tree_*() helpers to anon_rmap_tree_*(). No functional change intended. Link: https://lore.kernel.org/20260710-b4-pre-scalable-cow-v2-11-2a5aa403d977@kernel.org Signed-off-by: Lorenzo Stoakes Reviewed-by: Gregory Price Reviewed-by: Vlastimil Babka (SUSE) Reviewed-by: Zi Yan Cc: Ackerley Tng Cc: David Hildenbrand (Arm) Cc: Kai Huang Cc: Marek Szyprowski Cc: Pedro Falcato Cc: SJ Park Cc: Thomas Zimmermann Cc: Liam R. Howlett (Oracle) Signed-off-by: Andrew Morton

mm/rmap: parameterise anon_vma_interval_tree_*() by anon_vma

2026-08-05T02:18:58+00:00

Similar to what we did with mapping_rmap_tree*(), let's declare anon_vma_interval_tree*() in terms of anon_vma rather than rb_root_cached. In each case the rb tree referenced is &anon_vma->rb_root, so just pass anon_vma and the functions can figure this out themselves. Also update the VMA userland tests to reflect the change. No functional change intended. Link: https://lore.kernel.org/20260710-b4-pre-scalable-cow-v2-9-2a5aa403d977@kernel.org Signed-off-by: Lorenzo Stoakes Reviewed-by: Pedro Falcato Reviewed-by: Gregory Price Reviewed-by: Vlastimil Babka (SUSE) Reviewed-by: Zi Yan Cc: Ackerley Tng Cc: David Hildenbrand (Arm) Cc: Kai Huang Cc: Marek Szyprowski Cc: SJ Park Cc: Thomas Zimmermann Cc: Liam R. Howlett (Oracle) Signed-off-by: Andrew Morton

mm/rmap: rename vma_interval_tree_() to mapping_rmap_tree_()

2026-08-05T02:18:58+00:00

The family of vma_interval_tree_() functions manipulate the address_space (which, of course, is generally referred to as 'mapping') reverse mapping, but are named the 'VMA' interval tree. VMAs may be mapped by an anon_vma, an address_space, or both. Therefore calling the mapping interval tree a 'VMA' interval tree is rather confusing. This is also inconsistent with the anon_vma_interval_tree_*() functions which explicitly reference the rmap object to which they pertain. Rename the vma_interval_tree_*() functions to mapping_rmap_tree_*() to correct this. We will rename the anon rmap functions similarly in a subsequent patch. No functional change intended. Link: https://lore.kernel.org/20260710-b4-pre-scalable-cow-v2-8-2a5aa403d977@kernel.org Signed-off-by: Lorenzo Stoakes Reviewed-by: Gregory Price Reviewed-by: Vlastimil Babka (SUSE) Reviewed-by: Zi Yan Cc: Ackerley Tng Cc: David Hildenbrand (Arm) Cc: Kai Huang Cc: Marek Szyprowski Cc: Pedro Falcato Cc: SJ Park Cc: Thomas Zimmermann Cc: Liam R. Howlett (Oracle) Signed-off-by: Andrew Morton

mm/rmap: parameterise vma_interval_tree_*() by address_space

2026-08-05T02:18:57+00:00

The file-backed mapping interval tree functions vma_interval_tree_*() accept a raw rb_root_cached pointer to determine the tree in which they are operating. However, in each case, this is always associated with an address_space data type. So simply pass a pointer to that instead to simplify the code, and more clearly differentiate between these operations and those concerning anonymous mappings. While we're here, make the generated interval tree functions static as they do not need to be used externally (any previously existing external users have now been removed). We also rename VMA parameters from 'node' to 'vma' as calling this a node is simply confusing, update the input index types to pgoff_t since they reference page offsets and rename the parameters to pgoff_start and pgoff_last. No functional change intended. Link: https://lore.kernel.org/20260710-b4-pre-scalable-cow-v2-6-2a5aa403d977@kernel.org Signed-off-by: Lorenzo Stoakes Reviewed-by: Pedro Falcato Reviewed-by: Gregory Price Reviewed-by: Vlastimil Babka (SUSE) Reviewed-by: Zi Yan Cc: Ackerley Tng Cc: David Hildenbrand (Arm) Cc: Kai Huang Cc: Marek Szyprowski Cc: SJ Park Cc: Thomas Zimmermann Cc: Liam R. Howlett (Oracle) Signed-off-by: Andrew Morton

mm/memory-failure: add panic option for unrecoverable pages

2026-08-05T02:18:28+00:00

Add a sysctl panic_on_unrecoverable_memory_failure (disabled by default) that triggers a kernel panic when memory_failure() encounters pages that cannot be recovered. This provides a clean crash with useful debug information rather than allowing silent data corruption or a delayed crash at an unrelated code path. Panic eligibility is intentionally narrow: only MF_MSG_KERNEL with result == MF_IGNORED panics. After the previous patch, MF_MSG_KERNEL covers PG_reserved pages and the kernel-owned pages promoted from get_hwpoison_page() via -ENOTRECOVERABLE (slab, page tables, large-kmalloc). All other action types are excluded: - MF_MSG_GET_HWPOISON and MF_MSG_KERNEL_HIGH_ORDER can be reached by transient refcount races with the page allocator (an in-flight buddy allocation has refcount 0 and is no longer on the buddy free list, briefly), and panicking on them would risk killing the box for what is actually a recoverable userspace page. - MF_MSG_UNKNOWN means identify_page_state() could not classify the page; that is precisely the wrong basis for a panic decision. Link: https://lore.kernel.org/20260630-ecc_panic-v10-4-c6ed5b62eea2@debian.org Signed-off-by: Breno Leitao Acked-by: Miaohe Lin Cc: David Hildenbrand (Arm) Cc: Jonathan Corbet Cc: Lance Yang Cc: Liam R. Howlett Cc: Lorenzo Stoakes Cc: "Masami Hiramatsu (Google)" Cc: Mathieu Desnoyers Cc: Michal Hocko Cc: Mike Rapoport Cc: Naoya Horiguchi Cc: Shuah Khan Cc: Steven Rostedt Cc: Suren Baghdasaryan Cc: Vlastimil Babka Signed-off-by: Andrew Morton

mm/memory-failure: report MF_MSG_KERNEL for unrecoverable kernel pages

2026-08-05T02:18:28+00:00

The previous patch teaches get_any_page() to return -ENOTRECOVERABLE for stable unhandlable kernel pages (PG_reserved, slab, page tables, large-kmalloc). memory_failure() still folds every negative return into MF_MSG_GET_HWPOISON, so callers that want to react to the unrecoverable cases (a panic option, smarter logging) cannot tell them apart from transient page-allocator races. Turn the post-call branch into a switch over the get_hwpoison_page() return code: map -ENOTRECOVERABLE to MF_MSG_KERNEL and any other negative return to MF_MSG_GET_HWPOISON. case 0 keeps the existing free-buddy / kernel-high-order handling and case 1 falls through to the rest of memory_failure() unchanged. The MF_MSG_KERNEL label and tracepoint string are kept as "reserved kernel page" to avoid breaking userspace tools that match on those literals; the enum value still adequately tags the failure even though it now also covers slab, page tables and large-kmalloc pages. Link: https://lore.kernel.org/20260630-ecc_panic-v10-3-c6ed5b62eea2@debian.org Signed-off-by: Breno Leitao Suggested-by: David Hildenbrand Acked-by: David Hildenbrand (Arm) Acked-by: Miaohe Lin Cc: Jonathan Corbet Cc: Lance Yang Cc: Liam R. Howlett Cc: Lorenzo Stoakes Cc: "Masami Hiramatsu (Google)" Cc: Mathieu Desnoyers Cc: Michal Hocko Cc: Mike Rapoport Cc: Naoya Horiguchi Cc: Shuah Khan Cc: Steven Rostedt Cc: Suren Baghdasaryan Cc: Vlastimil Babka Signed-off-by: Andrew Morton

mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE

2026-08-05T02:18:28+00:00

get_any_page() collapses every HWPoisonHandlable() rejection into a single -EIO via the __get_hwpoison_page() -> -EBUSY -> shake_page() -> retry path. That is correct for the transient case (a userspace folio briefly off LRU during migration or compaction, which a later shake can drag back), but wrong for stable kernel-owned pages: slab, page-table, large-kmalloc and PG_reserved pages will never become HWPoisonHandlable(), so the retry loop is wasted work and the final -EIO loses the "this is structurally unrecoverable" information. memory_failure() then maps -EIO into MF_MSG_GET_HWPOISON, which the panic-on-unrecoverable sysctl deliberately does not act on. Introduce is_kernel_owned_page(), a small predicate that positively identifies pages the hwpoison handler cannot recover from: is_kernel_owned_page(p) := PageReserved(p) || PageSlab(head) || PageTable(head) || PageLargeKmalloc(head) where head = compound_head(p). PG_reserved is a per-page flag (PF_NO_COMPOUND) and is tested on the page directly. The slab, page-table and large-kmalloc page-type bits are only stored on the head page, so those tests resolve the compound head first, then re-read compound_head(page) afterwards: a concurrent split or compound free that moves head invalidates the just-read flags and the loop retries. The lookup still takes no refcount, mirroring the rest of get_any_page(); the recheck closes the common split race, and a residual free->alloc->free in the same window can only mis-tag a genuinely poisoned page, never reclassify a handlable one. No MF_SOFT_OFFLINE / page_has_movable_ops() opt-out is needed: a movable_ops page is always PageOffline or PageZsmalloc, whose page_type is mutually exclusive with slab, page-table and large-kmalloc, and it never carries PG_reserved, so it can never match any of the checks above. The list is intentionally not exhaustive. vmalloc and kernel-stack pages, for example, do not carry a page_type bit and would need a different oracle; they keep going through the existing retry path unchanged. This is the smallest set we can identify with certainty by page type. Wire the helper into the top of get_any_page() to short-circuit those pages before the retry loop runs. On a hit, drop the caller's MF_COUNT_INCREASED reference (if any) and return -ENOTRECOVERABLE straight away. Pages outside the helper's positive list still take the existing retry path and return -EIO, leaving operator-visible behaviour for those cases unchanged. Extend the unhandlable-page pr_err() to fire for either errno and update the get_hwpoison_page() kerneldoc to document the new return. memory_failure() still folds every negative return into MF_MSG_GET_HWPOISON via its existing "else if (res < 0)" branch, so this patch on its own only changes the errno that soft_offline_page() can propagate to its callers. A follow-up wires -ENOTRECOVERABLE through memory_failure() and reports MF_MSG_KERNEL for the unrecoverable cases, which is what the panic_on_unrecoverable_memory_failure sysctl observes. Link: https://lore.kernel.org/20260630-ecc_panic-v10-2-c6ed5b62eea2@debian.org Signed-off-by: Breno Leitao Suggested-by: David Hildenbrand Suggested-by: Lance Yang Acked-by: Miaohe Lin Cc: Jonathan Corbet Cc: Liam R. Howlett Cc: Lorenzo Stoakes Cc: "Masami Hiramatsu (Google)" Cc: Mathieu Desnoyers Cc: Michal Hocko Cc: Mike Rapoport Cc: Naoya Horiguchi Cc: Shuah Khan Cc: Steven Rostedt Cc: Suren Baghdasaryan Cc: Vlastimil Babka Signed-off-by: Andrew Morton

mm/memory-failure: drop dead error_states[] entry for reserved pages

2026-08-05T02:18:27+00:00

Patch series "mm/memory-failure: add panic option for unrecoverable pages", v10. A multi-bit ECC error on a kernel-owned page that the memory failure handler cannot recover is currently swallowed: PG_hwpoison is set, the event is logged, and the kernel keeps running. The corrupted memory remains accessible to the kernel and either drives silent data corruption or surfaces seconds-to-minutes later as an apparently unrelated crash. In a large fleet that delayed, unattributable crash turns into significant engineering effort to root-cause; in a kdump configuration, by the time the crash happens the original error context (faulting PFN, MCE/GHES record, page state) is long gone. This series adds an opt-in sysctl, vm.panic_on_unrecoverable_memory_failure, that converts an unrecoverable kernel-page hwpoison event into an immediate panic with a clean dmesg/vmcore that still contains the original failure context. The default is disabled so existing workloads see no change. There is a selftest that test different cases, and I tested it using the following variants: ┌─────────┬──────────┬───────────────────────────────────────────────────────────┐ │ Variant │ PFN │ Result │ ├─────────┼──────────┼───────────────────────────────────────────────────────────┤ │ rodata │ 0x2600 │ Panic with "Memory failure: 0x2600: unrecoverable page" │ ├─────────┼──────────┼───────────────────────────────────────────────────────────┤ │ slab │ 0x100032 │ Panic with "Memory failure: 0x100032: unrecoverable page" │ ├─────────┼──────────┼───────────────────────────────────────────────────────────┤ │ pgtable │ 0x100000 │ Panic with "Memory failure: 0x100000: unrecoverable page" │ └─────────┴──────────┴───────────────────────────────────────────────────────────┘ Each one shows the same call trace, exactly the path the series builds: hard_offline_page_store → memory_failure → action_result → panic("Memory failure: %#lx: unrecoverable page") This patch (of 5): The first entry of error_states[], { reserved, reserved, MF_MSG_KERNEL, me_kernel }, is unreachable. identify_page_state() has two callers, and neither one can dispatch a PG_reserved page to me_kernel(): * memory_failure() reaches identify_page_state() only after get_hwpoison_page() returned 1. get_any_page() reaches that return only via __get_hwpoison_page(), which only takes a refcount when the page is HWPoisonHandlable(). HWPoisonHandlable() is an allowlist for LRU, free-buddy, and (for soft-offline) movable_ops pages -- PG_reserved pages do not satisfy any of these, so they fail with -EBUSY/-EIO long before identify_page_state() runs. * try_memory_failure_hugetlb() reaches identify_page_state() only via the MF_HUGETLB_IN_USED branch, where the page is necessarily a hugetlb folio. hugetlb folios don't carry PG_reserved at that point: hugetlb_folio_init_vmemmap() calls __folio_clear_reserved() during init, so the reserved entry would not match even if it were still present. me_kernel() never executes and the entry exists only to be matched against by code that cannot see it. Drop the entry, the me_kernel() helper, and the now-unused "reserved" macro. Leave the MF_MSG_KERNEL enum value in place: it remains part of the tracepoint and pr_err() string tables, and follow-on work to classify unrecoverable kernel pages can reuse it without churning the user-visible enum. No functional change. Link: https://lore.kernel.org/20260630-ecc_panic-v10-0-c6ed5b62eea2@debian.org Link: https://lore.kernel.org/20260630-ecc_panic-v10-1-c6ed5b62eea2@debian.org Signed-off-by: Breno Leitao Suggested-by: David Hildenbrand Acked-by: David Hildenbrand (Arm) Reviewed-by: Lance Yang Acked-by: Miaohe Lin Cc: Jonathan Corbet Cc: Liam R. Howlett Cc: Lorenzo Stoakes Cc: "Masami Hiramatsu (Google)" Cc: Mathieu Desnoyers Cc: Michal Hocko Cc: Mike Rapoport Cc: Naoya Horiguchi Cc: Shuah Khan Cc: Steven Rostedt Cc: Suren Baghdasaryan Cc: Vlastimil Babka Signed-off-by: Andrew Morton

mm: split out internal page_alloc.h

2026-07-31T02:40:41+00:00

internal.h is a bit bloated, seems like time for a page_alloc.h. Where it wasn't obvious, the heuristic for deciding what goes into this new header was "does it support/correspond to a definition in mm/page_alloc.c?" Only need to include it from ~20 .c files out of ~150 so this does seem like a genuine reduction in scopes, which is nice. And there's no circular internal.h<->page_alloc.h dependency, so it seems worthwhile to split this up before that inevitably emerges! Link: https://lore.kernel.org/20260703-alloc-trylock-v5-4-c87b714e19d3@google.com Signed-off-by: Brendan Jackman Suggested-by: "David Hildenbrand (Arm)" Link: https://lore.kernel.org/all/41e92bab-6882-401a-8de9-154adbdcfb36@kernel.org/ Reviewed-by: Vlastimil Babka (SUSE) Acked-by: Zi Yan Signed-off-by: Andrew Morton

kernel/git/stable/linux.git/mm/memory-failure.c, branch master

mm: introduce and use vma_filebacked_address()

mm/rmap: rename anon_vma_interval_tree_*() to anon_rmap_tree_*()

mm/rmap: parameterise anon_vma_interval_tree_*() by anon_vma

mm/rmap: rename vma_interval_tree_*() to mapping_rmap_tree_*()

mm/rmap: parameterise vma_interval_tree_*() by address_space

mm/memory-failure: add panic option for unrecoverable pages

mm/memory-failure: report MF_MSG_KERNEL for unrecoverable kernel pages

mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE

mm/memory-failure: drop dead error_states[] entry for reserved pages

mm: split out internal page_alloc.h

mm/rmap: rename anon_vma_interval_tree_() to anon_rmap_tree_()

mm/rmap: rename vma_interval_tree_() to mapping_rmap_tree_()