| Age | Commit message (Collapse) | Author |
|---|
|
https://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest.git
|
|
https://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git
# Conflicts:
# kernel/smp.c
|
|
https://git.kernel.org/pub/scm/linux/kernel/git/herbert/cryptodev-2.6.git
|
|
https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
|
|
mm-unstable into for-next
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
|
|
mm-hotfixes-unstable into for-next-fixes
Signed-off-by: David Hildenbrand (Arm) <david@kernel.org>
|
|
plist_requeue() is meant to move a node to the end of its own priority
run. When the node heads the *last* priority bucket it is instead placed
at the head of the whole list, leaving the plist unsorted:
built: A(prio 0) B(prio 1) C(prio 1)
requeue(B): B(prio 1) A(prio 0) C(prio 1)
expected: A(prio 0) C(prio 1) B(prio 1)
prio_list is a *headless* circular ring of the nodes that lead each
priority bucket. The shortcut added by commit 95d4b3450ebe ("lib/plist.c:
add shortcut for plist_requeue()") takes
iter = list_entry(iter->prio_list.next, struct plist_node,
prio_list);
node_next = &iter->node_list;
which from the last bucket wraps round to the *first* bucket, so node_next
ends up pointing at the head of the list rather than at its end. The
plist_for_each_continue() loop immediately below it computes the correct
answer (&head->node_list) for that case.
With any bucket after it the shortcut is correct, which is why this went
unnoticed: the benchmark in that commit measured elapsed time and never
checked the resulting order.
Keep the shortcut -- it is a real win -- but exclude the case where iter's
bucket is the last one, which is exactly when its ring successor is the
first bucket again.
Reachable from mm/swapfile.c, which rotates swap_avail_heads[] with
plist_requeue(). It takes three or more swap devices: at least two
distinct priorities, so that a later bucket exists for the ring to wrap
round from, and two or more devices sharing the lowest priority, so that
plist_requeue() does not return early. One device per priority returns
early at the node->prio != iter->prio test. A single priority is also
safe, but for a different reason worth stating: with one bucket no node is
ever linked onto prio_list at all -- plist_add() skips it for the first
node and for every node whose predecessor shares its priority -- so
list_empty(&iter->prio_list) holds and the shortcut is never entered.
Tested by driving three implementations -- the pre-95d4b3450ebe code,
current mainline, and this patch -- through 1,084,492 identical random
add/del/requeue operations over 24 nodes and 1..5 distinct priorities,
comparing the resulting node_list node for node after every operation:
variant differs from pre-95d4b3450ebe left list unsorted
pre-95d4b3450ebe -- (reference) 0
mainline 289,297 276,660
this patch 0 0
Link: https://lore.kernel.org/20260903222456.1881786-1-handyhandyman.adam@gmail.com
Fixes: 95d4b3450ebe ("lib/plist.c: add shortcut for plist_requeue()")
Signed-off-by: Adam Harshbarger <handyhandyman.adam@gmail.com>
Assisted-by: Claude:claude-opus-5
Cc: I Hsin Cheng <richard120310@gmail.com>
Cc: <stable@vger.kernel.org> # v6.15+
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Keep variable assignments in the same order throughout the function to
make the code easier to follow.
No functional changes.
Link: https://lore.kernel.org/20260811121831.577848-3-yuntao.wang@linux.dev
Signed-off-by: Yuntao Wang <yuntao.wang@linux.dev>
Cc: Jason Baron <jbaron@akamai.com>
Cc: Jim Cromie <jim.cromie@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Patch series "dyndbg: fix incorrect mod_ct value in dynamic_debug_init()".
Fix and clean up dynamic_debug_init().
This patch (of 2):
Suppose all `struct _ddebug` instances belong to the same module, mod_ct
should be 1, but it is currently 0.
mod_ct is incremented only when iter->modname changes, i.e. when the loop
encounters the first _ddebug entry of a new module:
if (strcmp(modname, iter->modname)) {
mod_ct++;
...
}
If all _ddebug entries belong to the same module, strcmp() never returns
nonzero, so mod_ct remains 0.
However, the last (and in this case only) module is added after the loop:
di.num_descs = mod_sites;
di.descs = iter_mod_start;
ret = ddebug_add_module(&di, modname);
Thus, mod_ct should be incremented before adding this final module.
The bug only affects the diagnostic message printed by vpr_info():
"%d prdebugs in %d modules, ..."
It reports one fewer module than the actual number of modules. There is
no userspace-visible runtime effect; the dynamic debug tables themselves
are initialized correctly.
Fix it.
Link: https://lore.kernel.org/20260811121831.577848-1-yuntao.wang@linux.dev
Link: https://lore.kernel.org/20260811121831.577848-2-yuntao.wang@linux.dev
Signed-off-by: Yuntao Wang <yuntao.wang@linux.dev>
Cc: Jason Baron <jbaron@akamai.com>
Cc: Jim Cromie <jim.cromie@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Calling __decompress() or unxz() with fill == NULL && flush == NULL && in
== NULL is invalid, thus there were no memory leaks even though it might
have looked like that. Move the conditional free() calls so that it's
obvious that there are no leaks.
Link: https://lore.kernel.org/20260901014138.22699-1-skunkolee@gmail.com
Signed-off-by: Ivy Lopez <skunkolee@gmail.com>
Closes: https://bugzilla.kernel.org/show_bug.cgi?id=207113
Link: https://lore.kernel.org/lkml/20241006072542.66442-2-t.v.s10123@gmail.com/T/
Link: https://lore.kernel.org/lkml/20260825191333.34276-1-skunkolee@gmail.com/T/
Reviewed-by: Lasse Collin <lasse.collin@tukaani.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Since the AVX-512 optimized RAID6 code uses ZMM registers, execute
vzeroupper before returning from it. This is needed to avoid degrading
the performance of any later SSE code that may happen to be executed.
Link: https://lore.kernel.org/20260831212316.213896-1-ebiggers@kernel.org
Fixes: e0a491c12968 ("lib/raid6: Add AVX512 optimized gen_syndrome functions")
Fixes: 13c520b2993c ("lib/raid6: Add AVX512 optimized recovery functions")
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Cc: Christoph Hellwig <hch@lst.de>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Since the AVX2 optimized RAID6 code uses YMM registers, execute vzeroupper
before returning from it. This is needed to avoid degrading the
performance of any later SSE code that may happen to be executed.
Link: https://lore.kernel.org/20260831212308.213855-1-ebiggers@kernel.org
Fixes: 2c935842bdb4 ("lib/raid6: Add AVX2 optimized gen_syndrome functions")
Fixes: 7056741fd9fc ("lib/raid6: Add AVX2 optimized recovery functions")
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Cc: Christoph Hellwig <hch@lst.de>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Since the AVX optimized XOR code uses YMM registers, execute vzeroupper
before returning from it. This is needed to avoid degrading the
performance of any later SSE code that may happen to be executed.
Link: https://lore.kernel.org/20260831212248.213805-1-ebiggers@kernel.org
Fixes: ea4d26ae24e5 ("raid5: add AVX optimized RAID5 checksumming")
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Cc: Christoph Hellwig <hch@lst.de>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
The errseq_t infrastructure (lib/errseq.c) underpins writeback error
reporting but has no regression tests. Its semantics are subtle enough to
have needed fixing before: commit b4678df184b3 ("errseq: Always report a
writeback error once") changed how unseen errors reach new samplers.
Add a KUnit suite covering the documented single-threaded semantics:
- a zeroed errseq_t is the "no error yet" epoch
- errors are recorded, overwrite one another, and both ends of the
valid errno range round-trip exactly
- an error nobody has seen samples as zero, so a check against a
fresh sample still reports it
- errseq_check_and_advance() reports a given error exactly once per
cursor and leaves the cursor in place when nothing has changed
- once an error has been seen, a fresh sample is current and a check
against it reports nothing
- the same error recorded again after being seen is reported again,
even to a cursor that consumed the first occurrence while another
cursor marked the repeat as seen
- independent cursors each observe each error
The lockless behaviour of errseq_t under concurrent updates and the WARN
path for invalid error values are deliberately out of scope.
Tested with ./tools/testing/kunit/kunit.py run, with a kunitconfig
enabling CONFIG_KUNIT=y and CONFIG_ERRSEQ_KUNIT_TEST=y; all 13 tests pass
under ARCH=um.
Link: https://lore.kernel.org/20260812-errseq-kunit-v1-1-312be4c3aa0d@gmail.com
Signed-off-by: Thomas Maarseveen <maarseveent@gmail.com>
Acked-by: Jeff Layton <jlayton@kernel.org>
Cc: David Gow <david@davidgow.net>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
These 7 DECOMPRESS_ options (e.g. DECOMPRESS_GZIP) currently have the
tristate type, but can never be set to M. Their only valid values are Y
and N, making them effectively booleans.
Let's make their types more accurate by changing them to 'bool'. Note
that this is only a code cleanup, there is no functional change.
These bistates were found by kconfirm, a static analysis tool for Kconfig.
Link: https://lore.kernel.org/20260817210311.2142999-1-julianbraha@gmail.com
Signed-off-by: Julian Braha <julianbraha@gmail.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Jani Nikula <jani.nikula@linux.intel.com>
Cc: Julia Lawall <julia.lawall@inria.fr>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
group_cpus_evenly() builds the managed-IRQ affinity spread used by
multi-queue devices such as NVMe. That spread is meant to be a property
of the static CPU topology: it walks cpu_present_mask and then
cpu_possible_mask so every hardware queue owns a fixed set of CPUs,
including CPUs that are offline at the time. A driver depends on that
partition staying stable across re-computation - the CPUs a queue is given
at probe must still describe the same queue after the device is later
reset and its affinity recomputed.
On an AMD system that stability breaks across an s2idle cycle. With CPUs
3-11 offlined and only CPUs 0-2 left online, the machine is suspended to
s2idle and resumed. The NVMe controller uses the simple-suspend quirk, so
resume fully re-initialises it and recomputes the affinity spread. The
system then hangs for roughly two minutes and stays sluggish afterwards,
the controller only making progress through its command-timeout poll:
nvme nvme0: I/O tag 898 (3382) QID 9 timeout, completion polled
nvme nvme0: I/O tag 398 (618e) QID 11 timeout, completion polled
QID 9 and QID 11 are the queues whose CPUs were offline when the spread
was recomputed. "completion polled" means the commands did finish in
hardware, but their interrupts were never delivered to a CPU that was
watching the queue, so nothing reaped them until the timeout fired.
It happens because commit 89802ca36c96 ("lib/group_cpus: make group CPU
cluster aware") derives the cluster groups from
topology_cluster_cpumask(), which lists only the cluster siblings that are
online when it is called. The resulting partition therefore depends on
the transient online mask rather than on the topology alone. Recomputed
on resume while the non-boot CPUs are still offline, it no longer matches
the boot-time partition, and a queue is left with an affinity that does
not cover the CPU it is meant to serve once that CPU comes back online.
The dependence is on the online mask, not on any AMD-specific behaviour,
so the same stall is reproducible on Intel platforms as well.
Make the cluster grouping depend on the complete cluster topology rather
than on whichever CPUs happen to be online. Snapshot the cluster masks
once while every present CPU is online and reuse that view for every later
spread. Every spread then groups from the same masks, so the partition
computed when the controller is reset matches the one computed at probe
and each queue's IRQ still covers the CPUs it serves. If the snapshot was
never taken, the cluster path is skipped and the plain present/possible
spread is used.
Link: https://lore.kernel.org/20260826171537.4167367-1-Vishal.Badole@amd.com
Fixes: 89802ca36c96 ("lib/group_cpus: make group CPU cluster aware")
Signed-off-by: Vishal Badole <Vishal.Badole@amd.com>
Cc: "Borislav Petkov (AMD)" <bp@alien8.de>
Cc: Radu Rendec <radu@rendec.net>
Cc: Thomas Gleixner <tglx@kernel.org>
Cc: Tim Chen <tim.c.chen@linux.intel.com>
Cc: Wangyang Guo <wangyang.guo@intel.com>
Cc: Tianyou Li <tianyou.li@intel.com>
Cc: Dan Liang <dan.liang@intel.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
klist_remove() waits until a node is unreferenced so that its caller can
free the containing object. klist_release() currently publishes
waiter->woken and wakes the waiter before its final accesses to the waiter
and node. klist_remove() can then return, allowing its stack waiter and
the containing object to be freed or reused while klist_release() is still
running. In particular, bus_remove_driver() can free drv->p while
__device_attach() walks the same bus klist with bus_for_each_drv().
On an arm64 Cortex-A72 system, an unpatched 7.2.0-rc3 kernel with
CONFIG_PREEMPT_RT=y and CONFIG_KASAN=y reproduced the bug through the
in-tree I2C/at24 path. KASAN reported a use-after-free in
klist_dec_and_del() reached from klist_next()/bus_for_each_drv() while
at24 was being unregistered.
Clear n_klist and take a task reference before publishing woken. Use
release/acquire accesses for that publication and wake the referenced
task. The task reference keeps the waiter task alive if it returns and
exits before wake_up_process().
Link: https://lore.kernel.org/20260827041708.31682-1-kmehltretter@gmail.com
Fixes: 8b0c250be489 ("[PATCH] add klist_node_attached() to determine if a node is on a list or not.")
Fixes: 210272a28465 ("driver core: Remove completion from struct klist_node")
Assisted-by: LLM
Signed-off-by: Karl Mehltretter <kmehltretter@gmail.com>
Cc: Danilo Krummrich <dakr@kernel.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: "Rafael J. Wysocki" <rafael@kernel.org>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Enabling the (possibly long-running benchmarks) by default may cause a big
delay in boot time in case of built-in tests. However, they can still
safely be enabled by default if all tests are modular, as they would only
run when requested explicitly by the system administrator.
Link: https://lore.kernel.org/64c6e0191bd8ccef0074ffbb09bd0584680d710b.1787584360.git.geert@linux-m68k.org
Signed-off-by: Geert Uytterhoeven <geert@linux-m68k.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Acked-by: Ard Biesheuvel <ardb@kernel.org>
Cc: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
fault_create_debugfs_attr() has always taken an extra dentry reference on
the created directory (attr->dname = dget(dir)) so that fail_dump() could
print the name via %pd from any context. Nothing anywhere in the tree
ever calls dput() on attr->dname.
For callers with a matching teardown, that unmatched reference causes one
dentry plus its attached inode to leak per fault_create_debugfs_attr /
debugfs_remove_recursive cycle. simple_recursive_removal() drops
debugfs's own +1 ref on the child dentry, but the dget()'d ref keeps its
refcount at 1: the dentry ends up unhashed but pinned, and its inode is
never freed.
Boot-once callers (mm/failslab, block/blk-core, etc.) leak exactly once at
init and never destroy the tree, so the impact there is bounded. But
per-lifecycle callers (drivers/nvme, drivers/infiniband/hw/hfi1,
drivers/mmc, drivers/iommu/iommufd, drivers/media, drivers/misc,
drivers/gpu/drm/msm, drivers/crypto, net/sunrpc) leak on every
create/destroy cycle.
We observed this in production: an NVMe/RDMA host repeatedly reconnecting
to a target that rejected the CRTO Property Get went through ~50 nvme
controller create/destroy cycles per second, and dentry and inode_cache
grew by ~13k pinned objects per 240 s -- unrecoverable through
drop_caches. Byte math matched a per-cycle 1-dentry / 1-inode leak from
the "fault_inject" directory dentry.
Fix this by not holding any external reference in fault_attr. Embed the
directory name as a fixed-size char array (FAULT_ATTR_DNAME_LEN, 64 bytes)
inside struct fault_attr, copied by strscpy() at
fault_create_debugfs_attr() time. fail_dump() prints it via %s.
Advantages of an embedded array over kstrdup() + kfree() paired with a new
destroy API:
- Zero API footprint. No new export and no caller changes required:
callers already own their fault_attr's memory and free it when
they are done, and now that suffices.
- No allocation on the create path.
- fault_create_debugfs_attr() cannot fail from the name-copy step.
- No lifetime coupling between attr->dname and debugfs; the string
is valid for exactly as long as the containing struct.
The 64-byte length accommodates every in-tree caller with generous
headroom (the longest current name is "fail_dma_array_full", 19 chars).
The user-visible fail_dump() format changes from "name %pd" to "name %s",
but the printed content is identical -- %pd on the created directory
renders the same string that was passed in as @name.
drivers/infiniband/hw/hfi1/fault.c drops a now-invalid "attr.dname = NULL"
statement; the surrounding kzalloc() already zero-initialises the array.
Link: https://lore.kernel.org/20260821181527.3271414-1-mliang@purestorage.com
Fixes: 6adc4a22f20b ("fault-inject: add ratelimit option")
Signed-off-by: Michael Liang <mliang@purestorage.com>
Reviewed-by: Andrew Morton <akpm@linux-foundation.org>
Cc: Akinbou Mita <akinobu.mita@gmail.com>
Cc: Dennis Dalessandro <dennis.dalessandro@cornelisnetworks.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: Leon Romanovsky <leon@kernel.org>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Patch series "alloc_tag: fix a leak and a deadlock around
shutdown_mem_profiling()", v2.
Two fixes for issues reported by sashiko:
1. percpu counter leak on modules loaded after profiling is disabled.
2. AB-BA deadlock between module load and /proc/allocinfo readers.
This patch (of 2):
After shutdown_mem_profiling() clears mem_profiling_support,
needs_section_mem() returns false, so later modules have their codetag
section placed as regular data and never enter the alloc_tag maple tree.
codetag_load_module() still called load_module(), which allocated a percpu
counter for every tag; release_module_tags() could not find these modules
on unload, so the counters leaked.
Return -EOPNOTSUPP from load_module() when profiling is off:
codetag_module_init() drops the module's cmod, no counters are allocated
and the module loads without its tags. codetag_unload_module() now always
calls free_section_mem(), since a module whose module_load() returned
-EOPNOTSUPP is not in the idr but may still hold a reserved section.
Link: https://lore.kernel.org/20260817062726.106511-1-hao.ge@linux.dev
Link: https://lore.kernel.org/20260817062726.106511-2-hao.ge@linux.dev
Fixes: 4835f747d3ed ("alloc_tag: support for page allocation tag compression")
Signed-off-by: Hao Ge <hao.ge@linux.dev>
Reported-by: Sashiko <sashiko-bot@kernel.org>
Suggested-by: Suren Baghdasaryan <surenb@google.com>
Acked-by: Suren Baghdasaryan <surenb@google.com>
Cc: Kent Overstreet <kent.overstreet@linux.dev>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
A bug in the XArray iterator xas_find() causes the iterator's index
(xas->xa_index) to jump backwards when iterating over a multi-index entry
(like a THP) that resides in a non-leaf node and is concurrently split.
When iterating over a multi-index entry in a non-leaf node, xas_load()
sets xas->xa_offset to the base offset of the entry, but leaves
xas->xa_index at the requested index. When the caller subsequently wants
to advance to the next entry, xas_find() is called. xas_find() attempts
to synchronize xas->xa_offset with xas->xa_index before advancing.
However, the fixup logic was incorrectly restricted to leaf nodes
(!xas->xa_node->shift). Because the THP resides in a non-leaf node, the
fixup is skipped.
As a result, xas_find() simply increments xas->xa_offset and recalculates
xas->xa_index based on this new offset. This causes xas->xa_index to jump
backwards. If the THP was concurrently split, the entry at the new offset
is a node pointer, so xas_find() descends into it and returns the folio at
the backwards index. The caller (filemap_map_pages()) then calculates the
PTE pointer based on this backwards index, resulting in an invalid memory
access such as an out-of-bounds read or use-after-free on a page-table
page freed via tlb_remove_table_rcu().
A userspace access that faults in a file-backed mapping can trigger this
path. When the index moves backwards, filemap_map_pages() can calculate a
PTE outside the page locked for fault-around and dereference a freed
page-table page, resulting in a KASAN-detected use-after-free read.
To fix this, check if xas->xa_offset matches get_offset(xas->xa_index,
xas->xa_node). If it does not and the node is a non-leaf node, set
xas->xa_offset to get_offset(xas->xa_index, xas->xa_node) before
advancing. Also add test cases in test_xarray to verify xas_find()
behavior when iterating over and splitting multi-index entries.
Link: https://lore.kernel.org/20260904121301.200049-1-krystianmkaniewski@gmail.com
Fixes: b803b42823d0 ("xarray: Add XArray iterators")
Assisted-by: Gemini:gemini-3.7-flash syzbot
Reported-by: syzbot+b72767277f29b6407083@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=b72767277f29b6407083
Link: https://syzkaller.appspot.com/ai_job?id=a01c56bd-74d0-411c-afb4-ee6f0cb6cb61
Signed-off-by: Krystian Kaniewski <krystianmkaniewski@gmail.com>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
This was moved into mm/, but the original lib/ file somehow remained.
Remove it.
Reported-by: Suren Baghdasaryan <surenb@google.com>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
This is another run of the Coccinelle script for converting kmalloc()
family of allocations to kmalloc_obj() via the existing rules in
scripts/coccinelle/api/kmalloc_objs.cocci
This catches both the set of kmalloc() uses added since the first
kmalloc_obj() conversions in v7.0 and adds a large group missed in the
first pass due to Coccinelle not interacting well with the cleanup.h
scoped_...() family of macros[1]. I worked around this with spatch's
"--macro-file" argument to a file with all the scoped_...() macros mapped
to Coccinelle's YACFE_ITERATOR[2] as that was the closest viable control
flow indicator I could find.
Build tested allmodconfig on x86, arm64, arm, loongarch, mips, powerpc,
riscv, and s390 with no new warnings.
Link: https://lore.kernel.org/lkml/202609021314.8A9C0B8@keescook/ [1]
Link: https://github.com/coccinelle/coccinelle/blob/master/standard.h [2]
Signed-off-by: Kees Cook <kees+treewide@kernel.org>
|
|
This patch builds on
Commit: 060d4e94b8d4 ("rhashtable: give each instance its own lockdep class")
to allow lockdep to see each rhashtable as unique with respect to all of
the locks, not just the ht->mutex.
This is needed if rhashtable is to be used tracepoint BPF which could
run while an rhashtable lock (in a different table) is held - see Link
below.
Rather then adding extra keys for the other locks, we use distinct
sub-classes for the different locks.
rhashtable->lock uses class 0
rhashtable->mutex uses class 1
bucket bitlocks, which are the only ones that are ever nested, use
classes 2 and 3.
Currently rht_lock() and rht_lock_nested() are quite separate code
despite the near-identical function. This patch moves rht_lock() to
after rht_lock_nested(), and simply calls that other function with a
nesting level of zero.
Link: https://lore.kernel.org/all/20260801-fix-rhashtable-bucket-lockdep-v1-1-15a0f8ae094c@gmail.com/
Closes: https://syzkaller.appspot.com/bug?extid=ef8d17bae14efb960935
Tested-by: quanyeyang <quanyeyang@proton.me>
Signed-off-by: NeilBrown <neil@brown.name>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
No caller tests the return value of this function.
The integer return type is still a remnant from when the function
was executed as initcall. This usage however was removed in
commit 8c0d884986ba ("init: main: add KUnit to kernel init")
Link: https://lore.kernel.org/r/20260901-kunit-void-v1-1-c91952391d66@linutronix.de
Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
Reviewed-by: David Gow <david@davidgow.net>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
|
|
This reverts commit e8eef69a99f185e75909adb24ab93d706e07bf27.
While DO_ONCE_SLEEPABLE() is used from sleepable/process context, callers
may still be holding arbitrary subsystem locks.
For instance, __inet_hash_connect() uses get_random_sleepable_once() which
invokes DO_ONCE_SLEEPABLE() while holding the socket lock (sk_lock):
lock_sock(sk)
__inet_hash_connect()
get_random_sleepable_once()
DO_ONCE_SLEEPABLE()
__do_once_sleepable_done()
static_branch_disable()
static_key_disable()
cpus_read_lock()
Calling static_branch_disable() directly from __do_once_sleepable_done()
causes static_key_disable() to synchronously acquire cpus_read_lock()
(cpu_hotplug_lock) and jump_label_mutex inside the caller's lock context.
This introduces an unwanted lockdep dependency:
sk_lock -> cpu_hotplug_lock
Because cpu_hotplug_lock depends on fs_reclaim (via workqueue CPU bringup
allocating memory with GFP_KERNEL), and storage/block layers (such as
NVMe-TCP) acquire sk_lock during I/O dispatch, lockdep reports circular
locking dependencies:
set->srcu -> sk_lock -> cpu_hotplug_lock -> fs_reclaim -> q_usage_counter -> elevator_lock -> set->srcu
This false positive previously prompted commit 19bdb70c77d3 ("nvme-tcp:
lockdep: use dynamic lockdep keys per socket instance") to work around the
warning using per-socket dynamic keys in NVMe-TCP. That in turn broke
asynchronous socket teardown and caused syzbot warnings in
tcp_tsq_handler().
Restoring once_disable_jump() in __do_once_sleepable_done() ensures that
static_branch_disable() is executed asynchronously from a system workqueue
without holding the caller's locks.
Link: https://lore.kernel.org/20260825142515.1965654-1-edumazet@google.com
Fixes: e8eef69a99f1 ("once: don't use a work queue to reset sleepable static key")
Signed-off-by: Eric Dumazet <edumazet@google.com>
Closes: https://lore.kernel.org/lkml/ao0mwtt8ePAINFni@shinhome/
Reported-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Reinette Chatre <reinette.chatre@intel.com>
Cc: Keith Busch <kbusch@kernel.org>
Cc: Nilay Shroff <nilay@linux.ibm.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
Pull more MM updates from Andrew Morton:
- "mm/rmap: index MAP_PRIVATE file-backed folios by anonymous pgoff"
(Lorenzo Stoakes)
Index MAP_PRIVATE file-backed folios by their anonymous page offset
to resolve confusion around reverse mapping for zeroed and CoW'd
file-backed memory.
Use this new VMA anonymous page offset tracking to eliminate index
conflicts and lay the foundation for scalable CoW performance
improvements.
- "promote mapped executable folios after first usage for MGLRU"
(Baolin Wang)
Make MGLRU's protection of mapped executable file folios more
reliable. Follow the classical LRU's logic, promoting mapped
executable file folios after their first usage to give executable
code a better chance to stay in memory and improve workload
performance.
- "mm: vmscan: fix node reclaim ignoring swappiness parameter" (Ridong
Chen)
Fix per-node proactive reclaim interface's ignoring the swappiness
parameter when CONFIG_MEMCG is disabled by consolidating
sc_swappiness() into a single function that checks
proactive_swappiness regardless of kernel configuration.
- "mm/vmscan: reduce lru_lock contention via vmstat-derived
scan-balance cost" (Usama Arif)
Reduce lru_lock contention in the reclaim path by deriving
scan-balance costs from vmstat counters rather than lock-acquired
producer updates.
Read and decay these cost signals on the reclaim side under a
dedicated per-lruvec lock, reducing total LRU lock wait time by over
60% without impacting scan throughput.
- "zram: fix zram issues reported by sashiko" (Sergey Senozhatsky)
Fix two low-risk zram bugs which Sashiko spotted in drive-by review.
- "Honor XA_FLAGS_ACCOUNT in xas_split_alloc() and charge to folio's
memcg" (Zi Yan)
Fix xas_split_alloc() by enabling target folio memcg charging during
splits and adding the missing __GFP_ACCOUNT flag for proper XArray
node memory accounting.
- "selftests/mm: use pattern matching in .gitignore" (Pratyush Mallick)
Replace hardcoded binary names in selftests/mm/.gitignore with a
generic pattern-matching rule to automatically ignore generated test
files and avoid manual updates when adding new tests.
- "mm/page_ext: remove pgdat_page_ext_init()" (Sang-Heon Jeon)
Make the incompatibility between FLATMEM and NUMA explicit in
mm/Kconfig and remove the unused pgdat_page_ext_init() function.
- "zram: fix zstd error paths and add parameter validation" (Haoqin
Huang)
Clean up zram compression backends by removing redundant error
cleanup, adding parameter and dictionary validation, auto-prefixing
algorithm error logs, and resetting parameters prior to
reinitialization.
- "zram: fix stale scan bounds after reinitialization" (Longlong Xia)
Prevent out-of-bounds slot accesses during concurrent zram resets by
moving table scan bound calculations under dev_lock in
writeback_store() and read_block_state().
- "add anon mTHP collapse test cases" (Baolin Wang)
Extend selftests helper functions to support arbitrary page orders
and add new test cases and options for mTHP collapse in khugepaged.
- "selftests/mm: Handle unsupported and transient test conditions"
(Muhammad Usama Anjum)
Update MM selftests to report a SKIP status instead of a failure when
required kernel or filesystem features are unsupported, while adding
retry logic for transient page migration errors.
- "mm/zswap: Fixes and improves the zswap shrink" (Hao Jia)
Fix the missing zswap global shrinker when CONFIG_MEMCG is disabled
and extend shrink_memcg() to support batch writeback for improved
writeback efficiency.
- "alloc_tag: introduce IOCTL-based filtering for MAP" (Suren
Baghdasaryan)
Introduce an IOCTL-based binary interface for memory allocation
profiling that enables kernel-side filtering before per-CPU counter
aggregation.
This eliminates the text-parsing overhead of /proc/allocinfo and
provides up to a 20x speedup by transferring only filtered allocation
data to userspace.
- "better block swap batching and a different take on swap_ops v5"
(Christoph Hellwig)
Refactor block swap I/O to use swap_iocb for batching instead of
single-bio requests and rebase the swap_ops interface, achieving
faster swap throughput during kernel builds.
- "mm: kmemleak: reduce transient false positives by confirming leaks"
(Catalin Marinas)
Reduce false-positive kmemleak reports by combining two kmemleak
enhancements that add a second confirmation scan and a configurable
minimum unreferenced scan count module parameter.
- "mm: kmemleak: default min_unref_scans to 2 for verbose kernels"
(Breno Leitao)
Auto-scanning kernels can generate false-positive memory leak reports
on single scans, so this patch defaults min_unref_scans to 2 when
CONFIG_DEBUG_KMEMLEAK_VERBOSE is enabled to require a second
confirming scan.
- "swap_ops updates" (Christoph Hellwig)
Batching I/O for synchronous swap devices causes performance
regressions and filesystem-based swap suffers from double-indirection
overhead. This series resolves both issues by reintroducing per-folio
writes for synchronous swap and allowing filesystems to directly
export their own swap_ops.
- "mm/khugepaged: several cleanups" (Nico Pache)
khugepaged accumulated redundant state-checking patterns and outdated
comments following mTHP integration. Introduce dedicated helpers for
PTE validation and event counting while refreshing the internal
documentation.
- "maple_tree: lock checking and clean ups" (Liam Howlett)
Syzbot reports incorrectly blame memory management exit paths for
locking bugs, maple tree erase operations risk allocation failures
without gfp flags and internal documentation lacks clarity.
Improve lock error detection, update docs, fix race and allocation
edge cases and optimize erase allocations using a fallback to
GFP_KERNEL | GFP_NOFAIL.
* tag 'mm-stable-2026-08-26-15-22' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (172 commits)
selftests/proc: make proc-maps-race work with READ_IMPLIES_EXEC
memcg: move LRU size accounting on reparenting instead of copying it
mm/vmscan: fix comment logic in balance_pgdat
maple_tree: add helper mas_make_walkable()
maple_tree: avoid extra gap calculation
maple_tree: fix argument name in header
maple_tree: change two GFP flags in tests
maple_tree: document erase and allocations better
maple_tree: avoid mas_erase() and mtree_erase() failures
maple_tree: document that erase may use GFP_KERNEL for allocations
maple_tree: catch race in mas_alloc_cyclic()
maple_tree: add bulk parent set helper
maple_tree: micro optimisation of mas_wr_store_type()
maple_tree: optimise mas_wr_node_store() when not in rcu mode
maple_tree: use prefetched value in mas_wr_store_type()
maple_tree: clarify comments on mas_nomem()
maple_tree: drop MAPLE_ALLOC_SLOTS
maple_tree: drop dead code from mas_extend_spanning_null()
maple_tree: documentation fix
maple_tree: add write lock checking with lockdep sequence numbers
...
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc
Pull char/misc/IIO/etc driver updates from Greg KH:
"Here is the big set of char, misc, iio, counter, fpga, and other small
driver subsystems for 7.3-rc1.
Overall, due to some driver removals we only added a bit more code
than removed, which was a nice change. Highlights in this merge
request are:
- Loads of IIO driver updates and additions
- binder driver updates (more on that below...)
- Removal of the SGI XP and GRU drivers as they are not used anymore
and turn out to be pretty insecure overall
- Removal of the obsolete ibmasm driver as it's not being used
anymore
- Coresight driver updates and additions
- Mei driver udpates
- Counter driver updates
- FPGA driver updates
- ICC driver updates
- lots and lots of other tiny driver updates to resolve reported
issues
All of these have been in linux-next for a while"
* tag 'char-misc-7.3-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc: (513 commits)
iio: chemical: atlas-sensor: use iio_trigger_poll_nested() to fix remove UAF
iio: adc: pac1921: fix wrong channel used in trigger handler read
iio: light: gp2ap002: re-enable irq if runtime suspend fails
iio: light: gp2ap002: Fix unbalanced runtime PM on repeated event writes
iio: light: apds9306: fix PM reference leak in apds9306_read_data()
iio: gyro: mpu3050: fix sign of raw angular velocity readings
iio: srf04: fix pm_runtime handling on probe error path
iio: adc: ad4080: configure backend data size
iio: adc: adi-axi-adc: add data size support for AD408X backend
iio: chemical: atlas-sensor: fix PM reference leak in buffer postenable
iio: dac: ad5446: fix OF module device table
iio: light: opt4001: Fix reversed GENMASK() arguments in fault count mask
iio: light: opt4001: Reject integration times with a non-zero seconds part
iio: light: opt4001: Fix incompatible pointer type passed to div_u64_rem()
iio: light: opt4001: Fix power down clearing bits of the wrong register
iio: light: opt4060: Fix incorrect register name in threshold read error message
iio: light: opt4060: Fix pointer type passed to div_u64_rem()
iio: light: opt4060: Reject integration times with a non-zero seconds part
iio: light: ltrf216a: fix runtime PM reference leak in error path
iio: pressure: dps310: fix NULL pointer dereference on ACPI probe
...
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace
Pull bootconfig updates from Masami Hiramatsu:
- Support build-time command line building for embedded bootconfig
- Fix xbc_snprint_cmdline() to render descendant keys when the root has
both a value and subkeys, and treats empty subtrees correctly.
- Add build-time pipeline using tools/bootconfig -C to render the
embedded bootconfig "kernel" subtree into .init.rodata as a cmdline
string.
- Clean build-time tools/bootconfig from make clean
- Add helper to prepend embedded bootconfig cmdline into
boot_command_line early before parse_early_param()
- Wire early prepend helper in x86 setup_arch() so early_param handlers
see values from the embedded bootconfig (currently x86 only)
- Avoid duplicating "kernel" keys in setup_boot_config()
- Refactor setup_boot_config() to share bootconfig_cmdline_requested()
- Document CONFIG_CMDLINE_FROM_BOOTCONFIG usage, requirements, and
precedence
* tag 'bootconfig-v7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace:
init/main.c: use bootconfig_cmdline_requested() for the runtime opt-in
bootconfig: skip runtime kernel.* render once prepended early
x86/setup: prepend embedded bootconfig cmdline before parse_early_param
Documentation: bootconfig: document build-time cmdline rendering
bootconfig: add xbc_prepend_embedded_cmdline() helper
bootconfig: clean build-time tools/bootconfig from make clean
bootconfig: render embedded bootconfig as a kernel cmdline at build time
bootconfig: render descendant keys when xbc_snprint_cmdline() root has a value
|
|
A check in mas_walk() was incorrect and caused inefficient use of the
maple state. The same issue existed in mas_erase(), but was left unfixed.
Making a helper function is the obvious answer.
Link: https://lore.kernel.org/20260821192627.4085470-20-liam@infradead.org
Signed-off-by: Liam R. Howlett (Oracle) <liam@infradead.org>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Breno Leitao <leitao@debian.org>
Cc: Chris Mason <clm@meta.com>
Cc: Chuck Lever <cel@kernel.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: Joe Perches <joe@perches.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@surriel.com>
Cc: Waiman Long <longman@redhat.com>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Prior to ending the ascension loop of larger operations like split,
rebalance, and spanning store the gap in the node had been calculated.
Once the node is inserted into the tree, the gap is recalculated in
mas_update_gap(). This can be avoided by creating a helper for
mas_update_gap() that accepts the known gap value, which reduces the
operations required for gap updating path.
Link: https://lore.kernel.org/20260821192627.4085470-19-liam@infradead.org
Signed-off-by: Liam R. Howlett (Oracle) <liam@infradead.org>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Breno Leitao <leitao@debian.org>
Cc: Chris Mason <clm@meta.com>
Cc: Chuck Lever <cel@kernel.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: Joe Perches <joe@perches.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@surriel.com>
Cc: Waiman Long <longman@redhat.com>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Failures to remove entries using the two APIs to erase the entries may
result in allocation failures. The failures may go unnoticed and an
unexpected entry may remain. Instead, fall back to retrying with
GFP_KERNEL | __GFP_NOFAIL so that the entry will be removed.
Link: https://lore.kernel.org/20260821192627.4085470-15-liam@infradead.org
Signed-off-by: Liam R. Howlett (Oracle) <liam@infradead.org>
Cc: Rik van Riel <riel@surriel.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Breno Leitao <leitao@debian.org>
Cc: Chris Mason <clm@meta.com>
Cc: Chuck Lever <cel@kernel.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Joe Perches <joe@perches.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Waiman Long <longman@redhat.com>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
State that the mas_erase() and mtree_erase() functions may use GFP_KERNEL
on allocation retry. Don't just depend on people reading the
documentation by adding a check that will warn of the use.
Link: https://lore.kernel.org/20260821192627.4085470-14-liam@infradead.org
Signed-off-by: Liam R. Howlett (Oracle) <liam@infradead.org>
Reviewed-by: Rik van Riel <riel@surriel.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Breno Leitao <leitao@debian.org>
Cc: Chris Mason <clm@meta.com>
Cc: Chuck Lever <cel@kernel.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Joe Perches <joe@perches.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Waiman Long <longman@redhat.com>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
If mas_alloc_cyclic() is called during a low memory situation, it is
possible the lock may be dropped so reclaim can occur. There is a window
where some other task may allocate the same id and cause the mas_insert()
to fail with -EEXIST. In this scenario the function will return -EEXIST,
which is not expected.
Modifying the retry on mas_nomem() to re-search for a slot means that any
race with other writes will not matter as the lock will be held between
finding the index and writing the index.
Moving the flag logic avoids cases where the flag is modified on drop
lock/reacquire or when the write fails after clearing the flag.
No existing users are exposed to this issue.
Link: https://lore.kernel.org/20260821192627.4085470-13-liam@infradead.org
Fixes: 9b6713cc7522 ("maple_tree: Add mtree_alloc_cyclic()")
Signed-off-by: Liam R. Howlett (Oracle) <liam@infradead.org>
Reported-by: Chris Mason <clm@meta.com>
Reviewed-by: Chuck Lever <cel@kernel.org>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Breno Leitao <leitao@debian.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: Joe Perches <joe@perches.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@surriel.com>
Cc: Waiman Long <longman@redhat.com>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Instead of calculating the parent pointer each time for a child, cache the
majority of the parent pointer and only change the slot per child.
Drop the mas_set_parent() function since the last user has been removed.
Testing on a tree containing 2048 entries of height 4 had an increased
gain of 3.51% on nodes tracking gaps.
Link: https://lore.kernel.org/20260821192627.4085470-12-liam@infradead.org
Signed-off-by: Liam R. Howlett (Oracle) <liam@infradead.org>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Breno Leitao <leitao@debian.org>
Cc: Chris Mason <clm@meta.com>
Cc: Chuck Lever <cel@kernel.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: Joe Perches <joe@perches.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@surriel.com>
Cc: Waiman Long <longman@redhat.com>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Use three new local booleans instead of reading other structures. This
has shown an increase of 0.62% on a 2048 entry tree of height 4.
Link: https://lore.kernel.org/20260821192627.4085470-11-liam@infradead.org
Signed-off-by: Liam R. Howlett (Oracle) <liam@infradead.org>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Breno Leitao <leitao@debian.org>
Cc: Chris Mason <clm@meta.com>
Cc: Chuck Lever <cel@kernel.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: Joe Perches <joe@perches.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@surriel.com>
Cc: Waiman Long <longman@redhat.com>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Clearing the entire node on the stack is unnecessary since most of the
node will be overwritten anyways. Just clear what isn't used after the
data is in place.
Benchmarking shows a speedup of 0.67% on a height 4 tree with 2048
entries.
Link: https://lore.kernel.org/20260821192627.4085470-10-liam@infradead.org
Signed-off-by: Liam R. Howlett (Oracle) <liam@infradead.org>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Breno Leitao <leitao@debian.org>
Cc: Chris Mason <clm@meta.com>
Cc: Chuck Lever <cel@kernel.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: Joe Perches <joe@perches.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@surriel.com>
Cc: Waiman Long <longman@redhat.com>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
The slot contents exist in wr_mas->content, which has less overhead than
reading the slot again.
Link: https://lore.kernel.org/20260821192627.4085470-9-liam@infradead.org
Signed-off-by: Liam R. Howlett (Oracle) <liam@infradead.org>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Breno Leitao <leitao@debian.org>
Cc: Chris Mason <clm@meta.com>
Cc: Chuck Lever <cel@kernel.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: Joe Perches <joe@perches.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@surriel.com>
Cc: Waiman Long <longman@redhat.com>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
When an allocation completely fails, the return is false. If the
allocation succeeds or partially succeeds, return true to indicate a retry
of the operation. Note that since the lock may have been dropped, the
operation is retried from the start - including potentially allocating
more memory.
Link: https://lore.kernel.org/20260821192627.4085470-8-liam@infradead.org
Signed-off-by: Liam R. Howlett (Oracle) <liam@infradead.org>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Breno Leitao <leitao@debian.org>
Cc: Chris Mason <clm@meta.com>
Cc: Chuck Lever <cel@kernel.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: Joe Perches <joe@perches.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@surriel.com>
Cc: Waiman Long <longman@redhat.com>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
mas_extend_spanning_null() had a clause if the end of the range being
written (mas->last) is the same as the end of the existing range it is
overwriting (wr_mas->r_max), action will be taken.
This code path is not possible because the only calling function
increments mas->last (unless it's ULONG_MAX) to walk to one beyond the
write and then resets the value back to the initial value.
In the case of mas->last == ULONG_MAX, then the second part of the
statement will always be false - mas->last cannot be less than the node
max.
This code never executed and is flawed anyways (the arguments are
incorrectly ordered), so removing it is the safest action. Since the code
never executes, it is not fixing any issue so Fixes tag is not given.
Link: https://lore.kernel.org/20260821192627.4085470-6-liam@infradead.org
Signed-off-by: Liam R. Howlett (Oracle) <liam@infradead.org>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Breno Leitao <leitao@debian.org>
Cc: Chris Mason <clm@meta.com>
Cc: Chuck Lever <cel@kernel.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: Joe Perches <joe@perches.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@surriel.com>
Cc: Waiman Long <longman@redhat.com>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Use the lockdep sequence numbers to ensure the write lock is not dropped
between write operations. The lockdep sequence is recorded on any walk
that starts from the top of the tree and re-checked prior to any operation
using an active node.
When lockdep detects an issue, it sets debug_locks to 0 disabling further
reports. __lock_sequnece() will return u32 ~0 when debug_locks is zero,
and the real sequnece count cannot return such a high value as it is less
than 32bits.
By always updating the sequence number, regardless of lock state and by
ignoring ~0 value in the sequence number will avoid ever printing a
WARN_ON when lockdep sets debug_locks to 0.
Link: https://lore.kernel.org/20260821192627.4085470-4-liam@infradead.org
Signed-off-by: Liam R. Howlett (Oracle) <liam@infradead.org>
Cc: Breno Leitao <leitao@debian.org>
Tested-by: Breno Leitao <leitao@debian.org>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Chris Mason <clm@meta.com>
Cc: Chuck Lever <cel@kernel.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: Joe Perches <joe@perches.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@surriel.com>
Cc: Waiman Long <longman@redhat.com>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Patch series "maple_tree: lock checking and clean ups", v3.
In this series:
1. Try to detect lock issues
A number of syzbot reports are incorrectly pointing to the mm exit as
a source of the locking error. The first three patches attempt to
help users detect errors in their locking - but they still have to use
LOCKDEP. I guess it's still down to hope and prayers.
2. Documentation fixes
The documentation was lacking clarity, there are updates to try and
help the users, especially around the erase() cases.
3. Two benign issues
The cyclic allocator may have a race, although no in-kernel user can
hit it.
The erase functions may cause allocation issues if used with the
incorrect locking type, but none are present in-tree.
4. The erase gfp uses
mas_erase() and mtree_erase() do not take a gfp argument. To improve
reliability of the erase, the first attempt to allocate will be
GFP_NOWAIT, followed by a retry (if necessary of GFP_KERNEL |
GFP_NOFAIL. This will ensure the data is gone. I've updated the
documentation to make it more clear as well. mas_store() is not
addressed in the same way, but may need to be updated at a later date,
but that may require changing callers so it is out of scope here.
Beyond these goals there are some test fixes, some general speed-up
patches targeting extra work and cycles, and dropping dead code.
This patch (of 19):
When CONFIG_LOCKDEP and CONFIG_RCU_STRICT_GRACE_PERIOD is enabled, check
for rcu locking issues by recording the grace period in the maple state
and checking the rcu window is still valid whenever the maple state is
reused with a state that is not MA_START or MA_PAUSED.
Link: https://lore.kernel.org/20260821192627.4085470-1-liam@infradead.org
Link: https://lore.kernel.org/20260821192627.4085470-2-liam@infradead.org
Signed-off-by: Liam R. Howlett (Oracle) <liam@infradead.org>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Breno Leitao <leitao@debian.org>
Cc: Chris Mason <clm@meta.com>
Cc: Chuck Lever <cel@kernel.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: Joe Perches <joe@perches.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@surriel.com>
Cc: Waiman Long <longman@redhat.com>
Cc: Will Deacon <will@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Move pr_debug() inside the `if (dpage)` block to avoid printing garbage
pfn for NULL dpage, and correct the direction label from "sys to dev" to
"dev to sys".
Link: https://lore.kernel.org/20260812092856.55296-1-liuqiangneo@163.com
Signed-off-by: Qiang Liu <liuqiang@kylinos.cn>
Assisted-by: Qoder:Qwen-3.8-MAX-Preview
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: Leon Romanovsky <leon@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
alpha requires percpu variables in modules to be defined as weak so that the
compiler generates GOT based external references for them. This puts two
extra restrictions on percpu variable definitions. The symbol must be
globally unique even when static and a static percpu variable can't be
defined inside a function. DEBUG_FORCE_WEAK_PER_CPU exists to give generic
code build coverage for these restrictions without building for alpha.
MEM_ALLOC_PROFILING defines a static percpu counter at each allocation call
site and thus can't be built with weak percpu definitions, so it depends on
!DEBUG_FORCE_WEAK_PER_CPU. As allmodconfig enables DEBUG_FORCE_WEAK_PER_CPU,
this knocks MEM_ALLOC_PROFILING out of allmodconfig build coverage.
allmodconfig coverage for MEM_ALLOC_PROFILING is worth more than build
coverage for restrictions which only matter to alpha module builds. Drop
DEBUG_FORCE_WEAK_PER_CPU. Restriction violations will now show up only on
alpha builds.
Link: https://lore.kernel.org/178656406317.2437052.7257990869957704195@slm.duckdns.org
Signed-off-by: Tejun Heo <tj@kernel.org>
Reported-by: Andrew Morton <akpm@linux-foundation.org>
Reviewed-by: Suren Baghdasaryan <surenb@google.com>
Acked-by: Gabriele Monaco <gmonaco@redhat.com> [include/rv/da_monitor.h]
Cc: Dennis Zhou <dennis@kernel.org>
Cc: Kent Overstreet <kent.overstreet@linux.dev>
Cc: Steven Rostedt <rostedt@goodmis.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Fix a spelling mistake n a code comment. This is a comment only change
with no functional impact.
Link: https://lore.kernel.org/20260813183847.474357-1-sercombe.joel.mark@gmail.com
Signed-off-by: Mark Sercombe <sercombe.joel.mark@gmail.com>
Reviewed-by: Andrew Morton <akpm@linux-foundation.org>
Reviewed-by: Liam R. Howlett (Oracle) <liam@infradead.org>
Cc: Alice Ryhl <aliceryhl@google.com>
Cc: Andrew Ballance <andrewjballance@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Patch series "alloc_tag: introduce IOCTL-based filtering for MAP", v8.
Currently, memory allocation profiling data is primarily exposed through
/proc/allocinfo. While useful for manual inspection, this text-based
interface poses challenges for production monitoring and large-scale
analysis:
1. Userspace must parse large amounts of text to extract specific
fields.
2. To find specific tags, userspace must read the entire dataset,
requiring many context switches and high data copying.
3. The kernel currently aggregates per-CPU counters for every allocation
size, even those the user intends to filter out immediately.
This series introduces a new IOCTL-based binary interface for allocinfo
that supports kernel-side filtering. By allowing the user to specify a
filter mask, we significantly reduce the work performed in-kernel and the
amount of data transferred to userspace. The IOCTL mechanism was chosen
for allocinfo to address the per-CPU counter aggregation bottleneck. A
traditional read() operation must report the total allocation count and
sizes for every code tag in the system. Doing so requires iterating
across all CPUs to sum their per-CPU counters for thousands of tags, which
introduces substantial runtime overhead.
The IOCTL interface allows userspace to push selective filtering criteria
directly into the kernel before the per-CPU counter aggregation. The
kernel aggregates per-CPU counters only for a small subset of tags that
match the filter. This results in significant performance improvement.
Beyond fast filtered retrieval, the IOCTL foundation allows introducing a
context capture mechanism in the future to capture the context for
specific allocations.
Performance measurements were conducted on an Intel Xeon Platinum 8481C
(224 CPUs) with caches dropped before each run.
The IOCTL mechanism shows a ~20x performance improvement for
filtered queries. The kernel avoids the expensive per-CPU counter
aggregation (alloc_tag_read) for any tags that fail the initial string
or location filters.
Scenario 1: Specific File Filtering (arch/x86/events/rapl.c)
1. Traditional (cat /proc/allocinfo | grep): 22ms (sys)
2. IOCTL Interface: 1ms (sys)
Scenario 2: Compound Filtering (Filename + Size)
1. Traditional: (cat ... | grep | awk): 21ms (sys)
2. IOCTL Interface: 1ms (sys)
Scenario 3: Size-Based Filtering (min_size = 1MB)
1. Traditional: (cat ... | awk): 21ms (sys)
2. IOCTL Interface: 14ms (sys)
This patch (of 6):
Add the following ioctl commands for /proc/allocinfo file:
ALLOCINFO_IOC_CONTENT_ID - gets content identifier which can be used to
check whether the file content has changed specifically due to module
load/unload. Every time a module is loaded / unloaded, the returned value
will be different. By comparing the identifier value at the beginning and
at the end of the content retrieval operation, users can validate
retrieved information for consistency.
ALLOCINFO_IOC_GET_AT - gets the record at the specified position. This is
the position of a record in /proc/allocinfo.
ALLOCINFO_IOC_GET_NEXT - gets the record next to the last retrieved one.
If no records were previously retrieved, returns the first record.
Note, function file and module names often have the same prefixes,
therefore when filtering for them, we compare the last 64 characters to
minimize the chances of name collisions.
[akpm@linux-foundation.org: include compat.h, per Suren]
Closes: https://lore.kernel.org/oe-kbuild-all/202607091820.qbjlGhKK-lkp@intel.com/
Link: https://lore.kernel.org/cover.1783532853.git.abhishekbapat@google.com
Link: https://lore.kernel.org/15596de2607ef13e7c77c6d74763f4ae992ec475.1783532853.git.abhishekbapat@google.com
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
Signed-off-by: Abhishek Bapat <abhishekbapat@google.com>
Acked-by: Hao Ge <hao.ge@linux.dev>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Kent Overstreet <kent.overstreet@linux.dev>
Cc: Sourav Panda <souravpanda@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
The last callers of mas_is_root_limits() were removed by commit
b8852ef30c67 ("maple_tree: remove maple big node and subtree structs"),
together with the maple subtree state (mast_*) code that used it. As a
static inline it does not trigger -Wunused-function, so it went unnoticed.
Remove it. No functional change.
Link: https://lore.kernel.org/20260805070529.4118794-1-zhanxusheng@xiaomi.com
Signed-off-by: Zhan Xusheng <zhanxusheng@xiaomi.com>
Reviewed-by: Liam R. Howlett (Oracle) <liam@infradead.org>
Cc: Alice Ryhl <aliceryhl@google.com>
Cc: Andrew Ballance <andrewjballance@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
XArray operations that allocate xa_nodes, such as xas_nomem() and
xas_alloc(), add __GFP_ACCOUNT when the array has XA_FLAGS_ACCOUNT set.
This charges the allocated memory and avoids the workingset convergence
issue described by commit 7b785645e8f13 ("mm: fix page cache convergence
regression").
xas_split_alloc() does not add _GFP_ACCOUNT when XA_FLAGS_ACCOUNT is
present. Fix it.
Link: https://lore.kernel.org/20260804-add-gfp_account-to-xas_split_alloc-v3-2-38cb3ff325c5@nvidia.com
Fixes: 6b24ca4a1a8d ("mm: Use multi-index entries in the page cache")
Signed-off-by: Zi Yan <ziy@nvidia.com>
Reviewed-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Barry Song <baohua@kernel.org>
Cc: David Hildenbrand <david@kernel.org>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Lance Yang <lance.yang@linux.dev>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: William Kucharski <william.kucharski@oracle.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/vbabka/slab
Pull slab updates from Vlastimil Babka:
- Add kfree_rcu_nolock() that can be used from contexts where spinning
on a lock might be unsafe, such as a BPF program attached to an
arbitrary function, or in NMI context. This complements the existing
kfree_nolock() support (Harry Yoo)
- Runtime instead of compile-time slabobj_ext sizing.
Avoid wasting memory when memory allocation profiling is compiled but
not enabled, with initial partial support to also avoid wasting
memory for objcg pointers when those are not needed, while profiling
is enabled (Vlastimil Babka)
- Various non-urgent fixes, cleanups and optimizations (Hao Li,
Hongling Zeng, Li RongQing, Li Xiasong, Seongjun Hong, Shengming Hu)
* tag 'slab-for-7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/vbabka/slab: (31 commits)
mm/slab, kfence, memcg: completely remove obj_ext for kfence objects
mm/slab: stop allocating objcg pointers when unnecessary
mm/slab: add cache_ and slab_needs_objcg() helpers
mm/slab: stop exporting kvfree_rcu_barrier[_on_cache]()
slub_kunit: extend the test for kfree_rcu_nolock()
mm/slab: introduce kfree_rcu_nolock()
mm/slab: introduce struct kvfree_rcu_head for kvfree_rcu batching
mm/slab: reduce slabobj_ext memory with allocation profiling disabled
mm/slab: introduce slab_obj_ext_has_codetag()
mm/slab: allow kfree_rcu_sheaf() on PREEMPT_RT
mm/slab: extend deferred free mechanism to handle rcu sheaves
mm/slab: use call_rcu() in unknown context if irqs are enabled
mm/slab: handle the !allow_spin case in kfree_rcu_sheaf()
mm/slab: change struct slabobj_ext to a union
mm/slab: replace slab.stride with obj_exts_in_object
mm/slab: abstract slabobj_ext.ref access
mm/slab: abstract slabobj_ext.objcg access
mm/slab: make slab_obj_ext() determine object index
mm: move struct slabobj_ext to mm/slab.h
mm/slab: remove objs_per_slab()
...
|
|
Add test_csd_lock, a module that keeps one CPU from answering an IPI for
as long as its stall_ms parameter says, so that the CSD-lock debug code
has a stall to report. With in_handler=1 the CPU stalls inside a CSD
handler instead, which is the case where the IPI is not re-sent.
The module needs CONFIG_CSD_LOCK_WAIT_DEBUG and csdlock_debug=1. Loading
it runs one stall and then fails the load with -EAGAIN, the way
test_lockup does, so that nothing is left loaded afterwards.
Signed-off-by: Breno Leitao <leitao@debian.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
