| Age | Commit message (Collapse) | Author |
|---|
|
into clk-pile
Pull clk patches from Brian Masney:
- New clock controller drivers for the Cix Sky1 audio subsystem (AUDSS),
UltraRISC DP1000, and MediaTek MT8173 MFG_TOP, along with their devicetree
bindings. Si549 support was added to the existing si544 driver.
- New clock and reset support for the Aspeed AST2700 PECI controller and Airoha
EN7523 PCIe PERSTOUT reset lines.
- The clk core gains devm_clk_bulk_get_enable() as the mandatory counterpart to
the existing optional variant, and exports
devm_clk_hw_register_composite_pdata() for modular drivers.
- Tegra gets a proper EMC clock implementation for Tegra114, 48MHz pll_p_out1
support needed for UEFI on Surface2, and the Xilinx clocking-wizard gains PLL
charge pump/lock parameter programming during dynamic reconfiguration.
- Bug fixes for a NULL pointer dereference from uninitialized clk_init_data in
the eswin driver, an IO remap leak in the MediaTek pllfh error path, inverted
gate control for MT8135 devapc_ck, a missing OF node put in tegra124-emc on
registration failure, a prepare reference leak in the palmas driver,
unregistered PLLs on MT6735 probe failure, a missing kasprintf NULL check in
pmc_atom, PCIe warm boot instability in the Airoha EN7523 driver, and a
clocking-wizard clock difference detection bug.
- Various cleanups across tegra, st, and mvebu providers to stop misusing the
consumer clock API, along with other minor fixes and improvements.
Signed-off-by: Brian Masney <bmasney@redhat.com>
* tag 'clk-misc-round-two-for-v7.3' of ssh://github.com/masneyb/linux: (45 commits)
clk: tegra: set up proper EMC clock implementation for Tegra114
clk: clocking-wizard: remove 20kHz restriction
clk: clocking-wizard: optimize clock search
clk: clocking-wizard: fix clock difference detection
clk: mediatek: mt8135: Fix inverted gate control for devapc_ck
clk: clocking-wizard: Program PLL CP/RES and lock parameters on reconfig
clk: tegra: support 48MHz clock for pll_p_out1
clk: aspeed: add AST2700 PECI clock
dt-bindings: clock: ast2700: add PECI clock
clk: mediatek: Add mt8173-mfgtop driver
dt-bindings: clock: mediatek: Add mt8173 mfgtop
clk: tegra: clean-up simple provider misuse of the consumer API
clk: st: clean-up simple provider misuse of the consumer API
clk: mvebu: clean-up simple provider misuse of the consumer API
clk: remove conditional return with no effect
clk: Add devm_clk_bulk_get_enable()
clk: en7523: add support for dedicated PCIe PERSTOUT reset
dt-bindings: clock: airoha: Add additional reset for PCIe PERSTOUT
arm64: dts: cix: sky1: add audss cru
reset: cix: add sky1 audss auxiliary reset driver
...
|
|
Pull clk patches picked up by Brian Masney:
Here's various improvements and fixes for the clk subsystem that was posted prior
to the opening of the last merge window.
- Add spread spectrum clock (SSC) support to the clk framework, including a new
assigned-clock-sscs devicetree property and clk_hw_set_spread_spectrum()
API with KUnit tests (Peng Fan)
- Add SCMI clock OEM extensions for the i.MX95 clock driver and introduce a
common SCMI clock header (Peng Fan)
- Add clock, reset, and devicetree bindings for the ESWIN EIC7700 HSP clock and
reset generator (Xuyang Dong)
- Add clk_determine_rate_noop() helper for clock drivers that can do any rate,
and convert existing open-coded implementations across hisilicon, imx, qcom,
renesas, rp1, samsung, scpi, sprd, mediatek phy, and mediatek pmdomain
drivers (Brian Masney)
- Add kernel-doc documentation for struct clk_core and the core clock flags,
and wire up clk identifiers into the documentation build (Brian Masney)
- Fix clk_divider_bestdiv() returning the minimum rate instead of the maximum
rate for large rate requests, with KUnit tests (Lad Prabhakar)
- Fix Nuvoton MA35D1 PLL frequency calculation including ignored div_u64
return values, incorrect PLL_CTL1_FRAC bit field width, and broken
determine_rate logic (Joey Lu)
- Support unique clock names for multi-socket Tegra platforms (Jon Hunter)
- Allow COMPILE_TEST builds for HiSilicon clock drivers (Rosen Penev)
- Various fixes and cleanups from Akari Tsuyukusa, Alexander A. Klimov,
Brian Masney, David Carlier, David Laight, Min zhang, Myeonghun Pak,
Pavel Löbl, Randy Dunlap, Rob Herring, Rosen Penev, Uwe Kleine-König,
William Theesfeld, Xuyang Dong, and Yu-Chun Lin
Signed-off-by: Brian Masney <bmasney@redhat.com>
[sboyd@kernel.org: Fix allmodconfig modpost failure in scmi]
* tag 'clk-misc-for-v7.3' of ssh://github.com/masneyb/linux: (53 commits)
pmdomain: mediatek: mtk-mfg: use clk_determine_rate_noop()
pmdomain: mediatek: airoha: use clk_determine_rate_noop()
phy: mediatek: phy-mtk-hdmi-mt2701: use clk_determine_rate_noop()
clk: sprd: use clk_determine_rate_noop()
clk: scpi: use clk_determine_rate_noop()
clk: samsung: acpm: use clk_determine_rate_noop()
clk: rp1: use clk_determine_rate_noop()
clk: renesas: rzg2l-cpg: use clk_determine_rate_noop()
clk: qcom: smd-rpm: use clk_determine_rate_noop()
clk: qcom: rpmh: use clk_determine_rate_noop()
clk: qcom: rpm: use clk_determine_rate_noop()
clk: imx: scu: use clk_determine_rate_noop()
clk: hisilicon: hi3660-stub: use clk_determine_rate_noop()
clk: add clk_determine_rate_noop()
clk: imx: scu: drop redundant init.ops variable assignment
clk: test: convert constants to use HZ_PER_MHZ
docs: clk: include some identifiers to keep documentation up to date
clk: add kernel docs for struct clk_core
clk: add kernel docs for the core flags
clk: hisilicon: allow COMPILE_TEST builds
...
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
Pull failfs filesystem from Christian Brauner:
"Add failfs and expose a FD_FAILFS_ROOT sentinel.
This allows userspace to shed their filesystem state completely. A
process with its root or working directory in failfs must anchor every
path lookup at an explicit file descriptor. Absolute paths, absolute
symlinks and AT_FDCWD-relative lookups simply fail.
Failfs is the counterpart to nullfs. nullfs says adds a permanently
empty, immutable directory whose lookups fail with ENOENT but which
can be opened, read, stat'd and mounted upon. Failfs on the other hand
fails every operation. The root cannot be opened at all. A single
instance is mounted during early boot via kern_mount(), which makes it
logically distinct from every mount namespace.
This is accompanied by a new fchroot() system call which makes
chrooting via a file descriptor a first class concept. It's possible
to chroot into failfs as an unprivileged user provided the task has no
new privileges set"
* tag 'vfs-7.3-rc1.failfs' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs:
Documentation: add failfs documentation
selftests/filesystems: add failfs selftests
arch: hookup fchroot() system call
fs: support FD_FAILFS_ROOT in fchroot()
fs: add fchroot()
fs: support FD_FAILFS_ROOT in fchdir()
fs: add failfs
|
|
Every LOCALIO read and write is currently bounced through the dedicated
!WQ_MEM_RECLAIM nfslocaliod_workqueue. That bounce is only actually
required when the submitting context is a memory-reclaim context: LOCALIO
issues IO directly into a stacked local filesystem (e.g. XFS) which may in
turn flush its own !WQ_MEM_RECLAIM workqueue. Doing that from a
WQ_MEM_RECLAIM worker (most importantly writeback's wb_workfn on bdi_wq) or
an explicit PF_MEMALLOC reclaim task trips check_flush_dependency() and
risks a forward-progress deadlock, which is why commit b9f5dd57f4a5
("nfs/localio: use dedicated workqueues for filesystem read and write")
introduced the intermediate workqueue.
Outside of reclaim context -- ordinary application/task submission such as
O_DIRECT or fsync-driven writeback -- the workqueue hop buys nothing and
merely adds a context switch and scheduling latency per IO while discarding
the NFS client's inherent application-context parallelism.
Add current_is_workqueue_mem_reclaim(), which reports whether %current is a
WQ_MEM_RECLAIM worker using the same predicate check_flush_dependency()
warns on. Use it, together with the PF_MEMALLOC check, in the new
nfs_local_defer_io() helper to decide per-IO whether nfs_local_do_read()
and nfs_local_do_write() must defer to nfslocaliod_workqueue or may issue
the IO inline. Buffered writeback continues to bounce (wb_workfn is a
WQ_MEM_RECLAIM worker); O_DIRECT and app-context submission now run inline.
Running nfs_local_call_write() inline is safe: it already saves and
restores current->flags around the PF_LOCAL_THROTTLE|PF_MEMALLOC_NOIO it
sets and scopes the file opener's creds. The async O_DIRECT completion
path is likewise unaffected: when the underlying filesystem returns
-EIOCBQUEUED, the kiocb ki_complete callback (nfs_local_read_aio_complete /
nfs_local_write_aio_complete) can run in bottom-half context and so must
still defer the pgio completion (nfs_local_pgio_release -> rpc_call_done) to
nfsiod_workqueue via nfs_local_pgio_aio_complete(). That completion hop is
independent of how the IO was submitted, and this change leaves it as-is;
only the submission side stops unconditionally hopping through
nfslocaliod_workqueue.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Mike Snitzer <snitzer@kernel.org>
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
|
Honor the per-file UNCACHEABLE_FILE_DATA attribute by transparently
opening such regular files with O_DIRECT, so reads and writes bypass the
page cache as the attribute requires, without the application having to
request O_DIRECT itself.
This follows the model the specification describes: the attribute is
"similar in intent to O_DIRECT" and clients "retain flexibility in how
they satisfy the requirements" (draft-ietf-nfsv4-uncacheable-files
Section 4.4, "Relationship to Direct I/O"), and its Implementation
Status (Section 6) describes a prototype Linux client that "treats the
attribute as an indication to use O_DIRECT-like behavior for file
access".
Introduce an NFS_CONTEXT_O_DIRECT open-context flag: nfs4_atomic_open()
sets it when the resolved inode has uncacheable_file_data set (and the
open is not O_APPEND), and the open paths nfs_atomic_open() and
nfs4_file_open() apply O_DIRECT to the file when the flag is set.
The I/O mode is thus selected at open time and is not changed for an
already-open file: a later change to the attribute takes effect on the
next open. The specification permits this -- a client that has already
opened a file MAY continue with its existing caching behavior and apply
the updated attribute to subsequent operations (Section 5).
The delegation interaction in Section 4.3 was considered: it permits read
caching to remain when another NFSv4.2 mechanism, such as a delegation,
already ensures a consistent view of the file. That relaxation is
optional ("may remain appropriate") and read-only -- it does not relax
write-behind suppression (Section 4.1) or the WRITE durability invariant
(Section 4.2). This implementation deliberately does not take it: an
uncacheable file is opened O_DIRECT regardless of any delegation held,
which is compliant (read caching is simply suppressed more aggressively
than the Section 4.3 minimum) and avoids decoupling read vs write caching
behind a single open flag. Relaxing reads under a delegation is left as
a possible future optimization.
Section 6 observes the benefit holds "for applications that issue
well-formed I/O requests". That alignment caveat does not constrain the
Linux NFS client's over-the-wire path: the client readily issues
misaligned I/O using O_DIRECT over SunRPC to the remote NFS server. The
only place a fallback from O_DIRECT to buffered I/O for misaligned I/O
applies is NFS LOCALIO (fs/nfs/localio.c), which detects non-DIO-aligned
I/O and falls back internally; that path is unaffected by this change.
Link: https://datatracker.ietf.org/doc/draft-ietf-nfsv4-uncacheable-files/
Signed-off-by: Mike Snitzer <snitzer@kernel.org>
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
|
Recognize the NFSv4.2 per-file UNCACHEABLE_FILE_DATA attribute (attr 87,
draft-ietf-nfsv4-uncacheable-files): decode it via GETATTR, track per-
exported-filesystem support, and record on the inode whether a regular
file's data must not be cached. Acting on the attribute (opening such
files O_DIRECT) is done by a subsequent change.
If the NFSv4 server reports a regular file's UNCACHEABLE_FILE_DATA as
true, it indicates the file's data must not be cached; the client records
this in NFS_I(inode)->uncacheable_file_data for use by the I/O paths.
The UNCACHEABLE_FILE_DATA attribute applies only to regular files
(NF4REG); per the draft a server MUST reject a query of it on any other
object type with NFS4ERR_INVAL. A subsequent commit gates the client
accordingly.
Link: https://datatracker.ietf.org/doc/draft-ietf-nfsv4-uncacheable-files/
Signed-off-by: Tom Haynes <loghyr@hammerspace.com>
[snitzer: adapt Tom's original code focused on metadata for ABE]
Co-developed-by: Mike Snitzer <snitzer@hammerspace.com>
Signed-off-by: Mike Snitzer <snitzer@hammerspace.com>
Signed-off-by: Mike Snitzer <snitzer@kernel.org>
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
Pull efs and freevxfs removal from Christian Brauner:
"This removes the EFS and freevxfs filesystems:
- EFS was the read-only on-disk format SGI used on IRIX before XFS
- freevxfs provided compatibility with various old-school Unix
systems from the 1990s and was fun 25 years ago. Today it mostly
serves as fodder for automated bug checkers. There has been only
one known user and contributor in the last 15 years"
* tag 'vfs-7.3-rc1.efs' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs:
efs: Remove EFS
* tag 'vfs-7.3-rc1.freevxfs' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs:
freevxfs: remove the driver
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
Pull binfmt updates from Christian Brauner:
"This contains a bunch of work for binfmt_misc. It fixes a bunch of
old bugs, reworks the locking, and then extends the format registry
so a binary type can be matched programmatically and its interpreter
computed per exec instead of being a fixed string recorded at
registration time.
This allows nixos and other to e.g., implement relocatable binaries
meaning the interpreter/dynamic loader can be determined
programatically, say found relative to the binary. The mechanism is
flexible and can support other policies:
- Handler lookup is now an rcu walk. An exec that matches no
binfmt_misc entry should now never write to a shared cacheline
- remove the VERBOSE_STATUS and USE_DEBUG compile time toggles
- convert the entry file to a seq_file which simplifies things quite
a bit and kills a lot of custom logic
- make flags proper enums
- rename struct Node to binfmt_misc_entry
- allow entries to be removed with unlink(2)
- Add the ability to attach bpf programs to binfmt_misc entries so
it's possible to dynamically choose the execution environment such
as the loader or interpreter on a per binary basis.
A handler is an instance of a binfmt_misc_ops struct_ops with a
->match() and a ->load() program. match() decides from the entry
lookup walk whether the handler applies under the same
registration-order. It can read file content as needed not only the
prefetched 256 bytes in bprm->buf.
load() then selects the interpreter and stages it through the new
bpf_binprm_set_interp(), bpf_binprm_set_interp_arg() and
bpf_binprm_set_flags() kfuncs.
Handlers are published in a registry keyed by the registering
task's user namespace and activated through the existing text
interface with a new 'B' type carrying the handler name:
echo ':origin:B::::nix:' > /proc/sys/fs/binfmt_misc/register
The permission and namespacing model is unchanged. Activating a
handler requires the same write access to an instance as any other
registration. A container mounting its own instance escapes the
host's entries exactly as before. The computed interpreter is
opened with open_exec() under the caller's credentials and goes
through full LSM vetting as the next binprm level. A program can
only ever redirect the caller to something the caller could exec
anyway.
- Two dispatch modes are added. So far the chosen interpreter owns
the whole process identity (argv[0], /proc/pid/cmdline,
/proc/self/exe all name interpreter information). So relocatable
find the dynamic linker instead. Also a binary passed to execveat()
as an inaccessible O_CLOEXEC fd cannot run at all and gdb trips
because AT_ENTRY and AT_PHDR do not match the exe file. So PIE
symbols are unrelocated.
This adds transparent dispatch which allows the interpreter to load
the binary through AT_EXECFD and leaves the argument vector exactly
as the caller built it and labels mm->exe_file and comm with the
binary. It also raises the AT_FLAGS_TRANSPARENT_INTERP aux vector
bit. The interpreter keeps control of mapping the binary.
The second mode is loader substitution. This allows a binary to be
executed natively and only the interpreter to be changed.
- Last, interpreters can be bound at registration time. Each
interpreter is opened by its own write with the credentials the
entry file was opened with. The program picks one per exec with
bpf_binprm_select_interp().
Ucounts are used to properly account for pre-opened interpreters
via /proc/sys/user/max_binfmt_misc_interpreters"
* tag 'vfs-7.3-rc1.binfmt' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs: (63 commits)
binfmt_misc: document the pre-opened interpreter limit
selftests/exec: test the pre-opened interpreter limit
binfmt_misc: correctly account pre-opened interpreters
binfmt_misc: document interpreters bound by a 'B' entry
selftests/exec: test interpreters bound to a 'B' entry
binfmt_misc: let a 'B' entry bind its interpreters
binfmt_misc: carry pre-opened interpreters in struct binfmt_misc_interp
selftests/exec: share the bpf handler preconditions
binfmt_misc: document registering an entry disabled
selftests/exec: test registering an entry disabled
selftests/exec: let binfmt_flag_supported() return a bool
selftests/exec: check that a binfmt_misc instance cannot be pinned
binfmt_misc: let a register string create an entry disabled
binfmt_misc: document loader substitution
selftests/exec: test binfmt_misc loader substitution
binfmt_misc: let a bpf handler request loader substitution
binfmt_misc: add the 'L' loader substitution flag
binfmt_elf_fdpic: consume a stashed PT_INTERP substitute
binfmt_elf: consume a stashed PT_INTERP substitute
exec: carry a PT_INTERP substitute in struct linux_binprm
...
|
|
Implement zero-copy in fuse io-uring to eliminate memory copies between
the application, kernel, and server for read/write operations. The
server can directly access client pages or page cache folios without
copying data through an intermediary buffer. When a fuse request arrives,
the kernel registers the relevant pages into a sparse slot in the
server's io_uring registered buffer table. The server can then operate
on these pages directly using io-uring fixed buffer operations (eg
read_fixed/write_fixed) and the kernel unregisters these pages when the
request completes. Non-page-backed args (eg op out headers) will go
through the payload buffer as normal. The server can specify which open
files should have their reads/writes go through zero-copy, by setting
the FOPEN_IO_URING_ZERO_COPY flag when servicing opens.
This requires CAP_SYS_ADMIN and bufpools. This is gated behind
CAP_SYS_ADMIN because zero-copy allows the server direct access to the
client's underlying pages, rather than operating on an intermediary
buffer that the contents of the client's pages were copied into or on
page cache folios.
The request flow for the zero-copy direct-io write path (client writes
data, server reads it) is as follows:
=======================================================================
| Kernel | FUSE server
| |
| "write(fd, buf, 1MB)" |
| |
| >sys_write() |
| >fuse_file_write_iter() |
| >fuse_send_one() |
| [req->args->in_pages = true] |
| [folios hold client write data] |
| |
| >fuse_uring_copy_to_ring() |
| >copy_header_to_ring(IN_OUT) |
| [memcpy fuse_in_header] |
| >copy_header_to_ring(OP) |
| [memcpy write_in header] |
| |
| >fuse_uring_args_to_ring() |
| >setup_fuse_copy_state() |
| [skip_folio_copy = true] |
| |
| >fuse_uring_set_up_zero_copy() |
| [folio_get for each client folio] |
| [build bio_vec array from folios] |
| >io_buffer_register_bvec() |
| [register pages at
ent->zero_copy_index] |
| [ent->zero_copied = true] |
| |
| >fuse_copy_args() |
| [skip_folio_copy => return 0 |
| for page arg, skip data copy] |
| |
| >copy_header_to_ring(RING_ENT) |
| [memcpy ent_in_out] |
| >io_uring_cmd_done() |
| |
| | [CQE received]
| |
| | [issue io_uring READ at
| | ent->zero_copy_index]
| | [reads directly from
| |client's pages (ZERO_COPY)]
| |
| | [write data to backing
| | store]
| | [submit COMMIT AND FETCH]
| |
| >fuse_uring_commit_fetch() |
| >fuse_uring_commit() |
| >fuse_uring_copy_from_ring() |
| >fuse_uring_req_end() |
| >io_buffer_unregister(ent->zero_copy_index) |
| [unregister pages from index] |
| >fuse_zero_copy_release() |
| [folio_put for each folio] |
| [ent->zero_copied = false] |
| >fuse_request_end() |
| [wake up client] |
The zero-copy read path is analogous.
Some requests may have both page-backed args and non-page-backed args.
For these requests, the page-backed args are zero-copied while the
non-page-backed args are copied to the buffer selected from the buffer
pool:
zero-copy: pages registered via io_buffer_register_bvec()
non-page-backed: copied to payload buffer via fuse_copy_args()
For a request whose payload is zero-copied, the
registration/unregistration path looks like:
register: fuse_uring_set_up_zero_copy()
folio_get() for each folio
io_buffer_register_bvec(ent->zero_copy_index)
unregister: fuse_uring_req_end()
io_buffer_unregister(ent->zero_copy_index)
-> fuse_zero_copy_release() callback
folio_put() for each folio
Please note that on abort for in-flight zero-copied requests that have
been sent to userspace, the registered bvec slot remains occupied and
its folios remain pinned until the io-uring ring is destroyed, at which
point io-uring unregisters all buffers and the fuse_zero_copy_release()
callback drops the folio references. Unregistering at teardown would
require operating on the ring context directly, whose validity is hard
to ascertain; this is deemed not worth the complexity for the abort
race, since everything is freed when the ring is torn down.
The throughput improvement from zero-copy depends on how much of the
per-request latency is spent on data copying vs backing I/O. The gain
comes from eliminating the payload-buffer memcpy, but accessing the
zero-copied pages requires the server to issue the read/write as an
IORING_OP_READ/WRITE_FIXED operation. The benefit is largest when the
mempcy is a meaningful fraction of per-request latency while backing i/o
is still noticable enough that the extra io-uring op's overhead doesn't
dominate.
Benchmarked with passthrough_hp (--nopassthrough, q_depth=8) on a
2-socket Intel Xeon Gold 6138 (40 cores / 80 threads), using fio (sync
engine, bs=1M, O_DIRECT, numjobs=2, 30s run + 10s ramp, 3 runs) where
direct-I/O throughput is against a RAM-backed (tmpfs) source (backing
I/O is not the bottleneck):
baseline registered-buf zero-copy (zc vs base)
direct read ~5.1 GB/s ~5.4 GB/s ~8.9 GB/s (+75%)
direct write ~3.4 GB/s ~4.8 GB/s ~5.1 GB/s (+50%)
Reads end up higher than writes because the backing store reads faster
than it writes (the baseline shows the same read>write gap, and the raw
device does too). On a device-bound NVMe (~2 GB/s reads) the read gain
shrinks to ~10-16% (and no measurable gains for writes), as backing I/O
rather than the eliminated copy dominates latency.
The benefit overall scales with how much of the
per-request latency is the data copy versus backing I/O.
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Reviewed-by: Bernd Schubert <bernd@bsbernd.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
|
|
Right now, ents and buffers are tightly coupled in fuse io-uring where
each entry has its own dedicated payload buffer, requiring N buffers for
N entries where each buffer must be large enough to accomodate the
maximum payload size. This is suboptimal as most request types (lookup,
open, release, getattr, etc) require vastly less bytes than the maximum
payload size and some requests (unlink, rmdir, fsync, flush, etc) do not
require payload buffers at all.
Instead of requiring a 1:1 coupling between ents and payload buffers,
allow the server to pass in a buffer pool (a contiguous chunk of memory)
that the kernel will use as it wishes for servicing ents/requests.
Entries only reserve a "buffer" from the pool while actively processing
a request that requires a payload buffer. This decoupling and letting
the kernel delegate memory from the pool for requests allows the kernel to
optimize memory usage and reduces the memory usage requirements needed
to use fuse-over-io-uring.
A pool is registered per queue with the new
FUSE_IO_URING_CMD_ADD_BUFPOOL command. The server passes the pool's base
address and length in fuse_uring_cmd_req.bufpool.{uaddr,len}.
Internally, the kernel splits the region into buffers of
ring->max_payload_sz bytes each (nr_bufs = pool len / max_payload_sz). A
queue commits to a payload mode on first use: registering an entry that
carries its own payload selects the legacy per-entry mode, while
ADD_BUFPOOL selects pool mode. The two are mutually exclusive, so
ADD_BUFPOOL must be issued before any payload-carrying entries are
registered on that queue. The queue must have been created before the
bufpool is added, through the FUSE_IO_URING_CMD_ADD_QUEUE command.
The kernel tracks free buffers with a bitmap (a set bit marks a free
buffer). On dispatch, a request that needs a payload claims a free
buffer (find_first_bit + clear). A request that needs none claims
nothing. The buffer's byte offset within the pool is reported to the
server in the new fuse_uring_ent_in_out.offset field so that the server
can locate the payload. On completion the buffer is returned to the pool
or reused directly if the next request on that entry also has a payload.
The FUSE_HAS_IO_URING_BUFPOOL flag advertises kernel support to the
server for bufpools.
Buffer pool request flow
~~~~~~~~~~~~~~~~~~~~~~~~
| Kernel | FUSE daemon
| |
| [request arrives] |
| [claim a free pool buffer] |
| >fuse_uring_select_buffer() |
| [copy headers to ring] |
| [copy payload to buffer] |
| [report buffer offset in ent_in_out] |
| >io_uring_cmd_done() |
| | [read headers]
| | [read/write payload at offset]
| | [process request]
| | >io_uring_submit()
| | COMMIT_AND_FETCH
| >fuse_uring_commit_fetch() |
| [copy reply from ring] |
| [return buffer to the pool] |
| >fuse_uring_recycle_buffer() |
Reviewed-by: Bernd Schubert <bernd@bsbernd.com>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
|
|
fuse-over-io-uring queues are currently created lazily, as a side effect
of the first FUSE_IO_URING_CMD_REGISTER command for a given qid. This
ties queue creation to entry registration.
Add a FUSE_IO_URING_CMD_ADD_QUEUE command so a server can create a queue
explicitly, decoupling queue setup from entry registration. This is
additionally a prerequisite for FUSE_IO_URING_CMD_ADD_BUFPOOL, which
attaches a buffer pool to an existing queue and therefore needs the
queue to have been created first.
Reviewed-by: Bernd Schubert <bernd@bsbernd.com>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
|
|
Rename IO_IMU_DEST and IO_IMU_SOURCE to IO_BUF_DEST and IO_BUF_SOURCE
and export it so subsystems may use it.
This is needed by the io_buffer_register_bvec() path for callers who may
need the buffer to be both readable and writable.
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Link: https://patch.msgid.link/20260612184840.4058966-5-joannelkoong@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
|
|
Add io_buffer_register_bvec() for registering a bvec array.
This is a preparatory patch for fuse-over-io-uring zero-copy.
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Reviewed-by: Caleb Sander Mateos <csander@purestorage.com>
Link: https://patch.msgid.link/20260612184840.4058966-4-joannelkoong@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
|
|
Currently, io_buffer_register_bvec() takes in a request. In preparation
for supporting kernel-populated buffers in fuse io-uring (which will
need to register bvecs directly, not through a struct request), rename
this to io_buffer_register_request().
A subsequent patch will commandeer the "io_buffer_register_bvec()"
function name to support registering bvecs directly.
Rename io_buffer_unregister_bvec() to a more generic name,
io_buffer_unregister(), as both io_buffer_register_request() and
io_buffer_register_bvec() callers will use it for unregistration.
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Reviewed-by: Caleb Sander Mateos <csander@purestorage.com>
Link: https://patch.msgid.link/20260612184840.4058966-2-joannelkoong@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
|
|
Add SoC ID for Qualcomm Snapdragon SDM850.
Signed-off-by: David Heidelberg <david@ixit.cz>
Reviewed-by: Konrad Dybcio <konrad.dybcio@oss.qualcomm.com>
Acked-by: Krzysztof Kozlowski <krzysztof.kozlowski@oss.qualcomm.com>
Link: https://lore.kernel.org/r/20260627-sda850-v2-1-44bf46ade42e@ixit.cz
Signed-off-by: Bjorn Andersson <andersson@kernel.org>
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
|
|
Add new bpf_skb_adjust_room() decapsulation flags:
- BPF_F_ADJ_ROOM_DECAP_L4_GRE
- BPF_F_ADJ_ROOM_DECAP_L4_UDP
- BPF_F_ADJ_ROOM_DECAP_IPXIP4
- BPF_F_ADJ_ROOM_DECAP_IPXIP6
These flags let BPF programs describe which tunnel layer is being
removed, so later changes can update tunnel-related GSO state
accordingly during decapsulation.
This patch only introduces the UAPI flag definitions and helper
documentation.
Co-developed-by: Max Tottenham <mtottenh@akamai.com>
Co-developed-by: Anna Glasgall <aglasgal@akamai.com>
Signed-off-by: Max Tottenham <mtottenh@akamai.com>
Signed-off-by: Anna Glasgall <aglasgal@akamai.com>
Signed-off-by: Nick Hudson <nhudson@akamai.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Willem de Bruijn <willemb@google.com>
Link: https://lore.kernel.org/bpf/20260812083115.73100-4-nhudson@akamai.com
|
|
The existing anonymous enum for BPF_FUNC_skb_adjust_room flags is
named to enum bpf_adj_room_flags to enable CO-RE (Compile Once -
Run Everywhere) lookups in BPF programs.
Co-developed-by: Max Tottenham <mtottenh@akamai.com>
Co-developed-by: Anna Glasgall <aglasgal@akamai.com>
Signed-off-by: Max Tottenham <mtottenh@akamai.com>
Signed-off-by: Anna Glasgall <aglasgal@akamai.com>
Signed-off-by: Nick Hudson <nhudson@akamai.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Willem de Bruijn <willemb@google.com>
Link: https://lore.kernel.org/bpf/20260812083115.73100-2-nhudson@akamai.com
|
|
Scope and ptrace denials follow a different code path (a domain
hierarchy check) than access-right denials, so they need dedicated
tracepoints with type-specific TP_PROTO arguments. Complete the denial
coverage with:
- landlock_deny_ptrace: ptrace access denied by a domain hierarchy
mismatch.
- landlock_deny_scope_signal: signal delivery denied by
LANDLOCK_SCOPE_SIGNAL.
- landlock_deny_scope_abstract_unix_socket: abstract unix socket access
denied by LANDLOCK_SCOPE_ABSTRACT_UNIX_SOCKET.
TP_PROTO passes the raw kernel object (struct task_struct or struct
sock) for eBPF BTF access; the comm and sun_path string fields use
__print_untrusted_str() because they hold untrusted input. Unlike the
deny_access events, these omit the blockers field: each maps to exactly
one denial type named by the event, so the bitmask would always be zero.
Like the deny_access events they carry same_exec and logged.
Audit logs the task-targeted denials with generic field names (opid,
ocomm), but a strongly typed trace event can use role-prefixed names
(tracee_pid/tracee_comm, target_pid/target_comm) that match the mainline
task-name convention (sched_process_fork's parent_comm/child_comm) and
say whose name each field holds; a bare comm= would collide across
events. The abstract-unix-socket event reports peer_pid instead, a
tracepoint-only field with no audit counterpart.
A scope or ptrace verdict compares the subject domain against the other
party's domain, so each event also reports that other party's Landlock
domain (tracee_domain=, target_domain=, or peer_domain=); the subject
domain= alone does not let a consumer redo domain_is_scoped() or
domain_ptrace(). It is reported as a scalar ID rather than a domain
pointer: a domain object is immutable, but the other task can replace
its credential and free the domain that credential referenced, so a
stored foreign pointer could dangle before the event is consumed. The
scalar ID also honors the tracepoint no-nullable-pointer rule, since the
other party is frequently unsandboxed. Passing the foreign domain
hierarchy object so an eBPF consumer could walk the other party's
ancestry live would lengthen the RCU section on the shared denial path
and needs a deferred refcount put, so it is left as a future
enhancement. The relational domain-ID field (tracee_domain,
target_domain, or peer_domain) is trace-only and is not added to audit
records, so audit's denial format is unchanged by this series.
Cc: Günther Noack <gnoack@google.com>
Cc: Justin Suess <utilityemal77@gmail.com>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Tingmao Wang <m@maowtm.org>
Link: https://patch.msgid.link/20260811094338.288094-14-mic@digikod.net
Signed-off-by: Mickaël Salaün <mic@digikod.net>
|
|
Add per-type tracepoints emitted from landlock_log_denial() when an
access is denied: landlock_deny_access_fs for filesystem denials and
landlock_deny_access_net for network denials. They use the "deny_"
prefix (rather than "check_") to mark that they fire only on a denial,
and they complement the check_rule events by making the
denial-by-absence case explicit (when no rule matches, no check_rule
event fires).
Unlike the audit records, these events fire regardless of the audit
configuration and the domain's log flags: the user's "disable logging"
intent applies to audit records, not to kernel tracing. The logged
field records whether the domain's log policy would submit the denial to
audit; it is the decision computed once by landlock_log_denial() and
passed to both the audit and the tracing emitter, so a stateless ftrace
filter can select the audit-visible denials with logged==1.
TP_PROTO passes the denying hierarchy node, not the task's current
domain, so domain_id reports the specific node that blocked the access,
matching audit record semantics. (check_rule instead passes the current
domain, which it needs to size its per-layer array.) same_exec is also
passed explicitly because it is computed from the credential bitmask and
is not derivable from the hierarchy pointer alone. The denial field is
named blockers to match the audit record field.
The filesystem path comes from the request's audit data. Its type
selects which union member holds the object, exactly as
dump_common_audit_data() selects it (a path, a file's path, an ioctl
op's path, or a bare dentry); reading the wrong member would dereference
garbage, so every reachable type has an explicit case and an unexpected
one is flagged with WARN_ONCE() instead of misread. Path-backed types
resolve via d_absolute_path() (as landlock_add_rule_fs does) and the
bare-dentry case via dentry_path_raw().
The inode number is read defensively. A filesystem denial can carry a
negative dentry (no backing inode), for example a denied creation, so
the event mirrors the guard in dump_common_audit_data() and reports
inode 0 rather than dereferencing a NULL inode. The sibling fs
tracepoints do not need the guard: a dentry that matches a rule during
an access check, or one opened to add a rule, always has a backing
inode. Landlock tracepoints are reachable by unprivileged sandboxees,
so a denial on a negative dentry with the event enabled must not fault
the kernel.
Cc: Günther Noack <gnoack@google.com>
Cc: Justin Suess <utilityemal77@gmail.com>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Tingmao Wang <m@maowtm.org>
Link: https://patch.msgid.link/20260811094338.288094-13-mic@digikod.net
Signed-off-by: Mickaël Salaün <mic@digikod.net>
|
|
Merge landlock_find_rule() into landlock_unmask_layers() so rule
pointers stay inside the domain implementation while unmask checking
gets the matched rule it needs for the check_rule tracepoint.
landlock_unmask_layers() now takes a landlock_id and the domain instead
of a rule pointer. A rename or link evaluates the same dentry against
both renamed parents, so this path now looks the rule up once per
parent; collapsing that back to a single lookup is left to a follow-up.
Emit, via the per-type wrappers unmask_layers_fs() and
unmask_layers_net(), the rights each matching rule grants at every
domain layer. The events carry this as a dynamic per-layer array (up to
LANDLOCK_MAX_NUM_LAYERS entries) reserved from the trace ring buffer,
not the caller's stack, and rendered symbolically per layer. A
WARN_ON_ONCE() in __trace_landlock_fill_layers() flags a rule whose
layer levels fall outside the domain range or are unsorted, a
cannot-happen case; the zero-filled slots keep the rendered output and
the array bounds safe regardless.
Setting allowed_parent2 to true for non-dom-check requests when
get_inode_id() returns false preserves the pre-refactoring behavior: a
negative dentry (no backing inode) has no matching rule, so the access
is allowed at this path component. Before the refactoring,
landlock_unmask_layers() with a NULL rule produced this result as a side
effect; now the caller must set it explicitly.
Name the trace-only check_rule fields so each printk label equals its
ring-buffer field name and works directly as an ftrace filter: the
request field is labelled access_request= and the per-layer array is
named grants. Values audit also logs keep audit's label (domain=,
ruleset=) so a single filter works across trace and audit.
Cc: Günther Noack <gnoack@google.com>
Cc: Justin Suess <utilityemal77@gmail.com>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Tingmao Wang <m@maowtm.org>
Link: https://patch.msgid.link/20260811094338.288094-12-mic@digikod.net
Signed-off-by: Mickaël Salaün <mic@digikod.net>
|
|
The landlock_create_domain event records that a domain was created,
once, before thread-sync. It cannot tell which threads end up enforcing
it: a successful landlock_restrict_self(2) with
LANDLOCK_RESTRICT_SELF_TSYNC applies the domain to the caller and every
eligible sibling. Creation (the operation) and enforcement (the
per-thread outcome) are distinct.
Add landlock_enforce_domain(domain, complete, process_wide), emitted
once per thread the domain is applied to, strictly after that thread's
commit_creds(), so it fires only for a thread that is enforcing the
domain, never speculatively; an aborted operation emits none. The
lifecycle now reads create -> enforce* -> free.
The two booleans name properties, not the implementation:
- complete: marks the single event that concludes the operation. It
names the outcome, the set is now enforced, not which thread
finishes, which the contract leaves unspecified.
- process_wide: means every eligible thread of the process is
covered. It is set race-free by either establishing path,
thread-sync or a single-threaded process, so
complete && process_wide is the whole-process-enforced guarantee.
The requesting thread and source ruleset are not repeated here: they are
on create_domain (joined via domain->hierarchy->id) and on the immutable
domain->hierarchy->details. Source ruleset means the ruleset_id and
ruleset_version recorded on create_domain, not the ruleset object, which
the caller may close before enforcement.
Cc: Günther Noack <gnoack@google.com>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Tingmao Wang <m@maowtm.org>
Link: https://patch.msgid.link/20260811094338.288094-11-mic@digikod.net
Signed-off-by: Mickaël Salaün <mic@digikod.net>
|
|
Add a landlock_create_domain tracepoint emitted from
landlock_restrict_self() after the new domain is created, so a consumer
can correlate the source ruleset with the resulting domain. The
flags-only path (ruleset_fd == -1) creates no domain and emits no event.
Move the ruleset lock acquisition from landlock_merge_ruleset() to the
caller so the lock is held across both the merge and the tracepoint
emission, giving an eBPF program a consistent ruleset snapshot. Release
it before the thread-sync: holding ruleset->lock across
landlock_restrict_sibling_threads() would deadlock a sibling blocked on
the same lock. The event therefore fires before the (rare) thread-sync
failure path; when that path aborts the just-created domain, the
matching free_domain event fires so the create/free pair stays balanced.
Add a landlock_free_domain tracepoint that fires when a domain's
hierarchy node is freed. The hierarchy node is the lifecycle boundary
because it represents the domain's identity and outlives the domain's
access masks, which may still be active in descendant domains.
A domain freed without ever being committed to a credential was never
visible to user space, so free_domain is suppressed for it. This is
tracked by a new landlock_log_status value, LANDLOCK_LOG_UNCOMMITTED,
which is also the zero value so a hierarchy whose initialization failed
defaults to not observable. A hierarchy is born UNCOMMITTED and is
promoted to LANDLOCK_LOG_PENDING (or LANDLOCK_LOG_DISABLED when logging
is off) right after its create_domain event fires; a thread-sync failure
does not reset it, so an aborted domain that already emitted
create_domain still emits the matching free_domain. Promoting right
after the event, rather than at commit_creds() time, avoids a race: on a
successful thread-sync the sibling threads commit the new domain in
lockstep before landlock_restrict_self() returns, so the shared domain
may already have moved to LANDLOCK_LOG_RECORDED through a plain store,
and a late promotion would race that store and could unbalance the
domain allocation and deallocation audit records.
Cc: Günther Noack <gnoack@google.com>
Cc: Justin Suess <utilityemal77@gmail.com>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Tingmao Wang <m@maowtm.org>
Link: https://patch.msgid.link/20260811094338.288094-10-mic@digikod.net
Signed-off-by: Mickaël Salaün <mic@digikod.net>
|
|
Add tracepoints for Landlock rule addition, landlock_add_rule_fs for
filesystem rules and landlock_add_rule_net for network rules, so trace
consumers can correlate filesystem objects and network ports with their
rulesets. Both are emitted under the ruleset lock (asserted in
TP_fast_assign) so an eBPF program reads the ruleset, including the rule
just inserted, in a consistent snapshot.
Add a version field to struct landlock_ruleset, gated on
CONFIG_TRACEPOINTS like the id field and incremented under the ruleset
lock on each successful landlock_add_rule(2), including when it only
extends an existing rule's access rights. It fills the existing 4-byte
hole after usage, so the struct does not grow. Pairing the ruleset ID
with the version lets a later restrict_self event record the exact
ruleset revision merged into a domain.
Resolve the filesystem rule's absolute path with d_absolute_path()
rather than the d_path() audit uses: d_absolute_path() produces
namespace-independent paths that do not depend on the tracer's chroot
state, making trace output deterministic regardless of mount namespace
configuration. Distinguish the error cases as "<too_long>"
(-ENAMETOOLONG) and "<unreachable>" (anonymous files or detached
mounts).
Also add __trace_print_untrusted_str(), a static inline helper in the
header guarded by CREATE_TRACE_POINTS: it escapes separators, quotes,
backslashes, and non-printable bytes via string_escape_mem() so an
untrusted string (the path here, process names in later denial events)
cannot inject field separators or control characters into the ftrace
text output.
Cc: Christian Brauner <brauner@kernel.org>
Cc: Günther Noack <gnoack@google.com>
Cc: Justin Suess <utilityemal77@gmail.com>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Tingmao Wang <m@maowtm.org>
Link: https://patch.msgid.link/20260811094338.288094-9-mic@digikod.net
Signed-off-by: Mickaël Salaün <mic@digikod.net>
|
|
Add the first Landlock tracepoints, for ruleset lifecycle:
landlock_create_ruleset fires from the landlock_create_ruleset() syscall
handler, and landlock_free_ruleset fires in free_ruleset() before the
ruleset is freed.
These tracepoints, and the ones added by the following commits, share a
common design. Rather than one polymorphic event distinguished by a
status field (as audit uses a shared record type with a "status="
field), each lifecycle transition and denial type gets its own event
with a type-safe TP_PROTO, giving precise ftrace filtering by event name
and type-safe eBPF access. TP_PROTO passes the object pointer and the
fields are read from it in TP_fast_assign, so an eBPF program reads the
full object state (rules, access masks, hierarchy) via BTF from a single
pointer rather than from the flattened TP_STRUCT__entry fields. The
whole cost is paid only when a tracer is attached; the static branch is
not taken otherwise. Trace fields carry the bare access-right and scope
names (read_file), reusing the audit name tables; audit prepends the
category (fs.read_file), which the trace event name already conveys.
The trace header's DOC comment documents the consistency and locking
guarantees these events share.
create_ruleset needs no lock because the ruleset is not yet shared (its
file descriptor is not yet installed). The deallocation events use the
"free_" prefix, not "drop_", because they fire when the object is
actually freed.
Add trace.c, built for CONFIG_TRACEPOINTS, which defines
CREATE_TRACE_POINTS, and extend CONFIG_SECURITY_LANDLOCK_LOG to also be
selected by CONFIG_TRACEPOINTS so the common log framework is available
to a tracepoints-only build.
Add an id field to struct landlock_ruleset, gated on CONFIG_TRACEPOINTS
and assigned from landlock_get_id_range() at creation. Only the
tracepoints consume it (audit identifies domains, not rulesets), so it
does not exist in an audit-only build. The Landlock ID is a stable u64
that names the ruleset across the trace stream and uses the same scheme
as audit, so a ruleset can be correlated between trace and audit
records.
Cc: Günther Noack <gnoack@google.com>
Cc: Justin Suess <utilityemal77@gmail.com>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Tingmao Wang <m@maowtm.org>
Link: https://patch.msgid.link/20260811094338.288094-8-mic@digikod.net
Signed-off-by: Mickaël Salaün <mic@digikod.net>
|
|
Audit formats denial records with per-right name strings. A following
commit adds trace events that print the same access and scope masks with
__print_flags() and need the same names, but a trace event header cannot
include Landlock-internal headers, so the names cannot be shared from
the logging unit.
Define the filesystem, network, and scope names once, as the
_LANDLOCK_ACCESS_FS_NAMES, _LANDLOCK_ACCESS_NET_NAMES, and
_LANDLOCK_SCOPE_NAMES lists in the public Landlock header. Each entry
is a _LANDLOCK_NAME_ENTRY() the consumer expands: audit maps it to a
"[bit] = name" array slot for an O(1) lookup, the trace events map it to
a __print_flags() { mask, name } pair. The bit value comes only from
the LANDLOCK_* UAPI constant each entry references, so every bit-to-name
mapping has a single source and does not depend on entry order.
The shared names are unprefixed; blocker_prefix() prepends the
fs./net./scope. category for audit records, so the scope names move from
inline literals to the shared table too. Audit records are unchanged.
No functional change.
Cc: Günther Noack <gnoack@google.com>
Cc: Tingmao Wang <m@maowtm.org>
Link: https://patch.msgid.link/20260811094338.288094-7-mic@digikod.net
Signed-off-by: Mickaël Salaün <mic@digikod.net>
|
|
bpf_convert_ctx_accesses() turns a BPF_LDX into a BPF_PROBE_MEM one by
matching the type recorded for the insn against a list of exact pointer
types. The list cannot keep up with the flag combinations the verifier
produces, and a type which is missing from it ends up as a plain load
without an exception table entry, so a bad address panics the kernel
instead of being handled.
Two such types exist today and are reachable:
- PTR_TO_BTF_ID | PTR_UNTRUSTED | MEM_ALLOC | NON_OWN_REF
- PTR_TO_BTF_ID | PTR_UNTRUSTED | MEM_RCU
Rather than adding the two, just drop the list and state the property
itself in the default case of the switch. This is a superset of what
the list matched, the untrusted PTR_TO_MEM does not have to carry
MEM_RDONLY for it anymore, and it stays in sync with the verifier side
which uses the same match in save_aux_ptr_type() and reg_type_mismatch_ok().
Assert that a fault prone type which does not get the rewrite for whatever
reason is rejected at load time rather than left to fault at runtime to
catch any future cases.
Fixes: 1b12171533a9 ("bpf: Mark direct ld of stashed bpf_{rb,list}_node as non-owning ref")
Fixes: 6fcd486b3a0a ("bpf: Refactor RCU enforcement in the verifier.")
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260814215301.709827-4-daniel@iogearbox.net
|
|
When the same BPF_LDX instruction is reached through paths that yield
different pointer types, save_aux_ptr_type() merges them into a single
type which is later used by bpf_convert_ctx_accesses() to decide whether
the load has to be rewritten into a BPF_PROBE_MEM one.
Before f2362a57aeff ("bpf: allow void* cast using bpf_rdonly_cast()")
the merge only accepted two PTR_TO_BTF_ID pointers and unconditionally
fell back to PTR_TO_BTF_ID | PTR_UNTRUSTED, so the merged type was always
one that gets the BPF_PROBE_MEM rewrite. However, the mentioned commit
widened the merge to also cover a PTR_TO_MEM base and replaced the
fallback by a union of the PTR_UNTRUSTED and MEM_RDONLY flags.
A union of flags though cannot express the property the later rewrite
is built upon, some examples:
- PTR_TO_MEM merged with PTR_TO_BTF_ID | PTR_UNTRUSTED gets
PTR_TO_MEM | PTR_UNTRUSTED but only the MEM_RDONLY variant is valid
- PTR_TO_MEM merged with a plain PTR_TO_BTF_ID gets PTR_TO_MEM
dropping the rewrite the latter type would have gotten
- PTR_TO_MEM | MEM_RDONLY merged with a plain PTR_TO_BTF_ID gets
PTR_TO_MEM | MEM_RDONLY which is not rewritten either since only
its PTR_UNTRUSTED variant is
In all three cases a program can take the unsafe path at runtime with a
NULL or otherwise bad pointer and panic the kernel on the faulting load:
BUG: kernel NULL pointer dereference, address: 0000000000000038
RIP: 0010:bpf_prog_77531a87032eeaf1_mixed_mem_btf_id_type+0x4b/0x65
Call Trace:
<TASK>
bpf_test_run+0x20b/0x460
bpf_prog_test_run_skb+0x650/0xbe0
__sys_bpf+0xb96/0x3140
__x64_sys_bpf+0x2c/0x40
do_syscall_64+0xba/0x590
Kernel panic - not syncing: Fatal exception in interrupt
Note that the last two shapes have to be fixed right here, otherwise
the merged type retains nothing which marks the load as fault prone,
thus no rule in bpf_convert_ctx_accesses() can recover it. Fix it by
normalizing the merged type instead.
Reuse it in is_load_acq_unsafe() to avoid open coding, and trim the
overly verbose comment which is more of an implementation detail of
bpf_convert_ctx_accesses() anyway.
Fixes: f2362a57aeff ("bpf: allow void* cast using bpf_rdonly_cast()")
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260814215301.709827-1-daniel@iogearbox.net
|
|
|
|
Add a helper that returns PCI_IRQ_MSIX, PCI_IRQ_MSI, or PCI_IRQ_INTX
based on the interrupt type the PCI core selected after
pci_alloc_irq_vectors().
Several drivers already open-code this check against pdev->msix_enabled
and pdev->msi_enabled, or even open code this helper [1].
A common helper avoids the duplication and keeps drivers from accessing
the bitfield directly (see also [2]).
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Tested-by: John Hubbard <jhubbard@nvidia.com>
Link: https://elixir.bootlin.com/linux/v7.1/source/drivers/net/ethernet/aquantia/atlantic/aq_pci_func.c#L196 [1]
Inspired-by: John Hubbard <jhubbard@nvidia.com>
Link: https://lore.kernel.org/all/DKKG2QM3YJYB.Z2H2B2UXJ75N@kernel.org/ [2]
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260813165234.620555-5-dakr@kernel.org
[ Add missing pci_irq_type() stub for CONFIG_PCI=n. ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull timer fixes from Borislav Petkov:
- Detect a broken EL2 virtual timer in the bcm2712 SoC boards (RPi5)
and fallback to the physical one instead
- Fix a build error with ARM rpc_defconfig and function tracer enabled
* tag 'timers_urgent_for_v7.2' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
clocksource/drivers/arm_arch_timer: Workaround bcm2712 broken EL2 virtual timer
tick: Include ktime.h and jiffies.h in linux/tick.h
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull rseq fix from Borislav Petkov:
- Prevent a lockup when rseq grants a timeslice extension
* tag 'core_urgent_for_v7.2' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
rseq: Prevent hard lockup on granted time slice extension
|
|
The default core-sched ordering runs the longest waiting task first by
comparing p->scx.core_sched_at stamps. The stamp is maintained under two
rules. touch_core_sched() stamps when a task starts waiting for a CPU and
when its slice runs out. If the scheduler implements
ops.core_sched_before(), touch_core_sched_dispatch() re-stamps on every
dispatch.
A comparison can see one stamp taken under each rule, which isn't a
meaningful ordering. The dispatch rule also buys little - it only aligns
bypass-mode comparisons with the local DSQ order. Multiple schedulers make
the mixed comparisons more common.
Wait time is what p->scx.runnable_at already tracks for the stall watchdog.
Delete core_sched_at with both touch functions and compare runnable_at in
the scx_prio_less() fallback.
runnable_at is refreshed only on enqueue and goes stale while a task keeps
occupying its CPU. Instead of re-stamping, order a running task after every
waiting task as it is the most recently serviced.
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
It's not always possible for the user to predict during registration how
much memory zcrx will need to sustain the traffic. Allow to dynamically
add more areas with a new ctrl code ZCRX_CTRL_ADD_AREA.
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://patch.msgid.link/43495ca686713b3f0014c9f47c4c0276d704da4d.1786108672.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
blk_cgroup_congested() walks the current task's blkcg ancestor chain on
every readahead decision and, once swap is in use, on every anonymous and
shmem folio allocation. The answer is almost always "no", but finding that
out costs two loads per level on two cold cache lines, plus an out-of-line
kthread_blkcg() and an RCU read-side pair. On a fleet profile of hosts
running containers with 5-10 level hierarchies it costs about as much as
all of mutex_lock(), 99.4% of it under __folio_throttle_swaprate().
Gate the walk on a global count of blkcgs with a non-zero
congestion_count. The counter only moves on the 0 <-> 1 transitions of
each blkcg's congestion_count, so the extra atomic stays in the throttle
arm/disarm paths and never appears in steady state. When something is
throttled the counter is non-zero and the walk runs as before.
Signed-off-by: Usama Arif <usama.arif@linux.dev>
Acked-by: Tejun Heo <tj@kernel.org>
Link: https://patch.msgid.link/20260814165712.510132-4-usama.arif@linux.dev
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
In preparation to supporting dma-buf backed iterators and bios,
introduce bio_iov_iter_set() which attempts to set up the bio directly
from the given iterator. For now, it only supports bvec and expects
users to check the result and fall back to other means if fails, but
later we'll add more types.
Suggested-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/4686a0e47fc14f3f888967a80d45a6f66044f1e0.1785596451.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
io_req_local_work_add() signals the CQ ring eventfd inline when it is the
one to push the first entry onto ->work_list. For DEFER_TASKRUN rings that
add is frequently done from a waitqueue wakeup handler, where an
arbitrary waitqueue lock is held.
eventfd_signal_mask() only refuses to recurse when current->in_eventfd
is set, but that bit is set by eventfd_signal_mask() itself. If the wake
chain starts somewhere else, signal goes out inline and can feed back
into epoll.
Add IOU_F_TWQ_IN_WAKE, set it on the task_work add done from the three
waitqueue callbacks, and use it to force io_eventfd_signal() down the
existing call_rcu_hurry() deferral instead of signaling inline.
Fixes: 21a091b970cd ("io_uring: signal registered eventfd to process deferred task work")
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/all/20260813133843.2933127-1-4ncienth@gmail.com/
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
futex_exit_release() and futex_exec_release() are identical now. That means
also exit_mm_release() and exec_mm_release() are identical.
Consolidate the whole lot and remove the redundant copies.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Kyle Zeng <kylebot@openai.com>
Acked-by: Peter Zijlstra <peterz@infradead.org>
|
|
The check for private futexes whether the waiter's mm, which is stored in
the futex_key and copied into the pi_state, is the same as the owner's mm
is not sufficient for exec(). exec() has a gap where the mm check fails to
give the correct answer:
exec()
...
exec_release_mm()
futex_exec_release()
tsk::futex::exit_state = EXITING;
cleanup_robust_list();
1) tsk::futex::exit_state = OK;
...
old_mm = tsk::mm;
2) tsk::mm = ->mm;
Between #1 and #2 the check for the mm is wrong as that mm is about to be
swapped out and eventually freed.
Plug this gap by:
1) Setting tsk::futex::exit_state to FUTEX_STATE_DEAD in
futex_exec_release()
2) Setting tsk::futex::exit_state to FUTEX_STATE_OK after
the mm has been switched.
From a futex point of view the task is dead after it finished the robust
list cleanup up to the point where it sets the state to OK again.
Fixes: 80367ad01d93 ("futex: Add basic infrastructure for local task local hash")
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Kyle Zeng <kylebot@openai.com>
Acked-by: Peter Zijlstra <peterz@infradead.org>
Cc: stable@vger.kernel.org
|
|
Add BPF kfuncs that allow BPF LSM programs to create and use sockets for
sending data. This provides a mechanism for BPF programs to emit
telemetry. For this first patch set, it's restricted to SOCK_DGRAM
socket types with IPPROTO_UDP protocol but could be easily extended to
SOCK_STREAM and IPPROTO_TCP in the future.
The API consists of five kfuncs:
bpf_ksock_create() - Create a socket (sleepable)
bpf_ksock_connect() - Connect socket to remote address (sleepable)
bpf_ksock_send() - Send data through the socket (sleepable)
bpf_ksock_acquire() - Acquire a reference to a socket context
bpf_ksock_release() - Release a reference (cleanup via
queue_rcu_work since sock_release sleeps)
The setup kfuncs bpf_ksock_create, bpf_ksock_connect, can be called from
SYSCALL programs only. While bpf_ksock_acquire, bpf_ksock_release and
bpf_ksock_send can be called from SYSCALL and LSM programs.
The implementation follows the established kfunc lifecycle pattern
(create/acquire/release with refcounting, kptr map storage, dtor
registration). The kernel socket is wrapped in a refcounted bpf_ksock
struct. Cleanup is deferred via queue_rcu_work() because sock_release()
may sleep.
The kfuncs are only compiled when CONFIG_INET is enabled, as they
specifically support AF_INET and AF_INET6 sockets.
The socket operations go through the expected LSM hooks instead of
by-passing them like many kernel sockets since those are created by BPF
programs and thus system users. Thus, the bpf_ksock_send() kfunc, which
is exposed to LSM progs has a verifier filter protection to avoid
recursion so that the whole bpf_kfunc_set kfunc set cannot be called in
a program attached to security_socket_sendmsg(). Also, because of the
LSM checks, we prevent the use of the kfuncs from asynchronous workqueue
as the current value would then be invalid.
In bpf_ksock_create(), we copy the arg values to avoid TOCTOU races
since the kfunc can sleep and the arg values could be stored in a map
that could be re-written by BPF progs or even userspace programs if the
map is mmaped.
Signed-off-by: Mahe Tardy <mahe.tardy@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Acked-by: Song Liu <song@kernel.org>
Link: https://lore.kernel.org/bpf/20260813110540.103550-3-mahe.tardy@gmail.com
|
|
Add a helper that connects an existing socket while invoking the LSM
hook. Reuse it in __sys_connect_file() to avoid duplicating the connect
logic. Other socket operations have equivalent helpers that trigger the
appropriate LSM hooks that can be reused, this one was the only one
missing. This will be used in the next commit for a new BPF kfunc that
needs to connect a socket and trigger the LSM hook.
Signed-off-by: Mahe Tardy <mahe.tardy@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Reviewed-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Reviewed-by: Kuniyuki Iwashima <kuniyu@google.com>
Acked-by: Song Liu <song@kernel.org>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Link: https://lore.kernel.org/bpf/20260813110540.103550-2-mahe.tardy@gmail.com
|
|
Record material register and outgoing stack argument changes so diagnostics can
explain how a value reached its current type, bounds, or unreadable state.
Store old and new register types, scalar ranges, tnum value and mask, map and
BTF type identity, and basic operand metadata in the environment-owned
diagnostic event stream.
Record invalidations when packet data moves, references are released, or
borrowed references leave their protected region. Register-scoped history
starts at the latest matching modification and then shows later branch
outcomes.
Also record fixed stack spills and overwrites, and tag register fills from
stack so register-scoped history can follow value flow through spilled stack
slots.
The type_is_map_ptr() helper previously lived as a static function in
kernel/bpf/log.c since commit 0c95c9fdb696 ("bpf: emit map name in register
state if applicable and available"). Move it verbatim to
include/linux/bpf_verifier.h as a static inline, next to the other type
classifiers, so diagnostics.c can reuse it without duplicating the case list.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://patch.msgid.link/20260815064612.378577-6-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
|
|
Teach verifier diagnostics to annotate an instruction with BTF source
line information and nearby BPF instructions. The renderer keeps source
text in a fixed-width lane and prints instructions in a stable right-hand
gutter.
Wrap annotation text under the source line so long error labels remain
readable while the source and instruction lanes keep their fixed layout.
Keeping source and instruction context in one commit preserves the visual
layout contract that later diagnostic reports rely on.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://patch.msgid.link/20260815064612.378577-3-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
|
|
Add a landlock_restrict_self(2) flag to set the no_new_privs attribute
of the calling thread only after enforcement of the ruleset:
no_new_privs is set if and only if the call succeeds. This removes the
need for a prior prctl(2) PR_SET_NO_NEW_PRIVS call and guarantees that
a failed enforcement leaves the attribute unchanged.
Because no_new_privs is set by the call itself, the no_new_privs /
CAP_SYS_ADMIN requirement of landlock_restrict_self(2) is fulfilled by
construction, and the related EPERM check is skipped.
Unlike LANDLOCK_RESTRICT_SELF_LOG_SUBDOMAINS_OFF, this flag always
requires a valid ruleset: with a ruleset_fd of -1, such a call would be
nothing more than a Landlock-flavored prctl(2) PR_SET_NO_NEW_PRIVS, and
there is no valid use case for setting no_new_privs (possibly with
LANDLOCK_RESTRICT_SELF_TSYNC) without also enforcing Landlock
restrictions. Rejecting these calls also keeps the option of giving
them a meaning later.
The attribute is only set past the last point of failure, just before
committing the new credentials. When combined with
LANDLOCK_RESTRICT_SELF_TSYNC, no_new_privs is set on the sibling threads
as well, in their commit phase, with the same ordering.
Bump the Landlock ABI version to 11, and include the minimal related
test changes to keep the tests bisectable.
Cc: Mickaël Salaün <mic@digikod.net>
Signed-off-by: Justin Suess <utilityemal77@gmail.com>
Link: https://patch.msgid.link/20260809212459.2427878-1-utilityemal77@gmail.com
Signed-off-by: Mickaël Salaün <mic@digikod.net>
|
|
Whiteout objects are used in the upper layer of an OverlayFS to
indicate that the file with this name does not exist in the unified
view, even if it is present in one of the lower layer file systems.
For the userspace implementations of OverlayFS (fuse-overlayfs),
whiteout objects can be created from userspace as well:
* mknod(2) with S_IFCHR and makedev(0, 0)
* renameat2(2) with RENAME_WHITEOUT,
creating the whiteout in the old place of the moved file.
This commit guards whiteout creation in both of these cases with
LANDLOCK_ACCESS_FS_MAKE_REG. Whiteout objects are *not* considered
character devices and are not bound to a driver.
LANDLOCK_ACCESS_FS_MAKE_REG describes the same permission class as a
whiteout object: creating one is the only S_IFCHR creation that the VFS
exempts from CAP_MKNOD, so it is as unprivileged as creating a regular
file, while LANDLOCK_ACCESS_FS_MAKE_CHAR and
LANDLOCK_ACCESS_FS_MAKE_BLOCK keep meaning the creation of devices that
expose a kernel interface [1].
For the mknod(2) case, introduce a Landlock erratum. The creation of
whiteout objects through mknod(2) was previously guarded using
LANDLOCK_ACCESS_FS_MAKE_CHAR, and it is now guarded using
LANDLOCK_ACCESS_FS_MAKE_REG.
For the renameat2(2) case, fix a bug: Before this commit, renameat2(2)
with RENAME_WHITEOUT would create a directory entry even when all
LANDLOCK_ACCESS_FS_MAKE_* rights were denied.
This does not affect normal renames within layered OverlayFS mounts:
When doing a regular rename() on a mounted fuse-overlayfs, it is the
fuse-overlayfs daemon that exercises renameat2() with RENAME_WHITEOUT,
and only the Landlock domain of that daemon is checked there.
Suggested-by: Christian Brauner <brauner@kernel.org>
Suggested-by: Mickaël Salaün <mic@digikod.net>
Cc: stable@vger.kernel.org
Fixes: cb2c7d1a1776 ("landlock: Support filesystem access-control")
Depends-on: 49c9e09d9610 ("landlock: Fix handling of disconnected directories")
Depends-on: fe72ce6710cb ("landlock: Add errata documentation section")
Signed-off-by: Günther Noack <gnoack@google.com>
Link: https://patch.msgid.link/20260720.chow9ohYie5b@digikod.net [1]
Link: https://patch.msgid.link/20260813093157.1436894-3-gnoack@google.com
[mic: Record why LANDLOCK_ACCESS_FS_MAKE_REG is the matching right, and
add link(2) to the user doc]
Signed-off-by: Mickaël Salaün <mic@digikod.net>
|
|
Prepare input updates for 7.3 merge window.
|
|
Pull drm fixes from Dave Airlie:
"While this is large for rc8 time but also AI driven fixes is a lot of
it, we had a more traditional screw up, and a regression was just
found in the fair scheduling patches that went in back in rc1. This
reverts the fair scheduler back to an option and sets the default back
to what it should have been. We might have been a bit overly zealous
in switching over, but at least it feels more normal than the AI
driven fixes.
Apart from the scheduler, it's mostly amdgpu and xe fixes, with some
misc fixes to the log code and connector code.
scheduler:
- revert fair scheduler patches due to regression
- mark fair as experimental
connector:
- fix OOB read in hdmi audio infoframe
log:
- fix divide by 0 if module param is set to 0
- fix OOB read on empty message
- fix infinite loop for too large scale
xe:
- Fix DPT Allocation paths
- Fixes around UM queue BO
- Order ring writes before ring tail updates
- Add termination on resume for PXP
- Document Sentinel and make CTX_TIMESTAMP read TOCTOU-safe
- Fix sync entry leak on OA config emit failure
- Check managed mutex initilization errors
- Fix min frequency setting
- Fix xe_device_probe error path
amdgpu:
- Bounds checking fix in CS IOCTL
- Bounds checking fix in GEM IOCTL
- Display fixes
- GPUVM fix
- ASPM fix
- UVD bounds checking fixes
- VCE 3 fix
- BT.2020 fixes
- NBIF 6.3.1 fix
- IP discovery fix
radeon:
- Runtime pm fix
amdxdna:
- skip attempting to populate unmapped pages"
* tag 'drm-fixes-2026-08-15' of https://gitlab.freedesktop.org/drm/kernel: (51 commits)
drm/log: Fix infinite loop when scale is too large for display
drm/log: Fix out-of-bounds read on empty message length
drm/log: Fix division by zero when scale module parameter is 0
drm/xe: Fix xe_device_probe() failure
drm/xe: Fix a bug in pc_adjust_freq_bounds()
drm/xe/oa: Check managed mutex initialization errors
drm/xe/oa: Fix sync entry leak on OA config emit failure
drm/xe/lrc: document sentinel and make CTX_TIMESTAMP read TOCTOU-safe
drm/xe/pxp: add termination on resume
drm/xe: Order ring writes before ring tail updates
drm/xe/guc_ads: use uncached mapping for UM queue BO
drm/xe/guc_ads: allocate UM queues in VRAM on dGFX
drm/xe/guc_ads: allocate UM queues in a separate BO
drm/xe: Fix DPT allocation paths.
accel/amdxdna: Skip unmapped range in aie2_populate_range()
drm/amdgpu: Prefer default discovery offset
drm/amdgpu: Reject UVD message with invalid number of h265 refs
drm/amdgpu: fix nbif 6.3.1 l1 low power not functional
drm/amd/display: fix BT.2020 YCbCr output CSC matrices for DCE
drm/amd/display: fix BT.2020 YCbCr limited output CSC matrix
...
|
|
'rcu-tasks.2026.07.30a', 'srcu.2026.08.11a' and 'torture.2026.08.14a' into HEAD
Changes:
Make expedited grace periods expedite normal RCU callbacks
Miscellaneous fixes:
* Improve diagnostic output with character task states.
* Mark accesses to inform KCSAN of concurrency design.
* Move from kmalloc() to kmalloc_obj().
* Documentation updates.
* Improve handling of RCU deferred quiescent states.
* Clean up unused function arguments and structure fields.
* Reduce show_rcu_gp_kthreads() stack space.
Tasks RCU updates:
* Clean up after SRCU re-implementation of Tasks Trace RCU.
* Mark accesses to inform KCSAN of concurrency design.
* Add ->lazy_timer status to diagnostic output.
* Remove an unnecessary memory barrier.
* Fix a data race, courtesy of KCSAN.
* Documentation updates.
* Convert cond_resched_tasks_rcu_qs() from macro to static inline
function.
SRCU updates:
* Add Rust helpers for SRCU.
* Avoid losing queued work at cleanup_srcu_struct() time.
Torture-test updates:
* Preparation work for immediate RCU priority deboosting.
* Test RCU readers from real interrupt handlers (as opposed to softirq).
* Simplify code through use of cpumask_next_wrap().
* Improve diagnostic output with character task states.
* Add rcutorture.nwriters parameter to allow lightweight stall testing,
and rcutorture.stall_only to make doing so easier.
* Test an RCU Tasks Trace grace period implying an RCU grace period.
* Make RCU Tasks Trace torturing track reader batches.
* Fix a data race, courtesy of KCSAN.
* Plug a shuffle_tmp_mask memory leak on kthread spawn failure.
|
|
An mmap-able BPF array map (BPF_F_MMAPABLE) has its backing memory
vmalloc'ed up front at map creation time. array_map_mmap() then wired up
the whole mapping eagerly via remap_vmalloc_range(), which calls
vm_insert_page() for every page of the map. For large maps this makes
every mmap() O(number of pages): an 8MiB map inserts 2048 PTEs per
mmap() and tears them all down again on munmap(), even when user space
only touches a few pages (or none at all).
Populate the mapping lazily instead, the same way the arena map already
does. array_map_mmap() now only performs the bounds check and returns,
leaving the PTEs unpopulated; pages are inserted on demand by a new
array_map_mmap_fault() handler. Because the memory is already resident,
the fault handler simply resolves the vmalloc page and hands it to the
fault path. This makes mmap() O(1), and munmap() proportional to the
number of pages that were actually faulted in rather than to the size of
the map.
The handler is reached through a new optional ->map_mmap_fault callback.
Maps that provide it get a vm_operations_struct with a .fault handler;
maps that populate their mapping eagerly keep the one they had. Both
share the same open/close callbacks, so the existing VMA accounting
(VM_MAYWRITE write-active tracking, freeze handling) stays centralized
rather than each map installing its own vm_operations_struct.
Callers that want the pages populated up front can still request that
explicitly with MAP_POPULATE. Kernel-side access to the map (via the
vmalloc address) is unaffected.
Time for one mmap()+munmap() of an 8MiB mmap-able array map:
before after
no MAP_POPULATE, no access 226us 1.1us
no MAP_POPULATE, access all pages 236us 1341us
MAP_POPULATE, no access 312us 493us
MAP_POPULATE, access all pages 318us 519us
Mapping without touching the data, which is what this change targets,
gets ~160x cheaper. Faulting in the whole mapping one page at a time is
more expensive than the eager remap_vmalloc_range() loop, so users that
do touch every page should ask for MAP_POPULATE. Note that MAP_POPULATE
is not free before this change either: it adds ~85us (226us => 312us)
for no benefit, as the mapping is already fully populated.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Song Liu <song@kernel.org>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260814155623.111565-1-song@kernel.org
|
|
This commit adds the ->get_sp_seq and ->gp_diff fields to the
tasks_tracing_ops structure so that RCU Tasks Trace rcutorture runs will
track Reader Batch.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
The function used for obtaining the first "physical" device for which
the given ACPI one is the ACPI companion, acpi_get_first_physical_node(),
may return a stale device pointer (mostly in theory) because
acpi_unbind_one() may run as a whole after dropping the ACPI device's
physical_node_lock in acpi_get_first_physical_node() and before it
returns. The last reference to the "physical" device may be dropped
then before the pointer to it is returned to the caller.
If that happens and the acpi_get_first_physical_node() caller invokes
get_device() on the pointer obtained from it, which is done by the
majority of its callers, a use-after-free will occur.
To prepare for addressing this problem, introduce a new function for
getting the first "physical" device associated with the given ACPI one
(the "primary physical device") that will also reference count the
device in question before returning a pointer to it.
Make that new function and acpi_get_first_physical_node() share the
physical node list lookup code.
No intentional functional impact.
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Link: https://patch.msgid.link/2843318.mvXUDI8C0e@rafael.j.wysocki
|
