kernel/git/stable/linux.git/fs/fuse/file.c, branch master

treewide: refresh kmalloc_obj() conversions

2026-09-05T04:37:00+00:00

This is another run of the Coccinelle script for converting kmalloc() family of allocations to kmalloc_obj() via the existing rules in scripts/coccinelle/api/kmalloc_objs.cocci This catches both the set of kmalloc() uses added since the first kmalloc_obj() conversions in v7.0 and adds a large group missed in the first pass due to Coccinelle not interacting well with the cleanup.h scoped_...() family of macros[1]. I worked around this with spatch's "--macro-file" argument to a file with all the scoped_...() macros mapped to Coccinelle's YACFE_ITERATOR[2] as that was the closest viable control flow indicator I could find. Build tested allmodconfig on x86, arm64, arm, loongarch, mips, powerpc, riscv, and s390 with no new warnings. Link: https://lore.kernel.org/lkml/202609021314.8A9C0B8@keescook/ [1] Link: https://github.com/coccinelle/coccinelle/blob/master/standard.h [2] Signed-off-by: Kees Cook

Merge tag 'fuse-update-7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse

2026-08-25T14:59:44+00:00

Pull fuse updates from Miklos Szeredi: - Improve performance of the io-uring transport by introducing buffer pools and zero-copy (Joanne) - Fix lots of bugs (Baokun Li) - Fix io-uring initialization issues (Joanne, Bernd) - More prep work for large folios (Joanne) - Don't limit buffered read to 128k (Jim Harris) - Fix zeroing of page end (dirtied with mmap) on file size extension (Jimmy Zuber) - Improve performance in certain cases with wake_up_sync() when queuing request (Xuewen Yan) - Misc fixes and cleanups (Xuewen Yan) * tag 'fuse-update-7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse: (35 commits) fuse: zero the partial EOF page when extending a file io_uring: Add missing include for ITER_SOURCE and ITER_DEST fuse: Fix the condition to enable over-io-uring fuse: invalidate the correct range after O_APPEND direct write selftests/fuse: test post-EOF page zeroing when a file is extended fuse: wake one waiter per freed slot when raising max_background fuse: use min_not_zero() in fuse_init_server_timeout() fuse: copy request headers via a stack buffer for io-uring fuse: give wakeup hints to the scheduler for synchronous requests fuse: check for NULL root inode in fuse_fill_super_submount fuse: reject a duplicate fd= mount option cuse: wait for pending RCU callbacks on module exit fuse: fix invalidate lock leak on open O_TRUNC DAX failure fuse: fix invalidate lock leak on setattr writeback failure fuse: wait for FR_FINISHED on abort_on_kill to prevent use-after-free fuse: make dentry_tree_work static docs: fuse: document io-uring buffer pool and zero-copy uapi fuse: add zero-copy over io-uring fuse: support registered buffer pools in io-uring fuse: add io-uring buffer pools ...

fuse: zero the partial EOF page when extending a file

2026-08-24T14:54:04+00:00

Extending a fuse file past a non-page-aligned EOF does not zero the tail of the old last page. When that page is cached and has been mmap-dirtied beyond the old EOF, the now in-bounds tail is served to later reads as stale data rather than zeros, which violates POSIX file-extension semantics. Some file systems get this zeroing automatically at writeback time (block_write_full_folio() / iomap_writeback_handle_eof() zero the tail of the folio straddling i_size). A non-writeback caching fuse file system uses neither path, so it has to zero the tail itself from the size-extending paths, like XFS (xfs_file_write_zero_eof()) and ext4 (ext4_block_zero_eof()) do. Call truncate_pagecache_range() over the newly-exposed range up front from the three paths that extend a file, before the new size is published: - a buffered write whose position is past the old EOF (fuse_perform_write()); - a size-extending setattr/truncate (fuse_do_setattr()); - a size-extending fallocate (fuse_file_fallocate()). This unmaps the stale mappings and zeroes the partial tail of the old EOF folio, so a later read returns zeros. Truncating [old EOF, write start) before a buffered write keeps the dropped range disjoint from the written data, so a write that lands inside the old EOF folio is preserved. writeback_cache connections are unaffected, as their writes go through iomap_file_buffered_write(), which zeroes post-EOF folios. The bug is observable on a non-writeback_cache server that returns FOPEN_KEEP_CACHE on writable files (without FOPEN_DIRECT_IO), and is caught by the new write_extend_eof fuse selftest. Signed-off-by: Jimmy Zuber Signed-off-by: Miklos Szeredi

fuse: invalidate the correct range after O_APPEND direct write

2026-08-24T10:18:02+00:00

fuse_direct_write_iter() captures pos before generic_write_checks(), which moves ki_pos to EOF for O_APPEND writes: fuse_direct_write_iter() { pos = iocb->ki_pos; /* 0 (user-supplied) */ generic_write_checks(); /* ki_pos -> EOF */ fuse_direct_io(); /* writes at EOF, correct */ invalidate(pos, pos + res); /* [0, res) -- wrong */ } The post-write invalidation targets a stale range instead of the actual written range at EOF. This can cause data inconsistency when the file size is not page-aligned. The tail page straddling EOF has a valid portion before EOF that concurrent readers can fault back in during the DIO write window: Tail page (file size X not page-aligned): page_start X (EOF) page_end |--- valid data ----|-- stale --| CPU0 (O_APPEND DIO writer) CPU1 (buffered reader) -------------------------- ---------------------- invalidate [X, X+len) tail page evicted FUSE_WRITE in flight ... read [page_start, X) tail page re-faulted [X, page_end) = stale FUSE_WRITE completes i_size = X + len invalidate [0, len) <- WRONG tail page still cached read [X, X+len) hits stale tail page returns old data Fix by reading pos back from iocb->ki_pos after generic_write_checks(), as generic_file_direct_write() does. Also fix a typo in the comment ("may have" -> "may have competed"). Fixes: 2b0408d0284f ("fuse: invalidate page cache after DIO and async DIO writes") Signed-off-by: Baokun Li Reviewed-by: Bernd Schubert Signed-off-by: Miklos Szeredi

fuse: fix invalidate lock leak on open O_TRUNC DAX failure

2026-08-18T08:26:12+00:00

fuse_open() takes filemap_invalidate_lock() for a DAX truncate (dax_truncate = true) and releases it before the out_inode_unlock label. But when fuse_dax_break_layouts() fails, the goto out_inode_unlock skips the unlock and leaks the rwsem, so any later fault or truncate on the file stalls on the stale lock. fuse_dax_break_layouts() can fail with -ERESTARTSYS when a signal interrupts the wait for busy DAX pages to drain: open("file", O_RDWR | O_TRUNC) └─ fuse_open() ├─ filemap_invalidate_lock() # dax_truncate └─ fuse_dax_break_layouts() └─ dax_break_layout() └─ wait_page_idle() # TASK_INTERRUPTIBLE └─ fuse_wait_dax_page() # unlock, schedule, re-lock └─ signal → -ERESTARTSYS goto out_inode_unlock # <- lock leaked Fix this by moving filemap_invalidate_unlock() below the label so that all error paths release the lock, and rename the label to out_unlock as it now covers more than just the inode lock. Fixes: 2fdbb8dd0155 ("fuse: fix deadlock between atomic O_TRUNC and page invalidation") Cc: stable@vger.kernel.org # v6.0+ Signed-off-by: Baokun Li Signed-off-by: Miklos Szeredi

Merge tag 'vfs-7.3-rc1.iomap' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs

2026-08-17T17:03:55+00:00

Pull iomap updates from Christian Brauner: "The bulk of this is the conversion of iomap to a single ->iomap_next() callback and thus finishing the move to an iterator model. Every iomap operation drove its iteration through a struct iomap_ops holding ->iomap_begin() and ->iomap_end(). iomap_iter() only ever sees those as pointers. That means every step of every iteration is an indirect call. This collapses both into one ->iomap_next() callback that finishes the previous mapping and produces the next one. This lets callers inline the iteration loop and pass its ->iomap_next() as a compile time constant. That means the compiler can turn it into a direct and hence inlineable call. This also allows future callers to express custom logic to drive the iteration forward better. xfs, btrfs, ext4, ext2, erofs, f2fs, gfs2, hpfs, fuse, exfat, zonefs, ntfs, ntfs3 and the block device mapping are all converted. No functional changes are intended. This also adds a simple direct I/O path for small reads. On Gen5 NVMe the __iomap_dio_rw() dominates 4K random reads. The same single-core io_uring poll mode workload reaches ~3.2M IOPS against the raw block device but only ~1.92M through ext4 or XFS. __iomap_dio_rw(), iomap_iter(), iomap_dio_bio_iter() and kfree() were at the top of the profile. The new path is very lightweight if no special behavior is requested. The bio comes from a dedicated bioset and laid out so the whole request is a single cacheline aligned allocation. Completion runs inline. That takes ext4 from 1.92M to 2.19M IOPS in the original workload. fio shows around: - 4% at libaio queue depths of 64 and up - around 5% for io_uring - up to 10% for io_uring poll mode at depth 256 on both ext4 and xfs. A few other patches: - iomap_folio_mark_uptodate() lets a filesystem that writes into the page cache outside the iomap read and write paths keep iomap's internal uptodate bitmap in sync, which fuse needs for server-pushed notify stores before it can enable large folios; - two fixes for iomap_bio_read_folio_range_sync(): a potential crash when device integrity behavior is changed and a missing bio_uninit(). - a folio batch release fix on iomap callback failures - FGP_NOFS is dropped from iomap_get_folio() - documentation fix" * tag 'vfs-7.3-rc1.iomap' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs: (29 commits) iomap: iomap_bio_read_folio_range_sync is missing a call to bio_uninit iomap: don't free integrity payload that doesn't exist docs: fix grammatical error in iomap docs exfat: convert iomap ops to ->iomap_next() fuse: convert iomap ops to ->iomap_next() hpfs: convert iomap ops to ->iomap_next() gfs2: convert iomap ops to ->iomap_next() f2fs: convert iomap ops to ->iomap_next() block: convert iomap ops to ->iomap_next() ext2: convert iomap ops to ->iomap_next() zonefs: convert iomap ops to ->iomap_next() erofs: convert iomap ops to ->iomap_next() ext4: convert iomap ops to ->iomap_next() ntfs: convert iomap ops to ->iomap_next() ntfs3: convert iomap ops to ->iomap_next() btrfs: convert iomap ops to ->iomap_next() xfs: convert iomap ops to ->iomap_next() iomap: add ->iomap_next() iomap: use GFP_NOWAIT when application for iomap_dio_simple allocations iomap: decouple simple direct I/O reads from iomap_dio_rw ...

fuse: add zero-copy over io-uring

2026-08-17T15:02:37+00:00

Implement zero-copy in fuse io-uring to eliminate memory copies between the application, kernel, and server for read/write operations. The server can directly access client pages or page cache folios without copying data through an intermediary buffer. When a fuse request arrives, the kernel registers the relevant pages into a sparse slot in the server's io_uring registered buffer table. The server can then operate on these pages directly using io-uring fixed buffer operations (eg read_fixed/write_fixed) and the kernel unregisters these pages when the request completes. Non-page-backed args (eg op out headers) will go through the payload buffer as normal. The server can specify which open files should have their reads/writes go through zero-copy, by setting the FOPEN_IO_URING_ZERO_COPY flag when servicing opens. This requires CAP_SYS_ADMIN and bufpools. This is gated behind CAP_SYS_ADMIN because zero-copy allows the server direct access to the client's underlying pages, rather than operating on an intermediary buffer that the contents of the client's pages were copied into or on page cache folios. The request flow for the zero-copy direct-io write path (client writes data, server reads it) is as follows: ======================================================================= | Kernel | FUSE server | | | "write(fd, buf, 1MB)" | | | | >sys_write() | | >fuse_file_write_iter() | | >fuse_send_one() | | [req->args->in_pages = true] | | [folios hold client write data] | | | | >fuse_uring_copy_to_ring() | | >copy_header_to_ring(IN_OUT) | | [memcpy fuse_in_header] | | >copy_header_to_ring(OP) | | [memcpy write_in header] | | | | >fuse_uring_args_to_ring() | | >setup_fuse_copy_state() | | [skip_folio_copy = true] | | | | >fuse_uring_set_up_zero_copy() | | [folio_get for each client folio] | | [build bio_vec array from folios] | | >io_buffer_register_bvec() | | [register pages at ent->zero_copy_index] | | [ent->zero_copied = true] | | | | >fuse_copy_args() | | [skip_folio_copy => return 0 | | for page arg, skip data copy] | | | | >copy_header_to_ring(RING_ENT) | | [memcpy ent_in_out] | | >io_uring_cmd_done() | | | | | [CQE received] | | | | [issue io_uring READ at | | ent->zero_copy_index] | | [reads directly from | |client's pages (ZERO_COPY)] | | | | [write data to backing | | store] | | [submit COMMIT AND FETCH] | | | >fuse_uring_commit_fetch() | | >fuse_uring_commit() | | >fuse_uring_copy_from_ring() | | >fuse_uring_req_end() | | >io_buffer_unregister(ent->zero_copy_index) | | [unregister pages from index] | | >fuse_zero_copy_release() | | [folio_put for each folio] | | [ent->zero_copied = false] | | >fuse_request_end() | | [wake up client] | The zero-copy read path is analogous. Some requests may have both page-backed args and non-page-backed args. For these requests, the page-backed args are zero-copied while the non-page-backed args are copied to the buffer selected from the buffer pool: zero-copy: pages registered via io_buffer_register_bvec() non-page-backed: copied to payload buffer via fuse_copy_args() For a request whose payload is zero-copied, the registration/unregistration path looks like: register: fuse_uring_set_up_zero_copy() folio_get() for each folio io_buffer_register_bvec(ent->zero_copy_index) unregister: fuse_uring_req_end() io_buffer_unregister(ent->zero_copy_index) -> fuse_zero_copy_release() callback folio_put() for each folio Please note that on abort for in-flight zero-copied requests that have been sent to userspace, the registered bvec slot remains occupied and its folios remain pinned until the io-uring ring is destroyed, at which point io-uring unregisters all buffers and the fuse_zero_copy_release() callback drops the folio references. Unregistering at teardown would require operating on the ring context directly, whose validity is hard to ascertain; this is deemed not worth the complexity for the abort race, since everything is freed when the ring is torn down. The throughput improvement from zero-copy depends on how much of the per-request latency is spent on data copying vs backing I/O. The gain comes from eliminating the payload-buffer memcpy, but accessing the zero-copied pages requires the server to issue the read/write as an IORING_OP_READ/WRITE_FIXED operation. The benefit is largest when the mempcy is a meaningful fraction of per-request latency while backing i/o is still noticable enough that the extra io-uring op's overhead doesn't dominate. Benchmarked with passthrough_hp (--nopassthrough, q_depth=8) on a 2-socket Intel Xeon Gold 6138 (40 cores / 80 threads), using fio (sync engine, bs=1M, O_DIRECT, numjobs=2, 30s run + 10s ramp, 3 runs) where direct-I/O throughput is against a RAM-backed (tmpfs) source (backing I/O is not the bottleneck): baseline registered-buf zero-copy (zc vs base) direct read ~5.1 GB/s ~5.4 GB/s ~8.9 GB/s (+75%) direct write ~3.4 GB/s ~4.8 GB/s ~5.1 GB/s (+50%) Reads end up higher than writes because the backing store reads faster than it writes (the baseline shows the same read>write gap, and the raw device does too). On a device-bound NVMe (~2 GB/s reads) the read gain shrinks to ~10-16% (and no measurable gains for writes), as backing I/O rather than the eliminated copy dominates latency. The benefit overall scales with how much of the per-request latency is the data copy versus backing I/O. Signed-off-by: Joanne Koong Reviewed-by: Bernd Schubert Signed-off-by: Miklos Szeredi

fuse: convert iomap ops to ->iomap_next()

2026-07-31T10:28:51+00:00

Convert fuse iomap_ops to the new ->iomap_next() callback. Each callback is generated with the DEFINE_IOMAP_ITER_NEXT()/DEFINE_IOMAP_ITER_NEXT_END() macros, which wrap the iomap_iter_next() helper to finish the previous mapping if needed and produce the next one. No functional changes are intended. Signed-off-by: Joanne Koong Link: https://patch.msgid.link/20260729192737.3190206-19-joannelkoong@gmail.com Signed-off-by: Christian Brauner (Amutable)

fuse: use iomap helper to mark folio uptodate

2026-07-31T10:28:47+00:00

When fuse enables large folios, a large folio will be backed by iomap_folio_state that keeps track of uptodate and dirty state in an internal bitmap. Fuse writethrough and notify store paths currently set folio uptodate state with folio_mark_uptodate(), which touches only the folio-level flag, but on an iomap-backed folio, that leaves the uptodate bitmap out of sync. Use the iomap_folio_mark_uptodate() helper to update both the folio uptodate state and the iomap uptodate bitmap. Reviewed-by: Darrick J. Wong Signed-off-by: Joanne Koong Link: https://patch.msgid.link/20260707220450.1200943-4-joannelkoong@gmail.com Acked-by: Miklos Szeredi Reviewed-by: Christoph Hellwig Signed-off-by: Christian Brauner (Amutable)

fuse: don't clear folio uptodate on writethrough errors

2026-07-31T10:28:46+00:00

In the writethrough path (fuse_send_write_pages()), if the write to the server failed or was a short write, the uptodate flag on the folios are cleared. As explained by Matthew in [1], this is dangerous because the folio may be mapped into userspace. The mm code has the invariant that a non-uptodate folio must never be visible to userspace (to avoid potentially leaking confidental information to userspace) and has checks in place for this that if violated can bring down the whole machine. Practically speaking, the effect of this change for the fuse writethrough error path is that if an application does a write and then the server fails to persist the data or only services a short write, the page cache folio keeps the data the application wrote instead of being reverted to the server's contents on the next read. The failure is still reported to the application synchronously through the short count / error return of the write() syscall. Folios that were only partially written are unaffected since they were never marked uptodate in the first place (fuse_fill_write_page() only marks a folio as uptodate if the whole folio was written to). [1] https://lore.kernel.org/linux-fsdevel/ajtPMgO65FA1TXhi@casper.infradead.org/ Suggested-by: Matthew Wilcox Reviewed-by: Darrick J. Wong Signed-off-by: Joanne Koong Link: https://patch.msgid.link/20260707220450.1200943-2-joannelkoong@gmail.com Acked-by: Miklos Szeredi Reviewed-by: Christoph Hellwig Signed-off-by: Christian Brauner (Amutable)