kernel/git/next/linux-next.git/io_uring/rw.c, branch master

Merge branch 'non-rcu/next' of https://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git

2026-09-14T14:21:59+00:00

Merge branch 'io_uring-exit-cancel.7' into for-next

2026-09-11T22:25:44+00:00

* io_uring-exit-cancel.7: io_uring: wait for in-flight requests on ring release io_uring: drop registered files and buffers at release time io_uring: run cancelations synchronously on ring release io_uring/cancel: cancel and wait for all requests on process exit io_uring/notif: count pending zerocopy notifications per ring io_uring/uring_cmd: only cancel requests of the given task io_uring: put request files before posting the completions io_uring/rw: don't reap io-wq IOPOLL completions while io-wq has a reference io_uring: post io-wq completions from the last request reference io_uring/io-wq: put the request file before posting a completion

io_uring/rw: don't reap io-wq IOPOLL completions while io-wq has a reference

2026-09-11T22:24:53+00:00

IOPOLL requests issued from io-wq are reaped by io_do_iopoll() on the submitter, with the same window as the task_work completions: the CQE is posted while the worker may still hold its reference, and the file is only dropped once it does. Leave such a request on the iopoll list until the worker has dropped its reference, the next pass then reaps it and the put from io_free_batch_list() is the last one. Also move the reference handling for the task_work case into a helper. Signed-off-by: Jens Axboe

io_uring/rw: keep CQE flags on iopoll requests when adding kbuf flags

2026-09-09T15:59:02+00:00

io_do_iopoll() assigns the result of io_put_kbuf() to the request's CQE flags upon completion. This overwrites any CQE flags that may have been set by the opcode-specific layer. (For example, if __io_uring_cmd_done() had set IORING_CQE_F_32, it would be cleared.) Switch the = to an |= so the kbuf flags are added to the existing CQE flags rather than replacing them. io_req_rw_complete() does the same with the io_put_kbuf() result. Fixes: e26dca67fde1 ("io_uring: add support for IORING_SETUP_CQE_MIXED") Reported-by: sashiko-bot@kernel.org Link: https://sashiko.dev/#/message/20260827191705.D53C91F000E9%40smtp.kernel.org Signed-off-by: Caleb Sander Mateos Reviewed-by: Anuj Gupta Link: https://patch.msgid.link/20260902210200.2336720-1-csander@purestorage.com Signed-off-by: Jens Axboe

io_uring/rw: end write accounting from ->ki_complete

2026-09-09T01:03:26+00:00

Commit b000145e9907 moved both the fsnotify calls and the write accounting out of the kiocb completion handler and into the io_req_rw_complete() task_work. However, only the fsnotify part actually needed to move as it may sleep. Ending the write accounting is just a percpu_up_read() on the superblock writers sem. Deferring it is a problem, because it makes dropping SB_FREEZE_WRITE protection depend on the ring owner getting to running task_work. But the task may be blocked in freeze_super(), causing it to never get to that: task io-wq worker -------------------------------------------------------------- io_write() io_kiocb_start_write() (takes sb_writers, hidden from lockdep by __sb_writers_release) write_iter() -> -EIOCBQUEUED ioctl(FS_IOC_SHUTDOWN) bdev_freeze() freeze_super() percpu_down_write() <- waits for the reader above io_write() kiocb_start_write() percpu_down_read() <- queued behind the writer io_complete_rw() queues io_req_rw_complete() <- never runs, task is in D state End the write from io_complete_rw() instead, and leave only the fsnotify calls in task_work. Reported-by: syzbot+2eb3d983669d3e49d4fa@syzkaller.appspotmail.com Cc: stable@vger.kernel.org Fixes: b000145e9907 ("io_uring/rw: defer fsnotify calls to task context") Signed-off-by: Jens Axboe

io-uring/rw: Use accessor for hrtimer_sleeper ->task field

2026-09-03T17:56:08+00:00

The hrtimer_sleeper structure's ->task field is used as a flag to indicate that the associated hrtimer has expired. This means that the hrtimer handler can be storing to this field while other code is loading from it to check for expiry. Note that additional races appear for hrtimers that can be restarted, which could be argued to be a user error. However, that is no reason to let the compiler introduce additional confusion, and to this end, the hrtimer_sleeper_task_get() was introduced, use of which also has the benefit of avoiding open-code access to hrtimer_sleeper innards. Therefore, apply this accessor to the io_hybrid_iopoll_delay() function. KCSAN located this issue. Signed-off-by: Paul E. McKenney Cc: Jens Axboe Cc: Anna-Maria Behnsen Cc: Frederic Weisbecker Cc: Thomas Gleixner Cc: Cc: Cc: Cc:

Merge tag 'for-7.3/io_uring-20260819' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux

2026-08-20T20:47:56+00:00

Pull io_uring update from Jens Axboe: "On top of the usual cleanups and fixes, the bigger items in here are: - zcrx work, most of it centered around adding dynamic area provisioning, plus a bunch of prep and cleanups leading up to it: scale refilling with large pages, coalesce same-niov RQEs on refill, separate the RQ head/tail cache lines and cache the RQ tail, and rework the area creation locking. - Fix the futex inflight accounting so that only private futex waits are marked inflight, and don't mark wake requests as inflight at all. - Drop the custom iov copy in the buffer select prep and msg header copy paths, using the generic helpers instead. - Fix a folio size overflow in io_vec_fill_bvec(), and account the pages a compound region really uses in the memmap path. - Fix an iovec leak in uring_cmd when the async cmd isn't recycled, skip the blocking task work for io_uring_cmd_issue_blocking(), and don't skip completion for a synchronous multishot cmd - Defer eventfd signaling when queued from a wakeup handler - Fix io-wq worker accounting when canceling creation callbacks - Annotate remote tasks for kcoverage" * tag 'for-7.3/io_uring-20260819' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux: (30 commits) io_uring: Add missing include for ITER_SOURCE and ITER_DEST io_uring/uring_cmd: don't skip completion for a synchronous multishot cmd io_uring/memmap: account the pages a compound region really uses io_uring/zcrx: add dynamic area provisioning io_uring/zcrx: lock area creation with pp_lock io_uring/zcrx: keep array of areas io_uring/zcrx: move freelist lock to struct zcrx io_uring/zcrx: unmap under netdev lock io_uring/zcrx: split dmabuf unmap and release io_uring/zcrx: don't pass ifq_reg to area creation io_uring/zcrx: add helper for deriving area token io_uring/zcrx: don't reload skb_shinfo io_urint/zcrx: narrow var scope in io_zcrx_recv_skb() io_uring/zcrx: constify area_reg on import io_uring/zcrx: coalesce same-niov RQEs on refill io_uring/zcrx: cache RQ tail io_uring/zcrx: add RQ iterator io_uring/zcrx: move RQ head/tail to separate cache lines io_uring/zcrx: scale refilling with large pages io_uring/io-wq: fix worker accounting when canceling creation callbacks ...

io_uring/rw: Drop custom iov copy in io_iov_buffer_select_prep

2026-08-15T23:25:33+00:00

Similar to commit f4eaf8eda89e ("io_uring/rsrc: Drop io_copy_iov in favor of iovec API"), avoid custom copy and just rely on the iovec api. Signed-off-by: Gabriel Krisman Bertazi Link: https://patch.msgid.link/20260813004022.3514537-2-krisman@suse.de Signed-off-by: Jens Axboe

io_uring/rw: fix missing ERESTARTSYS conversion in read paths

2026-07-22T13:36:25+00:00

Both read and write may receive internal restart error codes from the filesystem layer and should be converted to -EINTR. However, when multishot read support was added, the error code normalization was lost for both io_read() and io_read_mshot(). Extract the conversion into io_fixup_restart_res() and apply it in all three locations: io_rw_done(), io_read(), and io_read_mshot(). Fixes: a08d195b586a ("io_uring/rw: split io_read() into a helper") Cc: stable@vger.kernel.org Signed-off-by: Yitang Yang Link: https://patch.msgid.link/20260722124551.130563-1-yi1tang.yang@gmail.com Signed-off-by: Jens Axboe

io_uring/rw: preserve partial result for iopoll

2026-06-16T12:20:01+00:00

A partial read will store the completed byte count in io->bytes_done. The regular completion path applies io_fixup_rw_res() so that, when the following operation reaches EOF, the number of bytes already read is returned. The iopoll completion path does not apply this fixup to the return value and can return zero instead. Use the fixup result when updating the CQE, and the raw result for the reissue check. Cc: stable@vger.kernel.org Fixes: 4d9cb92ca41d ("io_uring/rw: fix short rw error handling") Signed-off-by: Michael Wigham Link: https://patch.msgid.link/20260613225240.34032-1-michael@wigham.net Signed-off-by: Jens Axboe