kernel/git/stable/linux-stable.git/io_uring/rw.c, branch master

io_uring/rw: keep CQE flags on iopoll requests when adding kbuf flags

2026-09-09T15:59:02+00:00

io_do_iopoll() assigns the result of io_put_kbuf() to the request's CQE flags upon completion. This overwrites any CQE flags that may have been set by the opcode-specific layer. (For example, if __io_uring_cmd_done() had set IORING_CQE_F_32, it would be cleared.) Switch the = to an |= so the kbuf flags are added to the existing CQE flags rather than replacing them. io_req_rw_complete() does the same with the io_put_kbuf() result. Fixes: e26dca67fde1 ("io_uring: add support for IORING_SETUP_CQE_MIXED") Reported-by: sashiko-bot@kernel.org Link: https://sashiko.dev/#/message/20260827191705.D53C91F000E9%40smtp.kernel.org Signed-off-by: Caleb Sander Mateos Reviewed-by: Anuj Gupta Link: https://patch.msgid.link/20260902210200.2336720-1-csander@purestorage.com Signed-off-by: Jens Axboe

io_uring/rw: end write accounting from ->ki_complete

2026-09-09T01:03:26+00:00

Commit b000145e9907 moved both the fsnotify calls and the write accounting out of the kiocb completion handler and into the io_req_rw_complete() task_work. However, only the fsnotify part actually needed to move as it may sleep. Ending the write accounting is just a percpu_up_read() on the superblock writers sem. Deferring it is a problem, because it makes dropping SB_FREEZE_WRITE protection depend on the ring owner getting to running task_work. But the task may be blocked in freeze_super(), causing it to never get to that: task io-wq worker -------------------------------------------------------------- io_write() io_kiocb_start_write() (takes sb_writers, hidden from lockdep by __sb_writers_release) write_iter() -> -EIOCBQUEUED ioctl(FS_IOC_SHUTDOWN) bdev_freeze() freeze_super() percpu_down_write() <- waits for the reader above io_write() kiocb_start_write() percpu_down_read() <- queued behind the writer io_complete_rw() queues io_req_rw_complete() <- never runs, task is in D state End the write from io_complete_rw() instead, and leave only the fsnotify calls in task_work. Reported-by: syzbot+2eb3d983669d3e49d4fa@syzkaller.appspotmail.com Cc: stable@vger.kernel.org Fixes: b000145e9907 ("io_uring/rw: defer fsnotify calls to task context") Signed-off-by: Jens Axboe

Merge tag 'for-7.3/io_uring-20260819' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux

2026-08-20T20:47:56+00:00

Pull io_uring update from Jens Axboe: "On top of the usual cleanups and fixes, the bigger items in here are: - zcrx work, most of it centered around adding dynamic area provisioning, plus a bunch of prep and cleanups leading up to it: scale refilling with large pages, coalesce same-niov RQEs on refill, separate the RQ head/tail cache lines and cache the RQ tail, and rework the area creation locking. - Fix the futex inflight accounting so that only private futex waits are marked inflight, and don't mark wake requests as inflight at all. - Drop the custom iov copy in the buffer select prep and msg header copy paths, using the generic helpers instead. - Fix a folio size overflow in io_vec_fill_bvec(), and account the pages a compound region really uses in the memmap path. - Fix an iovec leak in uring_cmd when the async cmd isn't recycled, skip the blocking task work for io_uring_cmd_issue_blocking(), and don't skip completion for a synchronous multishot cmd - Defer eventfd signaling when queued from a wakeup handler - Fix io-wq worker accounting when canceling creation callbacks - Annotate remote tasks for kcoverage" * tag 'for-7.3/io_uring-20260819' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux: (30 commits) io_uring: Add missing include for ITER_SOURCE and ITER_DEST io_uring/uring_cmd: don't skip completion for a synchronous multishot cmd io_uring/memmap: account the pages a compound region really uses io_uring/zcrx: add dynamic area provisioning io_uring/zcrx: lock area creation with pp_lock io_uring/zcrx: keep array of areas io_uring/zcrx: move freelist lock to struct zcrx io_uring/zcrx: unmap under netdev lock io_uring/zcrx: split dmabuf unmap and release io_uring/zcrx: don't pass ifq_reg to area creation io_uring/zcrx: add helper for deriving area token io_uring/zcrx: don't reload skb_shinfo io_urint/zcrx: narrow var scope in io_zcrx_recv_skb() io_uring/zcrx: constify area_reg on import io_uring/zcrx: coalesce same-niov RQEs on refill io_uring/zcrx: cache RQ tail io_uring/zcrx: add RQ iterator io_uring/zcrx: move RQ head/tail to separate cache lines io_uring/zcrx: scale refilling with large pages io_uring/io-wq: fix worker accounting when canceling creation callbacks ...

io_uring/rw: Drop custom iov copy in io_iov_buffer_select_prep

2026-08-15T23:25:33+00:00

Similar to commit f4eaf8eda89e ("io_uring/rsrc: Drop io_copy_iov in favor of iovec API"), avoid custom copy and just rely on the iovec api. Signed-off-by: Gabriel Krisman Bertazi Link: https://patch.msgid.link/20260813004022.3514537-2-krisman@suse.de Signed-off-by: Jens Axboe

io_uring/rw: fix missing ERESTARTSYS conversion in read paths

2026-07-22T13:36:25+00:00

Both read and write may receive internal restart error codes from the filesystem layer and should be converted to -EINTR. However, when multishot read support was added, the error code normalization was lost for both io_read() and io_read_mshot(). Extract the conversion into io_fixup_restart_res() and apply it in all three locations: io_rw_done(), io_read(), and io_read_mshot(). Fixes: a08d195b586a ("io_uring/rw: split io_read() into a helper") Cc: stable@vger.kernel.org Signed-off-by: Yitang Yang Link: https://patch.msgid.link/20260722124551.130563-1-yi1tang.yang@gmail.com Signed-off-by: Jens Axboe

io_uring/rw: preserve partial result for iopoll

2026-06-16T12:20:01+00:00

A partial read will store the completed byte count in io->bytes_done. The regular completion path applies io_fixup_rw_res() so that, when the following operation reaches EOF, the number of bytes already read is returned. The iopoll completion path does not apply this fixup to the return value and can return zero instead. Use the fixup result when updating the CQE, and the raw result for the reissue check. Cc: stable@vger.kernel.org Fixes: 4d9cb92ca41d ("io_uring/rw: fix short rw error handling") Signed-off-by: Michael Wigham Link: https://patch.msgid.link/20260613225240.34032-1-michael@wigham.net Signed-off-by: Jens Axboe

io_uring/rw: drop unused attr_type_mask from io_prep_rw_pi()

2026-05-13T12:35:57+00:00

io_prep_rw_pi() never used the attr_type_mask argument. Callers already validate sqe->attr_type_mask before invoking the helper (only IORING_RW_ATTR_FLAG_PI is supported today). Remove the dead parameter to avoid implying further interpretation happens here. Signed-off-by: Yang Xiuwei Link: https://patch.msgid.link/20260513094303.866533-1-yangxiuwei@kylinos.cn Signed-off-by: Jens Axboe

io_uring/rw: add defensive hardening for negative kbuf lengths

2026-04-21T18:19:03+00:00

No real bug here, just being a bit defensive in ensuring that whatever gets passed into io_put_kbuf() is always >= 0 and not some random error value. Reviewed-by: Gabriel Krisman Bertazi Signed-off-by: Jens Axboe

io_uring/rw: clean up __io_read() obsolete comment and early returns

2026-04-02T12:55:50+00:00

After commit a9165b83c193 ("io_uring/rw: always setup io_async_rw for read/write requests") which moved the iovec allocation into the prep path and stores it in req->async_data where it now gets freed as part of the request lifecycle, this comment is now outdated. Remove it and clean up the goto as well. Signed-off-by: Joanne Koong Link: https://patch.msgid.link/20260401173511.4052303-1-joannelkoong@gmail.com Signed-off-by: Jens Axboe

io_uring/rw: use cached file rather than req->file

2026-03-17T20:35:00+00:00

In io_rw_init_file(), req->file is cached in file, yet the former is still being used when checking for O_DIRECT. As this is post setting the kiocb flags, the compiler has to reload req->file. Just use the locally cached file instead. Signed-off-by: Jens Axboe