kernel/git/stable/linux.git/drivers/virtio/virtio_ring.c, branch master

virtio_ring: fix stale descriptor flags after a failed packed add

2026-09-07T22:54:02+00:00

In a packed ring the AVAIL and USED bits sit in the descriptor itself, so writing them makes that descriptor available. Those bit combinations flip meaning on every round of the ring, tracked by a wrap counter, so invalidating or validating a descriptor means inverting both bits. Commit 1ce9e6055fa0 ("virtio_ring: introduce packed ring support") has virtqueue_add_packed() make every descriptor of a chain available as it maps the chain, and write the head last. The device consumes the ring in order and stops at a head that is not available yet, so it never reaches the rest. When vring_map_one_sg() fails partway, unmap_release unmaps the segments and restores avail_used_flags, but the descriptors it wrote to in the ring stay marked with AVAIL and USED bits. The head is now the only entry that keeps the device from consuming these stale entries. For example, the ring would look like this now. Z - pre-previous command A - previous command B - aborted command C - current command [A1 DONE] [A2 DONE] [B2] [B3] [Z1 DONE] When the driver now attempts to issue the C command, the next add starts at the same head as B. If C spans less descriptors than B, there is no end marker because AVAIL and USED bits were still in place. And that means the device will start interpreting these stale entries (B2/B3) as another command entry, which then blocks the queue. This effect typically happens in swiotlb configurations under memory pressure, because vring_map_one_sg() can then fail with larger I/O requests which then leads to command abortions. There are broadly 2 ways to avoid leaving those flags behind: 1) Defer those flags too until the chain is complete. 2) Rewrite those flags for the previous wrap counter. Implement the second option in both packed add paths. The first option traverses the chain a second time on every successful add. The second option invalidates all added descriptors when any add fails. With this patch applied, a packed virtqueue keeps completing requests after a failed add. Fixes: 1ce9e6055fa0 ("virtio_ring: introduce packed ring support") Fixes: f6a15d854986 ("virtio_ring: add in order support") Assisted-by: Kiro:claude-opus-5 checkpatch sparse Signed-off-by: Alexander Graf Signed-off-by: Michael S. Tsirkin Message-ID: <20260817223229.28954-1-graf@amazon.com>

virtio_ring: fix infinite loop in virtnet_poll_cleantx when device is broken

2026-08-19T10:38:47+00:00

virtnet_poll_cleantx() contains a do-while loop that cleans up transmitted TX buffers and calls virtqueue_enable_cb_delayed() to check whether more buffers need processing. When the virtio backend stops responding during guest reboot, used->idx is never updated, so virtqueue_enable_cb_delayed() always returns false and the loop never terminates. Then it will block reboot process, and the guest will hang. The problem occurs during guest reboot under network traffic: 1. kernel_restart() -> device_shutdown() traverses the device list 2. virtio_dev_shutdown() calls virtio_break_device() which sets vq->broken = true 3. virtio_dev_shutdown() then calls virtio_synchronize_cbs() to wait for in-flight callbacks to complete 4. A virtio interrupt fires, softirq is deferred to ksoftirqd which calls net_rx_action() -> virtnet_poll() -> virtnet_poll_cleantx() 5. virtnet_poll_cleantx() enters the do-while loop and never exits because the QEMU backend has stopped updating used->idx, despite vq->broken having been set to true in step 2. Since the loop runs inside ksoftirqd (a SCHED_OTHER kthread), it is visible to the scheduler and does not trigger a hard lockup. However, the kthread never leaves the loop, so RCU detects it as a CPU stall and reports it periodically. Meanwhile, the reboot process remains blocked in device_shutdown() because virtio_dev_shutdown() cannot complete its synchronization step, and the guest hangs permanently. This can be reproduced on a guest with a virtio-net device: run iperf3 traffic in the guest, then trigger reboot. The reboot occasionally hangs permanently with RCU stall on ksoftirqd. Observed on ARM64 KVM guest: CPU#1 RCU stall (ksoftirqd/1), repeated periodically: virtqueue_enable_cb_delayed_split <- virtnet_poll <- __napi_poll <- net_rx_action <- handle_softirqs <- run_ksoftirqd <- smpboot_thread_fn <- kthread Fix by adding a vq->broken check in virtqueue_enable_cb_delayed(), so that the loop exits immediately when the device is broken, allowing the device shutdown to proceed. Signed-off-by: Jinqian Yang Reviewed-by: Xuan Zhuo Signed-off-by: Michael S. Tsirkin Message-ID: <20260716115940.394832-1-yangjinqian1@huawei.com>

virtio_ring: Add READ_ONCE annotations for device-writable fields

2026-06-10T06:16:59+00:00

KCSAN reports data races when accessing virtio ring fields that are concurrently written by the device (host). These are legitimate concurrent accesses where the CPU reads fields that the device updates via DMA-like mechanisms. Add accessor functions that use READ_ONCE() to properly annotate these device-writable fields and prevent compiler optimizations that could in theory break the code. This also serves as documentation showing which fields are shared with the device. The affected fields are: - Split ring: used->idx, used->ring[].id, used->ring[].len - Packed ring: desc[].flags, desc[].id, desc[].len This patch was partially written using the help of Kiro, an AI coding assistant, to automate the mechanical work of generating the inline function definition. Signed-off-by: Alexander Graf [jth: Add READ_ONCE in virtqueue_kick_prepare_split ] Co-developed-by: Johannes Thumshirn Signed-off-by: Johannes Thumshirn Reviewed-by: Alexander Graf Signed-off-by: Michael S. Tsirkin Message-ID: <20260131102810.1254845-1-johannes.thumshirn@wdc.com>

dma-mapping: Clarify valid conditions for CPU cache line overlap

2026-03-20T10:33:24+00:00

Rename the DMA_ATTR_CPU_CACHE_CLEAN attribute to better reflect that it is debugging aid to inform DMA core code that CPU cache line overlaps are allowed, and refine the documentation describing its use. Signed-off-by: Leon Romanovsky Signed-off-by: Marek Szyprowski Link: https://lore.kernel.org/r/20260316-dma-debug-overlap-v3-3-1dde90a7f08b@nvidia.com

Convert 'alloc_obj' family to use the new default GFP_KERNEL argument

2026-02-22T01:09:51+00:00

This was done entirely with mindless brute force, using git grep -l '\

treewide: Replace kmalloc with kmalloc_obj for non-scalar types

2026-02-21T09:02:28+00:00

This is the result of running the Coccinelle script from scripts/coccinelle/api/kmalloc_objs.cocci. The script is designed to avoid scalar types (which need careful case-by-case checking), and instead replace kmalloc-family calls that allocate struct or union object instances: Single allocations: kmalloc(sizeof(TYPE), ...) are replaced with: kmalloc_obj(TYPE, ...) Array allocations: kmalloc_array(COUNT, sizeof(TYPE), ...) are replaced with: kmalloc_objs(TYPE, COUNT, ...) Flex array allocations: kmalloc(struct_size(PTR, FAM, COUNT), ...) are replaced with: kmalloc_flex(*PTR, FAM, COUNT, ...) (where TYPE may also be *VAR) The resulting allocations no longer return "void *", instead returning "TYPE *". Signed-off-by: Kees Cook

virtio: add virtqueue_add_inbuf_cache_clean API

2026-01-02T11:22:49+00:00

Add virtqueue_add_inbuf_cache_clean() for passing DMA_ATTR_CPU_CACHE_CLEAN to virtqueue operations. This suppresses DMA debug cacheline overlap warnings for buffers where proper cache management is ensured by the caller. Message-ID: Signed-off-by: Michael S. Tsirkin

virtio_ring: add in order support

2025-12-31T10:47:50+00:00

This patch implements in order support for both split virtqueue and packed virtqueue. Performance could be gained for the device where the memory access could be expensive (e.g vhost-net or a real PCI device): Benchmark with KVM guest: Vhost-net on the host: (pktgen + XDP_DROP): in_order=off | in_order=on | +% TX: 4.51Mpps | 5.30Mpps | +17% RX: 3.47Mpps | 3.61Mpps | + 4% Vhost-user(testpmd) on the host: (pktgen/XDP_DROP): For split virtqueue: in_order=off | in_order=on | +% TX: 5.60Mpps | 5.60Mpps | +0.0% RX: 9.16Mpps | 9.61Mpps | +4.9% For packed virtqueue: in_order=off | in_order=on | +% TX: 5.60Mpps | 5.70Mpps | +1.7% RX: 10.6Mpps | 10.8Mpps | +1.8% Benchmark also shows no performance impact for in_order=off for queue size with 256 and 1024. Reviewed-by: Eugenio Pérez Signed-off-by: Jason Wang Signed-off-by: Michael S. Tsirkin Message-Id: <20251230064649.55597-20-jasowang@redhat.com>

virtio_ring: factor out split detaching logic

2025-12-31T10:47:50+00:00

This patch factors out the split core detaching logic that could be reused by in order feature into a dedicated function. Acked-by: Eugenio Pérez Reviewed-by: Xuan Zhuo Signed-off-by: Jason Wang Reviewed-by: Eugenio Pérez Signed-off-by: Michael S. Tsirkin Message-Id: <20251230064649.55597-19-jasowang@redhat.com>

virtio_ring: factor out split indirect detaching logic

2025-12-31T10:39:18+00:00

Factor out the split indirect descriptor detaching logic in order to allow it to be reused by the in order support. Acked-by: Eugenio Pérez Reviewed-by: Xuan Zhuo Signed-off-by: Jason Wang Signed-off-by: Michael S. Tsirkin Message-Id: <20251230064649.55597-18-jasowang@redhat.com>