kernel/git/stable/linux.git/arch/x86, branch linux-rolling-stable

KVM: x86: Move enabling EFER.SVME and EFER.LMSLE to generic EFER setup

2026-09-11T09:51:07+00:00

commit 6ccc19d4c1eb97a994180af8afcab58422cb409d upstream. Move SVM-specific EFER bit enablement to generic x86 code, with the rest of EFER bit enablement. Unifying the code for EFER bit enablement allows for a later change to re-initialize EFER bits on module init. No functional change intended. Cc: stable@vger.kernel.org Suggested-by: Sean Christopherson Signed-off-by: Yosry Ahmed Link: https://patch.msgid.link/20260713181020.2735367-2-yosry@kernel.org Signed-off-by: Sean Christopherson Signed-off-by: Greg Kroah-Hartman

KVM: x86: Ensure runtime reads of disabled_quirks are resolved once

2026-09-11T09:51:07+00:00

commit ed15cb21999217e549414c128b4a0485debf6278 upstream. Wrap the sole reader of disabled_quirks with READ_ONCE(), and wrap the post-VM-creation write to disabled_quirks with WRITE_ONCE(), to ensure checking the status of a quirk doesn't re-read disabled_quirks *if* the caller needs such a guarantee. This will allow splitting the "fast" MMU zap into front and back halves, without potentially skipping the back half if SLOT_ZAP_ALL were concurrently disabled (which would be "fine" in the current code base, but far from ideal). Cc: stable@vger.kernel.org # 6.12.x Reviewed-by: Michael Roth Link: https://patch.msgid.link/20260709204948.1988414-7-seanjc@google.com Signed-off-by: Sean Christopherson Signed-off-by: Greg Kroah-Hartman

KVM: x86: Serialize writes to disabled_quirks using kvm->lock

2026-09-11T09:51:07+00:00

commit ba76b23ed36ab230fc2577aba24f65851114902f upstream. Protect writes to disabled_quirks with kvm->lock to ensure KVM doesn't clobber state in the unlikely scenario that userspace disables disparate quirks from multiple tasks. More importantly, this will allow wrapping accesses with {READ,WRITE}_ONCE without "needing" to also guard the writer with a useless and confusing READ_ONCE (since the RMW wouldn't be atomic anyways). Ideally, KVM would disallow disabling quirks once quirks are "live", but that would be a potentially breaking userspace ABI change, and while all existing quirks are fully live only after vCPUs have been created, several MMU-related quirks, IGNORE_GUEST_PAT and SLOT_ZAP_ALL, are partially live at all times. Because populating MMUs requires a vCPU, the guest-visible behavior of IGNORE_GUEST_PAT and SLOT_ZAP_ALL requires a vCPU, but for KVM itself, processing the quirk (or not) has functional impact, i.e. for all intents and purposes, KVM can't prevent those quirks from being disabled after they've been consumed. Cc: stable@vger.kernel.org # 6.12.x Reviewed-by: Michael Roth Link: https://patch.msgid.link/20260709204948.1988414-6-seanjc@google.com Signed-off-by: Sean Christopherson Signed-off-by: Greg Kroah-Hartman

KVM: x86: hyper-v: Clamp stimer deadline to avoid livelock

2026-09-11T09:51:06+00:00

commit 0ca49fbd2883cd53d32d85b50feef17fa04d0fbf upstream. Fix an issue where userspace or the guest can program an Hyper-V synthetic timer to have a deadline in the past via integer overflow, preventing the CPU from making progress and triggering an RCU stall. Hyper-V's SynIC exposes 4 per-vCPU synthetic timers to the guest, which are emulated by KVM. Each is programmed through the HV_X64_MSR_STIMERi_CONFIG and HV_X64_MSR_STIMERi_COUNT MSRs. Depending on CONFIG, COUNT represents either the absolute expiration time or the period of a periodic timer, both expressed in 100ns ticks. These timers may be set both by the guest (WRMSR) and the host (KVM_SET_MSRS). When the timer is enabled, stimer_start() translates COUNT to an absolute monotonic deadline and arms an hrtimer. If COUNT is set to a value close to U64_MAX, the deadline calculation can overflow. ktime_add_ns(ktime_now, 100 * (stimer->exp_time - time_now)) This can result in a CPU livelock. stimer_start() arms the timer via hrtimer_start() with a deadline in the past, which causes it to immediately fire. The stimer callback then raises KVM_RQ_HV_STIMER, with the intention of causing KVM to deliver a synthetic interrupt on the next vCPU guest enter. Then, once userspace issues KVM_RUN, vcpu_enter_guest() consumes the request, calling kvm_hv_process_stimers(). This would normally disable the timer via stimer_expiration() once the deadline is in the past. However, the deadline comparison is done between the KVM reference counter and stime->exp_time, which is a big value close to U64_MAX, so this never happens for a few thousand years. kvm_hv_process_timers() then re-arms the timer via stimer_start(), since it was not disabled, which again fires immediately. Before entering the guest, kvm_vcpu_exit_request() checks kvm_request_pending(), which returns true due to the newly raised KVM_REQ_HV_STIMER. Then vcpu_enter_guest() aborts the guest entry, returning early into vcpu_run(), which loops back again into vcpu_enter_guest(), restarting the cycle. Since there are no manual yields in this loop, a task with SCHED_FIFO may starve RCU grace-period kthreads, which exposes the stalls found by syzcaller: rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: rcu: (detected by 1, t=10502 jiffies, g=14269, q=1142 ncpus=2) rcu: All QSes seen, last rcu_preempt kthread activity 10500 (4294965239-4294954739), jiffies_till_next_fqs=1, root ->qsmask 0x0 rcu: rcu_preempt kthread starved for 10500 jiffies! g14269 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0 rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. ( ... ) Call Trace: __run_hrtimer kernel/time/hrtimer.c:1773 [inline] __hrtimer_run_queues+0x408/0xc30 kernel/time/hrtimer.c:1841 hrtimer_interrupt+0x45b/0xaa0 kernel/time/hrtimer.c:1903 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1045 [inline] __sysvec_apic_timer_interrupt+0x102/0x3e0 arch/x86/kernel/apic/apic.c:1062 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1056 [inline] sysvec_apic_timer_interrupt+0xa1/0xc0 arch/x86/kernel/apic/apic.c:1056 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:697 RIP: 0010:__raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:152 [inline] RIP: 0010:_raw_spin_unlock_irqrestore+0xa8/0x110 kernel/locking/spinlock.c:194 Code: 74 05 e8 0b f4 5f f6 48 c7 44 24 20 00 00 00 00 9c 8f 44 24 20 f6 44 24 21 02 75 4f f7 c3 00 02 00 00 74 01 fb bf 01 00 00 00 23 6b 27 f6 65 8b 05 7c 60 5a 07 85 c0 74 40 48 c7 04 24 0e 36 RSP: 0018:ffffc900040a7320 EFLAGS: 00000206 RAX: 5de15cb931505900 RBX: 0000000000000a06 RCX: 5de15cb931505900 RDX: 0000000000000007 RSI: ffffffff8daa9dc3 RDI: 0000000000000001 RBP: ffffc900040a73b0 R08: ffffffff8fc3d077 R09: 1ffffffff1f87a0e R10: dffffc0000000000 R11: fffffbfff1f87a0f R12: dffffc0000000000 R13: 0000000000000000 R14: ffff8880b8628240 R15: 1ffff92000814e64 hrtimer_start include/linux/hrtimer.h:259 [inline] stimer_start arch/x86/kvm/hyperv.c:682 [inline] kvm_hv_process_stimers+0xd0a/0x16a0 arch/x86/kvm/hyperv.c:893 vcpu_enter_guest arch/x86/kvm/x86.c:11193 [inline] vcpu_run+0x2240/0x76b0 arch/x86/kvm/x86.c:11639 kvm_arch_vcpu_ioctl_run+0x1148/0x1c90 arch/x86/kvm/x86.c:11984 kvm_vcpu_ioctl+0x99a/0xed0 virt/kvm/kvm_main.c:4492 vfs_ioctl fs/ioctl.c:51 [inline] __do_sys_ioctl fs/ioctl.c:597 [inline] __se_sys_ioctl+0xfc/0x170 fs/ioctl.c:583 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline] do_syscall_64+0xfa/0xf80 arch/x86/entry/syscall_64.c:94 entry_SYSCALL_64_after_hwframe+0x77/0x7f RIP: 0033:0x7f635278f749 Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 a8 ff ff ff f7 d8 64 89 01 48 RSP: 002b:00007f635365c038 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 RAX: ffffffffffffffda RBX: 00007f63529e5fa0 RCX: 00007f635278f749 RDX: 0000000000000000 RSI: 000000000000ae80 RDI: 0000000000000005 RBP: 00007f6352813f91 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000 R13: 00007f63529e6038 R14: 00007f63529e5fa0 R15: 00007ffd5b219358 Fix this by clamping the deadline computation to KTIME_MAX, which preserves the intent of arming a timer very far in the future. ktime_add_safe() already does this type of clamping, so use it after checking that that multiplying by the 100ns time tick also does not overflow. Reviewed-by: Vitaly Kuznetsov Reported-by: syzbot+3d5461510f8dc4adfe30@syzkaller.appspotmail.com Closes: https://syzkaller.appspot.com/bug?extid=3d5461510f8dc4adfe30 Fixes: 1f4b34f825e8 ("kvm/x86: Hyper-V SynIC timers") Cc: stable@vger.kernel.org Signed-off-by: Carlos López Link: https://patch.msgid.link/20260714133212.3916611-3-clopez@suse.de [sean: tag for stable] Signed-off-by: Sean Christopherson Signed-off-by: Greg Kroah-Hartman

KVM: x86/mmu: Consume the locked rmap value in the lockless rmap walk

2026-09-11T09:51:06+00:00

commit e428f9779a43737d830111238816f1928b07aefb upstream. __kvm_rmap_lock() deliberately elides the rmap lock when it observes an empty rmap. In that case kvm_rmap_lock_readonly() also re-enables preemption and returns zero, so the caller holds neither the rmap lock nor a preemption reference. The elision documents the invariant it relies on: * Elide the lock if the rmap is empty, as lockless walkers (read-only * mode) don't need to (and can't) walk an empty rmap, nor can they add * entries to the rmap. I.e. the only paths that process empty rmaps * do so while holding mmu_lock for write, and are mutually exclusive. kvm_rmap_age_gfn_range() ignores the returned value and unconditionally enters for_each_rmap_spte_lockless(). The iterator started with rmap_get_first(), which re-reads rmap_head->val rather than using the value returned by the lock. If a writer populates the rmap between the lock's read and the iterator's re-read, the aging path walks the newly installed rmap without holding its lock. For a KVM_RMAP_MANY rmap this leaves the walker following a pte_list_desc chain that it never locked. A writer holding mmu_lock for write may free that chain (e.g. kvm_zap_all_rmap_sptes() on the recycle path, or any rmap zap) via kmem_cache_free() while the walk is in progress, giving a slab use-after-free. Nothing serialises the two: the aging path runs without mmu_lock when CONFIG_KVM_MMU_LOCKLESS_AGING=y, and the rmap lock that would otherwise exclude the writer was elided. Because the empty path re-enables preemption, the interval between the two reads can span an arbitrary scheduling delay. Fix the class of bug by having the lockless walk consume the value returned by the lock instead of re-reading the rmap. Split rmap_get_first() into __rmap_get_first(), which starts an iterator from an already-read rmap value, and make for_each_rmap_spte_lockless() take that value and call __rmap_get_first() directly. kvm_rmap_age_gfn_range() passes the value returned by kvm_rmap_lock_readonly(): when the lock was elided the value is zero, __rmap_get_first() returns NULL, and the walk is skipped. No lockless walker re-reads the rmap, so the lock-elision invariant cannot be violated, and no lock()-without-paired-unlock() path is added to the aging code. Fixes: af3b6a9eba48 ("KVM: x86/mmu: Walk rmaps (shadow MMU) without holding mmu_lock when aging gfns") Suggested-by: Sean Christopherson Cc: stable@vger.kernel.org Signed-off-by: Phil Rosenthal Link: https://patch.msgid.link/20260720-rmap-age-elided-submit-v2-1-668973030d47@phil.gs Signed-off-by: Sean Christopherson Signed-off-by: Greg Kroah-Hartman

KVM: x86/mmu: Use CMPXCHG when clearing Accessed bit in TDP MMU

2026-09-11T09:51:06+00:00

commit 3d679b7cb31f74bf2303123ce4ee3266eeee999a upstream. Use LOCK CMPXCHG instead of LOCK AND to clear the Accessed bit when aging SPTEs in the TDP MMU, as doing a LOCK AND can corrupt a FROZEN SPTE and allow a third CPU to effectively overwrite the FROZEN SPTE. As pointed out by AI of some kind, because the magic FROZEN_SPTE value is a "full" SPTE, not a single bit, and includes the Accessed bit, clearing the Accessed bit in a FROZEN SPTE will result in is_frozen_spte() getting a false negative. E.g. if CPU0 freezes an SPTE, and CPU1 clears the Accessed bit in the frozen SPTE, then CPU2 could come along and overwrite the frozen SPTE with a shadow-present SPTE. Thankfully, the false negative is largely benign, because outside of TDX, which doesn't support aging, KVM only freezes leaf SPTEs when removing an upper level shadow page. So while KVM could clobber a frozen SPTE back to a shadow-present SPTE, and could even use the new SPTE, the subsequent TLB flush will make the orphaned, shadow-present SPTE unreachable. Failure to ever zap the orphaned leaf SPTE would show up in KVM's stats, but otherwise is benign (because KVM no longer keeps an elevated refcount for leaf SPTEs). Opportunistically add a comment to warn future developers away from using kvm_tdp_mmu_write_spte_atomic() and tdp_mmu_clear_spte_bits_atomic(), as they are generally unsafe. Keep the helpers, e.g. instead of open-coding the atomic64_fetch_and() in tdp_mmu_clear_spte_bits(), as scary warnings usually are more effective deterrent against recidivism than removal of the dangerous code. Alternatively, KVM could use different bits for the magic FROZEN_SPTE value, e.g. setting the Dirty bits (with effective IPAT and Global aliases) would likely be "ok", as IPAT/Global are extremely unlikely to be cleared without doing a full SPTE write, and KVM's clearing of Dirty bits shares logic with Write-Protection, which must do a full SPTE write (via cmpxchg64() in the TDP MMU) to ensure KVM isn't clobbering state. But there is zero reason to carry that risk (beyond stubbornness in wanting to preserve a "cute" idea), as the cost of LOCK CMPXCHG and LOCK AND are within 1-2 uops of each other on modern hardware. Fixes: b146a9b34aed ("KVM: x86/mmu: Age TDP MMU SPTEs without holding mmu_lock") Cc: stable@vger.kernel.org Reviewed-by: Kai Huang Reviewed-by: James Houghton Link: https://patch.msgid.link/20260728002236.869865-2-seanjc@google.com Signed-off-by: Sean Christopherson Signed-off-by: Greg Kroah-Hartman

KVM: nVM: Ensure INVVPID is emulated on the correct physical CPU

2026-09-11T09:51:06+00:00

commit 6d00e67326d831e6e610933a3800712f4ffe6ec1 upstream. When emulating INVVPID, KVM executes INVVPID on the physical CPU using vpid02 (instead of the L1 assigned VPID), after doing some validations on the operands. However, it is possible that the physical CPU KVM executes INVVPID on is different from the CPU L2 is running on. For example, in the following scenario: - L2 runs on CPU #1 and exits to L1 (vmx->nested.vmcs02.cpu=1) - L1 migrates to CPU #2 and executes INVVPID - KVM executes INVVPID on CPU #2 - L1 migrates back to CPU #1 and runs L2 (vmx->nested.vmcs02.cpu=1) The TLB entries on CPU #1 are never invalidated, because INVVPID was executed on CPU #2, and vmcs02 never ran on a different pCPU (i.e. vmx_vcpu_load_vmcs() will *not* request KVM_REQ_TLB_FLUSH). Ensure that INVVPID is being executed on the same pCPU that L2 last ran on, and if not, fallback to clearing last_vpid=0 to trigger a full VPID flush on the next nested VM-Enter (as KVM will detect L1 using a different VPID for L2). If L2 ends up running on a different pCPU, KVM will flush the TLB anyway through vmx_vcpu_load_vmcs(). Cc: stable@vger.kernel.org Signed-off-by: Yosry Ahmed Reviewed-by: Kai Huang Link: https://patch.msgid.link/20260616214652.2157032-4-yosry@kernel.org Signed-off-by: Sean Christopherson Signed-off-by: Greg Kroah-Hartman

KVM: nVMX: Service local TLB flushes on failed nested VM-Enter

2026-09-11T09:51:06+00:00

commit 05a0b701d1089fb57beeb8982f23c3bbafe0fa8b upstream. KVM services local TLB flushes on "full" nested VM-Exits (through __nested_vmx_vmexit()), but not if a nested VM-Enter fails (e.g. due to failed VMCS checks in nested_vmx_enter_non_root_mode()). However, it is possible that KVM had queued TLB flushes that need to be performed, even if the nested VM-Enter was not successful. For example, if VPID is disabled for L2 (via nested_vmx_transition_tlb_flush(), or if via the MSR load lists, as the SDM says: If any MSR is being loaded in such a way that would architecturally require a TLB flush, the TLBs are updated so that, after VM entry, the logical processor will not use any translations that were cached before the transition. The SDM is unclear about when the TLB flush should occur, and whether or not a failed VM entry would flush the TLB, so it is safer to always do the TLB flush in this case. More concretely, KVM also updates the last VPID L1 used for L2 in nested_vmx_transition_tlb_flush() (i.e. last_vpid), even if the VM entry ultimately fails. With the current code, KVM could miss a TLB flush if L1 changes L2's VPID, then does a failed VM entry followed by a successful one, as the failed VM entry would update last_vpid but not actually flush the TLB. Servicing local TLB flushes on failed VM entries makes sure that the TLB is always flushed when last_vpid is updated. Fixes: 5c614b3583e7 ("KVM: nVMX: nested VPID emulation") Cc: stable@vger.kernel.org Reported-by: Sashiko # Internal review Suggested-by: Sean Christopherson Signed-off-by: Yosry Ahmed Link: https://patch.msgid.link/20260722230128.1587363-1-yosry@kernel.org Signed-off-by: Sean Christopherson Signed-off-by: Greg Kroah-Hartman

KVM: nVMX: Ensure KVM_REQ_GET_NESTED_STATE_PAGES is cleared on VM-Exit

2026-09-11T09:51:06+00:00

commit 11722439fb206c88e6f31be54173efa9880b4ccb upstream. Always check and clear KVM_REQ_GET_NESTED_STATE_PAGES when emulating a nested VM-Exit to ensure the request is cleared, even when KVM was built with CONFIG_KVM_HYPERV=n, as KVM subtly relies on the "check" to clear the flag and thus avoid double-mapping the vmcs12 pages, e.g. if KVM manages to bail from VM-Enter without processing the request, and then emulates VMLAUNCH or VMRESUME. Fixes: b4f69df0f65e ("KVM: x86: Make Hyper-V emulation optional") Cc: stable@vger.kernel.org Reported-by: Yosry Ahmed Reviewed-by: Yosry Ahmed Link: https://patch.msgid.link/20260724004757.131420-2-seanjc@google.com Signed-off-by: Sean Christopherson Signed-off-by: Greg Kroah-Hartman

KVM: nVMX: Decouple INVVPID operand checks from flushing of vpid02

2026-09-11T09:51:06+00:00

commit 32912404b4b1ee98400744941c78f019a63d6e8f upstream. Separate the INVVPID operand checks from the actual flushing of vpid02 so the flushing can be adjusted to do the right thing when vmcs02 was last loaded on a different pCPU, without having to duplicate the logic across multiple case-statements. Opportunistically let the VM-Fail paths poke out past 80 chars. No functional change intended. Cc: stable@vger.kernel.org Signed-off-by: Yosry Ahmed Reviewed-by: Kai Huang Link: https://patch.msgid.link/20260616214652.2157032-3-yosry@kernel.org Signed-off-by: Sean Christopherson Signed-off-by: Greg Kroah-Hartman