New to ShortSpan? We distil the AI-security research that matters into practitioner takeaways — edited by Ben Williams (NCC Group).Get the weekly email
// Analysis
Shared KV Caches Leak Prompts; KVGov Stops It
Published: Tue, 11 Aug 2026 • By Rowan Vale
Enterprise
Shared Key-Value caches speed Large Language Model inference, but they leak timing signals. New research shows three attack families can recover private prompts on production-like stacks. KVGov isolates cache namespaces with per-tenant HMAC salts, cutting cross-tenant cache hits to zero in simulation while keeping most cache gains. Audits further reduce attacker payoff.
Shared Key-Value (KV) caches are the workhorse trick that makes hosted Large Language Model (LLM) inference feel snappy. They also leak. If two tenants share a KV cache, one can time the model’s Time To First Token (TTFT) to learn whether a prefix is already computed. That tiny delta is enough to confirm private prompt content.
The idea is simple: cache hits make TTFT drop. If I can send probes that line up with your prefix and watch TTFT, I can tell hit from miss and recover information. The paper looks at three attack families already in the wild: fingerprinting whole prompts, token-by-token reconstruction, and template-field extraction. On unprotected vLLM and SGLang the success rate hits 100% in some setups, with effectiveness shaped by cache design and how structured the victim’s prompt is.
This is not lab-only. On Qwen2.5-7B-Instruct over vLLM 0.26.0 on an NVIDIA A100, a long shared prefix produced a cold-to-cached TTFT ratio of 0.22. They reproduced the signal on a different stack too: llama.cpp on Apple Metal clocked a ratio of 0.093. That’s plenty of gap for a patient tenant to run O(1) membership tests against candidate fields.
Defence-wise, KVGov treats cache keys as a governance problem, not a hardware one. It seeds the per-block hash chain with a per-principal salt σ_p = HMAC(secret, principal_id). Result: your cache keys occupy a cryptographically disjoint namespace from mine, so my probes never collide with your entries. In simulation this took attack success to 0% across the evaluated prefix-cache scenarios, and an ablation showed the salt was the necessary and sufficient piece. Overhead is tiny at lookup time (about 1.33 microseconds per HMAC), with the real cost being lost cross-tenant reuse. They suggest injecting the salt at the point prompts diverge rather than at the root, keeping roughly 93% of the prefix-cache benefit, though that efficiency figure is extrapolated rather than end-to-end measured. Semantic caches based on embedding similarity are a different beast and need index partitioning.
There’s also an operational angle. ORIGAMI, a Stackelberg water-filling audit scheduler, trims attacker expected utility by 12.6% under realistic tenant skew (Gini about 0.63) and evens out payoff across audited tenants. An evolutionary analysis adds a useful knob: if adversaries make up more than 31.6% of the population, ephemeral per-session caching becomes the stable strategy; below that, global shared caching can hold.
Limitations are candid. The defence evaluation leans on simulation calibrated by hardware timings, and the “salt at divergence” win is modelled. Admission checks add latency on cache misses. Still, the takeaway is clear: TTFT is a signal, and in shared KV caches it says more than you think. The open questions are where this lands for semantic caches and how much real-world variance blurs those timing gaps at scale.
Additional analysis of the original ArXiv paper
📋Original Paper Title and Abstract
Governing the KV Cache: Preventing Timing Side-Channel Leakage in Multi-Tenant LLM Inference
Authors: Tejasvi C. Addagada
The key-value (KV) cache is the primary throughput optimization in modern large language model (LLM) inference, enabling prefix reuse across requests. In multi-tenant deployments this cache is shared across tenants, creating a timing side channel: an adversarial tenant can reconstruct another tenant's private prompt by probing cache-hit latency. Three published attacks exploit it -- PROMPTPEEK, EarlyBird and InputSnatch -- reaching up to 100% attack success rate against unprotected vLLM and SGLang, with rates varying by cache architecture and prompt structure. We present KVGov, a governance layer addressing all three attack families' prefix-cache paths under one mechanism. A per-principal salt sigma_p = HMAC_K(secret, principal_id) seeds the block-hash chain, making cache keys cryptographically disjoint across principals. An ablation (N=1000 trials, seed 2026, deterministic judges) isolates this salt as the necessary and sufficient component. KVGov adds ORIGAMI, a Stackelberg water-filling audit scheduler that reduces adversary expected utility by 12.6% at realistic tenant heterogeneity (Gini 0.63), and an evolutionary stability analysis giving a 31.6% adversary-prevalence tipping point below which global caching remains stable. On real hardware (Qwen2.5-7B-Instruct, vLLM 0.26.0, NVIDIA A100) we measure a gate-verified cold/cached TTFT ratio of 0.22, confirming the channel is exploitable at production scale; the defense itself is evaluated in simulation calibrated to those measurements. We replicate the channel on an independent stack (llama.cpp on Apple Metal, ratio 0.093). Finally, isolation and cache efficiency need not conflict: identifying information resides only where prompts diverge, so injecting the salt at that boundary rather than the chain root retains an estimated 93% of the prefix-cache benefit with no cross-principal signal.
🔍ShortSpan Analysis of the Paper
Problem
Modern LLM serving engines use a shared key-value cache to reuse prefix computations and dramatically reduce time-to-first-token. In multi-tenant deployments that shared KV cache creates a timing side channel: an adversarial tenant can probe whether particular cache entries exist by measuring TTFT and thereby infer another tenant's private prompt. Three independent attack families exploit this channel—fingerprinting, token-by-token reconstruction and template-field extraction—reaching very high success rates on unprotected systems and proving the leakage is practical on production-like hardware.
Approach
The paper proposes KVGov, a governance layer that prevents cross-principal timing leakage by cryptographic namespace isolation and operational auditing. The core mechanism seeds the block-hash chain with a per-principal salt sigma_p = HMAC_K(secret, principal_id), so cache keys are collision-resistant across tenants. KVGov also includes ORIGAMI, a Stackelberg water-filling audit scheduler that allocates limited audit budget proportionally to tenants' value-at-risk, and an admission gate for PII checks. Experiments combine deterministic simulation (seeded trials, attack-specific deterministic judges) with gate-verified hardware measurements on Qwen2.5-7B-Instruct running vLLM 0.26.0 on an NVIDIA A100 and an independent replication on llama.cpp via Ollama on Apple Metal. Ablations isolate which components provide protection and microbenchmarks measure per-request overheads.
Key Findings
HMAC per-principal salting alone eliminates cross-principal cache hits in simulation and reduces Attack Success Rate to 0% across the three published attack families in the evaluated prefix-cache scenarios.
ORIGAMI audits reduce adversary expected utility by 12.6% at realistic enterprise workload heterogeneity (Gini ≈ 0.63) compared with random audit; it equalises attacker payoff across audited tenants via a closed-form water-filling solution.
Evolutionary analysis identifies a tipping point at 31.6% adversary prevalence below which global shared caching is evolutionarily stable and above which ephemeral per-session caching becomes stable.
Hardware measurements confirm the channel is exploitable: with a 2119-token shared prefix on an A100 the cold/cached TTFT ratio is 0.22; a replication on Apple Metal produced a ratio of 0.093.
Ablations show partial mitigations perform unevenly: session-boundary flushing left substantial residual risk (PROMPTPEEK ≈ 19.5%, InputSnatch ≈ 9.8%), and Gaussian noise broke fingerprinting but not membership-based extraction.
Operational cost is low in CPU terms: HMAC derivation adds about 1.33 microseconds per cache lookup; the main efficiency cost is reduced cross-tenant reuse when keys are partitioned.
Injecting the salt at the divergence boundary rather than the chain root can retain most cache benefit; extrapolated estimates indicate roughly 92.8–93% of prefix-cache gain can be preserved while removing cross-principal signal.
Limitations
KVGov's salt construction addresses hash-equality based prefix and radix caches but does not by itself defend semantic caches that resolve by embedding similarity; those require index partitioning. The defence evaluation mixes hardware measurement for the channel with simulation for the full defence; boundary-salting efficiency figures are extrapolated rather than measured end to end. Simulations use fixed hit/miss latencies and deterministic judges, which simplifies but does not capture all real-world variance. Admission checks add non-trivial latency on miss paths.
Implications
An attacker with only legitimate API access and millisecond timing observations can confirm membership of template fields, enumerate served identifiers, or reconstruct prompts under realistic conditions. Membership queries are O(1) per candidate and therefore trivial when templates constrain unknown fields. The demonstrated exploitability on different hardware and software stacks means this is an actionable risk in multi-tenant deployments unless cache resolution is bound to principals or caches are partitioned.
- Why it may be relevant: It tackles a real timing-side-channel risk from shared KV caches during LLM inference, where an attacker could infer sensiti...
- The paper shows a practical side-channel risk in multi-tenant LLM services: shared caches can leak prompted content, not just model outputs.
- It in...
- It shows a new timing-based side channel in multi-tenant LLM serving stacks: shared KV and semantic caches, and GPU memory behavior, can leak privat...
- It exposes a new attack surface: bit-flips in a shared KV-cache used by LLM serving can subtly change model outputs.
- The attack can be hard to det...
- Addresses a real privacy risk in LLM inference: cross-user KV-cache reuse can leak user input via side channels, and this work aims to mitigate that...
- It tackles a real timing side-channel in multi-tenant LLM serving: how cached prefix reuse can leak others’ inputs through observable latency differ...
- Core idea: Semantic caches use embedding-based keys, which trade locality (fast hits) for collision resistance (security). This makes them vulnerabl...
- It reveals a new vulnerability in position-independent KV cache reuse for LLMs: an attacker can seed a prefix that, when a benign text later appears...
- Addresses a real security risk in RAG systems: cached answers can be stale, misleading, or poisoned, turning fast responses into safety/privacy prob...
- The paper shows that even with modern isolation, timing leaks can occur across containers and VMs because the host page cache is shared. This can re...
Get the weekly digest
The few AI-security papers that matter, with the practitioner takeaway. No spam.
