HijackKV exploits position-independent LLM cache reuse
New research shows position-independent Key-Value cache reuse in Large Language Models can be hijacked. The HijackKV attack seeds a cached chunk after an attacker prefix, then later steers a victim’s output when the same chunk is reused. Reported success rates are high and robust across recomputation, multi-turn chats, and cross-model transfer.
Latency tricks are never free. Large Language Model (LLM) serving stacks increasingly reuse Key-Value (KV) cache entries for identical token chunks wherever they appear, not just at the same position. That speeds up decoding. It also opens the door to cross-user integrity failures. A new study shows how a cached chunk, computed after an attacker-controlled prefix, can later be pulled into a different user’s query and quietly steer the model’s behaviour.
How the hijack works
KV entries are retrieved by token match, but the tensors encode the full context in which they were created. HIJACKKV weaponises that fact. The attacker picks a common benign chunk and searches for a prefix that imprints a specific objective into the KV produced for that chunk. They submit prefix plus chunk once, so the service caches the chunk-level KV. Later, when any user’s prompt happens to include the same chunk, position-independent reuse returns the contaminated KV, and generation tilts toward the attacker’s goal even though the victim’s input contains no malicious text.
The paper reports a Greedy Coordinate Gradient search over discrete tokens using a surrogate model to find strong prefixes. In a single attempt the attack averages 94% success, with targeted control reported at 89% on a Llama‑3.1‑8B model across four QA datasets under default settings. It holds up under realistic service behaviour: with a 10% cache hit ratio the authors still see 67% targeted success in some settings, and even when half of matched tokens are recomputed, targeted success persists at 39%. Influence lingers across long dialogues, with targeted rates of 55% after 1,024 filler tokens and 47% at 2,048. In code generation, average success reaches 95.5% across sampling temperatures. Prefixes transfer in black-box tests too, with 75% untargeted and 37% targeted success observed against a 70B model.
Why this matters for shared services
This is a cross-tenant problem by design. The attacker needs a serving stack that reuses position-independent KVs across users and an opportunity to insert the crafted cache entry before a victim hits the same chunk. That condition appears in places where content is naturally shared: documents, retrieval corpora, or standardised code scaffolds. The authors find that partial recomputation, KV compression, and heuristic pruning blunt but do not remove the risk, and adaptive search can recover effectiveness.
The clever bit here is not prompt injection in the usual sense. It is supply-chain style steering through cached state, triggered by innocuous text the victim would have written anyway. Performance engineering has quietly become a policy choice about isolation. If providers persist with cross-user, position-independent reuse, what guarantees should they offer about cache integrity, provenance, and scope? That is the uncomfortable but necessary question this work puts on the table.
Additional analysis of the original ArXiv paper
📋 Original Paper Title and Abstract
HijackKV: New Threat in Position-Independent KV Cache Reuse
🔍 ShortSpan Analysis of the Paper
Problem
This paper studies a new integrity threat in large language model serving caused by position-independent key-value (KV) cache reuse. Modern inference engines reuse cached KV tensors for identical token chunks regardless of their original position to reduce latency. Because KV entries encode the full preceding context, reusing a chunk that was computed after an attacker-controlled prefix can silently steer model generations for a different user who supplies the same chunk without any malicious text. The paper shows this breaks isolation in multi-tenant deployments and can produce attacker-desired outputs without direct prompt injection, creating a covert channel from attacker inputs to victim responses.
Approach
The authors design HIJACKKV, an attack framework that optimises an attacker-controlled prefix so that the KV vectors for a subsequent common benign chunk carry an adversarial objective. The attacker submits the crafted prefix plus the chunk to the service so the resulting chunk-level KV is cached. Later, when a victim’s query contains the same chunk, position-independent reuse returns the contaminated KV and the model produces attacker-chosen outputs. Optimization uses a Greedy Coordinate Gradient algorithm over discrete tokens with a surrogate model; the attack is evaluated in white-box and black-box settings across several open-source LLM families and QA and code benchmarks. Experiments vary cache chunk size, cache hit ratio, recomputation ratio, multi-turn context length and model size to characterise practicality and robustness.
Key Findings
- High single-attempt effectiveness: HIJACKKV attains an average success rate reported as 94% in a single attempt and achieves strong targeted attack success rates across models and datasets (for example, an average targeted success rate of 89% on Llama-3.1-8B across four QA datasets under default settings).
- Robust under realistic constraints: the attack remains effective at low cache hit rates (examples at 10% cache ratio yield a 67% targeted success rate in some settings) and survives recomputation up to substantial levels (with targeted success persisting at 39% when 50% of matched tokens are recomputed).
- Persistence across multi-turns and stochastic decoding: a single cached hijack retains influence after long unrelated contexts (T-ASR 55% at 1024 filler tokens and 47% at 2048) and achieves high attack success in code-generation tasks across sampling temperatures (average ASR 95.5% reported).
- Cross-model transferability: prefixes optimised on a proxy model transfer to larger or comparably capable targets in black-box settings (for example, 75% untargeted and 37% targeted success against a 70B model), though transfer to much smaller models reduces targeted success.
- Existing mitigations are insufficient: partial recomputation, KV compression and heuristic pruning reduce but do not eliminate the threat; adaptive optimisation that simulates recomputation during prefix search further restores attack efficacy.
Limitations
The attack requires the target infrastructure to use position-independent, cross-user KV reuse and for the attacker to be able to insert a cache entry for the chosen chunk before a victim hits it. The attacker has no direct cache access and cannot read or edit victim queries. Practical effectiveness depends on cache occupancy and replacement policy, so applicability is strongest for moderately shared content such as shared documents, RAG corpora or code skill files. Transfer to much smaller or less capable models reduces precise targeted control.
Implications
Offensively, an unprivileged user can stealthily influence other users’ model outputs without any adversarial text appearing in victim inputs. Attackers can poison shared artefacts to seed persistent, covert behaviour modifications that survive long dialogues and stochastic decoding, and that transfer to other deployed models. This enables cross-user integrity attacks in multi-tenant services, covertly steering responses, inducing misinformation, corrupting code completions or altering retrieval-augmented behaviour. The threat highlights that latency-optimising, position-independent KV reuse can create a practical attack surface for supply-chain style manipulation of model behaviour.
