MemPoison maps persistent memory attacks in LLM agents
MemPoison dissects how persistent external memory lets adversaries steer Large Language Model (LLM) agents long after the initial interaction. Across 1,227 hand-validated cases, it shows write-time filters blunt simple attacks but miss multi-record and trigger-activated poisons. The work argues for adaptive, context-aware memory defences grounded in how retrieval actually composes.
Agents built on Large Language Models (LLMs) now keep notes: user preferences, facts, working summaries, tool outputs. That memory makes them useful across days, not just turns. It also opens a quiet, durable attack surface. Poison goes in through normal channels, looks fine on the way in, sits in storage, then tilts behaviour later when the right context pulls it back out.
How the attacks work
MemPoison is a neat piece of kit for studying this. The authors assemble 1,227 hand-validated cases spanning four target types (fact, instruction, preference, state), three injection channels (user input, tool return, cross-agent), and three memory substrates (flat chunks, a fact store, hierarchical notes). Ten model families are exercised, and behaviour is scored using Clean Accuracy and a Behavioural Corruption Rate (BCR) that tracks how often the agent does the wrong thing under poison. Undefended, Clean Accuracy stays high, yet aggregate BCR often sits north of 50%.
The taxonomy is the clever bit. L1 is direct single-record corruption: one bad write, one bad outcome. L2 is compositional: two or more benign-looking fragments become harmful when co-retrieved; the limit here is how completely the system retrieves. L3 is context-triggered and dormant: a record lies inert until a specific prompt or tool output activates it. Channel and substrate shape risk. Externally mediated channels like tool return and cross-agent produce higher corruption than plain user input. Flat chunk storage is most exposed; fact stores and hierarchical notes blunt some effects, sometimes by diluting specifics through summarisation.
Why common defences miss
Write-time checks and consistency filters knock down L1 nicely. They have a stage-specific frontier, though: pointwise screens do little against L2 and L3 because the harm only appears after retrieval composes records or a later trigger fires. Even an ensemble of write-time defences (MIXed) drops BCR to about 10.7% but leaves residual corruption, especially for compositional and dormant cases.
To see why, the paper introduces Mechanistic Influence Decomposition (MID), a counterfactual probe that decomposes memory influence into single-record effects, pairwise interactions, and trigger-conditioned activation shifts. L1 shows up as a concentrated single-record effect. L2 reveals non-additive pairwise terms. L3 manifests as large activation shifts tied to specific contexts. That is a structural blind spot for filters that only judge records at admission time.
There are boundaries: this is text poisoning via normal channels, not weight-level attacks or side channels; only three storage abstractions; no temporal decay or adaptive summarisation policies. Still, the pattern is clear. Real agents will need defences that reason over provenance and the retrieval-time composition of memory. The open question is how to do that without crushing utility or overfitting to a single substrate.
Additional analysis of the original ArXiv paper
📋 Original Paper Title and Abstract
MemPoison: Uncovering Persistent Memory Threats and Structural Blind Spots in LLM Agents
🔍 ShortSpan Analysis of the Paper
Problem
This paper studies how persistent external memory in LLM agents creates a durable attack surface: adversarial content can be injected through normal interaction channels, retained across turns, and later distort agent behaviour. This matters because modern assistants rely on external memory to preserve continuity for preferences, facts, summaries and task state; persistence that improves utility also enables time-separated attacks that evade same-turn safeguards.
Approach
The authors introduce MemPoison, a benchmark and analysis framework built from a family-first pipeline that yields 1,227 hand-validated cases. Cases span four attack target types (fact, instruction, preference, state), three injection channels (user input, tool return, cross-agent), three memory substrates (flat chunk, fact store, hierarchical notes) and a three-level threat taxonomy: L1 single-record corruption, L2 compositional multi-record corruption, and L3 context-triggered dormant corruption. Ten models were evaluated, including seven open-weight and three closed-weight families. Behavioural metrics include Clean Accuracy and Behavioural Corruption Rate (BCR), plus Ambiguous and Unclear rates. The paper evaluates a suite of practical defences placed at write-time or retrieval-time and introduces Mechanistic Influence Decomposition (MID), a counterfactual diagnostic that decomposes memory influence into single-record effects, pairwise interactions and trigger-conditioned activation shifts.
Key Findings
- Undefended agents retain high clean utility but substantial vulnerability: Clean Accuracy remained high while mean BCRs were large across models, with aggregate BCR often above 50% in the undefended setting.
- Attack difficulty rises L1 to L3: Corruption generally increases from single-record attacks to compositional and trigger-conditioned attacks; L2 is retrieval-completeness limited while L3 activates only under specific contexts.
- Substrate and channel matter: flat chunk storage is most vulnerable, fact stores attenuate some effects, hierarchical notes partially dilute attacks via summarisation; externally mediated channels such as tool_return and cross_agent yield higher corruption than user_input.
- Write-time defences have a stage-specific frontier: simple write-time checks and consistency filters substantially reduce L1 but fail reliably for L2 and L3. A combined defence ensemble achieved the best overall reduction (MIXed reduced BCR to ~10.7% while keeping Clean Accuracy high), yet many defence families left residual corruption, especially for compositional and dormant attacks.
- MID explains failure modes: L1 manifests as concentrated single-record influence; L2 shows non-additive pairwise interaction where harmless fragments become harmful when co-retrieved; L3 shows large activation shifts where records remain inert until a trigger context appears.
Limitations
The study focuses on text-based poisoning via normal channels and excludes parametric weight poisoning, side channels and privileged access attacks. Only three substrate abstractions were tested and real-world memory management variations such as temporal decay, adaptive summarisation and fine-grained access control were not explored. The benchmark is evaluation intensive and built from curated scenario families rather than live deployment traces.
Implications
Offensive security implications are clear: an attacker with no model or database privileges can persistently influence an agent by writing benign-looking fragments or dormant instructions that later activate, or by exploiting lower-trust channels. Pointwise, write-time admission controls are insufficient against compositional and trigger-conditioned strategies. Practical attacks therefore exploit provenance, retrieval completeness and context-conditioned activation. Defenders should prioritise adaptive, context-sensitive, provenance-aware defences that reason across the full memory lifecycle rather than relying solely on static filters.
