Forged Reasoning Poisons LLM Agent Memory, Beats Defences
New research shows you can poison an LLM agent's reasoning memory, not just its facts. The FARMA technique forges and amplifies prior rationales to bypass keyword and consensus defences, hitting up to 100% success in tests. SENTINEL's layered filters cut this to 0% in some settings without false positives.
Agents with persistent memory are finally useful. They remember past choices, tool calls and why they did things, so they stop repeating themselves. That same memory is now an attack surface. The paper introduces a clean idea: do not poison the agent’s facts, poison its remembered reasoning. If the agent believes it already validated a risky step yesterday, it will probably skip the check today.
How the attack works
The authors call it FARMA: Forged Amplifying Rationale Memory Attack. Phase one is injection. The attacker writes a handful of seed traces into the agent’s memory that look like the agent’s normal logs. The text is neutral and evasive, no obvious red flags, and it matches the expected structure and metadata. These are not instructions; they are fake records of prior decisions. Phase two is amplification. The attacker appends corroborating entries that cite the forged traces, so retrieval pulls them more often and the forgeries look like established precedent.
This is why it bites. Many agents implicitly trust their own past reasoning. Consensus-style defences that look for outliers are easy to fool when the attacker manufactures a consensus. Keyword filters miss carefully phrased logs. In tests across three agent setups — electronic health record import, ReAct question answering, and an online shopping agent — and three Large Language Models (LLMs), FARMA reached up to 100% attack success under baseline conditions. It bypassed simple keyword blocklists and a consensus-deviation defence, A-MemGuard, in the reported trials.
Defence: SENTINEL
The countermeasure is SENTINEL, a write-time pipeline that screens candidate memory entries. Four lightweight layers handle keyword filtering, provenance labelling, taint-threshold checks and pattern screening. The load-bearing piece is the Reasoning Guard, which assigns a forgery score using five weighted signals: provenance anomaly, self-reference mismatch, suspicious pattern density, implausible claim checks and structural anomalies. Deployed in their evaluation, SENTINEL cut FARMA’s success to as low as 0% and showed no false positives across 326 benign agent traces. An ablation confirms the Reasoning Guard does most of the heavy lifting, with the other layers adding defence in depth.
The caveats matter. All results come from simulated environments and a single-agent, single-store design. The signals in SENTINEL are heuristic for speed, and the authors note that adaptive paraphrasing can slip past early versions. Multi-agent shared memory remains untested. They suggest plugging in a semantic LLM judge and running broader deployments next. The core insight stands: agent reasoning history is state, and state you can forge becomes a lever. Consensus is cheap when you write both sides of the argument.
Additional analysis of the original ArXiv paper
📋 Original Paper Title and Abstract
Your Agent's Memories Are Not Its Own: Forged Reasoning Attacks on LLM Agent Memory and Defenses
🔍 ShortSpan Analysis of the Paper
Problem
The paper studies a new attack surface in persistent-memory LLM agents: their stored reasoning history. Agents use persistent memory to retain decision logs, intermediate rationales and prior validations to improve continuity and reduce redundant checks. An attacker who can write to that memory can insert forged reasoning traces that the agent treats as evidence of prior safe behaviour, causing it to skip safety-relevant steps. This undermines agent integrity even when factual knowledge and model weights remain unchanged.
Approach
The authors introduce FARMA, a two-phase memory-poisoning attack that forges the agent’s decision logs. In the injection phase the attacker writes a small set of seed traces that use evasive, neutral phrasing and mimic the agent’s structural format and metadata. In the amplification phase the attacker appends corroborating entries that cite prior forged traces, increasing retrieval probability and making forgeries appear consensual. The threat model assumes write access to persistent memory (for example via a compromised plugin, shared store, or insider) but not control of model weights or prompts. To defend, the authors propose SENTINEL, a five-layer defence pipeline applied at write time. Four lightweight layers perform keyword filtering, provenance labelling, taint-threshold checks and pattern screening. The central component, the Reasoning Guard, computes a weighted forgery score from five signals — provenance anomaly, self-reference mismatch, suspicious pattern density, implausible claim checks and structural anomalies — and rejects entries above a threshold.
Key Findings
- FARMA successfully poisons reasoning traces by avoiding overt keywords and matching agent formatting, causing agents to treat forged entries as prior work rather than direct instructions.
- FARMA’s amplification makes forged traces dominant in retrieval and defeats consensus-based defences that rely on outlier detection.
- In experiments across three agent domains (electronic health record import, ReAct question answering, and an online shopping agent) and three models, FARMA achieved attack success rates up to 100% against undefended agents.
- FARMA bypassed simple defences such as keyword blocklists and a consensus-deviation defence (A-MemGuard) under the tested conditions.
- SENTINEL reduced FARMA’s attack success rate to as low as 0% and produced no false positives across 326 benign agent traces at the chosen threshold.
- An ablation study shows the Reasoning Guard is the primary, load-bearing defence; the other layers provide defence in depth against orthogonal attack variants.
Limitations
Evaluation was performed in simulated environments rather than live production deployments and focused on a single-agent single-store setting; multi-agent shared memory effects remain untested. SENTINEL currently uses heuristic signals for low latency, which preliminary tests show can be bypassed by an adaptive paraphrasing attacker. The authors note that integrating a semantic LLM judge and broader deployment trials are future work.
Implications
Offensively, an attacker with write access can cause agents to skip validation, safety or authorisation steps by implanting forged reasoning that appears to be the agent’s own prior decisions. In binary-decision domains such as clinical imports or transaction approvals this can be highly effective, enabling import of unvalidated records, bypass of compliance checks, or automated decisions based on manufactured precedent. Amplification over time turns isolated forgeries into apparent consensus, making detection by simple statistical defences unreliable.
