PIPES clamps down on agent state corruption

Problem

This paper studies how tool-using agents form beliefs about their environment from external tool responses and shows a novel attack class called state-corruption. Tool responses often omit explicit provenance and semantic expectations for each component, so low-trust content can exceed its informational authority or masquerade as higher-trust data. Attackers can exploit this perception gap to inject environmental claims that corrupt the agent’s internal state, causing actions that appear justified to existing guardrails. The risk is acute for open-ended response fields and for tool interfaces absent from the agent’s training, because the agent cannot reliably detect when a field violates its expected role.

Approach

The authors propose PIPES, Provenance-Informed, Prior-Enforced Screening, a response-level screening mechanism that inspects tool responses before they enter an agent’s reasoning context. PIPES decomposes responses into assessable units and applies two checks: prior consistency, which verifies that a unit conveys the kind of information its response component is expected to provide, and provenance hierarchy, which prevents lower-trust sources from contradicting or overriding higher-trust sources. For structured tools with stable fields PIPES uses static field contracts and deterministic checks for closed formats; for open-ended content it uses contextual priors conditioned on the pre-response trajectory and extractor-provided provenance metadata. An LLM-based assessor reports unit-level violation indicators for prior and provenance; deployments may remove, warn, block or escalate marked units. The experimental instantiation uses atomic removal: any unit flagged as violating is removed prior to agent consumption. Evaluation datasets are derived from VitaBench and AgentDyn benchmarks. The attacker is a reasoning-aware adaptive PAIR adversary that controls a single response component, must preserve the component’s syntactic type, cannot alter user requests or tools, and may iteratively optimise payloads with privileged feedback across up to ten attempts. Target agents include Gemma 4 31B IT and GPT-5.6 Luna.

Key Findings

• PIPES drastically reduces adaptive attack success: averaged across six benchmark splits, attack success falls for Gemma from 84.7% to 2.3% and for GPT-5.6 Luna from 21.6% to 1.1%.
• PIPES preserves or improves benign utility: aggregate benign utility moves from 90.6% to 92.5% for Gemma and from 84.0% to 86.5% for Luna under PIPES.
• PIPES outperforms other defences in many settings: on a static-path VitaBench split PromptArmor and DRIFT had higher attack success rates than PIPES, and on AgentDyn PIPES achieved 1.7% ASR with 100% utility compared with DRIFT’s 5.0% ASR and 81.7% utility.

Limitations

PIPES assesses semantic admissibility and source authority, not factual truth; false statements can pass if they fit priors and provenance. Effectiveness depends on accurate static contracts, informative trajectories and trustworthy provenance anchors. The evaluation covers two benchmarks, two target models, single-surface attacks and an atomic removal policy; coordinated multi-surface attacks, compromised provenance sources, alternate response policies and potential assessor failures are left to future work. Model-based assessments can miss violations or produce false positives.

Implications

Offensive security implications emphasise that an attacker can manipulate an agent’s perceived environment by injecting data-like claims into low-trust fields to bias decision-making and bypass guardrails because resulting actions align with the corrupted state. PIPES narrows this channel by enforcing per-unit semantic and provenance constraints, but attackers might adapt by exploiting multiple surfaces, compromising provenance anchors or crafting payloads that conform to priors while achieving goals. The work highlights that securing agents requires controlling which external claims enter perceived state, not only which instructions they follow.