Trajectory Poisoning Turns Agent Experience Into Backdoors

Problem

The paper studies a new attack surface in self-evolving skill (SES) systems where agent trajectories are distilled into persistent skills. These systems treat promoted evidence as trusted instruction, so an adversary who contributes seemingly ordinary interaction records can cause harmful behaviours to be written into durable skill artefacts. This matters because skills persist across sessions and become part of an agent's operating procedure, making the promotion of experience into instruction a security boundary distinct from retrieval-time or direct file poisoning.

Approach

The authors define PoisonedEvolution, an attribution-oriented trajectory-poisoning attack. The attacker is skill-visible and black-box: they can inspect public skills and submit a bounded fraction of trajectories but cannot view private pools, filters, evolver prompts, or edit the skill bank. Success requires three conditions: C1 Inclusion (poisoned records enter the evolver stream), C2 Evolution Attribution (the evolver credits the behaviour as reusable knowledge), and C3 Realization (the behaviour survives summarisation and merge into the generated skill). The attack transforms attacker-owned trajectories so the target behaviour appears causally relevant to task success or as the natural repair for failure. Payloads are inert canaries (temporary paths, loopback registries, CI flags) encoded into four security-effect families representing confidentiality, integrity, supply-chain, and operational-safety weakening. Primary evaluation uses SkillClaw with SpreadsheetBench-derived sessions; default experiments use batches of n=30 trajectories with k=3 attacker records (10% support). Transfer is tested on the structurally different Trace2Skill pipeline. Metrics focus on Skill Embedding Rate (SER), measuring whether a generated skill newly embeds the target behaviour in its diff.

Key Findings

• High embedding success in SkillClaw: at 10% attacker support across six evolvers and four behaviour families, PoisonedEvolution produced poisoned skill artefacts in 546 of 600 completed trials (91.0% SER).
• Cross-architecture transfer: on Trace2Skill under the same 10% setting, the attack embedded behaviours in 369 of 600 trials (61.5% SER), showing vulnerability beyond a single pipeline.
• Recurrence matters: three consistent attacker records in a 30-record batch reliably cause promotion, whereas a single record is often treated as accidental and much less effective. Increasing k from 1 to 3 raises SER sharply.
• Attribution and encoding determine success: causal framing that links the behaviour to outcomes substantially increases attribution success; domain-aligned, actionable encoding improves Realization. Fragmented or generic insertions perform poorly.
• Payload sensitivity and model variation: families that fit the workflow locus (cleanup, reporting) embed more easily than environment or package-source changes; evolvers varied from 70% to 100% embedding rates, indicating promotion policy matters.

Limitations

The study focuses on trajectory-grounded SES pipelines and stops after a single evolution cycle. It does not execute payloads, perform direct skill uploads, compromise models, or study credential theft. User-scoped dialogue-memory systems, verifier-only designs, and parameter-training coupled systems are outside the empirical scope. Longitudinal multi-cycle amplification effects are not measured.

Implications

Offensively, a modest, bounded contribution of consistent, domain-aligned trajectories can cause SES pipelines to author persistent instructions that embody attacker-chosen behaviours. An adversary who can submit or corrupt a small fraction of traces can turn experience into trusted instruction without ever editing the skill bank or compromising the evolver. This elevates evidence promotion into an exploitable trust boundary and suggests attackers could weaponise shared trajectory pools or upstream telemetry to seed long-lived harmful procedures.