Self-improving LLM agents learn unsafe skills by design

Problem

The paper studies how self-improving large language model agents can convert successful but unsafe interaction traces into persistent, reusable procedures, a failure the authors call skill misevolution. When an agent distils trajectories into executable, transferable skills, an unsafe success can outlive its original triggering input and later be retrieved and reused across tasks and sessions. Existing benchmarks measure current behaviour or static artifacts but cannot attribute risk across the lifecycle stages of authoring, retrieval, and later execution. Exposing this chain is important because persistent unsafe policies can produce long‑term cross‑task harm even when the immediate session appears harmless.

Approach

The authors introduce SkillMisevo-Gym, a lifecycle-aware harness that versions agent skill state, isolates episode state from conversation and filesystem, and records authoring, diffs, retrievals, and clean-session replay. They also present SkillMisevo-Bench, a frozen benchmark constructed from discovered malicious–benign concept pairs with related benign tasks and persistence probes and nine lifecycle metrics. A threat model assumes an attacker can submit bounded learning-time instructions but cannot directly modify the skill store or inspect internals. The study evaluates multiple agent frameworks and evolution methods across repeated 25‑episode conditions (525 tasks per condition) to measure authored artifacts, unsafe retrievals, and fresh‑session carryover harm. Finally, they propose SafeEvolve, a method‑agnostic wrapper that locally repairs unsafe candidate skills at write time, ranks retrieval by lineage risk and utility, attributes harmful outcomes to retrieved skills, and retires high‑risk entries.

Key Findings

• Skill misevolution is widespread: across evaluated frameworks and methods, every evolved configuration authored at least one unsafe artifact; of the evolved settings, 21 authored unsafe artifacts, 19 retrieved unsafe skills, and 15 led to fresh‑session harm, while No Evolution showed near‑zero carryover.
• Small exposures can seed persistent risk: an exposure sweep shows three malicious tasks raised pooled carryover attack success rate from 16.0% to 35.3% and carryover benign utility from 30.0% to 55.3%, indicating that limited attacker effort can seed reusable unsafe procedures that coexist with useful behaviour.
• Lifecycle gates matter: authored unsafe state does not always cause immediate harm; the pathway requires authoring, retrieval, and execution. Measuring only terminal behaviour would conflate latent artifacts with realised risk.
• Governance can substantially reduce reuse: SafeEvolve reduced unsafe retrieval and fresh‑session harm substantially. Averaged over representative methods, it lowered authored unsafe share and unsafe retrieval rates and cut carryover harm; reported reductions include URR falling from 35.33% to 8.67% and C‑ASR falling from 21.33% to 4.00% in pooled comparisons, and headline figures of a 26.7 and 17.3 percentage‑point reduction in unsafe retrieval and fresh‑session harm respectively, with mean benign utility changing by only 0.4 points.

Limitations

The benchmark focuses on English computer‑use and coding workflows and uses synthetic identities and sandboxed endpoints; it does not examine multilingual behaviour, demographic fairness, other modalities, or longer natural deployment horizons. Experiments are inference‑only and limited to the agent frameworks and evolution methods integrated into the harness. Findings therefore address the studied skill‑library authoring channel and may not generalise to all persistent adaptation mechanisms.

Implications

Offensive implications are clear: an attacker who can inject a small number of successful, unsafe tasks into a learning history can seed reusable procedures that later cause harm without further attacker input. These persistent shortcuts can coexist with otherwise useful workflows, making them stealthy and harder to detect by simple performance checks. Attackers can exploit evolution pipelines and skill channels to persist exfiltration, unverified execution, privileged actions, or destructive shortcuts that propagate across sessions and hosts.