Lifecycle Attacks Expose Fragility in Reusable Agent Skills
New research maps how reusable agent skills get compromised across their full lifecycle. Using 327 real-world skills, the study shows attackers win at repository admission, semantic retrieval, planner selection, execution and updates. Sybil-style retrieval hits 93.20% when undefended; metadata tricks outvote prompt injection; runtime guards miss paraphrased flows; unmoderated evolution inherits trust.
Reusable skills are the agent world’s plug-ins: convenient, composable, and a fresh seam for attackers. Most of us fixate on prompt injection or runtime misuse. This study steps back and tests the whole skill lifecycle and finds cracks at every stage. It is a familiar rhythm: once you modularise capability, the attack surface shifts into the supply chain.
How the lifecycle breaks
Start with repository admission. Structural checks catch the obvious, but they still allow a Malicious Admission Rate of 52.9%. Add semantic intent checks with a Large Language Model (LLM) and that drops to 7.9%, at the cost of a 20.0% false positive rate. The lesson is old: syntax is cheap; semantics are messy.
Semantic retrieval is the soft underbelly. In an undefended FAISS pipeline, Sybil-style clones dominate with a 93.20% Attack Success Rate. Padding skills with the right triggers and near-duplicates floods Top-N candidates. A retrieval validation layer helps a lot, pushing many attacks below 10% ASR and cutting Sybil clones to an average of 0.27 in the Top‑5. Yet Trigger Poisoning still lands at 17.50% and Sybil attacks hold 26.59% even after filtering. Once you bias the shortlist, the planner never sees the good options.
Planner selection then falls to persuasion over payload. Fake recommendation attacks hit 45.64% baseline success, while classic prompt injection trails at 4.69%. The planner follows endorsements and glossy descriptions. Pre-validating metadata curbs this decisively and eliminates misleading-description attacks outright. It’s a neat inversion: the attack is in the brochure, not the code.
Runtime execution guards matter, but they are not a panacea. With no monitoring, execution attacks succeed 100.0%. A sandbox and policy engine blocks 87.0% of unauthorised tool calls. Dynamic taint tracking adds friction, but with 66.67% accuracy it misses paraphrased or implicit flows, leaving a residual 23.0% execution success rate. Strings make poor lifelines when meaning shifts.
Evolution is the sleeper risk. Unmoderated updates inherit trust at 100%, so a benign skill can turn bad later. Treating updates like fresh admissions detects 92.5% of malicious revisions, but it pushes false positives to 37.0% and drops benign acceptance to 63.0%. Trust, once conferred, proves hard to recalibrate without breaking the developer experience.
What makes this work useful is not a single killer exploit but a map: repository, retrieval, planner, runtime, evolution. The weak link moves. As with every modular system before, the interesting failures cluster where components meet and where semantics, not syntax, decide the outcome.
Additional analysis of the original ArXiv paper
📋 Original Paper Title and Abstract
Agent Skill Security: Threat Models, Attacks, Defenses, and Evaluation
🔍 ShortSpan Analysis of the Paper
Problem
This paper examines security risks introduced by reusable agent skills across their full lifecycle. Reusable skills package executable workflows, semantic metadata and permission manifests that agents discover and compose dynamically. Because skills can be authored, stored, retrieved, selected, executed and updated independently of the host language model, new attack surfaces emerge beyond conventional prompt injection and runtime tool misuse. The work argues that threats can originate at any lifecycle stage and that existing defences focused on execution or package integrity are insufficient.
Approach
The authors introduce SkillSec-Eval, a modular, lifecycle-aware evaluation framework that decomposes the skill ecosystem into five managed stages: repository admission, semantic retrieval, planner selection, runtime execution and skill evolution. They formalise a skill manifest consisting of structural metadata, semantic metadata, permission manifests and behaviour definitions, and construct an empirical benchmark from 327 real-world skills across 15 capability clusters. Attack classes are instantiated for each stage and measured with stage-appropriate metrics. Defences evaluated include a hybrid repository admission pipeline combining structural rules and LLM-based semantic validation, a retrieval validation layer that filters vector search candidates, a metadata validation layer for planners, a sandboxed runtime with dynamic taint tracking and policy enforcement, and continuous revalidation of updates.
Key Findings
- Vulnerabilities appear at multiple lifecycle stages, not only at execution time: attacks can successfully exploit repository admission, retrieval, selection and evolution stages to influence agent behaviour.
- Repository admission requires semantic intent checks: structural checks alone reduced malicious admission to 52.9% but the hybrid defence dropped Malicious Admission Rate to 7.9% while introducing a 20.0% false positive rate, showing LLM-based semantic validation is necessary but noisy.
- Semantic retrieval is highly exploitable: an undefended FAISS pipeline saw Sybil-style clones dominate retrieval with 93.20% Attack Success Rate. The retrieval validation layer reduced many attacks to below 10% ASR and cut Sybil clones to an average of 0.27 per Top-5, but Trigger Poisoning and Sybil attacks still achieved 17.50% and 26.59% defended ASR respectively, indicating limits of post-retrieval filtering.
- Planner decisions are driven by persuasive metadata: Fake recommendation attacks achieved a 45.64% baseline planner success rate, while prompt injection had a much lower baseline of 4.69%. Metadata validation vastly reduced selection attacks and eliminated misleading-description attacks, demonstrating preprocessing effectiveness.
- Runtime guards are essential but incomplete: without monitoring execution attack success was 100.0%. The proposed runtime blocked 87.0% of unauthorised tool invocations but taint-tracking had only 66.67% accuracy, yielding a residual 23.0% execution success rate because paraphrased or implicit flows bypass string-based taints.
- Evolution is a critical vector: unmoderated updates let 100% of malicious revisions inherit trust. Treating updates as new admissions detected 92.5% of malicious updates but incurred a 37.0% false positive rate and reduced benign update acceptance to 63.0%.
Limitations
Evaluation used a controlled repository of 327 filtered public skills and deterministic LLM prompts; larger or heterogeneous production ecosystems may expose additional vulnerabilities. Longitudinal dependency histories and trust decay over long timeframes were not modelled. The study deliberately excludes attacks targeting the underlying LLM itself such as jailbreaks or adversarial fine-tuning.
Implications
From an offensive perspective, adversaries can abuse the lifecycle to gain execution of unintended behaviour: publish malicious skills or Sybil clones, manipulate vector-space retrieval via keyword stuffing or trigger phrases, deceive planners through fabricated endorsements or misleading descriptions, cause privilege escalation or covert data exfiltration at runtime, and introduce malicious functionality through subsequent updates or dependency compromise. These vectors permit supply-chain style compromises and semantic manipulation that may be difficult to detect without multi-stage, lifecycle-aware defences.
