BadSkill poisons agent skills with embedded backdoors

Problem

This paper studies a supply-chain threat in agent ecosystems where third-party skills can bundle learned model artefacts. Unlike prompt injection or ordinary plugin misuse, a malicious skill can hide a backdoor inside its embedded model so that seemingly benign skill invocations activate a hidden payload only when routine, structured skill parameters satisfy an attacker-chosen semantic conjunction. This model-in-skill threat surface is harder to spot with code inspection because the activation logic is encoded in model weights rather than explicit symbolic rules.

Approach

The authors formulate BadSkill, a two-stage backdoor pipeline tailored to skills that expose structured parameter interfaces. Stage one fine-tunes a compact classifier head on canonicalised structured invocations using a composite training objective combining weighted binary classification loss, a margin-based separation term, and a poison-focused upweighting term to preserve rare trigger signals. Training data mixes clean examples, trigger-positive examples produced by a trigger transformation that rewrites specific argument fields, and hard negatives that match all but one trigger field to force learning of the full conjunction. Stage two packages the trained classifier inside the skill artifact; at runtime the skill parses incoming queries, computes a trigger score, and routes execution to either the advertised benign branch or a hidden payload when the score exceeds a threshold. The threat model assumes a grey-box adversary acting at skill-development time with access to public interface details and a public pre-trained model, but not to host private prompts or runtime configuration. Evaluation uses an OpenClaw-inspired simulated skill environment to preserve installation and execution semantics while enabling controlled experiments.

Key Findings

• High effectiveness across models: Evaluated on eight instruction-tuned models from five families spanning 494M to 7.1B parameters, BadSkill achieves up to 99.5% average attack success rate (ASR) across eight triggered skills while keeping strong benign accuracy on negative-class queries.
• Poison efficiency: In poison-rate sweeps, ASR rises sharply between 1% and 3% poison and largely saturates by around 7%; a 3% poison rate yields 91.7% ASR on the standard test split.
• Benign-side preservation: Clean skills show 100% benign accuracy baseline; after poisoning the largest benign accuracy drop observed is 4.2 percentage points and many model-skill pairs retain perfect benign accuracy, showing the backdoor can remain covert under routine use and lightweight inspection.
• Trigger design matters: Compositional triggers spanning two to three structured parameters give the best trade-off, consistently achieving average ASR at or above 95%; single-field triggers are too weak and very large conjunctions may overfit.
• Robustness and training design: The attack remains effective under five types of text perturbation, though character-level corruption and heavy typos degrade performance more than surface reorderings. Ablations show the margin and poison-focused loss terms improve stability and ASR relative to plain classification loss.

Limitations

Experiments are limited to models up to 7.1B parameters and a lightweight simulated environment rather than production agent stacks. The study uses English triggers and a benign canary payload, does not evaluate real-world payload variety, and does not include a dedicated evaluation of defensive mechanisms.

Implications

An attacker can publish a seemingly legitimate skill that, after installation, behaves normally while containing a backdoored model that routes specific combinations of ordinary-looking parameters to a hidden payload. Because the trigger is encoded in model weights and can be realised with a small poisoned fraction of training data, the compromise can be stealthy, robust to moderate input variation, and difficult to detect by code review or simple behavioural checks. The threat therefore enables covert conditional execution and exfiltration or other malicious actions whenever the host invokes the skill with attacker-aligned parameter conjunctions, highlighting a distinct model supply-chain risk in agent ecosystems.