Adaptive tools amplify agent prompt-injection risk
New research on agentic Large Language Models shows adaptive indirect prompt injection can steer tool-using agents past task-relevance checks. By choosing plausible tools and iterating attack prompts, success rates more than double and utility drops notably. Defences help but leave large gaps, especially on local open-source models.
Agent frameworks keep wiring Large Language Models, or LLMs, into external tools through protocols such as the Model Context Protocol. That is useful for getting real work done. It also broadens the blast radius when an attacker slips hostile instructions into the data those tools return. The latest work on adaptive indirect prompt injection shows how easily that gap between a tidy research demo and a 3am incident can close.
What the research shows
The AdapTools framework targets agentic LLMs that fetch data or call tools. It builds two advantages for the attacker. First, it iteratively refines prompts using agent reasoning traces, then distils those strategies into a reusable library. Second, it selects a tool that looks like the agent’s next sensible move, using a simple transition model and semantic similarity, so task-relevance checks are less likely to block it.
Across evaluated models, this approach roughly doubles attack success. The authors report a 2.13× increase in success rate and a 1.78× degradation in utility. On commercial systems, attack success rises into the teens and twenties in percentage terms depending on the model. Open-source local models fare worse in places, with some groups above fifty per cent. The stealthy tool chooser adds several percentage points on top of the base attack. Runtime defences reduce success for many baselines, but the adaptive method remains effective, cutting less deeply into its rates. Iteration matters: one round delivers modest gains, while multiple rounds push success much higher before levelling off; five iterations are used as a practical balance.
The team also releases a benchmark of thousands of benign agent Trajectory" target="_blank" rel="noopener" class="term-link">trajectories and hundreds of high-authority tools to test against, and they consider both grey-box attackers who can influence a server-side controller and black-box third parties who only control content.
Where this bites in real environments
This is not an abstract language game. It maps neatly to production stacks where an agent reads from endpoints and then fires tools: browse a vendor page, parse an API response, edit a document, file a ticket, run a query, send an email. The attack nests in the returned content and nudges the agent to pick a powerful tool at exactly the moment it seems reasonable. Your comforting system prompt that says use only relevant tools does little when the attacker makes their tool choice look relevant.
At the orchestration layer, think about the broker that maps model intent to tools. This paper’s trick of modelling the next-likely tool for stealth can be inverted defensively. Learn expected tool transitions from your own benign traffic and flag improbable jumps, especially into high-authority tools like payment, code execution or mass data export. You already do something similar for API abuse; do it here.
For endpoints and data pipelines, treat every external response as untrusted. Sanity-check and segment what you inject into the model context. Do not put raw HTML, unvetted JSON, or third-party text straight into the same context window as your privileged instructions. If you must, annotate and isolate it so the agent cannot easily confuse user content with policy.
On model serving, log every tool call with arguments and outcome, and bind those calls to a service that enforces policy. Give each tool its own identity. Keep credentials short-lived and tightly scoped. The dataset used in the paper includes hundreds of high-authority tools; assume attackers will aim there first. Do not let the model see raw secrets; the broker should handle tokens and sign requests on the model’s behalf.
On GPU clusters and inference nodes, the model is not the crown jewel; the tools are. Limit egress from model pods to approved tool endpoints, and block direct Internet access where that is operationally feasible. If you use Model Context Protocol servers or custom connectors, put them on their own auth boundary and apply the same controls you would to any integration tier.
Finally, be cautious with reasoning traces. The attack method learns from them to get better. If you store traces for observability, treat them as sensitive and avoid spraying them into third-party analytics.
There are limits. The results depend on the chosen models, the new benchmark and the attacker model that crafts prompts. Building an adaptive strategy library is costlier than past one-shot attacks. Still, the direction of travel is clear: adaptive, tool-aware injection survives current detectors and makes agents misbehave in ways that look plausible. If you run tool-using agents in production, move your controls to the tool boundary, watch the sequences, and stop assuming a polite system prompt will save you. It will not.
Additional analysis of the original ArXiv paper
