GrantBox tests LLM agents with real-world privileges
GrantBox puts Large Language Model (LLM) agents in a sandbox with genuine tool privileges, then hits them with prompt-injection tricks. Across four models, attacks succeed in around 85% of crafted scenarios. Pre-planned agents do better than reactive ones, but at a cost to flexibility. The work releases tooling for broader testing.
We keep wiring Large Language Model (LLM) agents into systems that can do real work. Calendars, file shares, ticketing, build pipelines. The promise is convenience. The price is privileges. A new study, GrantBox, asks a blunt question: what happens when you test those agent privileges with real tools and real mischief, not toy demos?
What GrantBox changes
GrantBox is a sandbox that integrates actual Model Context Protocol (MCP) servers and their tools, then lets agents act with genuine privileges. It supports two familiar agent patterns: ReAct, where the model chooses tools step by step, and Plan-and-Execute, where it sketches a full plan first. The framework deploys servers in containers, normalises their endpoints through an SSE-Stdio proxy, and logs outbound service calls so you can see which privileges were used and where they went.
The authors built 100 benign privileged requests and 50 malicious prompt-injection cases, mixed them, and executed up to 5,000 attack instances across four LLMs against 10 integrated MCP servers exposing 122 privilege-sensitive tools. These are not simple flows: a benign request touches on average 3.15 servers and 5.67 tools, with 96 unique tool combinations across the set.
The results are sobering. In carefully crafted scenarios, attacks succeed around 84.80% of the time. Mode matters: ReAct agents record an average attack success rate of 90.55%, while Plan-and-Execute agents come in lower at 79.05%. Planning appears to help the model spot or ignore injected detours, but it also curbs adaptability. Stronger models that follow complex instructions well can be the most brittle in dynamic modes; their obedience becomes a liability when the instructions carry a payload.
Attack flavour matters too. Data exfiltration makes up 36% of the malicious set, infrastructure disruption 28%, and workspace tampering 16%. The last often lands the highest success rates. Obviously destructive actions can trigger confirmation behaviour, which lowers but does not erase risk. As always, the quiet edit to a workspace trumps the noisy smash of a server.
Why it rhymes with the past
If this all feels familiar, it is. In the 1990s, Office macros gained convenient access to files and network shares, and we discovered that convenience routes are also attack paths. Early web mashups happily trusted third-party script with your cookies until same-origin policies and content security rules caught up. In Unix shops, handing out a permissive sudo rule to save a support call often bought an incident later. The pattern is stable: when we bundle powerful actions behind language-like interfaces, the social layer of instruction and persuasion becomes part of the attack surface.
GrantBox’s most useful observation is not that agents are fallible. It is that the shape of fallibility tracks with how we structure execution. A pre-declared plan functions like a change ticket: it gives you hooks for validation and breaks the trance of the moment. The ReAct loop, by contrast, resembles an admin at a terminal, moving fast and trusting context, which is great until someone slips a bad line into the runbook.
Practically, the work argues for layered controls. Use strict, narrow privileges on tools. Monitor execution and outbound requests, not only local logs. Validate plans before you let them execute, and contain tool use so that an injected step cannot wander through the whole estate. The authors note that GrantBox currently evaluates native model behaviour without external defences, and they point to future benchmarks for filters, plan validators and finer-grained privilege controls. There is setup overhead as some MCP servers depend on external services, and the team plans simulated responses to ease that burden. The tooling and datasets are released to help others probe the same fault lines.
History suggests we can get this under control. Smartphone platforms tamed wild app permissions with scoped access and review. Web platforms fenced in cross-site trickery with policies and headers. Agents will need the same: precise scopes, visible plans, and hard edges. GrantBox’s message is plain: if you grant real privileges, test them in a world that looks like yours, not a toybox.
Additional analysis of the original ArXiv paper
