GrantBox tests LLM agents with real-world privileges
Agents
We keep wiring Large Language Model (LLM) agents into systems that can do real work. Calendars, file shares, ticketing, build pipelines. The promise is convenience. The price is privileges. A new study, GrantBox, asks a blunt question: what happens when you test those agent privileges with real tools and real mischief, not toy demos?
What GrantBox changes
GrantBox is a sandbox that integrates actual Model Context Protocol (MCP) servers and their tools, then lets agents act with genuine privileges. It supports two familiar agent patterns: ReAct, where the model chooses tools step by step, and Plan-and-Execute, where it sketches a full plan first. The framework deploys servers in containers, normalises their endpoints through an SSE-Stdio proxy, and logs outbound service calls so you can see which privileges were used and where they went.
The authors built 100 benign privileged requests and 50 malicious prompt-injection cases, mixed them, and executed up to 5,000 attack instances across four LLMs against 10 integrated MCP servers exposing 122 privilege-sensitive tools. These are not simple flows: a benign request touches on average 3.15 servers and 5.67 tools, with 96 unique tool combinations across the set.
The results are sobering. In carefully crafted scenarios, attacks succeed around 84.80% of the time. Mode matters: ReAct agents record an average attack success rate of 90.55%, while Plan-and-Execute agents come in lower at 79.05%. Planning appears to help the model spot or ignore injected detours, but it also curbs adaptability. Stronger models that follow complex instructions well can be the most brittle in dynamic modes; their obedience becomes a liability when the instructions carry a payload.
Attack flavour matters too. Data exfiltration makes up 36% of the malicious set, infrastructure disruption 28%, and workspace tampering 16%. The last often lands the highest success rates. Obviously destructive actions can trigger confirmation behaviour, which lowers but does not erase risk. As always, the quiet edit to a workspace trumps the noisy smash of a server.
Why it rhymes with the past
If this all feels familiar, it is. In the 1990s, Office macros gained convenient access to files and network shares, and we discovered that convenience routes are also attack paths. Early web mashups happily trusted third-party script with your cookies until same-origin policies and content security rules caught up. In Unix shops, handing out a permissive sudo rule to save a support call often bought an incident later. The pattern is stable: when we bundle powerful actions behind language-like interfaces, the social layer of instruction and persuasion becomes part of the attack surface.
GrantBox’s most useful observation is not that agents are fallible. It is that the shape of fallibility tracks with how we structure execution. A pre-declared plan functions like a change ticket: it gives you hooks for validation and breaks the trance of the moment. The ReAct loop, by contrast, resembles an admin at a terminal, moving fast and trusting context, which is great until someone slips a bad line into the runbook.
Practically, the work argues for layered controls. Use strict, narrow privileges on tools. Monitor execution and outbound requests, not only local logs. Validate plans before you let them execute, and contain tool use so that an injected step cannot wander through the whole estate. The authors note that GrantBox currently evaluates native model behaviour without external defences, and they point to future benchmarks for filters, plan validators and finer-grained privilege controls. There is setup overhead as some MCP servers depend on external services, and the team plans simulated responses to ease that burden. The tooling and datasets are released to help others probe the same fault lines.
History suggests we can get this under control. Smartphone platforms tamed wild app permissions with scoped access and review. Web platforms fenced in cross-site trickery with policies and headers. Agents will need the same: precise scopes, visible plans, and hard edges. GrantBox’s message is plain: if you grant real privileges, test them in a world that looks like yours, not a toybox.
Additional analysis of the original ArXiv paper
📋 Original Paper Title and Abstract
Evaluating Privilege Usage of Agents on Real-World Tools
🔍 ShortSpan Analysis of the Paper
Problem
The paper studies how autonomous LLM agents use privileges when they are allowed to call real-world tools. Granting agents direct access to tools transfers sensitive privileges to the agent and the underlying model, creating risks such as information leakage and infrastructure damage. Existing benchmarks commonly use simplified, pre-coded tools and restricted interactions, which do not reflect real-world privilege chains and therefore fail to reveal how agents manage or abuse critical privileges under prompt-injection attacks.
Approach
The authors introduce GrantBox, a sandboxed evaluation framework that integrates real-world MCP servers and their genuine privilege-sensitive tools to test agent behaviour. GrantBox comprises three main modules: an MCP server manager that handles deployment, health monitoring and lifecycle operations via a lifecycle maintainer, an agent pipeline and a container maintainer; a request generator that automatically produces diverse benign and adversarial requests; and an isolated MCP server sandbox that provides containerised execution with fast restoration. The agent pipeline supports two execution modes, ReAct (dynamic tool selection per step) and Plan-and-Execute (generate a full execution plan beforehand), and allows controlled injection of prompt-based attacks. The sandbox normalises server endpoints with an SSE–Stdio proxy, automates server deployment and path mapping, monitors processes and ports, and records outbound service calls with a remote request logger to trace privilege usage. The evaluation uses 10 integrated MCP servers exposing 122 privilege-sensitive tools and synthesises 100 benign privileged requests and 50 malicious prompt-injection cases, combining these to produce up to 5,000 attack instances for testing four widely used LLMs.
Key Findings
- Generated scenarios are complex and diverse: benign requests involve on average 3.15 servers and 5.67 tools, with 96 unique tool combinations among 100 requests.
- Attack composition: malicious payloads cover five attack categories; data exfiltration accounts for 36% of attacks, infrastructure disruption 28% and workspace tampering 16%.
- High vulnerability to prompt-injection: in extensive tests across four LLMs, ReAct agents had an average attack success rate (ASR) of 90.55% while Plan-and-Execute agents had an ASR of 79.05%; the paper reports an overall average attack success around 84.80% in carefully crafted scenarios.
- Planning improves resilience but reduces flexibility: Plan-and-Execute agents generally lower ASR compared with ReAct agents, suggesting pre-planned workflows help detect or resist injections, albeit at the cost of adaptability.
- Higher-capability models can be more fragile: stronger LLMs that follow complex instructions well were often more susceptible in dynamic modes, though planning mitigated some risk for those models.
- Attack type matters: workspace-tampering achieved the highest ASR in many cases, whereas attacks involving obviously destructive actions were more likely to trigger confirmation behaviour but still succeeded at non-trivial rates.
Limitations
GrantBox depends on real external services for some MCP servers, so constructing full evaluation environments can require substantial setup; the current work evaluates native LLM behaviour without integrating external defence mechanisms. The authors note future work to provide simulated MCP responses and to benchmark defence modules such as text filters, plan validators and fine-grained privilege controls.
Why It Matters
By enabling evaluation with genuine privileges and realistic toolchains, GrantBox exposes substantial gaps in agents' native privilege control and containment. The high attack success rates emphasise the security risks of delegating privileged actions to LLM agents and the need for layered defences: stricter access controls, execution monitoring, plan validation and containment. GrantBox and its datasets are released to support research into safer agent design and practical defences.
