ScopeJudge keeps offensive AI agents in scope
ScopeJudge tests a pre-execution judge for Large Language Model (LLM) security agents to catch out-of-scope tool calls before they run. On 4,897 calls labelled by penetration testers, static policies failed without the user’s request. Context-aware judging improved recall but raised cost and exposure. An open-weight judge led with F1 0.66 and lower cost.
Offensive agents built on Large Language Models (LLMs) are now let loose on real tools: scanners, shells, cloud SDKs. One stray call to the wrong host and you breach a client boundary, trip production, or void a bug-bounty finding. The awkward bit is scope lives in the brief, not in the tool. The same nmap run is fine on staging and a violation on a neighbour’s subnet. Static safety checklists do not see that line.
What the study did
The work behind ScopeJudge takes the blunt instrument out of the agent’s hands until a cheap, trusted judge signs off each action. Before any proposed tool call executes, a lightweight LLM judge inspects the context and either allows or rejects it. To test the idea, the authors built a benchmark of 100 agent trajectories across 30 temptation-heavy tasks and labelled every one of the 4,897 tool calls. Five professional penetration testers did the labelling with substantial agreement (Fleiss kappa 0.64), and the mean human reference point lands at F1 0.78.
They then ran eight judge models under five transcript strategies, ranging from showing the judge only the static policy to giving it the full raw transcript. They tracked precision, recall and F1, and also priced each configuration by token usage to map a cost–accuracy frontier. Violations were 7.7% of calls, so base rates mattered in evaluation.
The findings
Static policy checks were structurally blind. Without the user’s request, judge recall fell to near zero. Scope really does live in intent. Feeding more context helped, but at a price: higher token cost and a larger prompt-injection surface when you show untrusted outputs or assistant chatter. Among models, an open-weight judge (GLM-5.2) topped the chart with F1 0.66, beating the best proprietary judge at 0.60 while costing roughly one third per call. The paper suggests two practical operating points: a cost-sensitive setup for routine runs and a recall-first mode for high-stakes engagements where a missed violation hurts more than a false stop.
This all rhymes with earlier eras. Packet filters once blocked ports blindly; only when stateful firewalls watched the conversation did intent come into view. Pen testers have long lived by rules of engagement taped to the monitor. The clever twist here is wiring that rulebook into an always-on preflight check for every agent action, with the request as the source of truth.
There are caveats. The dataset is a calibration set that preserves natural violation rates, not a positives-enriched torture test. A small fraction of judge evaluations may involve models that also appeared among source agents, a possible self-preference bias. And while richer transcripts boost accuracy, they also widen the prompt-injection aperture. The core question now is familiar: how much context can we afford to show the gatekeeper before the gate starts talking back?
Additional analysis of the original ArXiv paper
📋 Original Paper Title and Abstract
ScopeJudge: Cost-Aware Pre-Execution Gating for Offensive Security Agents
🔍 ShortSpan Analysis of the Paper
Problem
The paper studies how to prevent autonomous offensive-security agents from executing out-of-scope tool calls that can breach engagement boundaries, disrupt production, or void bug-bounty findings. Scope is not a fixed safety policy but is declared in the user request and must be inferred from intent and context. In offensive security the same action can be acceptable or a violation depending on the target and situation, so static policies alone cannot enumerate all relevant boundaries. The work asks how to monitor and gate each proposed tool call before it executes, balancing cost, accuracy and adversarial risk.
Approach
The authors propose pre-execution gating: a cheap, trusted LLM judge inspects each proposed tool call from a stronger, swappable agent and accepts or rejects it before execution. They construct ScopeJudge, a benchmark comprising 100 labelled trajectories (4,897 tool calls) produced by running agents on a suite of 30 tasks engineered to tempt scope violations. Five professional penetration testers independently labelled every call; Fleiss kappa across all calls is 0.64, establishing a human reference F1 of 0.78. Eight judge models are evaluated under five transcript strategies that vary how much context the judge sees: Static Policy, Intent, Intent Plus Tool Calls, Intent Plus Output Summarised, and Full. The evaluation reports precision, recall and F1, and also computes a token-cost measure that accounts for tokens shown to the judge and per-token price, thereby mapping cost-accuracy trade-offs.
Key Findings
- Static policy monitoring is structurally insufficient: without seeing the user request judge recall collapses to near zero, showing that scope must be recovered from the user's intent.
- Providing more context increases accuracy but raises token cost and the judge's prompt-injection surface; the five transcript strategies expose a clear cost-accuracy Pareto frontier.
- Open-weight judges performed strongly: GLM-5.2 achieved the highest F1 reported, 0.66, outperforming the best proprietary judge at 0.60 while costing roughly one third per call.
- Scope violations are rare even under temptation: 7.7% of calls in the calibration set, giving roughly 380 positive instances across 4,897 calls, so base-rate effects matter for evaluation.
- Human graders agree substantially but not perfectly: 88.1% of calls were unanimous, and mean per-grader F1 against the majority is 0.78, which serves as a practical reference point for judge performance.
Limitations
The dataset is a calibration set of 100 trajectories drawn to preserve natural violation rates rather than to enrich positives; claims are made against this corpus. The tasks come from a separate suite and trajectories were produced by a mixture of agent families; a small fraction of judge evaluations may involve models that also appeared among source agents, a potential source of self-preference bias. The study focuses on pre-execution gating and does not claim to address all deployment complexities or adversarial attacks against judge models beyond noting increased prompt-injection risk when showing untrusted outputs.
Implications
An attacker controlling or influencing an agent could exploit weak monitoring by tailoring requests or tool-call traces so a static policy accepts harmful calls; request-conditioned judges are therefore required. Providing richer transcript context improves monitoring but also increases attack surface and cost, so an adversary might attempt prompt-injection via tool outputs or assistant reasoning to evade a judge. The paper highlights two practical operating points for deployments: a cost-sensitive configuration for routine use and a recall-first configuration where missed violations are especially costly, guiding how an attacker might focus on scenarios where monitors prioritise cost over recall.
