Draft model veto cuts agent coding failures

Problem

The paper studies how to predict costly failures of agentic large language models used for software engineering before those agents execute actions. In production, agents often emit confident but incorrect code or tool calls whose failure only becomes apparent after execution, causing wasted API calls, retries and latency. Practical constraints in deployed settings include black-box access to the agent (no logits, weights or activations) and prohibitive cost of repeated sampling, which rule out many standard uncertainty estimation methods.

Approach

The authors introduce Speculative Uncertainty (SU), a black-box method that uses a small open-weight draft model to score an agent's already-generated trajectory in a single teacher-forced forward pass. The draft computes token-level speculative surprisal, gap and entropy by evaluating its next-token distribution at each token the agent emitted. Trajectories are split into reasoning and action spans and features are summarised separately over a short lookback window. A sparse linear calibrator maps 50 phase-aware features to a failure-likelihood score for the next action against a verifiable binary objective; in this study the objective is code-execution success (no exceptions). The paper evaluates a pre-execution veto gate policy that blocks an action when predicted success probability falls below a threshold and triggers a cheaper replan instead of executing and retrying.

Key Findings

• The phase split matters: separating reasoning and action spans improves prediction by about 5–6 AUROC points compared with pooled features.
• SU recovers most of the white-box ceiling using only output tokens: distilled drafts yield AUROC around 0.77 versus 0.82 for a white-box upper bound method.
• Applied as a veto gate on two agents (an open-weight Qwen3-Coder-480B and closed-source Claude 3.5 Sonnet), SU reduced per-call execution error rates by 6–8 percentage points and cut average token cost per task by 14–19% in deployment-style evaluations.
• The draft model need not be large or trained with an elaborate objective: a distilled 4B draft was effective, and training on trajectories from one agent transfers to other agents, including closed-source models, improving AUROC substantially over an untrained baseline.
• Gains transfer zero-shot to out-of-distribution benchmarks without retraining the calibrator.

Limitations

Results are derived solely from code-execution as the verifiable objective; no quantitative evidence is provided for other domains. Reported numbers are single-run point estimates without seed variance or confidence intervals. SU assumes short-window, phase-identifiable trajectories and that the draft forward pass is inexpensive relative to agent cost, which may not hold for all workloads. The linear calibrator shows residual miscalibration; the veto gate relies on ranking more than absolute probabilities.

Implications

Offensive security implications centre on the ability to influence or spoof the veto signal and the draft model. An attacker who can manipulate prompts or the agent's output tokens could induce false confidence or false alarms. Reliance on a draft model creates a supply-chain and integrity risk: if the draft or its training data are poisoned or replaced, the speculative signal can be degraded or exploited. Because SU enables pre-execution intervention, adversaries may target the gating policy or attempt calibration attacks to force costly retries or suppress detection of failing actions. These risks warrant threat modelling, careful governance and integrity controls for draft models and gating logic in deployed systems.