Deterministic guardrails block LLM prompt injection
A new paper moves authority out of the Large Language Model (LLM) and into deterministic code, stopping low-trust text from authorising actions. Using an unmodified Gemma-4-26B and a “cascade” wrapper, defended rate jumps from 27% to 94% with a small quality hit, and holds 87% under adaptive red-teaming. Attribution also improves sharply.
Prompt injection thrives on a simple fact: instructions and data share one token stream, so a Large Language Model (LLM) treats a phishing email and a system directive as equally loud unless you force it not to. This paper stops pretending the model will behave and shifts the power elsewhere.
What they built
Authority to act lives outside the model in deterministic code. Inputs arrive tagged by a trust “ring” (think system, user, content, web). A non-model monitor enforces simple rules: ring assignment happens out of band, combining values never raises integrity, actions require thresholds, writes are stamped, and the operation is bound to the highest-trust ring present. The model itself runs inside a conditioned cascade. Each lower-trust channel is passivated in descending order: embedded commands are stripped while factual content survives, conditioned on higher-trust context. A wrapper tags each passivated block with a nonce and ring label, then a final perform call executes. No model weights are changed; prompts for passivation and the wrapper are tuned via gradient-free search.
The boundary that matters for operators is provable: no lower-trust input can rebind the operation or authorise an action, by construction over a finite integrity lattice. The rest is measured, not proved. On a one-shot held-out set with Gemma-4-26B, the cascade raises the genuine-leak defended rate from 27% to 94% at roughly a 4% quality cost (Qrel = 0.96). Under adaptive red-teaming, the proof still holds and the defended rate lands at 87%; successful attacks only changed generated text (format or language), not authority. Attribution improves from 0% to 92%, and the system follows higher-trust sources on conflicts.
Does it matter
Yes, if your LLM can press buttons. The monitor makes action authorisation a closed door; content alone cannot unlock it. That cuts off whole classes of actuator abuse. But do not confuse this with a cure for content steering. The defence against text leaks is empirical and depends on prompt-tuned passivation, bespoke to a model and ring scheme. Residual leaks persist, sample sizes are modest, and attackers will target the ranking, the passivation prompts, or the wrapper. For pure text applications, format flips and subtle steering still bite, just without a clean “gotcha” signal.
The practical questions are where this will creak in the real world: who assigns rings, how brittle is passivation across model updates, and what happens when enterprise context gets messy. The boundary proof is solid where it applies. The rest is engineering, measurement and upkeep.
Additional analysis of the original ArXiv paper
📋 Original Paper Title and Abstract
Composable Trust for Language Models: A proven boundary and a measured defense
🔍 ShortSpan Analysis of the Paper
Problem
The paper addresses prompt injection and integrity in large language models where instructions and data share the same token stream, so untrusted text can steer generation. The goal is to ensure that authority to act is determined by the source's trustworthiness rather than by content, preventing low-trust inputs from re-binding the operation or authorising external actions while still allowing lower-trust material to inform answers in an attributed way.
Approach
The authors place authority outside the model in deterministic code: inputs are labelled into a finite integrity lattice of rings (for example system, user, content, web). A trusted monitor enforces five invariants including out-of-band ring assignment, a low-water-mark meet rule when values are combined, action thresholds, stamped writes, and binding the operation to the highest ring present. Unmodified models run inside a conditioned cascade: each below-system channel is passivated in descending trust order, passivation strips embedded commands while preserving factual content conditioned on higher-trust context, and a wrapper tags each passivated block with a nonce and ring label. The only calls to the model are per-ring passivation and a final perform call; the prompts for passivation and the wrapper are tuned by a gradient-free, multi-objective prompt search that preserves short, principle-based prompts without changing model weights. Evaluation uses an unmodified Gemma-4-26B model, a bespoke attack dataset of delivery forms crossed with payload goals, and human-validated automated judges to measure genuine text leaks, attribution and task quality.
Key Findings
- The authority boundary is provable: exhaustive enumeration over the finite lattice and a monotonicity argument show that no lower-ring input can rebind the operation or authorise an action, so action-authorization failures are 0 by construction.
- The measured in-pipeline defence is large: on a held-out one-shot evaluation the cascade raised the genuine-leak defended rate against instruction injection from 27% to 94% while retaining roughly 96% relative clean-task quality (Qrel = 0.96).
- Under adaptive red-teaming the proved authority boundary still held unconditionally and the measured defended rate declined modestly to 87%; every successful adaptive attack only changed generated text (format or language) and did not gain authority.
- The cascade improves provenance handling: attribution of web-sourced facts rose from 0% with the base to 92% with the cascade, and on cross-ring conflict the system follows the higher-trust source consistently.
Limitations
The paper proves only the authority decision; the defence against content steering and text leaks is empirical and requires prompt tuning. Residual text leaks remain (for example format or language changes), sample sizes are modest, and the measured rates are model- and configuration-specific. The passivation and wrapper reduce but do not formally guarantee zero text leakage and require re-tuning for different models or ring definitions.
Implications
Offensive implications are bounded: an attacker cannot use injected content to reassign authority or to authorise actions, so exploits that would trigger external actuators or change privileged operations are prevented. However, attackers can still sometimes steer the model's textual output, causing misformatted, translated or otherwise altered answers and surfacing content that may mislead readers; such text-only leaks give no direct signal of success and do not grant control over actions. Attackers may therefore focus on finding weaknesses in ranking, passivation prompts, wrapper tagging, or the tuning process and may mount adaptive attacks to raise the residual text-leak rate; these are measurable but not proved away by the architecture.
