Fixed action lists, not guardrails, contain hijacked SOC agents
Reviewing Constrained-Action AI Remediation for SIEM/XDR via a NeMo-Guardrails Proxy · Georgios Koutidis, Nikolaos Kekatos, Tom Nianios et al.
Researchers wrap an alert-triage LLM in guardrail rails and confine it to a closed list of remediation intents with validated arguments. The rails catch most injected log lines but add heavy latency, and the simple, constrained action layer is what actually stops a misled model from touching production hosts.
A log line is attacker-controlled text. Feed it to an LLM that can block IPs or kill processes, and a single crafted alert becomes a way to run commands on your own hosts. The researchers behind this study set out to close that path for AI tools in the security operations centre (SOC).
Their design has two layers. The first, and to my mind the important one, is that the model never writes a command. It picks from a closed list of intents, such as blocking an address or quarantining a file, and supplies an argument. A manager expands that intent into a fixed, OS-specific command template. Each argument is checked on its own: binaries against a whitelist, strings scanned for shell metacharacters.
The second layer is a NeMo Guardrails proxy around the model. It runs three checks. One looks for prompt injection in incoming log content, one blocks personal and infrastructure data in replies, and one refuses off-topic requests. It sits as a drop-in API intermediary, so the application needs no changes.
The headline figure belongs to the proxy. On a released corpus of adversarial and benign log lines, injection recall rose from 25.0% to 94.5% while flagging almost no benign lines. For stock rails with barely any tuning, that is a solid result.
The boring layer did the work
Look closer and the proxy is not carrying the load. A deterministic pre-filter with no model at all simply undid the corpus's Obfuscation" target="_blank" rel="noopener" class="term-link">obfuscations. It recovered every adversarial marker with no false positives. Placed in front of the proxy, it nudged recall slightly higher for negligible cost.
The proxy, meanwhile, is slow. Median response time went from a fraction of a second to the better part of a quarter of a minute. Anything that passes the input check triggers three sequential model calls. The researchers rule out inline control themselves and recommend a human in the loop.
The live red-team exercise tells the same story. Across three simulated attack scenarios drawn from a standard catalogue of real-world adversary techniques, the analyst model mis-extracted a target and drifted from the intended action. The validator rejected the bad target. The closed vocabulary contained the drift. Nothing was dispatched without manual approval anyway.
The limits are clear. The rails were tested against a fixed corpus, not an attacker who knows they are there and adapts. Each request is judged alone, so an injection split across several log lines is out of scope. The claims about critical infrastructure are an architectural argument, not a measured deployment.
The remaining attack surface is where you would expect it. An attacker can craft arguments that pass the schema but point at the wrong asset. A well-formed address that happens to belong to a critical server is still a valid argument. An attacker can also write alerts aimed at persuading the human who clicks approve.
What I take from this is that the rails are a useful filter, but a filter an adaptive attacker will probe. The boundary is the short list of things the model is allowed to ask for.
Further analysis
Problem
Security Operations Centers face two coupled challenges: a high volume of correlated alerts that outstrip analyst capacity and cognitive limits that make triage slow and error prone. Large Language Models are being proposed to triage and in some deployments to trigger active remediations such as blocking IPs or killing processes. This creates new risks because adversarial content in logs can serve as a remote code path through the LLM’s reasoning, causing it to recommend harmful actions, and logs contain sensitive data that must not be exfiltrated to model providers. The paper frames three attacker goals: induce the LLM to follow embedded instructions in telemetry, make it recommend remediation against benign assets, and leak sensitive log contents via model replies.
Approach
The authors propose a constrained-action architecture with two coordinated layers. The first is a SIEM/XDR control plane that correlates events into incidents, surfaces a structured incident object to a SOC-analyst LLM, and confines model outputs to a closed intent vocabulary. Intents are expanded into OS-specific, parameterised command templates by a manager, with argument-by-argument validation, binary whitelisting and metacharacter scanning before any command reaches endpoints. The second layer is a NeMo-Guardrails proxy that wraps the LLM with three rails: an input rail to detect prompt injection in log content, an output rail to block PII and infrastructure-data leakage, and a topical dialog rail to refuse off-task queries. The proxy uses stock rail primitives and is deployed as an OpenAI-compatible intermediary so no application code change is required. Evaluation uses a released SOC-specific adversarial corpus of 200 variants derived from published jailbreak benchmarks and 1,000 benign log lines, plus a live red-team exercise across three MITRE ATT&CK scenarios.
Key Findings
- The NeMo Guardrails proxy raised adversarial injection recall from 25.0% to 94.5% at a 0.1% false-positive rate on the 200 adversarial variants and 1,000 benign logs; most adversarial variants were caught by the input rail.
- A deterministic, zero-LLM pre-filter that inverts the SOC augmentations recovered all adversarial markers at 0.0% false positives and in front of the proxy raised recall to 95.0% while adding negligible cost.
- Latency increased substantially: raw inference median was 0.31s versus 14.14s through the three-rail proxy, yielding a bimodal latency distribution where blocked inputs are fast but passed inputs incur three sequential LLM calls.
- In a live red-team over 54 alerts the SOC-analyst produced 41 action records; the manager’s argument validator rejected a mis-extracted target and the closed intent vocabulary contained intent drift, and none of the actions were dispatched without manual approval.
Limitations
The study uses stock rail primitives with only modest SOC-specific prompt phrasing and does not co-evolve rails with the corpus. Multi-turn injections spanning multiple log lines are not modelled since the proxy treats each request independently. The evaluated implementation targets host and web telemetry; OT and ICS applicability is presented as an architectural fit but not measured on live field devices or protocols. Latency makes inline automated control impractical, so the loop is recommended human-in-the-loop or delayed.
Implications
An attacker who controls at least one log source can attempt prompt injection, induce intent drift, or try to exfiltrate sensitive fields. With the constrained-action stack the attacker’s ability to cause harm is reduced: they cannot supply arbitrary command syntax, only intent and argument, and those arguments are validated and checked against whitelists. Offensive strategies therefore shift toward crafting arguments that pass schema checks or designing multi-request chains that evade per-request rails. The architecture limits but does not eliminate these avenues, so an attacker seeking self-inflicted denial-of-service or lateral-movement opportunities would focus on argument manipulation, ambiguous prompts that evade single-call self-checks, or attempts to induce human operators to approve unsafe actions.
The paper title, authors and abstract
Constrained-Action AI Remediation for SIEM/XDR via a NeMo-Guardrails Proxy
Security Operations Centers (SOCs) for information technology and operational technology share one incident-response problem: a flood of correlated alerts and too few analysts. Large Language Models (LLMs) are increasingly proposed as reasoning engines that triage alerts and, in autonomous deployments, issue commands that block IPs, kill processes, or quarantine files on production hosts. This coupling introduces a new risk: a single adversarial alert can become a remote code path through the LLM's reasoning, leading it to recommend an action the SOC then executes. We present a constrained-action architecture with two coordinated layers: (i) a SIEM/XDR control plane that grounds remediation in correlated host events and confines the LLM's output to a closed intent vocabulary whose templated commands are executed by thin endpoint agents, backstopped by an argument validator; and (ii) a NeMo-Guardrails proxy that wraps the SOC-analyst LLM with input- and output-rail policies, evaluated out-of-the-box against a SOC-specific adversarial corpus we release. The stock proxy lifts injection recall from 25.0% to 94.5% at a 0.1% false-positive rate, and a live red-team exercise confirms that the closed intent vocabulary and argument validator contain the observed LLM failure modes before any command crosses the trust boundary. As an architectural fit (not yet a measured operational-technology deployment), the constrained-action property suits critical-infrastructure settings where a wrong remediation has physical, not merely operational, consequences. The loop is best run human-in-the-loop or delayed: the measured rail latency keeps inline control out of scope.
