Whitelist prompts to harden agentic LLMs

Defenses

Published: Wed, Sep 24, 2025 • By Adrian Calder

Whitelist prompts to harden agentic LLMs

Researchers propose LLMZ+, a prevention‑first defence that enforces contextual prompt whitelisting for agentic Large Language Models (LLMs). The approach blocks unauthorised or out‑of‑scope prompts before they reach the agent, showing near zero false positives and negatives in test settings with larger models, while preserving legitimate workflows. Practical tradeoffs and upkeep remain.

Researchers introduce LLMZ+, a practical prevention layer that enforces contextual prompt whitelists for agentic Large Language Models (LLMs). It matters because agentic LLMs are high value targets: they juggle nondeterministic reasoning and have privileged access to data and APIs, so a successful jailbreak can hit both operational and information security.

For practitioners and decision makers the stakes are straightforward. Many current defences try to detect and block malicious intent after it appears. LLMZ+ flips that model by defaulting to deny and only allowing messages that match explicit, business‑centred criteria, shrinking the attack surface before the agent sees the input.

How it works: a guard component, itself driven by an auxiliary LLM, performs ingress and egress checks. Inbound messages must be interpretable, resemble authorised conversational patterns, and stay relevant to the agreed use case. Outbound replies get screened against the same contextual rules and can be constrained further with a data exposure or data loss prevention layer. Optional contextual retrieval augmented generation helps the guard decide which data categories are allowed.

The team tested the idea on an on‑prem fintech chatbot and reported strong resilience to common jailbreak prompts. Larger guard models — examples include 70B and 405B parameter variants — delivered near zero false positives and false negatives in the described experiments. Smaller models, such as an 8B variant, needed simple prefilters like message length checks to reach comparable performance.

Impact and limits are clear. LLMZ+ can meaningfully reduce prompt injection risk and support policy enforcement, but it is not a silver bullet. It does not replace network hardening or traditional software security, adds latency and compute overhead, and requires ongoing whitelist governance to avoid blocking legitimate use or becoming brittle as contexts evolve.

What to do next

Start by trialling a deny‑by‑default guard in non‑critical flows, use a larger model for the guard where feasible, add DLP and contextual RAG, and log guard decisions for audit and tuning. Maintain a governance process to review and expand allowed contexts. Expect an operational trade off: better prevention for a bit more maintenance and compute, but a substantially smaller attack surface.

Additional analysis of the original ArXiv paper

📋 Original Paper Title and Abstract

LLMZ+: Contextual Prompt Whitelist Principles for Agentic LLMs

Authors: Tom Pawelek, Raj Patel, Charlotte Crowell, Noorbakhsh Amiri, Sudip Mittal, Shahram Rahimi, and Andy Perkins

Compared to traditional models, agentic AI represents a highly valuable target for potential attackers as they possess privileged access to data sources and API tools, which are traditionally not incorporated into classical agents. Unlike a typical software application residing in a Demilitarized Zone (DMZ), agentic LLMs consciously rely on nondeterministic behavior of the AI (only defining a final goal, leaving the path selection to LLM). This characteristic introduces substantial security risk to both operational security and information security. Most common existing defense mechanism rely on detection of malicious intent and preventing it from reaching the LLM agent, thus protecting against jailbreak attacks such as prompt injection. In this paper, we present an alternative approach, LLMZ+, which moves beyond traditional detection-based approaches by implementing prompt whitelisting. Through this method, only contextually appropriate and safe messages are permitted to interact with the agentic LLM. By leveraging the specificity of context, LLMZ+ guarantees that all exchanges between external users and the LLM conform to predefined use cases and operational boundaries. Our approach streamlines the security framework, enhances its long-term resilience, and reduces the resources required for sustaining LLM information security. Our empirical evaluation demonstrates that LLMZ+ provides strong resilience against the most common jailbreak prompts. At the same time, legitimate business communications are not disrupted, and authorized traffic flows seamlessly between users and the agentic LLM. We measure the effectiveness of approach using false positive and false negative rates, both of which can be reduced to 0 in our experimental setting.

🔍 ShortSpan Analysis of the Paper

Problem

Agentic large language models carry privileged access to data sources and application programming interfaces, making them a high value target for attackers. Unlike conventional software, agentic LLMs rely on the AIs nondeterministic path selection to reach a final goal, which creates substantial operational and information security risks. Traditional defence methods largely rely on detecting malicious intent and blocking jailbreak or prompt injection attempts, but such approaches struggle against adaptive adversaries and require continual updates. The paper proposes a prevention based defence, LLMZ+, which enforces a contextual prompt whitelist so that only contextually appropriate and safe messages interact with the agentic LLM, ensuring exchanges stay within predefined use cases and boundaries. The aim is to streamline the security framework, improvelong term resilience and reduce ongoing security maintenance. The authors report empirical results indicating strong resilience against common jailbreak prompts, preservation of legitimate business communications, and zero false positives and false negatives in their experimental setting.

Approach

LLMZ+ implements a guard prompt that sits within a whitelist style architecture and uses an auxiliary LLM as a whitelist guard for both inbound prompts and outbound replies. In line with a firewall like principle, the system denies by default and only permits messages that satisfy strict criteria. Ingress checks require messages to be fully interpretable by the guard prompt, to resemble natural customer service conversations, and to be relevant to the agents business use case. Egress checks ensure outbound messages remain consistent with the intended business use case. An information scope layer or data loss prevention mechanism can be added to restrict what the agent may disclose. A contextual retrieval augmented generation component may be used to inform the guard about allowed data categories. The result can be a binary yes no decision or a risk score from 0 to 10, guiding decision thresholds. The evaluation used a commercial fintech chatbot in a highly regulated retail market and on premises Llama based models, including 3.18 8B, 3.70 70B, and 3.1405 405B configurations. The primary task was a simple customer login and balance confirmation scenario, with experiments comparing legitimate messages and jailbreak style prompts. The study also discusses pre processing steps and potential deployment considerations for latency and resource use, and notes that larger models improve detection performance while smaller models may require additional filtering.

Key Findings

LLMZ+ provides strong resilience against common jailbreak prompts while preserving legitimate business communications.
Using larger models (for example 70B and 405B variants) enabled near zero false positives and near zero false negatives across a range of decision thresholds in the tested configuration.
For the smaller 8B model, some false positives occurred unless a simple pre filtering step such as message length filtering was employed, which combined with a larger model achieved zero false rates across thresholds 1 to 10 in the reported setup.
The guard prompt plus an information scope or simple data exposure controls effectively narrows the allowable content, reducing the attack surface and blocking malicious prompts before they reach the agentic LLM.
The approach is not a replacement for a complete information security architecture but an additional safeguard that can be deployed alongside existing controls and can be extended with contextual RAG to enhance assessment of responses.

Limitations

The study notes that real world effectiveness may vary as attack techniques evolve and contexts change, and maintaining the whitelist may require ongoing effort to avoid overly constraining legitimate use. LLMZ+ focuses on prompt based threats and does not address other layers of the deployment stack such as network vulnerabilities or traditional software exploits. Deployment considerations include potential latency and increased resource use, particularly if parallel processing is used to improve user experience. The findings come from on premises testing in a fintech setting with specific model configurations, so results may differ in other environments or with different LLMs or use cases.

Why It Matters

LLMZ+ represents a prevention based defence for agentic LLMs by enforcing contextual prompt whitelisting, potentially reducing the attack surface from jailbreaks and prompt injections while preserving legitimate workflows. The approach offers a streamlined security framework that can lower ongoing maintenance and be adapted to business specific contexts. In practice it supports governance and policy enforcement around AI systems by providing explicit context boundaries and real time evaluation during inference. The study suggests the method is scalable with larger LLMs and can be complemented by additional safeguards such as RAG or internal content restrictions, representing a meaningful advance in securing agentic AI systems.

Attribution Original paper on arXiv