Single Rule Poisons Policy-Aware RAG Agents
New research on a Policy-Aware LLM-RAG for battlefield IoT shows a single injected policy rule can dominate retrieval and corrupt the Large Language Model’s context across most operator prompts. The authors propose CLD-KB, a lightweight dual-detector that flags poisoned rules using category-aware signals, outperforming common anomaly detectors with millisecond overhead.
We have been so busy staring at prompt injection that we missed the quieter door: the knowledge base. A new study of a Policy-Aware Retrieval-Augmented Generation (RAG) pipeline for Internet of Battlefield Things mission control shows how little an attacker needs to nudge an agent off course. Not prompt tampering. Just one politely written rule added to the policy store.
The target is a policy-aware Large Language Model (LLM) agent that grounds decisions on a 60-rule knowledge base split into three categories: Workflow, Rules of Engagement, and Capability. The system uses nomic-embed-text for retrieval and runs Llama 3.2 locally. The authors craft a Query-Agnostic Semantic Retrieval Poisoning attack: a rule stuffed with high-frequency operational vocabulary from all three categories so its embedding sits near the centroid of typical operator queries. That placement wins retrieval across many prompts without seeing any of them.
The attack
In tests, a single injected rule — only 1.6% of the knowledge base — corrupted the LLM’s context 85% of the time, affecting 17 of 20 prompts. Impact saturates around 7.7% poisoning, where the context averages roughly 2.65 poisoned rules; more injections add little. The trick is not exotic vector calculus but social engineering for embeddings: sound like everything, to everyone, so the retriever picks you first.
This has a familiar ring. In the 1980s, small mis-specified rules in expert systems like XCON quietly flipped large swathes of configuration advice. And in 2008, DNS cache poisoning taught us that a tiny, well-placed record could misroute a universe of queries. Here, the embedding index plays the cache: compromise a few entries and you steer the conversation.
The filter
Defence comes as CLD-KB, a light, in-line filter between retriever and LLM. It pairs a One-Class SVM boundary detector with a Member-Based Category Spread test that checks each candidate rule’s minimum distance to members of all three categories. The retriever proposes ten candidates; CLD-KB passes three verified rules onward and blacklists anything flagged. On the authors’ data, CLD-KB achieved perfect rule-level detection and preserved 100% context integrity from 1.6% to 25% poisoning. Baselines struggled: DBSCAN and LOF failed; K-Means and Isolation Forest were partial (F1 around 0.68–0.70); One-Class SVM alone caught everything but threw a false positive (F1 0.98). Overhead was about 7 ms per mission, so edge-friendly.
The catch: this result leans on structure. The spread signal assumes a three-category taxonomy and a 60-rule base. Flat or fast-changing stores may blunt it, and a patient adversary with prompt knowledge could adapt. Still, the rhyme is clear: the layer you trust to be neutral often isn’t. Retrieval is policy by other means; treat it with the same suspicion you reserve for prompts.
Additional analysis of the original ArXiv paper
📋 Original Paper Title and Abstract
Knowledge Base Poisoning Attacks and Defense for Policy-Aware LLM-RAG Framework
🔍 ShortSpan Analysis of the Paper
Problem
This paper analyses the security of Policy-Aware LLM Retrieval-Augmented Generation (PA-LLM-RAG) for Internet of Battlefield Things mission control, focusing on the integrity of the knowledge base used to ground LLM decisions. It shows that retrieval-augmented systems introduce a novel attack surface at the retrieval layer: an adversary with write access can inject semantically crafted policy rules that are preferentially retrieved and thereby corrupt LLM context and mission actions. The study is motivated by the real-world risk that even small, targeted compromises of a rule base can produce unsafe or incorrect behaviour in safety-critical autonomous systems.
Approach
The authors design a Query-Agnostic Semantic Retrieval Poisoning attack and evaluate it against an edge-deployed PA-LLM-RAG pipeline built from a 60-rule structured knowledge base partitioned into three categories: Workflow, Rules of Engagement, and Capability. The system uses nomic-embed-text embeddings and Llama 3.2 hosted locally. The attack crafts poisoned rules that pack high-frequency operational vocabulary across all three categories so their embeddings sit near the centroid of the query distribution, achieving high retrieval rank for any operator prompt without knowledge of runtime prompts. Defences are implemented in CLD-KB (Cyber-Layered Defence for Knowledge Base), a dual-detector filter placed between retrieval and the decision LLM. CLD-KB combines One-Class SVM boundary detection with a Member-Based Category Spread analysis that measures minimum distances to members of each category. The retriever returns a candidate pool of 10 rules and the defence selects three verified rules for the LLM context, blacklisting flagged rules permanently.
Key Findings
- The query-agnostic attack achieves 85% LLM context corruption from a single injected rule, corresponding to a 1.6% poisoning rate, affecting 17 of 20 evaluated prompts.
- Attack impact saturates at approximately 2.65 poisoned rules per LLM context around 7.7% overall poisoning; further injections provide limited extra benefit up to 25% poisoning.
- CLD-KB attains perfect rule-level detection on the evaluated data: Recall = 1.000, Precision = 1.000, F1 = 1.000, with zero false positives across 24 crafted poisoned rules.
- CLD-KB preserves context integrity across poisoning levels: 100% Context Integrity Rate from 1.6% to 25% poisoning, preventing any poisoned rule from reaching the decision LLM in tests.
- By contrast, baseline detectors fail or perform partially: DBSCAN and LOF failed to detect poisoned rules; K-Means and Isolation Forest produced partial detection (F1 around 0.68 to 0.70); One-Class SVM alone had full recall but one false positive (F1 = 0.98).
- CLD-KB adds only about 7 ms overhead per mission, suitable for edge deployment in real-time orchestration.
Limitations
Results rely on a three-category, 60-rule structured knowledge base; the Member-Based Category Spread signal may not generalise to flat or differently structured KBs without recalibration. Defence training is performed once at startup and may require incremental retraining in dynamic environments. Experiments use a Gazebo simulation and a single mission scenario; broader mission types, larger KBs, hardware-in-the-loop validation, and multi-modal KBs remain future work. The paper also notes that a targeted adversary with prompt knowledge or adaptive strategies could overcome the evaluated ceiling.
Implications
An adversary who gains write access to a policy knowledge base can, with very little effort, steer retrieval for a wide range of operator queries and corrupt mission decisions without interacting with runtime prompts. The query-agnostic construction means a single crafted rule can influence many mission prompts, potentially causing unsafe engagement or asset misallocation. Attack vectors include compromised synchronization channels, untrusted operators, or supply chain insertion. CLD-KB demonstrates that taxonomy-aware filtering can detect such injections before they reach LLM decision logic, but defenders must ensure KB structure, retraining processes, and provenance controls are robust because attackers only need minimal KB compromise to achieve large impact.
