Single Rule Poisons Policy-Aware RAG Agents

Problem

This paper analyses the security of Policy-Aware LLM Retrieval-Augmented Generation (PA-LLM-RAG) for Internet of Battlefield Things mission control, focusing on the integrity of the knowledge base used to ground LLM decisions. It shows that retrieval-augmented systems introduce a novel attack surface at the retrieval layer: an adversary with write access can inject semantically crafted policy rules that are preferentially retrieved and thereby corrupt LLM context and mission actions. The study is motivated by the real-world risk that even small, targeted compromises of a rule base can produce unsafe or incorrect behaviour in safety-critical autonomous systems.

Approach

The authors design a Query-Agnostic Semantic Retrieval Poisoning attack and evaluate it against an edge-deployed PA-LLM-RAG pipeline built from a 60-rule structured knowledge base partitioned into three categories: Workflow, Rules of Engagement, and Capability. The system uses nomic-embed-text embeddings and Llama 3.2 hosted locally. The attack crafts poisoned rules that pack high-frequency operational vocabulary across all three categories so their embeddings sit near the centroid of the query distribution, achieving high retrieval rank for any operator prompt without knowledge of runtime prompts. Defences are implemented in CLD-KB (Cyber-Layered Defence for Knowledge Base), a dual-detector filter placed between retrieval and the decision LLM. CLD-KB combines One-Class SVM boundary detection with a Member-Based Category Spread analysis that measures minimum distances to members of each category. The retriever returns a candidate pool of 10 rules and the defence selects three verified rules for the LLM context, blacklisting flagged rules permanently.

Key Findings

• The query-agnostic attack achieves 85% LLM context corruption from a single injected rule, corresponding to a 1.6% poisoning rate, affecting 17 of 20 evaluated prompts.
• Attack impact saturates at approximately 2.65 poisoned rules per LLM context around 7.7% overall poisoning; further injections provide limited extra benefit up to 25% poisoning.
• CLD-KB attains perfect rule-level detection on the evaluated data: Recall = 1.000, Precision = 1.000, F1 = 1.000, with zero false positives across 24 crafted poisoned rules.
• CLD-KB preserves context integrity across poisoning levels: 100% Context Integrity Rate from 1.6% to 25% poisoning, preventing any poisoned rule from reaching the decision LLM in tests.
• By contrast, baseline detectors fail or perform partially: DBSCAN and LOF failed to detect poisoned rules; K-Means and Isolation Forest produced partial detection (F1 around 0.68 to 0.70); One-Class SVM alone had full recall but one false positive (F1 = 0.98).
• CLD-KB adds only about 7 ms overhead per mission, suitable for edge deployment in real-time orchestration.

Limitations

Results rely on a three-category, 60-rule structured knowledge base; the Member-Based Category Spread signal may not generalise to flat or differently structured KBs without recalibration. Defence training is performed once at startup and may require incremental retraining in dynamic environments. Experiments use a Gazebo simulation and a single mission scenario; broader mission types, larger KBs, hardware-in-the-loop validation, and multi-modal KBs remain future work. The paper also notes that a targeted adversary with prompt knowledge or adaptive strategies could overcome the evaluated ceiling.

Implications

An adversary who gains write access to a policy knowledge base can, with very little effort, steer retrieval for a wide range of operator queries and corrupt mission decisions without interacting with runtime prompts. The query-agnostic construction means a single crafted rule can influence many mission prompts, potentially causing unsafe engagement or asset misallocation. Attack vectors include compromised synchronization channels, untrusted operators, or supply chain insertion. CLD-KB demonstrates that taxonomy-aware filtering can detect such injections before they reach LLM decision logic, but defenders must ensure KB structure, retraining processes, and provenance controls are robust because attackers only need minimal KB compromise to achieve large impact.