RAGSentinel blocks poisoned documents with geometric consensus

Problem

This paper studies knowledge poisoning in retrieval-augmented generation systems, where an attacker inserts adversarial documents into the retrieval corpus so they appear in the LLM context and steer outputs to a targeted incorrect answer. Such attacks require only write access to the database and no access to the deployed model, and they are hard to detect because poisoned documents are crafted to be topically relevant and superficially plausible. Existing post-retrieval defences operate on signals attackers can directly optimise against, such as token outputs, parametric knowledge conflicts or text-level consistency, leaving a gap for robust, black-box defences.

Approach

RAGSentinel is a training-free, label-free post-retrieval defence compatible with black-box LLMs. It uses a separately chosen surrogate encoder to extract query-conditioned hidden-state shifts for each retrieved document: compute a baseline embedding for the query, compute document-conditioned embeddings, take their differences and preprocess these residuals by selecting an active subspace of responsive dimensions, applying adaptive norm clipping, and removing the shared topic direction via mean-centering. A robust global anchor is estimated as the geometric median of the residuals; a local consistency score is computed from nearest neighbours. The two signals are combined with an adaptive weight and a query-adaptive radius is derived to select a trusted, majority-sized context (top ceil(k/2) documents). The final LLM call is a single zero-shot generation using this filtered context. The authors prove a certifiable guarantee: under an honest-majority assumption and a representation-level separation condition, RAGSentinel exactly recovers a poison-free majority-sized context.

Key Findings

• RAGSentinel recognises poisoned documents as geometric outliers in the surrogate encoder's topic-removed residual space; this signal persists even for retrieval-optimised adversarial documents.
• Empirical evaluation on Natural Questions, HotpotQA and MS-MARCO across three LLM families (Mistral-7B, Llama-3.1-8B, Qwen-2.5-7B) and multiple attacks shows consistently low attack success rates while preserving competitive utility accuracy. For example, on MS-MARCO with Mistral-7B RAGSentinel matches Vanilla RAG accuracy (0.82) under no attack and keeps ASR below approximately 0.11 across settings.
• RAGSentinel is robust to variations in k (number of retrieved documents), surrogate encoder choice and pooling strategy, degrades gracefully as the number of poisoned documents approaches the honest-majority threshold, and retains defensive advantage under adaptive attacks that target specific filtering components.
• Computation cost is modest: the defence needs k+1 surrogate forward passes and a single LLM call, incurring modest latency and outperforming several baselines in throughput.

Limitations

The method relies on an honest-majority assumption that poisoned documents are fewer than half of the retrieved set and on a representation-level separation condition for the surrogate encoder. The defence assumes the surrogate encoder is inaccessible to the attacker; if the attacker gains direct query access to that encoder the threat model is unresolved. Adaptive attacks that approximate the surrogate raise attack success but RAGSentinel still outperforms baselines. Extending to higher poison ratios and the stronger surrogate-access attacker are left for future work.

Implications

Offensive implications: an attacker with write access can still attempt tailored evasion strategies. Practical evasion paths identified include generating many poisoned documents to violate honest-majority, approximating the defender's surrogate to perform anchor mimicry, keeping shift norms within clipping bounds, or minimising projection onto the active subspace to camouflage residuals. Adaptive attackers with knowledge of the full pipeline can raise attack success by targeting these components. Incorrect filtering may also yield confidently wrong answers to end users. The defence therefore raises the bar for poisoning but does not eliminate risks when an attacker can control representation-level signals or surpass the honest-majority threshold.