HoneyRoute shunts hostile prompts to LLM honeypots

Problem

This paper studies how to protect LLM inference fleets from probing, jailbreaks, extraction and resource-exhaustion attacks at the serving tier. Existing defences either embed traps inside a single model or deceive at the protocol layer, leaving the API gateway and model farm exposed and providing little feedback to improve detection. The serving tier needs a low‑latency way to isolate adversarial requests without degrading benign user quality and while collecting forensic signals for adaptation.

Approach

HoneyRoute inserts a detection-and-routing layer in front of production models. It uses a frozen 0.8B embedding backbone with small per-domain MLP heads to score each request in real time, a session-level tracker to detect soft escalation across turns, and two honeypot implementations: a rule/prompt-engineered code honeypot that produces short, instrumented replies with near-zero GPU cost, and a dedicated same‑family trained replica that mirrors production responses but records full sessions. Trapped interactions are converted into fingerprints (tactic signature, content objective, behavioural profile) and used to retrain heads in a continuous loop. Experiments use benign production traces, a seven‑domain attack-seed corpus with 13 adversarial transformations, JailbreakBench prompts, and concurrent stress flooding with real payloads. Router heads are tiny (≈0.07M parameters) so daily incremental updates retrain only affected heads while keeping the backbone frozen.

Key Findings

• The router achieves precision 0.903, recall 0.918 and F1 0.911 with AUROC 0.975 and a median added latency of 38 ms, matching 96% of a two‑tier guard‑LLM cascade's F1 at 1/385 of its latency. FPR at 95% recall is 0.
• Under a 13‑transformation red team harness there was 0% evasion; a score‑aware black‑box evader produced 2.5% evasion in one test.
• Diversion of malicious traffic during concurrent flooding reduces production-model token consumption by 97.8% (708,077 → 15,351 tokens, about 46× reduction) while benign median latency rises only modestly.
• Honeypot fidelity and traceability trade off. A trained replica agrees with production on 92.9% of benign holdout requests. A naive unconditional baited code honeypot drops fidelity to 7.6%, while selective, camouflaged bait recovers fidelity to 88.9% and preserves attacker-side traceability in the 0.83–1.0 band.
• The continuous analysis loop yields positive gains: a correction head trained from loop feedback reduces misrouting of legitimate security research 9× (FPR from 30% → 3.3%) and raises pooled detection F1 to 0.933. Embedding fingerprints plus multi-view fusion achieve substantial campaign retrieval (top-1 56.7%, top-5 80.7% among 30 anchors).
• Session‑trend features separate soft‑escalation attacks from benign multi‑turn traffic: a simple slope rule detected attacks at 100% with ≤7% benign false‑flag in experiments; an online prefix rule detected attacks early with low false positive rate.

Limitations

The fidelity versus traceability tension is intrinsic: more faithful honeypots leak fewer forensic signals. Evaluations cover several attack families but do not exhaustively explore white‑box, gradient‑access evasion or specialised weight‑extraction adversaries. Selective bait operating points and threshold settings are deployment dependent. Some fidelity judgements exhibit judge‑position sensitivity and judge-family variance, and the loop's measured gains are demonstrated mainly on the false‑positive side.

Implications

Offensive implications include that attackers diverted to honeypots can be observed and linked across identity rotations, aiding defender attribution. However, attackers aware of per-request routing might probe to detect honeypots or attempt threshold estimation; white‑box access to the frozen backbone is not covered. The system shows that serving‑tier interception can both conserve production resources under floods and provide a rich harvest of attacker behaviour, enabling continuous adaptation of detection models.