Researchers expose inference-time backdoors in chat templates

Problem

Open weight language models are increasingly deployed in sensitive contexts, raising security concerns about backdoor attacks. A prominent threat is backdoors that activate under specific conditions. Prior research often assumes attackers have access to training pipelines or deployment infrastructure; this work identifies a new attack surface that requires neither, exploiting chat templates. Chat templates are executable Jinja2 programs invoked at every inference call, positioned between user input and model processing and thus occupying a privileged place in the inference pipeline. An adversary who distributes a model with a maliciously modified template can implant an inference time backdoor without modifying model weights, poisoning training data, or controlling runtime infrastructure. The study evaluates two objectives by constructing template backdoors: degrading factual accuracy and prompting emission of attacker controlled URLs, across eighteen models from seven families and four inference engines. Benign inputs show no measurable degradation, while triggered conditions produce substantial harm. The work also shows that backdoors generalise across runtimes and evade automated security scans on large open weight platforms, highlighting a supply chain security gap.

Approach

The authors introduce a template based inference time backdoor that relies on modifying the chat template bundled with GGUF style open weight models. They adapt injection syntax to different model families and inference engines while keeping the core mechanism intact. The evaluation spans eighteen models from seven families and four inference engines, using four configurations: clean template with no trigger, clean template with trigger, backdoored template with no trigger, and backdoored template with trigger. The trigger conditions activate a hidden directive that manipulates the model input hierarchy without altering weights. Two payload types are studied: integrity degradation by inducing plausible but incorrect factual answers in single hop factoid questions, and forbidden resource emission by causing the model to output attacker controlled URLs. Triggers use short phrases appended to prompts, and the backdoored templates insert instructions that steer the model’s responses when triggered. The assessment uses fixed decoding settings, evaluates factual accuracy against ground truth, and measures attack success rates for URL emissions across explicit, hidden, and encoded variants. The evaluation also tests cross engine generalisability by running experiments on llama cpp based stacks and additional engines, showing consistent results.

Key Findings

• Identification of a new inference time backdoor surface: chat templates bundled with open weight models can carry conditional behavioural backdoors without training access or infrastructure control.
• Demonstrated across eighteen models from seven families and four inference engines, backdoors activate under designated triggers while remaining dormant for benign inputs.
• Factual accuracy under trigger conditions collapses from around 0.90 to about 0.15 on average, with some models near complete failure; for example, some configurations yield outputs with correct structure but incorrect factual content.
• Attacker controlled URLs are emitted with high success rates when triggered, with explicit, hidden and encoded payload variants showing robust effectiveness across models and payload types.
• Backdoors generalise across inference runtimes and evade automated security scans on the largest open weight distribution platform, exposing a systemic defence gap in current ecosystems.
• Backdoors rely on the privileged status of template driven instructions rather than exploiting model failure modes, making them difficult to detect by surface level checks and conventional guardrails.
• The work notes a constructive use: templates can also be used as inference time safety controls; combining safety oriented templates with system prompts can improve malicious prompt refusal rates without harming benign performance.

Limitations

The study focuses on GGUF style open weight deployments and a particular class of template based backdoors, evaluating eighteen models across four engines; results may not capture all possible template formats or domain specific models. While cross engine and cross model family generalisation is demonstrated, the authors do not claim exhaustive coverage of all attack spaces. They do not provide a definitive causal link between instruction following and backdoor success beyond observed correlations, and they do not thoroughly evaluate mitigations beyond a qualitative discussion. The evaluated mitigations are not implemented at scale and the work does not test all potential defence strategies or provenance mechanisms.

Why It Matters

The findings establish chat templates as a security relevant artifact in LLM deployments, shaping threat modelling for the open weight ecosystem. The results show a stable, adversarially controllable backdoor that survives across models and runtimes and can bypass current automated scans, underscoring the need for template provenance, integrity verification, and auditing. Practically, these attacks pose risks of misinformation, output manipulation, or phishing like links, especially in high stakes domains. High level mitigations include signing and versioning templates, strict access control and code review, sandboxing or isolating template execution, runtime and CI CD checks with attestation of deployed templates, and monitoring for anomalous URL emissions or degradation in factual accuracy. From a safety perspective, the work highlights that improvements in instruction following may inadvertently raise the reliability of inference time backdoors, reinforcing the importance of defending at the template layer as part of holistic alignment and governance strategies. Overall, the study argues for including chat templates in threat models and defence planning for LLM deployments.