Researchers Suppress Harmful Output by Editing Latents

Defenses

Published: Wed, Oct 15, 2025 • By Elise Veyron

Researchers Suppress Harmful Output by Editing Latents

A new inference‑time method called CALM edits last‑layer latent representations to suppress harmful concepts in Large Language Models (LLMs) without retraining. The approach combines concept whitening and projection to reduce unsafe outputs with small computational overhead. It improves safety metrics in many tests but introduces new attack surfaces and governance trade‑offs.

Large Language Models (LLM) remain vulnerable to jailbreak attacks that coax models into producing harmful content despite built‑in guardrails. Recent research proposes an inference‑time intervention called CALM that edits the model's final latent representations to suppress unwanted concepts, aiming to reduce exploitability without retraining or new labelled data.

At a high level CALM combines Concept Whitening (CW) from computer vision with a projection approach similar to Projection Filter for Subspaces (ProFS). The method learns a rotation that aligns top concept directions with canonical axes, applies a diagonal projection to zero out axes associated with harmful concepts, and inverses the transform before the softmax. That means the model itself is unchanged; only the last‑layer activations are adjusted at inference. The paper reports reduced harmful outputs and improvements on toxicity and harmfulness metrics across several model families, with only a modest inference overhead (the per‑step cost is O(d^2) in the latent dimension).

Concept whitening here refers to decorrelating and rotating the activation space so that interpretable concept directions become separable. The projection then suppresses those specific directions. Because the axes are aligned to identifiable concepts, CALM also yields some interpretability: practitioners can inspect which axes relate to violent or identity‑based harms. The technique therefore offers a lightweight, post‑deployment defence that can be deployed alongside prompts and other guardrails.

That convenience carries trade‑offs. The method relies on large amounts of embedding data for whitening (the authors note whitening needs on the order of a million token embeddings), and its effectiveness varies by model family. The paper flags instability in singular value decomposition steps, English‑only evaluations, and uneven generalisation. Crucially, altering latent space at inference creates a new attack surface: adversaries might craft inputs that circumvent or exploit the projection, or attempt to poison the signals used to identify harmful axes. There is also a governance risk. Latent suppression can be used benignly to reduce harm, but it could also be misused to mute safety signals or censor content in opaque ways. That raises questions about auditability, accountability, and compliance with regulatory expectations around transparency.

Policy and practical trade‑offs

From a policy perspective, CALM sits at the intersection of technical control and governance. Regulators and auditors will want records showing what was suppressed and why, plus evidence that suppression does not create dangerous blind spots. Relying on a single post‑hoc control to claim compliance is risky; defenders should avoid performative configurations that look safe but are brittle under adversarial pressure.

Immediate steps this quarter and what to plan later

In the next quarter organisations can test CALM‑style controls in staging: run red teaming that targets latent edits, benchmark safety and utility across representative prompts, and add inference logging and monitoring so any latent transform is auditable. Pair automated suppression with human oversight for edge cases and expand evaluation beyond social‑media toxicity tools to include domain‑specific harms and multiple languages.

Over the longer term embed such controls into safety cases and governance frameworks: require independent audits of latent‑space interventions, standardise transparency on what concepts are suppressed, invest in multi‑model and adversarially robust defences, and work with regulators on acceptable disclosure. CALM offers a practical tool, but it is not a silver bullet. Treat it as a component in a layered safety strategy rather than a single proof of compliance.

Additional analysis of the original ArXiv paper

📋 Original Paper Title and Abstract

Keep Calm and Avoid Harmful Content: Concept Alignment and Latent Manipulation Towards Safer Answers

Authors: Ruben Belo, Claudia Soares, and Marta Guimaraes

Large Language Models are susceptible to jailbreak attacks that bypass built-in safety guardrails (e.g., by tricking the model with adversarial prompts). We propose Concept Alignment and Concept Manipulation \textbf{CALM}, an inference-time method that suppresses harmful concepts by modifying latent representations of the last layer of the model, without retraining. Leveraging \gls*{cw} technique from Computer Vision combined with orthogonal projection, CALM removes unwanted latent directions associated with harmful content while preserving model performance. Experiments show that CALM reduces harmful outputs and outperforms baseline methods in most metrics, offering a lightweight approach to AI safety with no additional training data or model fine-tuning, while incurring only a small computational overhead at inference.

🔍 ShortSpan Analysis of the Paper

Problem

Large Language Models are vulnerable to jailbreak attacks that bypass safety guardrails. The paper studies how to safely suppress harmful content at inference time without retraining, to reduce exploitability while maintaining performance, addressing the need for post deployment safety and resilience.

Approach

The authors propose CALM, an inference time method that suppresses harmful concepts by modifying latent representations in the last decoder layer. It combines Concept Whitening CW from computer vision with Projection Filter for Subspaces ProFS to align and suppress harmful concepts. The method uses an offline learned rotation QQ to align top concept directions with canonical axes, followed by a diagonal projection P to zero out harmful dimensions, allowing inverse transform to recover embeddings before passing to softmax. It avoids retraining and uses precomputed whitening and rotation matrices. A variant without alignment removes explicit alignment and uses direct projection. Data sources include Harmful Q and A, Harmful Chat, and Alpaca for neutral data; datasets LLM LAT Harmful, AdvBench; models include LLaMA 3 8B Instruct, Phi-3 Mini, and Gemma variants with Abl iterations. Evaluation employs perplexity for safe and unsafe outputs, Unsafe Win Rate, Detoxify toxicity, and harmfulness via Harmful Chat and Harmful Q and A.; complexity is O(d^2) per step for inference, with training complexity O(max(N d^2, T d^3)).

Key Findings

CALM reduces harmful outputs and outperforms baselines in most metrics across multiple model families, with a small inference overhead and without additional training data or fine tuning.
Whitening and decorrelation improve concept separability, enabling more accurate suppression; CALM increases perplexity for unsafe answers while preserving perplexity for safe answers, yielding improved Unsafe Win Rates in many configurations. For example, in Llama 3 8B Instruct, CALM reduces positive perplexity and increases negative perplexity as the number of concepts grows, indicating better separation; in Phi-3 Abl with ten concepts, CALM attains lower positive perplexity than ProFS.
CALM demonstrates strong performance on Harmful Chat across model families, with some configurations achieving perfect Unsafe Win Rate for certain prompts; overall CALM matches or surpasses ProFS in toxicity and harmfulness metrics, with better gains on Llama and Phi-3 than Gemma; combining CALM with a safe prompt yields the strongest overall results.
Interpretability is enabled by aligning concept directions with a small set of axes, allowing inspection of which concepts influence responses; activations mapped to axes illustrate identifiable concepts such as violent content or identity related harm.
Computational considerations include per step cost of O(d^2); experiments used NVIDIA hardware (A100 40GB for inference and a GTX 1050 Ti 4GB for rotation training), with timings reported in supplementary material.

Limitations

Limitations include variability in toxicity and harmlessness counts which can complicate interpretation; Detoxify training data is biased toward social media content; evaluating with LLaMA 3 70B may be affected by intrinsic model biases; generalisation across model families can be uneven and SVD stability can vary; attempting Isomap for intrinsic dimensionality offered limited actionable insights; whitening relies on large volumes of data (approximately one million token embeddings); evaluations are conducted in English only; potential new attack surface exists around latent space edits and inference time interventions; model specific tuning may still be required.

Why It Matters

CALM offers a lightweight post deployment defence that suppresses harmful content without retraining, enabling safer deployment of LLMs with small inference overhead and compatibility with existing guardrails. It provides interpretable concept axes which support transparency and governance. However the approach raises ethical and policy considerations including risks of over censorship and misuses that suppress safety signals, highlighting the need for robust guardrails, transparency, and accountability. Future work should address broader language coverage and guard against evasion strategies targeting latent space edits.

Attribution Original paper on arXiv