Triggered MoE routers turn one GPU into a straggler

Problem

The paper studies a supply-chain attack on Mixture-of-Experts (MoE) large language models served with expert parallelism across multiple GPUs. Because routers choose which experts execute each token and experts are sharded to devices, a malicious model provider could alter router weights in a published checkpoint so that a private trigger concentrates work on experts co-located on one GPU. This can make that GPU a straggler, increasing latency and reducing throughput while leaving ordinary inputs looking benign, creating a stealthy scheduling attack surface in distributed serving.

Approach

The attacker modifies only router parameters of a clean MoE checkpoint and keeps a private trigger. The goal is trigger-conditional concentration of token-to-expert assignments onto a target expert set hosted entirely on a victim EP rank, while keeping routing on ordinary inputs near the clean reference. Direct optimisation to increase target usage on triggered inputs also pushes ordinary routing toward the same experts, so the authors develop a three-stage training procedure: Stage 1 uses an anti-target loss and language-model objectives to reduce ordinary target mass; Stage 2 trains on paired ordinary and triggered inputs with a trigger loss plus a gap loss that forces the triggered input to have substantially larger target mass than its ordinary counterpart; Stage 3 restores ordinary routing by matching the poisoned router to the clean router via a KL teacher-matching loss and applies layer-level uniformity and cap losses computed with a straight-through estimator to eliminate residual concentration. Experiments use LoRA on routers only and evaluate across three MoE model families and four corpora, plus live expert-parallel serving on four GPUs.

Key Findings

• Trigger-conditioned concentration is achievable: across three MoE families and four corpora the attack directs between 92.3% and 95.6% of triggered token top-k assignments to the target experts.
• Stealth on ordinary inputs: benign routing shares remain close to the clean reference, within about 1.1 percentage points in reported cases, and downstream task metrics (perplexity, HellaSwag, ARC-Challenge) degrade only marginally.
• Serving degradation in live EP: triggered traffic raised tail time-to-first-token to 1.43× benign and reduced throughput to 0.86×; device-level imbalance estimates showed a victim-to-peer compute ratio rising to about 49× and SM-utilisation ratio to 1.60× under fully triggered traffic.
• Simple defences have limits: a rank-skew runtime audit with a fixed threshold readily detects dense triggered waves but can be evaded by sufficiently sparse triggered traffic; router-only fine-tuning with a load-balancing objective repaired the poisoned router and restored triggered shares to near-clean levels, taking less time to repair than the original poisoning training.

Limitations

The threat model assumes the attacker knows the expert-to-rank placement and that deployments use a fixed contiguous placement; dynamic remapping or unknown placements are outside scope. The attack relies on access to a proxy corpus and ability to modify router weights only; results use LoRA on routers. The defence evaluation used a particular audit threshold and EP configuration, so detection sensitivity depends on deployment parameters.

Implications

An attacker who publishes a poisoned MoE checkpoint can act as a trigger-controlled device scheduler, selectively turning a chosen GPU into a straggler to increase latency and reduce throughput for triggered requests while preserving normal-looking behaviour. Offensive uses include covertly degrading service quality for targeted inputs, imposing intermittent denial-of-service style slowdowns, or creating platform-specific performance instability that is hard to spot from model outputs alone. These findings expand the adversary model for distributed LLM serving and motivate integrity checks of router parameters, secure checkpoint distribution, and runtime monitoring of per-rank routing and load.