Public comments can poison LLM pretraining data
New research shows that public comment sections can seed Large Language Model (LLM) pretraining data with malicious text. Using a pipeline-level analysis called HalfLife, the authors estimate that 0.13% of injected pages make it into training sets. Small contamination can shift model behaviour, and ad slots offer little poisoning leverage.
Most of us worry about prompt injection at inference time. This paper looks upstream and asks a tougher question: how easy is it to slip poison into the raw web that Large Language Models (LLMs) learn from? The answer is uncomfortably practical. Public discussion features on third‑party sites are a workable injection vector, and a small amount of what gets posted survives crawling and filtering to become training data.
HalfLife: measuring what sticks
The neat idea here is HalfLife, a decomposition of the full pipeline into three measurable stages: injectable (can an attacker post text), captured (does a crawler extract it), and not filtered (does it pass deduplication, language, and quality gates). The team scanned 181,857 pages, detected common comment platforms, and simulated attacks by swapping detected comment text with adversarial content. They used Common Crawl WARC files to stand in for the crawled web, Resiliparse for text extraction, and an open curation pipeline drawn from Dolma 3 for filtering.
The numbers matter. About 3.4% of sampled pages exposed detectable comment interfaces. Of the injected text on those pages, 71.9% was captured by extraction. A non‑trivial 5.5% of that captured content passed filters. Multiply through and you get an inclusion probability of roughly 0.13% for injected pages. To land around 250 poisoned documents, a prior backdoor threshold, an attacker would need on the order of 100k to 1M attempted posts. That is noisy but feasible with automation.
They also probed ad tech as a poisoning route. In static HTML, 92.9% of ad slots were empty placeholders; in rendered pages, 75.9% of ads lived in cross‑origin iframes, hiding advertiser text from typical scrapers. In short, programmatic ads look like a poor DOM‑based poison vector.
Do these traces matter during pretraining? In controlled runs mixing tiny poison rates at 0.1%, 0.01%, and 0.001% of tokens into web subsets, models from 65M to 1.3B parameters shifted towards attacker‑favoured completions. Instruction tuning damped retention, especially for larger models, but contamination persisted for more natural formats like Q/A. A “No‑label” format weakened after instruction tuning and scale, though not uniformly.
I particularly liked the synthetic rewriting test: paragraph‑level rephrasing removed most surface‑form attacks yet left belief‑manipulation style poisons intact in about 65.3% of documents. That is a crisp demonstration that semantics, not just strings, are the problem.
There are caveats. Common Crawl stands in for proprietary crawlers, and injections were simulated rather than posted live, so server‑side defences may be undercounted. Filter results reflect one open pipeline; others will differ. Still, HalfLife gives us a measurable way to reason about inclusion risk. The open questions now are provenance at scale, how to audit third‑party page features, and what level of residual contamination we are willing to tolerate in foundation models.
Additional analysis of the original ArXiv paper
📋 Original Paper Title and Abstract
Pretraining Data Can Be Poisoned through Computational Propaganda
🔍 ShortSpan Analysis of the Paper
Problem
This paper studies whether adversaries can poison large-scale language model pretraining data by injecting content into third-party webpage features, especially public discussion interfaces. This matters because pretraining corpora are vast and heterogeneous, so small amounts of malicious text that survive crawling and curation can introduce persistent harmful behaviours, belief manipulation, or quality degradation that are hard to detect and costly to remove.
Approach
The authors introduce HalfLife, an analysis that decomposes the end-to-end probability that an injected fragment appears in a final training corpus into three stages: injectable (ability to post), captured (survival through crawling and text extraction), and not filtered (survival through deduplication and quality and language filters). They use Common Crawl WARC files as a proxy for the crawlable web, detect comment platforms across 181,857 pages, and simulate injections by replacing identified comment text with adversarial content. Text extraction uses Resiliparse and filtering mimics an open pipeline drawn from a released Dolma 3 toolchain including heuristic, English-detection, and quality classifiers. They also simulate rendered crawls to evaluate programmatic ad creatives, and run controlled pretraining experiments by mixing trace amounts of poison into web-subset pretraining data for model sizes 65M to 1.3B and token poison rates of 0.1%, 0.01% and 0.001% to measure downstream behaviour changes.
Key Findings
- Third-party comment interfaces are a viable injection vector: 3.4% of sampled pages contained detectable comment platforms (P(injectable)=0.034).
- Injected comments largely survive extraction: P(captured | injectable)=0.719, and a non-trivial fraction pass filtering so P(not filtered | captured, injectable)=5.5%, yielding an estimated inclusion probability P(include)=0.13% of injected pages entering the final corpus.
- At that inclusion rate, an attacker aiming for n=250 poisoned documents (a previously reported threshold for backdoors) would need on the order of 100k–1M attempted injections, which automated commenting can plausibly achieve.
- Programmatic advertisements are not a practical vector for DOM-based text poisoning: 92.9% of ad slots in static HTML are empty placeholders, and in rendered pages 75.9% of ads resolve to cross-origin iframes that do not expose advertiser text to scrapers.
- Controlled pretraining experiments show that small token-level contamination biases base models toward attacker-preferred completions across formats; instruction tuning reduces retention of the effect, especially for larger models. More naturalistic formats (Q/A and No-label) can still produce contamination, though No-label effects diminish after instruction tuning and at larger scale.
- Synthetic rewriting (paragraph-level rephrasing) largely removes surface-form poison but preserves belief-manipulation style poisons: belief-manipulation claims survive rephrasing in about 65.3% of documents.
Limitations
Analysis uses Common Crawl as a proxy for production crawlers and relies on simulated injections rather than live posting, which may overestimate the injection surface if sites employ unseen server-side protections. Inclusion estimates are specific to the chosen open quality-filter pipeline and may differ for other pipelines. Experiments avoid live harm by sandboxing and therefore do not capture adversary adaptations that rely on privileged knowledge of private pipelines.
Implications
An attacker with modest automation can mass-post to public comment forms to probabilistically seed pretraining corpora with tailored claims that bias model outputs, implant backdoors, or degrade trust. Open-data projects that publish sources and curation tools are easier to target. Offensive outcomes include targeted belief manipulation, reputational attacks, and model reliability degradation, with particular risk for low-resource languages or smaller open-model efforts where relative saturation is easier to achieve.
