ASCII framing slips past LLM safety training

Problem

This paper studies a novel recontextualisation attack that exploits a mismatch between model capability and safety alignment. Safety training concentrates on refusing plainly stated harmful requests, but that supervision is shallow with respect to surface form. The ASCII Attack embeds a fully legible harmful request inside an assembly of ASCII-art characters, frames it as artwork and asks for artistic feedback, thereby changing the speech act without hiding the operational content. The attack is single-turn and black-box, so an attacker needs only one user message and no access to model internals. The study assesses whether this aesthetic register bypasses existing refusal behaviour and how broadly it generalises.

Approach

The authors formalise the ASCII Attack as a five-part prompt template and evaluate it with a paired design: every ASCII-framed prompt is matched to a direct-question control that states the same operational request plainly. They generated two independent corpora (Alpha and Bravo) across eleven target models spanning roughly 1B to a mixture-of-experts Scout model, and eight harm topics including account-takeover, ransomware, chem-synthesis and an Amanita phalloides poisoning scenario. Responses were scored by a panel of automated harm-aware judges (five primary classifiers) and additional robustness graders; a blind LLM-adjudicated panel was used as a proxy for human review on samples. Decoding temperatures were swept and generation was single-turn with no system prompt.

Key Findings

• Framing uplift: a harm-aware classifier judges 62% of framed prompts harmful versus 42% for matched direct-question controls, an uplift in judged attack-success rate of roughly 20 percentage points under the principal judge.
• Extreme susceptibility: on the most vulnerable target the ASCII-framed prompt succeeded 93.3% of the time.
• Comparative strength: a single black-box ASCII query matches or exceeds published single-query attacks under four of five harm judges and often equals or outperforms a multi-query iterative attack on most models; a best-of-three single-query portfolio attains a pooled union near 77% on the frozen snapshot.
• Architecture domination: variance in vulnerability is driven more by model identity than by topic; susceptibility does not decrease with model scale and in some cases larger or reasoning-enabled models are more susceptible.
• Judge disagreement and measurement validity: judges disagree substantially; at least one judge dissents from the panel majority on 62.9% of ASCII-framed rows, and the uplift range across judges was from about +7.8 to +20.1 percentage points in the main corpus.
• Hidden deliberation traces: for reasoning models a nontrivial subset showed harmful content in hidden think traces even when the visible response appeared clean, although the visible reply was overall more likely to be harmful.

Limitations

Main constraints are that the paired ASCII prompts are substantially longer and include scene descriptions and enumerated sub-questions, so the measured effect is a bundled framing plus information-priming treatment rather than an isolated aesthetic-channel coefficient. Judgements are automated LLM classifiers and therefore judge-rated rather than human ground truth, although a blind LLM-adjudicated panel corroborated a positive uplift in sampled draws. Generation provenance and serving runtime varied across rows, and runtime-supplied chat templates can alter generation; the study is English-only and restricted to single-turn attacks.

Implications

Offensive implications are clear: an end-user attacker can craft a single black-box message that reframes operationally precise harmful requests as art critique and elicit operational detail that would likely have been refused if asked plainly. Such prompts require no optimisation, no role-play instruction and no access to model internals, can be effective across model families, and combine easily with other single-query surfaces to increase overall success. The attack also leverages serving and runtime variability and can exploit reasoning-model deliberation traces, so defenders cannot assume uniform behaviour across deployments.