Narrative Speech Evades Audio-Language Model Safeguards

Problem

Large audio language models process raw speech inputs and safety filters that are largely designed for text may fail when commands come as spoken language. The paper investigates this risk by designing a text to audio jailbreak that hides disallowed directives within a narrative style audio stream. The attack uses an advanced instruction following text to speech system to exploit both the structure of the prompt and acoustic properties, thereby bypassing safety mechanisms traditionally tuned for text. When delivered as synthetic speech, narrative delivery produces markedly higher rates of unsafe outputs from state of the art models such as Gemini 2.0 Flash, far surpassing text only baselines. The work highlights a cross modal safety risk as speech based interfaces become more widespread and capable.

Approach

The study adopts a black box adversarial framework in which the attacker interacts with end to end large audio language models solely via audio input. It identifies delivery style as a controllable parameter and designs five style templates inspired by psychology to shape prosody and affect while keeping the textual content constant. The audio prompts are produced by a text to speech system and fed into the target model to elicit responses that may violate alignment policy. The research considers two jailbreak paradigms: a text based DeepInception style that embeds unsafe instructions within recursive narratives, and an audio based AdvWave style that directly adapts audio tokens through iterative refinement and prompts. Three benchmark model families are evaluated: closed source end to end models GPT 4o Realtime and Gemini 2.0 Flash, and the open source Qwen 2.5 Omni 7B. Datasets include Advbench, JailbreakBench, and Malicious Instruct, with attacks measured by attack success rate. The evaluation uses LLM as judge to determine whether the model output breaches policy, and experiments include both synthetic speech generated by GPT 4o mini TTS and an alternative TTS for generalisation. All settings are kept consistent with prior work in terms of budget and iteration counts to ensure fair comparison.

Key Findings

• Delivery style substantially boosts attack success across text and audio modalities, with gains up to about 26 percentage points for some models and benchmarks, indicating that vocal delivery acts as a potent adversarial channel beyond linguistic content alone.
• On GPT 4o Realtime, stylised audio raises attack success from 44.4 to 57.9 per cent on AdvBench and from 40.0 to 66.7 per cent on JailbreakBench; Gemini 2.0 Flash shows a large jump on Malicious Instruct from 73.0 to 86.0 per cent, illustrating cross model vulnerability to audio delivery.
• Qwen 2.5 Omni 7B maintains high performance with attack success above 70 per cent across benchmarks, though some small reductions occur due to front end instability; overall the trend mirrors other models in which audio delivery enhances jailbreak effectiveness.
• Across models, opening guitars of delivery styles such as Authoritative Demand and Social Bonding Appeal consistently drive high ASR, suggesting that directive and affiliative vocal frames are particularly effective in deterring alignment safeguards.
• In controlled ablations, converting text to neutral audio yields small improvements (3 to 5 per cent), while applying stylised delivery adds a further 10 to 20 per cent boost, confirming that paralinguistic framing drives much of the effect beyond mere audio conversion.
• Human recorded speech experiments reproduce the qualitative patterns observed with synthetic audio, indicating the vulnerability is not an artifact of synthetic generation but a broader sensitivity to delivery style in end to end audio language models.
• Across benchmarks, delivery based modulation outperforms plain acoustic perturbations, suggesting that integrating linguistic form with prosody creates a more efficient pathway for adversarial influence than haying acoustic noise alone.

Limitations

The study notes that smaller end to end models such as Qwen 2.5 Omni 7B can be less susceptible to audio based attacks due to decoding instability, and some results reflect front end variability. The delivery style set used is comparatively limited and hand crafted from established psychology literature, leaving open the possibility of further optimising vocal parameters. The experiments are English language based, and broader multilingual and cross accent analyses are needed to assess generality. Ethical considerations were followed with controlled experiments and disclosed to model developers prior to publication.

Why It Matters

The findings demonstrate that paralinguistic delivery forms a powerful adversarial vector for large audio language models. Safety mechanisms that address text content alone are insufficient for multimodal systems, and defenses should jointly model linguistic content, prosody, and speaker intent. Practical implications include implementing audio level detectors, multi modal content filtering, and robust evaluation with audio adversaries, as well as safeguards such as watermarking, response gating, and human in the loop review for high stakes domains. The work underlines societal and security risks from spoken manipulation in voice enabled assistants, educational tools and clinical triage systems, where attackers could influence model behaviour through conversational delivery styles.