Hidden visual prompts steer multimodal LLMs

Multimodal Large Language Models (MLLMs) are turning up in products that read images and text together. Most prompt injection stories have focused on obvious text tricks. This paper looks at something sneakier: hide the instruction in the pixels. The model then follows the buried cue, while a human sees a normal-looking picture.

The authors propose CoTTA, a targeted attack that combines a small, covert text overlay with an imperceptible perturbation. They iteratively optimise the image so that its visual features line up with both a malicious text prompt and a dynamically refined target image. In practice, they use surrogate encoders such as CLIP-like models to guide this process and keep the total pixel change within a tight bound. The evaluation is black-box: the attackers do not need access to model weights or gradients for the commercial systems under test.

They test two common tasks, image captioning and visual question answering (VQA), across several closed-source models, including GPT-4o, GPT-5, Claude-4.5 and Gemini-2.5. On captioning, the attack achieves high targeted success on GPT-4o and Gemini-2.5, around 80% under both strict and similarity-based scoring. On VQA, it reaches adversarial success rates of 82% and 79% on GPT-4o and Gemini-2.5 respectively, with strong similarity to the target sentence. Against the strongest prior baseline, FOA-Attack, CoTTA substantially lifts success across models, including a large gain on GPT-5. One model, Claude-4.5, is notably harder to move. Ablation studies suggest the image-to-image feature alignment is doing most of the work; removing it halves success, while removing the covert overlay hurts results more modestly.

What the researchers did

The core idea is to make the attacked image look, in feature space, like it contains the attacker’s instructions, without making that visible to the user. The covert overlay provides a seed for the semantics, while the perturbation tunes the global and token-level features toward the malicious target. A dynamically updated target image, instantiated from a text-rendered image and refined each step, improves transfer to different closed models. All of this sits under a small l-infinity pixel budget, so the result still looks clean to humans and likely survives basic pre-processing.

Why it matters

If your product lets users submit images to an MLLM, an attacker could ship a photo that silently steers the model to produce a chosen answer, including policy-violating content, while your UI and reviewers see nothing unusual. Because the method is black-box and uses common surrogate encoders, it does not rely on insider access to the target model. That widens the attack surface to any workflow that trusts third-party images: support tickets, forms, content uploads, or partner feeds.

There are caveats. The paper evaluates two tasks and a handful of models and prompts; it does not prove generalisation to other settings. It does not test defences. The overlay is subtle rather than invisible and its detectability will vary with rendering, compression and any input transformations. Robustness also varies across models, as Claude-4.5’s higher resistance suggests.

Commercially, the signal is simple: images are not passive inputs for MLLMs. Treat them like untrusted code paths. If you run multimodal inference on unvetted images, assume prompt injection risk even when nothing looks amiss. In the near term, this argues for targeted evaluation rather than grand fixes. Test your specific pipelines with visual prompt attacks, including variants that survive your resizing, cropping and compression. If you are building foundational defences, the obvious candidates to explore are multimodal input screening, adversarial training that includes visual prompt attacks, and randomised or lossy transforms to disrupt precise feature alignment. The paper does not claim these will work; they are simply where a sensible team would start measuring. As for a definitive mitigation, we are not there yet. Watch this space.