Adversarial patches make edge VLMs draw wrong boxes
Reviewing A Unified Evaluation Framework of Physical Adversarial Robustness in the Edge VLM Era · Eric Liang, Alan Luo, Kaiwen Yuan
Compact vision-language grounding models built for edge devices turn out to be as vulnerable to adversarial patches as conventional detectors. The more serious finding is how they fail: many return confident boxes in the wrong place rather than no box at all, and a downstream system cannot tell those from correct detections.
Put an adversarial patch on a person's shirt and ask a small vision-language model to find the person. Sometimes it returns nothing. Sometimes it returns a neat, confident box around the patch itself, or around an empty patch of scene.
The second outcome is the one that interests me. The researchers tested compact grounding models, small enough to run on a device at the edge, against conventional object detectors. A grounding model takes a text prompt such as "person" and generates the coordinates of a box. These models are attractive because they find objects they were never explicitly trained on. They are being considered for embedded, safety-critical systems.
The attack itself is familiar. For each model, the researchers trained a patch with full knowledge of its internals, optimising it to stop the model locating a person. Random rotations, brightness changes and printability constraints were added so the patch would plausibly survive being worn.
Comparing the two families fairly takes care, because any patch hides part of the body. So they first swapped the patch for a plain grey square of the same size. An attack only counts on images where the model still finds the person behind the grey square. That separates adversarial effect from simple occlusion. It is a sensible control.
The small grounding models proved as vulnerable as the detectors. The worst reached about 58 per cent attack success, against 78 per cent for the weakest detector. Robustness varied by model rather than by family. One grounding model and one open-vocabulary detector both held up comparatively well.
The failure that matters is the wrong box
The turn is in how they fail. The researchers split every successful attack three ways: no box, a box on the patch, or a box on neither. Which one a model produces tracks its output format more than its architecture. Models that gate their answer, such as a detector with a confidence threshold, mostly fail by returning nothing. Models that generate free text keep producing coordinates, and those coordinates land in the wrong place.
Only the missing box is visible downstream. A planner can notice that nothing was found. A well-formed box on the patch looks exactly like a correct detection.
It gets worse off the training distribution. On driving images from Waymo, several models that had failed by returning nothing began failing by misplacing the box instead. The attack kept working. It just stopped announcing itself.
The preconditions are worth stating. The attacker here had white-box access, and the patches were simulated digitally rather than printed and filmed. Physical transfer and black-box attacks are left for later work. Even so, if a grounding model feeds a safety decision, I would not treat a returned box as proof it found the right object.
A missing box is a warning. A wrong box is an answer, and the system downstream has no reason to doubt it.
Further analysis
Problem
This paper studies the physical adversarial robustness of compact vision-language grounding models (VLMs) deployed at the edge, comparing them directly to traditional object detectors. This matters because grounding VLMs output single bounding boxes in a form indistinguishable from correct detections to downstream systems, and they are being considered for safety-critical, embedded applications. Prior work evaluated physical attacks on detectors but not on lightweight grounding VLMs nor with a unified evaluation that controls for occlusion effects and differing output formats.
Approach
The authors train image-agnostic adversarial patches in a white-box setting against each target model to suppress person localisation. They select compact grounding VLMs (under 4 billion parameters) and a range of closed-set and open-vocabulary detectors. Training uses 1,324 single-person images drawn from COCO, PASCAL VOC and AdvT-Shirt, with patches of original resolution 160x160 placed at scale a=0.5 relative to the person. Optimisation runs 500 epochs with Expectation Over Transformation including rotations uniformly in [-20,20] degrees and brightness jitter +/-10 percent; printability and smoothness terms constrain the patch. Evaluation is performed on a 331-image validation set and 999 out-of-distribution Waymo images. To make detector and VLM outputs comparable they normalise outputs to a single xyxy box per image, read detectors at standard confidence thresholds, and decode VLMs greedily.
Key Findings
- Unified metric: The authors introduce conditional attack success rate (cASR), which scores an adversarial patch only on images where the model still localises the person under a matched grey occluder. This controls for occlusion-caused failures and baseline errors.
- Failure partitioning: Every adversarial failure counted by cASR is partitioned into omission (no box emitted), substitution (box lands on the patch), or displacement (box lands on neither the person nor the patch); these three exhaustive categories sum to cASR.
- VLM vulnerability: Grounding VLMs can be as vulnerable as detectors. In the benchmark VLM cASR spanned 9.2% to 57.7% while detectors spanned 18.2% to 78.3%; specific models InternVL3-2B and PaliGemma 2-3B reached high cASR values and, on some subsets, near-total failure.
- Failure mode tied to output format: Whether attacks induce omission versus mislocalisation is associated more with a model's output grammar than with architecture family. Gated VLMs and score-gated detectors fail mostly by omission, while free-text VLMs emit boxes that are displaced or substitute onto the patch.
- Dangerous failures and distribution shift: Substitution and displacement are more dangerous because they present confident, well-formed but incorrect boxes to downstream systems. Under distribution shift (Waymo), several models that omitted in-distribution shifted toward substitution/displacement, reducing visible failure signals despite persistent cASR.
- Robust examples: Some models remained relatively robust (e.g. MiniCPM-V 4.6 with 9.2% cASR and YOLO-World at 18.2%), indicating robustness is model-specific rather than family-specific.
Limitations
Attacks are digital simulations optimised for printability and pose robustness but not physically printed or camera-captured; only white-box, suppression-objective attacks are evaluated. Results do not measure real-world physical transfer or black-box attack efficacy.
Implications
An attacker who can train or transfer a patch could cause grounding VLMs to either hide a person (omission) or to produce confident incorrect boxes (substitution or displacement). The latter modes are especially hazardous because downstream planners or safety systems may act on a plausible but wrong localisation, misdirecting attention or action. Distribution shifts can convert visible omissions into stealthier mislocalisations, increasing real-world risk.
The paper title, authors and abstract
A Unified Evaluation Framework of Physical Adversarial Robustness in the Edge VLM Era
Recent advancements in Vision-Language Models (VLMs) enable end-to-end visual grounding via language prompts and generative decoding, demonstrating largely improved zero-shot and open-vocabulary detection capabilities. As a result, VLMs are rapidly being deployed at the edge in safety-critical real-world systems. However, the physical adversarial robustness of edge VLMs has not been systematically studied, and the fundamental differences in output with modern object detectors create methodological challenges for fair evaluation. To compare the two families directly, we evaluate every selected representative model on one outcome: whether an adversarial patch worn on a person evades detection of the model. We report this by introducing a unified metric, conditional attack success rate (cASR), counted only on images the model still localizes under a matched gray occluder, and partition every failure into omission (no box), substitution (a box on the patch), and displacement (a box on neither). We benchmark the physical adversarial vulnerability of compact grounding VLMs against a range of object detectors, and find that the VLMs can be as vulnerable as the detectors. We disclose that failure mode is largely associated with the output format of a model rather than its architecture family. Of the three, only omission is visible to a downstream system; displacement and substitution return confident, well-formed boxes, making them potentially more dangerous failures. Further, we find that, under distribution shift, VLMs that primarily failed by omission instead shift to failing more by displacement/substitution. Thus, a grounding VLM can inherit the detector's vulnerability while showing the possibility to adopt more dangerous failure modes. Our code and data are available at https://github.com/ericliang1/physical-adversarial-robustness.
