Near-infrared twin points mislead AV map builders

Problem

This paper studies the security of online high-definition (HD) map construction for autonomous vehicles and shows how physical-world interference can cause targeted, safety-relevant map deformations. The authors identify a cross-boundary compensation effect: when a single attack perturbs a target road boundary, a separate, visible compensation boundary can preserve enough geometric cues for the model to recover the original road geometry. This makes single-point attacks unreliable in many asymmetric scenes and can allow corrupted map elements to propagate into unsafe motion planning decisions.

Approach

The authors propose TwinIR, a mechanism-guided, at-most-two-point physical attack framework. TwinIR performs an offline black-box optimisation using pre-collected multi-view camera observations to select a primary attack position on the target boundary and conditionally a secondary position on the compensation boundary when it improves the objective. Two targeted objectives are considered: Road Straightening Attack (RSA) to remove turns and Early Turn Attack (ETA) to shift a turn toward the ego vehicle. To keep multi-point interference inconspicuous, TwinIR models camera responses to near-infrared (NIR) illumination and maps optimised BEV points to feasible roadside deployment locations. Physical realisations use near-infrared laser emitters placed so they remain largely invisible in full-colour views while producing a clear response in vehicle cameras. Evaluation is performed digitally on the nuScenes asymmetric-scene subset against three state-of-the-art online map models and validated on a controlled real-world autonomous-vehicle testbed.

Key Findings

• TwinIR yields substantially larger degradation than single-point blinding: under RSA it reduces mean AP by 8.18 to 8.96 percentage points across victim models, and under ETA by 2.84 to 5.62 points, relative to clean inputs.
• Attack impact on planning is significant: TwinIR increases unreachable-goal and unsafe-planned-trajectory metrics; the abstract reports increases of 25 to 28 percentage points for unreachable goals and 19 to 20 points for unsafe trajectories over clean inputs. Comparisons with single-point attacks show consistent additional harm (for example, UGR increases of about 11 to 12 points in some comparisons).
• The marginal benefit is mostly realised by adding a second point: moving from one to two infrared points produces the largest additional mAP drop, while adding three to five points yields only small further gains, indicating saturation after two points.
• NIR realisation is stealthy in visible images yet clearly captured by infrared-sensitive cameras, and the digital optimisation transfers to physical deployment where TwinIR induced road straightening and early-turn deformations on a test vehicle.
• TwinIR affects not only boundary AP but also related map elements such as dividers and pedestrian crossings, so boundary deformation can propagate through the map representation.

Limitations

The attack assumes offline access to pre-collected scene data and black-box query capability to the victim model during optimisation. Effectiveness depends on camera spectral sensitivity to near-infrared and on feasible roadside deployment with line of sight. Experiments focus on an asymmetric-scene subset and a controlled test field with specific camera hardware and without sensor fusion; results may vary with different sensors, IR-cut filters, multi-sensor fusion, or broader scene types.

Implications

Offensively, an attacker can use a small number of concealed near-infrared emitters to coordinate dual-point interference that is hard for humans to notice but degrades HD map construction and downstream planning. By targeting both the primary target boundary and a compensation boundary, TwinIR can reliably induce straightening or early-turn errors and thereby create unreachable goals or unsafe planned trajectories. The threat exploits camera NIR sensitivity and map-construction reliance on geometric relations, and can evade defences that only monitor the visible spectrum or assume single-point perturbations.