Adversarial tweaks mislead binary code similarity detectors

Security teams lean on binary code similarity detection to cluster malware, spot code reuse, and support reverse engineering when source code is unavailable. As these tools adopt machine learning, they promise speed and scale. They also inherit a familiar problem in adversarial machine learning: small, carefully chosen changes can flip a model’s verdict without changing what the program actually does.

A new study probes that gap. The authors evaluate six machine learning models for binary code similarity detection (BCSD) against semantics-preserving transformations using a framework they call asmFooler. They transform 620 baseline samples into 9,565 binary variants with eight diversification techniques, including in-place code randomisation, inter–basic block reordering, semantic NOPs, junk code insertion, and compiler-driven obfuscation via Obfuscator-LLVM. The threat model is black box: the attacker only mutates the raw binary and observes the model’s outcome. To reflect real model limits, a transformation budget caps added instructions or bytes based on each model’s input constraints.

The headline result is straightforward and uncomfortable. With modest edits, attackers can induce both false negatives and false positives. The details matter. Robustness is not a single metric but a property of the whole pipeline: preprocessing and normalisation, feature choice, and architecture. Models that use control-flow graph (CFG) structure, such as Gemini and Genius, or dynamic signals like micro-traces in Trex, withstand some perturbations better than instruction-level models. Approaches that rely mainly on instruction sequences, such as BinShot, are more exposed to basic block reordering. Yet LLVM-based obfuscation dents performance broadly; interestingly, BinShot shows relative robustness there. Different models lean on different cues, and each cue can be targeted.

Attack potency is notable. The study reports that well-crafted edits can drive false positives to high rates with small changes, on the order of tens of instructions and budgets up to 100 bytes, and that false negatives often require even less. To understand why, the authors use explainability tools. Under attack, attention and feature importance disperse: SHAP values collapse towards zero and saliency on key tokens drops. The model becomes less anchored to the truly informative parts of the function, increasing the chance of a wrong match.

Transferability is mixed. Perturbations that fool one model sometimes carry over to others, especially when input representations are similar. Graph-based models are generally harder to fool this way, though there are notable exceptions; Trex shows transfer in some pairings. This variability is useful operationally: it suggests ensembles drawn from diverse representations can buy time and accuracy, though they are not a silver bullet.

For defenders, two implications stand out. First, ML-as-a-service pipelines that gate decisions on a single BCSD score are at risk. Small edits can push borderline samples over a threshold, enabling evasion or misattribution. That matters for malware triage, code provenance claims, and cross-binary vulnerability matching in firmware or supply chains. Second, robustness is designable. The study points towards practical defences: adversarial training or data augmentation with semantics-preserving transforms; combining static and dynamic features; and integrating multiple analysis methods so that an attacker must defeat more than one representation at once. Because the effectiveness of attacks is bounded by model-specific input limits, exposing and stress-testing those limits is part of engineering, not an afterthought.

There is a policy angle here too. If public bodies or critical suppliers procure automated analysis for attribution or compliance, they should require evidence of performance under semantics-preserving transformations, not just clean benchmarks. Independent evaluations using datasets like asmFooler can inform certification and auditing. Vendors should document preprocessing, input constraints, and model assumptions, so operators understand how decisions can fail. And organisations should avoid treating BCSD outputs as dispositive in enforcement or legal contexts without corroboration.

The work has limits, as the authors note: no formal verification of every transformation, a focus on 64-bit ELF, and partial coverage of obfuscation and protections. Still, it offers a concrete path to measure and improve robustness. If we standardise such testing and set expectations in procurement and oversight, we can nudge this class of tools toward safer defaults without stalling innovation.