Grace Hopper page migration leaks across GPU partitions
Reviewing PAGE404: Breaking GPU Isolation Through Hardware-assisted Page Migration · Ravan Nazaraliyev, Yicheng Zhang, Nael Abu-Ghazaleh
Researchers show that hardware-assisted page migration on NVIDIA Grace Hopper leaks information between Multi-Instance GPU partitions through shared access counters and a shared driver pipeline. Co-located tenants can build covert channels and identify a neighbour's LLM and prompt length, which undercuts the isolation multi-tenant GPU providers rely on.
Two tenants share a Grace Hopper module, the combined CPU and GPU board made by NVIDIA, the graphics chipmaker, each in its own Multi-Instance GPU (MIG) partition. Neither can read the other's memory. Yet one can tell which large language model the other is running, and roughly how long its prompt was. The leak comes from a feature built to make things faster.
Grace Hopper puts an Arm CPU and a Hopper GPU on one module with a cache-coherent link, so both see a single pool of memory. To keep data close to where it is used, the hardware counts accesses. When a region of host memory gets hot, the system migrates those pages into GPU memory. The driver services each migration through interrupts, in stages.
MIG is meant to carve the GPU into slices with hardware-enforced boundaries. The access counters do not respect those slices. There are only a few hundred of them, grouped into small sets, and every process on the module shares them. Physical addresses map onto those sets through a fixed function, which the researchers recovered.
That is all an attacker needs for an old trick. Fill a counter set with your own addresses, let the victim touch memory, then check whether your entries were pushed out. It is the eviction-set technique from CPU cache attacks, pointed at a new piece of shared hardware. As a covert channel between cooperating processes in different partitions, it moved about 8,500 bits per second with few errors. As a side channel it told ten GPU kernels apart with high accuracy.
The second channel interests me more, because it is not in silicon at all. Migration interrupts flow through one driver pipeline shared by every tenant. When a neighbour is busy, your own migrations take longer, and you can measure that delay. From those timings the researchers identified a neighbour's LLM with up to 94% accuracy. They estimated prompt length to within about five tokens on average.
The preconditions are real. The attacker must be co-located on the same physical module. Mapping the counters took a modified driver that an ordinary tenant would not have, though I would expect that mapping to be a property of the hardware rather than of each machine. The delay channel is unstable and worked in about half of runs. Some techniques depend on huge-page behaviour and allocation order that may vary, and results degraded across providers.
None of that undoes the point. A model identity or a prompt length is not a stolen secret, but it is reconnaissance. A covert channel is also a ready exfiltration route for code already inside one partition. What strikes me is that MIG did its job on memory and compute. The leak sits in the shared machinery underneath, the counters and the driver that every slice depends on. Anyone selling these slices as strong isolation should treat that claim as unproven.
The partition stops above the counters.
Further analysis
Problem
This paper examines whether NVIDIA’s Grace Hopper (GH200) hardware-assisted page migration and its Unified Virtual Memory pipeline introduce cross-tenant side channels and covert channels that break Multi-Instance GPU (MIG) isolation. The architecture uses per-region access counters to promote hot pages from host DRAM to GPU DRAM to improve locality; the authors ask whether the design of those counters and the driver’s interrupt handling leak information across software or hardware isolation boundaries in multi-tenant deployments.
Approach
The authors reverse-engineer the GH200 page-migration subsystem and the UVM driver pipeline using GH200 nodes on CloudLab and Lambda and a modified NVIDIA driver for controlled experiments. They characterise the access-counter hardware, its mapping and eviction policy, the migration granularity and prefetch behaviour, and the driver’s interrupt bottom-half servicing stages. Based on these observations they design two exploitation primitives: an eviction-set-based covert channel that targets shared access counters, and a migration-delay channel that exploits contention in the driver’s interrupt/migration pipeline. Proof-of-concept attacks run across distinct MIG partitions and include workload fingerprinting and prompt-length estimation for large language model inference.
Key Findings
- The access-counter hardware comprises 256 counters organised as 8 sets of 32 counters, with mapping determined by XOR functions operating on physical address bits 21–38, so every 2MB physical region maps to a single counter set while migration operates on 64KB pages.
- Counters are shared across processes and across MIG instances; the counter state persists until the driver is reinitialised, so counters can survive process teardown when a persistent GPU context exists.
- The shared counter structure enables an eviction-based covert channel achieving about 8,500 bits per second with a 1.5% error rate for a single set and about 15,500 bits per second at roughly 5% error when using two sets; set-based side channels distinguish 10 CUDA kernels with 94% accuracy.
- The driver interrupt and migration pipeline introduce measurable migration delay variability dominated by software-stage contention; a migration-delay covert channel achieves roughly 8,100 bits per second at a 4% error rate but is less stable, succeeding in about half of runs.
- Side channels can fingerprint workloads: up to 94% accuracy for LLM identification, up to 89% accuracy for SPEC2017 CPU benchmarks, and token-length estimation for LLM inference with R2=0.92 and mean absolute error 4.6 tokens.
- Practical attack techniques exploit platform features such as 512MB huge pages, probe sampling rates up to ~17,000 samples per second with parallel probing, and methods to bypass UVM prefetching by striding accesses.
Limitations
Experiments used modified drivers and controlled setups to recover detailed mappings; some reverse-engineering steps required driver instrumentation not generally available to unprivileged attackers. The migration-delay channel is inherently less stable and succeeds only in a subset of runs. Several synchronisation and allocation techniques used by the covert channel rely on huge-page behaviour and allocation ordering that may be environment dependent. Cross-provider generalisation degraded compared with same-host performance in some evaluations.
Implications
Offensively, an adversary co-located on separate MIG instances can create high-bandwidth covert channels and perform cross-tenant fingerprinting of GPU and CPU workloads, including identifying LLM models and estimating processed prompt length to within a few tokens. These vectors exploit shared hardware counters and a shared driver pipeline, calling into question current MIG isolation guarantees in multi-tenant GH200 deployments and enabling data-exfiltration and reconnaissance that could aid further attacks.
The paper title, authors and abstract
PAGE404: Breaking GPU Isolation Through Hardware-assisted Page Migration
NVIDIA's Grace Hopper (GH200) superchip represents a new generation of heterogeneous computing, tightly integrating a 72-core Arm-based CPU with a Hopper GPU on a single module via a high-bandwidth, cache-coherent interconnect. By eliminating the conventional PCIe bottleneck for CPU-GPU data movement, this design establishes a unified memory domain that scales to the capacities demanded by generative AI and high-performance computing (HPC) workloads. The architecture also introduces hardware-assisted page migration: driven by access counters, the system dynamically promotes frequently accessed pages from host CPU memory to GPU DRAM to improve memory locality. We study the security of this emerging architecture against new side- and covert-channel attacks targeting its page migration support. We analyze counter-driven migration across both the hardware mechanisms and the UVM driver's interrupt-handling pipeline, and identify two side-channel vulnerabilities: one rooted in the shared structure of access counters, and the other in the driver's response to migration interrupts. We further show that both channels are exploitable across Multi-Instance GPU (MIG) partitions, a feature intended to provide strong, hardware-enforced isolation between co-located tenants. Using proof-of-concept attacks, we build covert channels that reach roughly 8,500 bits per second through eviction-based signaling and 8,100 bits per second through migration-delay-based signaling across MIG instances. The interrupt-based channel also fingerprints inference workloads across distinct large language models (LLMs) and CPU SPEC2017 benchmarks, and estimates the prompt length processed during LLM inference with high accuracy. These results expose a security gap in hardware-managed heterogeneous memory systems and call into question the isolation guarantees of current MIG-enabled multi-tenant GPU deployments.
