An NVIDIA H100 GPU costing tens of thousands of dollars sits idle, waiting for data to arrive. In AI inference, this is an everyday occurrence. Mark Menger, a solutions architect at F5, puts it this way: "The GPU is rarely the bottleneck. It's waiting for data." While GPU compute power has grown roughly 18x from the V100 to the B200, HBM bandwidth growth has only kept pace at about 9x. The gap between compute speed and data supply speed keeps widening, and the pathway from storage to GPU memory has become the deciding factor in the overall efficiency of AI infrastructure.
The root of this problem lies in a historical design decision in server architecture. In traditional x86 servers, the CPU held both the control path and the data path for storage access. The GPU was merely a "subordinate accelerator" that received data prepared by the CPU.
The Bounce Buffer: A 30-Year-Old Intermediary
When a GPU reads data from storage, the conventional path works like this: first, a DMA transfer moves data from storage to the CPU's system memory (DRAM), and from there it is copied again into GPU memory. This temporary copy region in CPU memory is called a bounce buffer. While CPU mediation guarantees memory consistency and access control, it inevitably introduces an extra copy and CPU processing delay.
According to measurements NVIDIA published in 2019, on a DGX-2 system the bandwidth from CPU system memory to the GPU was about 12 GB/s per PCIe tree, totaling around 50 GB/s. In contrast, direct transfer from NVMe drives to GPU memory using GPUDirect Storage reached a total of 215 GB/s. This confirmed roughly a 4x improvement in bandwidth and up to a 3.8x improvement in latency. CPU load also dropped to one-third.
| Metric | Via Bounce Buffer | GPUDirect Storage (cuFile) |
|---|---|---|
| Peak bandwidth on DGX-2 | ~50 GB/s | Up to 215 GB/s |
| Latency for 80 GB data transfer | Baseline | 3.8x reduction |
| CPU utilization | Baseline | Reduced to ~1/3 |
| Accessible data volume | Limited to CPU memory capacity (~1 TB) | Storage capacity (petabyte-scale) |
GPUDirect Storage was officially released in July 2021 alongside CUDA Toolkit 11.4, with the cuFile API serving as its core. cuFile transports data via DMA directly from storage devices such as NVMe drives to GPU memory. Because it bypasses the CPU and system memory, the extra copy disappears, and the GPU can fetch data in parallel across hundreds of thousands of threads.
However, GPUDirect Storage had a structural limitation. While the data path was connected directly to the GPU, the CPU was still the one issuing I/O "commands."
Inference Workloads Expose the Control-Path Bottleneck
In AI training, large blocks are read in bulk, so control-path delay is relatively small compared to the whole. The problem is inference. Retrieval-augmented generation (RAG), agentic AI, and Mixture-of-Experts models read small blocks under 4 KB randomly and in massive volume. Referencing KV caches, retrieving tool-call results, and loading sparse expert weights all fall into this pattern.
Chander Chadha, director of flash storage product marketing at Marvell, points out: "Current SSDs can't respond adequately in terms of IOPS for datasets under 4 KB, leaving the PCIe bus idle. The GPU wastes cycles waiting for data." In SCADA environments, the GPU issues I/O simultaneously across 1,000 or more parallel threads, but as long as the CPU holds the control path, this parallelism cannot be fully conveyed to the storage side.
SCADA Wrests the Control Path Away from the CPU
NVIDIA's answer to this problem is SCADA (Scaled Accelerated Data Access). SCADA is an architecture in which the GPU itself initiates and controls storage I/O. Whereas GPUDirect Storage moved the data path from the CPU to the GPU, SCADA moves the control path to the GPU as well. GPU threads access storage directly using load/store instruction semantics, and the NVMe driver itself runs inside the GPU.
The academic foundation for this architecture is the BaM (Big Accelerator Memory) paper presented at ASPLOS in 2023. Proposed by a research team at the University of Illinois Urbana-Champaign, BaM was the first system architecture in which GPU threads access storage on demand without going through the CPU. It builds a software cache in GPU memory to consolidate redundant accesses and communicates directly with storage devices via highly parallel submission/completion queues. In experiments, it achieved a 5.3x speedup compared to CPU-initiated storage access and reduced hardware costs by up to 21.7x compared to holding data entirely in DRAM.
SCADA incorporates this BaM design philosophy into NVIDIA's product stack. At the SC25 conference in November 2025, Micron presented a demonstration using the SCADA programming model. On an H3 Platform Falcon 6048 server equipped with 44 Micron 9650 PCIe Gen6 SSDs, it recorded 230 million IOPS for 512-byte random reads. With each SSD delivering 5.4 million IOPS, the 44 drives scaled linearly, measuring a value close to the theoretical maximum of 237.6 million IOPS.
The Difficult Challenge of Balancing Security and Speed
Direct storage access comes with a cost. When an application interacts directly with a drive, there is a risk of overwriting another process's memory—the so-called "clobber" problem. Traditional operating systems have prevented this through privilege-level separation, page tables, and file system protocols. Allowing direct access risks bypassing protection mechanisms such as encryption and access control as well.
SCADA resolves this problem through architectural separation. According to NVIDIA's official blog, SCADA splits the work into two parts. The user application portion that needs speed sits outside the trusted computing base, gaining raw speed. Meanwhile, a separate privileged component exists that sets up protected access between the user application and approved storage, following standard Linux protocols. The privileged path is used only during setup, while the runtime data path is controlled directly by the GPU. This separation maintains security without sacrificing speed.
The Goal of Open-Sourcing and the Full Picture of Storage-Next
Open-sourcing the cuFile API is not simply about releasing code. NVIDIA's official blog states that "this API becomes a new hub for accepting open contributions," specifying that Google, Intel, NVIDIA, and Meta will participate as initial maintainers. cuFile is an interface that allows GPUs to read from and write to storage directly, not just the CPU, leveraging hundreds of thousands of GPU threads and fast high-bandwidth memory to achieve microsecond-level data access from storage.
Announced simultaneously, Storage-Next is a cross-industry initiative involving more than 40 storage and flash memory vendors. DDN, Kioxia, and Micron are among the participants, bringing together storage manufacturers, controller vendors, thermal design and cooling providers, orchestration providers, and standards bodies. The goal is to reach agreement on how GPU-driven storage should behave and to codify this into an interoperable, open industry standard.
Sven Oehme, CTO of DDN, states: "AI success isn't determined by how much infrastructure an organization owns, but by how productively they use it." DDN has announced plans to integrate SCADA into its software-defined data intelligence platform, Infinia.
Remaining Questions
How far will the open-sourcing of cuFile ripple beyond the CUDA ecosystem? At present, cuFile is tightly coupled to NVIDIA GPUs and the CUDA runtime, with no guarantee that AMD or Intel GPUs will adopt the same interface. Given that Storage-Next champions an "open industry standard," the formulation of vendor-neutral specifications is unavoidable—but to what extent will the structure in which NVIDIA effectively holds the specification-setting authority persist?
SCADA-compatible SSD controllers also remain immature. Marvell's Chadha states that, alongside faster PCIe Gen6 and Gen7 buses, a new generation of controllers is needed—ones equipped with SCADA accelerator functionality and error-correction schemes suited to small payloads. The Micron 9650 has begun shipping as the world's first PCIe Gen6 SSD, but the timeline for mass production of SCADA-optimized controllers spreading across the industry remains undetermined.
On the security front, verification is still in progress. No third-party evaluation has yet been published on how robust SCADA's separation architecture actually is against real multi-tenant environments and adversarial workloads. In a world where GPUs hold the control path for storage, the boundary of trust shifts from the CPU to the GPU and DPU. How that boundary is designed will determine the reliability of AI infrastructure for years to come.