NVIDIA opens cuFile: Open-Source Storage Stack for AI GPUs

The cuFile technology itself is not new: it is part of GPUDirect Storage and allows applications to transfer data between compatible storage and GPU memory without routing each data block through a classic CPU bounce buffer in system RAM. This is particularly important for AI and HPC tasks, where modern GPUs generate many parallel memory requests simultaneously. If every access requires the CPU to act as an intermediary, additional data copying, latency, and CPU load occur.

However, one should not take the phrase "storage becomes memory" too literally. SSDs do not replace HBM memory: even very fast NVMe systems remain far from local GPU-HBM in terms of latency and bandwidth. Storage is interesting as an additional level for data that does not need to constantly occupy expensive accelerator memory.

The novelty lies in opening the interfaces. At the Future of Memory and Storage (FMS) conference, NVIDIA announced that the cuFile API and part of the vertically integrated storage software stack would be available as open-source. The new GitHub organization xio-sig serves as the foundation.

In this organization, cuFile is intended to be a unified programming interface that can be extended for various platforms and manufacturers. Additionally, a conformance test package, the multi-platform library libxFile, and xioLinux as an Linux base are planned. Google, Intel, Meta, and NVIDIA are named as original maintainers of the project.

This is strategically more interesting than simply publishing source code: if storage manufacturers implement the same interface and pass joint conformance tests, theoretically the risk of incompatible data paths for each file system, platform, and accelerator is reduced. However, the current status of the project on GitHub shows that it is in an early stage: components are announced, but the code is not yet fully populated.

Parallel to this, NVIDIA is building a larger industrial program called Storage-Next. According to the company, more than 40 manufacturers and suppliers are participating, including DDN, KIOXIA, and Micron. The goal is to jointly develop interfaces and requirements for GPU-driven storage systems and create as interoperable standards as possible.

Another component is SCADA (Scaled Accelerated Data Access). It allows massively parallel GPUs to specifically retrieve only those data from storage that an application needs at the moment and transfer them directly into fast GPU memory. This could be beneficial for retrieval systems, large databases, and long AI contexts where it is not necessary to hold huge datasets entirely in HBM or system RAM.

NVIDIA also links this strategy with Vera and BlueField technologies. For example, for a two-stage compression and encryption pipeline using a Vera CPU, up to 3.21x throughput is claimed compared to an x86 system. However, this is NVIDIA's own benchmark and should be viewed as a manufacturer claim.

The key point here is not to replace HBM: technically this would be unrealistic due to fundamental differences in latency and bandwidth. NVIDIA is rather trying to unify the memory hierarchy of an AI system. HBM remains critical for current tensors, KV caches, and other time-critical data. Large datasets, model information, or context data can lie on cheaper storage levels and be loaded as needed as efficiently as possible.

As agent-based AI systems and their context volumes grow, this data path becomes increasingly important. If Intel, Meta, Google, storage manufacturers, and NVIDIA are indeed working towards compatible interfaces, this could create significant infrastructure below the level of the AI models themselves.

SSDs will not suddenly become HBM, but if data reaches the GPU in microseconds instead of unnecessary detours, and different manufacturers can use the same software path, then mass storage for AI systems will be much more interesting than just a large archive behind the server.