Modal Claims NVIDIA Vera CPUs Offer 2X Throughput Over Intel And AMD
Cloud provider Modal, which offers on-demand infrastructure for AI developers, has released performance data for the new NVIDIA Vera CPUs. The company claims that in high-concurrency workloads, the Vera architecture can achieve up to twice the throughput of current Intel (Granite Rapids) and AMD (Zen 5 EPYC) systems.
Throughput vs. Latency and the Memory Factor
To evaluate the hardware, Modal developed the Agentless Agent Bench, a suite of 50 tasks simulating AI agent activities, such as code compilation, file searching, Git operations, and device emulation. Crucially, the tests focused on throughput—measuring how many tasks a system can complete within a fixed timeframe under heavy concurrent load—rather than single-task serial latency.
Modal attributes this performance leap primarily to memory bandwidth. The Arm-based Vera CPUs offer up to 1.2 TB/s of LPDDR5X bandwidth. In comparison, Intel Granite Rapids provides approximately 844 GB/s, while AMD's Zen 5 EPYC platform delivers 576 GB/s. This increased bandwidth allows Vera to host more concurrent sandbox environments per core without the performance degradation seen in competing architectures when workloads scale.
Context and Competitive Landscape
The results, however, require careful interpretation. When testing serial latency on idle systems, Vera did not win every test, and Modal itself noted that the advantage was often modest. The performance gap between Arm and x86 architectures may also reflect varying levels of software optimization for different instruction sets rather than raw hardware superiority alone.
It is also important to note that Vera is a highly specialized processor optimized for the low-latency, high-concurrency demands of AI agents and reinforcement learning. This differs from the general-purpose nature of Intel and AMD server CPUs designed for diverse workloads like databases or high-performance computing (HPC). Nevertheless, the competition is moving fast: AMD is expected to bring up to 1.6 TB/s of bandwidth with its Venice processors, and Intel aims for a significant boost with its Diamond Rapids lineup expected in 2027.
For now, Modal intends to integrate Vera alongside its existing Intel and AMD fleets rather than replacing them entirely. While the throughput claims are compelling for specific AI-centric workloads, the industry awaits independent third-party benchmarks to validate these findings outside of Modal's proprietary testing environment.
Гика
