NVLink

NVLink
Name	NVLink
Developer	NVIDIA
Introduced	2016
Type	High-speed interconnect
Use	GPU-to-GPU and GPU-to-CPU communication

NVLink NVLink is a proprietary high-speed interconnect developed to accelerate data movement between processors and memory in high-performance computing and artificial intelligence platforms. It was introduced by NVIDIA to complement PCI Express and to address bandwidth and latency limitations in multi-processor systems used by organizations such as Oak Ridge National Laboratory, Lawrence Livermore National Laboratory, Argonne National Laboratory, and corporations like Amazon Web Services, Microsoft, Google, and Facebook. NVLink has been integrated into systems from vendors including IBM, Dell Technologies, HP Enterprise, Lenovo, and Supermicro for workloads in research, cloud, and enterprise environments.

Overview

NVLink provides point-to-point and mesh interconnect topologies that enable coherent and non-coherent memory access between devices such as NVIDIA Tesla, NVIDIA Volta, NVIDIA Turing, NVIDIA Ampere, and NVIDIA Hopper GPUs as well as select CPU products like IBM POWER9 and partnerships involving Intel initiatives. Designed to overcome bottlenecks witnessed in systems during projects like the Summit (supercomputer) and Perlmutter (supercomputer), NVLink targets workloads from institutions such as Los Alamos National Laboratory, CERN, and NASA centers. It interfaces with software ecosystems including CUDA, OpenACC, OpenMP, and MPI used by researchers at Stanford University, Massachusetts Institute of Technology, University of California, Berkeley, and Carnegie Mellon University.

History and Development

Development began within NVIDIA amid collaborations with partners such as IBM, Cray Research, and system integrators responding to demands from projects like Exascale Computing Project and national initiatives in the United States Department of Energy. Early demonstrations targeted scientific workloads developed with libraries like cuBLAS, cuDNN, and frameworks such as TensorFlow, PyTorch, and MXNet. NVLink iterations accompanied GPU microarchitecture generations from Pascal (microarchitecture) through Volta (microarchitecture), Turing (microarchitecture), Ampere (microarchitecture), to Hopper (microarchitecture), reflecting design trade-offs informed by research at Lawrence Berkeley National Laboratory and collaborations with companies like NVIDIA DGX customers and consortiums including OpenAI, DeepMind, and Microsoft Research.

Architecture and Technical Specifications

NVLink uses multiple physical lanes aggregated into links with differential signaling and proprietary protocols governed by NVIDIA hardware teams and validated in labs such as IARPA-funded facilities and corporate R&D centers in Santa Clara, California and Cambridge, UK. Implementations specify lane counts, link widths, and per-link bandwidths that evolved across generations providing per-link bandwidth improvements compared to PCI Express 3.0 and PCI Express 4.0. NVLink supports cache coherence in specific host integrations like IBM POWER9 systems and implements topology-aware routing used in clusters like Summit and Selene. It interoperates with system software stacks including Linux, NVIDIA CUDA Toolkit, NVIDIA Collective Communications Library, and orchestration platforms such as Kubernetes for deployment in environments maintained by companies like Nutanix and Red Hat.

Performance and Scalability

NVLink’s performance claims were benchmarked against multi-socket and multi-GPU setups in studies by teams at Argonne National Laboratory and universities including University of Illinois Urbana–Champaign and Georgia Institute of Technology. It improves effective bandwidth and reduces latency in multi-GPU training used by projects at Stanford AI Lab, Berkeley AI Research, and industrial groups at Microsoft Azure and Amazon EC2. Scalability patterns influenced system topologies in supercomputers such as Summit, Perlmutter, and enterprise AI appliances like NVIDIA DGX A100 and HPE Apollo. Real-world scaling for models trained with BERT, GPT, and other transformer architectures benefited from NVLink-enabled aggregation of memory and synchronization using libraries like Horovod and NCCL.

Implementations and Supported Products

NVLink has been implemented in NVIDIA product lines including NVIDIA Tesla V100, NVIDIA A100, NVIDIA H100, and in workstation and server products like NVIDIA Quadro GV100, NVIDIA DGX Station, NVIDIA DGX-1, NVIDIA DGX-2, and integrated platforms by IBM Power Systems. Cloud offerings from NVIDIA Cloud, Amazon Web Services EC2 P4d, Google Cloud TPU-adjacent GPU instances, and Microsoft Azure ND A100 series leveraged NVLink in hardware designs by OEMs such as Dell EMC, Hewlett Packard Enterprise, Lenovo ThinkSystem, and Inspur.

Use Cases and Applications

NVLink targets large-scale deep learning in projects by OpenAI, DeepMind, and academic groups at MIT CSAIL, large-scale simulation at Los Alamos National Laboratory and Sandia National Laboratories, medical imaging workloads at universities like Johns Hopkins University, financial modeling at firms like Goldman Sachs and JPMorgan Chase, and graphics rendering pipelines at studios using Autodesk and Blender Foundation tools. It accelerates distributed training in frameworks including TensorFlow, PyTorch, MXNet, and HPC applications employing libraries such as PETSc and Trilinos used at national labs and research centers.

Criticisms and Limitations

Critiques highlight NVLink’s proprietary nature compared with open standards championed by consortia like PCI-SIG and the OpenCAPI initiative; organizations such as European Commission funded projects emphasize open interconnects. Integration is limited to select CPUs such as IBM POWER9 and specific NVIDIA GPUs, raising concerns for heterogeneous clusters using Intel Xeon or AMD EPYC processors. Some research groups at University of Cambridge and ETH Zurich noted complexity in topology management and cost considerations for adoption in smaller institutions and startups. Competition and alternative approaches include efforts by AMD Infinity Fabric, Intel Omni-Path Architecture, and ecosystem work from Mellanox Technologies (now part of NVIDIA), affecting procurement choices by hyperscalers like Meta Platforms, Alibaba Group, and Tencent.

Category:Computer buses