Chapter 1. Architecture and technical description
17
For a list of supported CAPI adapters, see 1.13.5, “CAPI enabled Infiniband adapters” on
page 27.
1.7.9 NVLink
NVLink is NVIDIA’s new high-speed interconnect technology for GPU-accelerated computing.
Supported on SXM2 based Tesla P100 accelerator boards, NVLink significantly increases
performance for both GPU-to-GPU communications, and for GPU access to system memory.
Today, multiple GPUs are common in workstations as well as the nodes of HPC computing
clusters and deep learning training systems. A powerful interconnect is extremely valuable in
multiprocessing systems. Our vision for NVLink was to create an interconnect for GPUs that
would offer much higher bandwidth than PCI Express Gen3 (PCIe), and be compatible with
the GPU ISA to support shared memory multiprocessing workloads.
Support for the GPU ISA means that programs running on NVLink-connected GPUs can
execute directly on data in the memory of another GPU as well as on local memory. GPUs
can also perform atomic memory operations on remote GPU memory addresses, enabling
much tighter data sharing and improved application scaling.
NVLink uses NVIDIA’s new High-Speed Signaling interconnect (NVHS). NVHS transmits data
over a differential pair running at up to 20 Gb/sec. Eight of these differential connections form
a
Sub-Link
that sends data in one direction, and two sub-links - one for each direction - form a
Link
that connects two processors (GPU-to-GPU or GPU-to-CPU). A single Link supports up
to 40 GB/sec of bidirectional bandwidth between the endpoints. Multiple Links can be
combined to form
Gangs
for even higher-bandwidth connectivity between processors. The
NVLink implementation in Tesla P100 supports up to four Links, allowing for a gang with an
aggregate maximum theoretical bandwidth of 160 GB/sec bidirectional bandwidth.
Summary of Contents for S822LC
Page 2: ......
Page 10: ...THIS PAGE INTENTIONALLY LEFT BLANK...
Page 14: ...xii IBM Power Systems S822LC for High Performance Computing...
Page 74: ...60 IBM Power Systems S822LC for High Performance Computing...
Page 78: ...64 IBM Power Systems S822LC for High Performance Computing...
Page 79: ......
Page 80: ...ibm com redbooks Printed in U S A Back cover ISBN REDP 5405 00...