NVIDIA announced last week that with IBM they will be building two supercomputers for the US Department of Energy. The supercomputers will be installed in 2017 at the Oak Ridge National Laboratory and in the Lawrence Livermore National Laboratory. The supercomputers will use NVIDIA’s Tesla GPU accelerators and using NVIDIA NVLink GPU interconnect technology. NVLink allows the GPUs a direct channel to communicate rather than going back to the PCIe bus. This video explains and visualizes NVLink:
The following chart shows the gains for different applications using NVLink compared to using the PCI-Express bus. Applications could see an increase in performance from 1.25x to 2.25 times.