Driver Software and Firmware
Rev 1.6
Mellanox Technologies
40
# cat /sys/module/ib_core/parameters/gpu_direct_enable => 1
The number of shared pages by GPUDirect is reported under:
# cat /sys/module/ib_core/parameters/gpu_direct_shares
4.4.2.3
NVIDIA Driver Installation:
Install NVIDIA Development Driver for Linux x86_64, available under:
http://developer.nvidia.com/object/cuda_3_0_downloads.html
For example:
- Run: devdriver_3.0_linux_64_195.36.15.run
- Follow the installation wizard instructions
To make sure that the NVIDIA driver was installed successfully:
- Load nvidia driver:
# modprobe nvidia
- Check the driver version, for example:# cat /proc/driver/nvidia/version => version 195.36.15 (or later)
4.5
CORE-Direct (Collectives Offload Resource Engine)
CORE-Direct (Collectives Offload Resource Engine) provides the most advanced solution for
handling collectives operations, ensures maximum scalability, minimizes the CPU overhead and
provides the capability for overlapping communications with computations. Mellanox Con-
nectX®-2 adapters address the collectives communication scalability problems by offloading the
communications to the adapters and switches.
4.5.1 Hardware and Software Requirements
Software:
Operating system
RHEL 5.4 or later
Mellanox OFED 1.5.1 or later
Hardware:
Mellanox ConnectX®-2 adapter card
Disk Space for Installation 400 MB
4.5.2 Installation
For installation instructions visit:
http://www.mellanox.com
=> Support => VPI SW/Driver.
4.6
RDMA over Converged Ethernet (RoCE)
ConnectX-2 with RoCE utilizes advances in Data Center Bridging (DCB) to enable efficient and
low cost implementations of RDMA over Ethernet, supporting the entire breadth of RDMA and
