13–NVMe-oF Configuration with RDMA
Testing the NVMe-oF Devices
236
BC0154501-00 P
Testing the NVMe-oF Devices
Compare the latency of the local NVMe device on the target server with that of the
NVMe-oF device on the initiator server to show the latency that NVMe adds to the
system.
To test the NVMe-oF device:
1.
Update the Repository (Repo) source and install the Flexible Input/Output
(FIO) benchmark utility on both the target and initiator servers by issuing the
following commands:
#
yum install epel-release
#
yum install fio
Figure 13-4. FIO Utility Installation
2.
Run the FIO utility to measure the latency of the initiator NVMe-oF device.
Issue the following command:
#
fio --filename=/dev/nvme0n1 --direct=1 --time_based
--rw=randread --refill_buffers --norandommap --randrepeat=0
--ioengine=libaio --bs=4k --iodepth=1 --numjobs=1
--runtime=60 --group_reporting --name=temp.out
FIO reports two latency types: submission and completion. Submission
latency (slat) measures application-to-kernel latency. Completion latency
(clat), measures end-to-end kernel latency. The industry-accepted method is
to read
clat percentiles
in the 99.00th range.
In this example, the initiator device NVMe-oF latency is 30µsec.
3.
Run FIO to measure the latency of the local NVMe device on the target
server. Issue the following command:
#
fio --filename=/dev/nvme0n1 --direct=1 --time_based
--rw=randread --refill_buffers --norandommap --randrepeat=0
--ioengine=libaio --bs=4k --iodepth=1 --numjobs=1
--runtime=60 --group_reporting --name=temp.out
