How to config and look up your RDMA deivce

Posted on

RDMA technology is built upon Infiniband NIC, and OFED package is the basic software stack for RDMA. I will show some methods to look up and test your Infiniband NIC and make sure your RDMA library works well.

  1. Using lspci to look up your IB’s Firmware version.

    1
    lspci | grep Mellanox

  2. Start your opensm service (a service for running RDMA).

    1
    /etc/init.d/opensmd start[restart...]

  3. Get information of the whole Infiniband network.

    1
    2
    3
    ibhosts
    iblinkinfo
    ibswitches

  4. A useful tools works to check whether your IB’s status.

    1
    2
    3
    4
    5
    6
    ibstat
    ibstatus
    ib_devinfo
    ib_device
    ifconfig ibx
    cat /sys/class/infiniband/mlx4_0/*

  5. Tools to benchmark the Throughput(Mpps)/Latency(us)/Bandwidth(Gb) of RNIC.

    1
    ib_[read|write|atomic|send]_[bw|lat]

  6. pingpong test

    1
    ib_[rc|uc|srq|task|ud|xsrq|cc]_pingpong