HP XC System Software Administration Guide Version 3.2

You can try to ping other nodes that are connected to the network.
8. You can find additional information about InfiniBand in the /proc/voltaire directory.
Use the find command to display it:
# find /proc/voltaire -type f -print -exec cat {} \;
21.4.4 OFED Troubleshooting Procedures
Starting with Version 3.2, the HP XC System Software uses the OpenFabrics Enterprise Distribution
(OFED) InfiniBand software stack.
OFED is an open software stack supported by the major InfiniBand vendors as the future of
InfiniBand support. OFED offers improved support of multiple HCAs per node. The OFED stack
has a different structure and different commands from the InfiniBand stack that was used in
previous HP XC releases.
See the following Web page for more information about OFED:
http://www.openfabrics.org/
To determine if your HP XC system is configured properly, perform the following steps on any
node on which you suspect a problem:
1. Use the lspci command to ensure that your system has InfiniBand boards installed, and
that the operating system detects them:
[root@n1 ~]# lspci -v
44:00.0 InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx HCA] (rev 2 0)
Subsystem: Hewlett-Packard Company: Unknown device 170a
Flags: bus master, fast devsel, latency 0, IRQ 233
Memory at fdf00000 (64-bit, non-prefetchable) [size=1M]
Memory at fd000000 (64-bit, prefetchable) [size=8M]
Capabilities: [40] Power Management version 2
Capabilities: [48] Vital Product Data
Capabilities: [90] Message Signalled Interrupts: 64bit+ Queue=0/5 Enable -
Capabilities: [84] MSI-X: Enable- Mask- TabSize=32
Capabilities: [60] Express Endpoint IRQ 0
The information reported may differ, depending on the type of InfiniBand board used, but
you should see output for a board with the string "InfiniBand".
If you do not see output for an InfiniBand board, verify the following:
• An InfiniBand board is installed.
• The board is seated properly.
• The board functions properly. You might have to swap it with an identical working
board to verify this.
2. Run the ibv_devinfo command to verify that the InfiniBand system interconnect is
connected and that it is operating correctly:
[root@n1 ~]# ibv_devinfo
hca_id:mthca0
fw_ver:1.2.0
node_guid:0017:08ff:ffd1:33b4
sys_image_guid:0017:08ff:ffd1:33b7
vendor_id:0x02c9
vendor_part_id:25204
hw_ver:0xA0
board_id:HP_0010000001
phys_port_cnt:1
port:1
state:PORT_ACTIVE (4)
max_mtu:2048 (4)
active_mtu:2048 (4)
sm_lid:1
port_lid: 3
port_lmc: 0x00
21.4 System Interconnect Troubleshooting 257