HP Fabric Clustering System for InfiniBand™ Interconnect Performance on HP-UX 11iv2

Confidential Page 6 1/28/2005
Bi-directional 1-way latencies on a rx2600 in point-to-point configuration are 5.6usec on each stream in the RDMA
programming model for 8-byte messages, and 7.1usec on each stream in the Send/Receive programming model for
1-byte messages. The bi-directional latencies for short messages, up to 512 bytes, remain pretty close to the
respective uni-directional latencies. On a rx2600 in point-to-point configuration, the bi-directional 1-way latencies for
512 byte messages are 7.6usec on each stream in RDMA programming model, and 9.1usec on each stream in the
Send/Receive programming model.
Bandwidth
The HP-UX Fabric Clustering System interconnect solution offers high bandwidth rates in the order of 700MB/s right
from messages of length 16KB and more. Service demand requirements on the server falls exponentially as the
message size increases providing the applications with a high bandwidth at lower CPU utilization rates.
Point-to-Point Bandwidth on rx2600 (2 CPUs/1.5GHz/4GB RAM)
200
250
300
350
400
450
500
550
600
650
700
750
800
850
1 10 100 1000 10000
Message Length (KB)
Bandwidth (MB/s)
0
5
10
15
20
25
30
35
CPU Utilization (%
)
Uni-Directional Bandw idth on
rx2600
Bi-Directional Bandw idth on
rx2600
CPU Utilization - Uni-Directional
per-CPU Utilization - Bi-
Directional
Under bi-directional traffic loads, an acid test for any system performance, HP-UX Fabric Clustering System
interconnect solution offers excellent performance results. The bi-directional bandwidth on a rx2600 crosses 731MB/s
at a short message length of 8KB with a service demand of only 13% per-CPU utilization. The bi-directional
bandwidth is almost equally split across each individual stream for all message sizes.
HP-UX Fabric Clustering System interconnect solution scales almost linearly (approximately 2X for 2 HCAs), in both
uni-directional as well as bi-directional bandwidth rates on a rx4640 with 2 HCAs.