HP Fabric Clustering System for InfiniBand™ Interconnect Performance on HP-UX 11iv2

Confidential Page 4 1/28/2005
Cluster Solution Performance – Point-to-Point Configuration
A pair of rx2600/rx4640 Integrity servers are connected via a single HCA on each server. One port on each HCA is
connected to the other using a 4X cable.
VGA CONSOLE / R EMOTE / UPS
LAN 10/100
LAN Gb
SCSI LVD/SE
USB
CONSOLE
SERIAL A
SERIAL B
PWR
1
PWR
2
Management Card
LAN 10/100
GSP RESE TS
SOFT HARD
TOC
PCI-X 133
PCI-X 133
PCI-X 133
PCI-X 133
Infiniband
rx2600 rx2600
IB 4X Cable
VGA CONSOLE / REMOTE / UPS
LAN 10/100
LAN Gb
SCSI LVD/SE
USB
CONSOLE
SERIAL A
SERIAL B
PWR
1
PWR
2
Management Card
LAN 10/100
GSP RESETS
SOFT HA RD
TOC
PCI-X 133
PCI-X 133
PCI-X 133
PCI-X 133
Infiniband
!
!
1
3
4
5
6
7
8
2
System Input Power
VAC 200-240 8 A ma x
50/60 Hz
01
M
P
LA
N
U
S
B
1
U
S
B
2
M
P
l
o
c
a
l
a
u
x
i
li
a
r
y
r
e
m
o
t
e
PCI-X 66 PCI-X 66 PCI-X 66 PCI-X 66
PCI-X 13 3 PCI-X 13 3
PCI- X 66 (LAN)
P CI- X 66 ( SC S I)
Dual SCSI
1000T LAN
Infiniband
IB 4X Cable
rx4640 rx4640
!
!
1
3
4
5
6
7
8
2
Sys tem Input Power
VAC 200-24 0 8 A max
50/60 Hz
01
M
P
LA
N
U
S
B
1
U
S
B
2
M
P
l
o
c
a
l
a
u
x
i
li
a
r
y
r
e
m
o
t
e
PCI-X 66 PCI-X 66 PCI-X 66 PCI-X 66
PCI-X 13 3 PCI-X 13 3
PCI- X 66 (LAN)
P CI- X 66 ( SC S I)
Dual SCSI
1000T LAN
rx4640
Infiniband
For the HCA scalability tests, 2 HCAs on rx4640 are connected in point-to-point configuration to two HCAs on
another rx4640.
!
!
1
3
4
5
6
7
8
2
System Input Power
VAC 200-240 8 A ma x
50/60 Hz
01
M
P
LA
N
U
S
B
1
U
S
B
2
M
P
l
o
c
a
l
a
u
x
i
li
a
r
y
r
e
m
o
t
e
PCI-X 66 PCI-X 66 PCI-X 66 PCI-X 66
PCI-X 13 3 PCI-X 13 3
PCI- X 66 (LAN)
PCI- X 66 ( SC S I)
Dual SCSI
1000T LAN
Infiniba nd
IB 4X Cable
rx4640 rx4640
!
!
1
3
4
5
6
7
8
2
Sys tem Input Power
VAC 200-24 0 8 A max
50/60 Hz
01
M
P
LA
N
U
S
B
1
U
S
B
2
M
P
l
o
c
a
l
a
u
x
i
li
a
r
y
r
e
m
o
t
e
PCI-X 66 PCI-X 66 PCI-X 66 PCI-X 66
PCI-X 13 3 PCI-X 13 3
PCI- X 66 (LAN)
PCI- X 66 ( SC S I)
Dual SCSI
1000T LAN
rx4640
Infiniba nd Infiniba nd Infiniba nd
IB 4X Cable
Latency
On rx2600 servers in point-to-point configuration, HP-UX cluster delivers 1-way latency of 4.3usec for 8-byte
messages using the RDMA programming model and 7.0usec for 1-byte messages using the Send/Receive
programming model. The latency remains low for larger messages. On rx2600 servers in point-to-point configuration,
a 512 byte message gets a 1-way latency of 7.1usec using the RDMA programming model and 9.0usec using the
Send/Receive programming model. Cumulative service demand on the available processors is 100% as the itperf
application polls for completion events.