HP InfiniBand Solution for Oracle RAC Environments white paper

• Set the host_mode for each connection to Linux. In a redundant configuration, each node in the

cluster should have 2 connections, one for each port on the HBA.

• If Selective Storage Presentation (SSP) is used to control access to the LUNs, make sure to present

the LUNs used for storage by the database to ALL of the nodes in the cluster.

Installing HP PDC for Oracle10

RAC on Linux

Follow the procedure described in the Installing HP PDC for Oracle 10g RAC on Linux IA32 (PDC

Installation) document. The PDC installation procedure outlines all of the steps necessary to install and

configure the operating system, servers, and storage to run in an Oracle RAC environment. The PDC

kit provides a set of scripts that automate the installation of HP and Oracle software pieces. Once the

procedure has been completed, the system should be ready to accept an Oracle10g installation. Prior

to proceeding with the Oracle installation, the InfiniBand components should be configured.

Installation of InfiniBand software components

Follow the instructions in the Installing the HCA Drivers section of the Dual-Port 4x Adapter User

Guide to install the HCA drivers. The drivers will automatically be installed with support for all of the

protocols used by Oracle10g RAC including uDAPL and IPoIB.

Once the HCA drivers have been installed, follow the instructions in the Configuring IPoIB Drivers

section of the Dual-Port 4x Adapter User Guide to configure an IB interface for use as the cluster

interconnect. In most cases, the PDC installation scripts will have configured eth1 as the cluster

interconnect. The interface should be disabled and the ib interface should be configured as a

replacement. In order to disable eth1 and configure the virtual InfiniBand NIC, use the following

procedure on each node:

1. Unmount any OCFS volumes that are currently mounted.

2. Shutdown the eth interface using the following command:

ifdown ethX

3. Edit the /etc/sysconfig/network-scripts/ifcfg-ethX file, setting the ONBOOT parameter to the value

NO in order to prevent the eth device from starting up automatically on boot. Below are the

contents of an example configuration file:

# Broadcom Corporation|NetXtreme BCM5703 Gigabit Ethernet

DEVICE=eth1

BOOTPROTO=static

HWADDR=00:02:A5:EF:5F:8E

IPADDR=192.168.0.1

NETMASK=255.255.255.0

ONBOOT=no

TYPE=Ethernet

4. Create the /etc/sysconfig/network-scripts/ifcfg-ib0 file for the virtual InfiniBand NIC. Make sure

to use the same networking configuration for the ib device as the eth device. Below are the

contents of an example configuration file:

DEVICE=ib0

BOOTPROTO=static

IPADDR=192.168.0.1

NETMASK=255.255.255.0

ONBOOT=yes

TYPE=Ethernet

5. Startup the ib interface using the following command:

ifup ib0