HP InfiniBand Solution for Oracle RAC Environments white paper

ManualsBrandsHP ManualsServerHP ProLiant DL320 G3 Server

HP InfiniBand solution for Oracle RAC environments

Overview............................................................................................................................................ 2

Oracle RAC overview....................................................................................................................... 2

Protocols used by Oracle RAC in an InfiniBand environment.............................................................. 3

Internode synchronization in an Oracle RAC environment ................................................................. 4

Advantages of InfiniBand in an Oracle RAC environment.................................................................. 5

HP InfiniBand solution for Oracle RAC................................................................................................ 5

Solution overview ......................................................................................................................... 5

Sample 4-node configuration ......................................................................................................... 6

Deployment of the HP InfiniBand solution for Oracle RAC......................................................................... 7

HP PDC Installation Service ............................................................................................................... 7

Service benefits............................................................................................................................ 8

HP PDC manual installation process ................................................................................................... 8

Hardware setup and configuration ................................................................................................. 8

Installing HP PDC for Oracle10g RAC on Linux................................................................................ 9

Installation of InfiniBand software components ................................................................................. 9

Oracle installation ...................................................................................................................... 11

Oracle uDAPL support................................................................................................................. 11

Verifying that the uDAPL protocol is being used for inter-node communication.................................... 12

Conclusion........................................................................................................................................ 12

For more information.......................................................................................................................... 13

Summary of content (13 pages)

PAGE 1
HP InfiniBand solution for Oracle RAC environments Overview............................................................................................................................................ 2 Oracle RAC overview....................................................................................................................... 2 Protocols used by Oracle RAC in an InfiniBand environment..............................................................
PAGE 2
Overview Oracle® Real Application Clusters (RAC) has a scale out, shared-disk clustering architecture. In a shared-disk architecture, all nodes in the cluster have access to, and can manipulate, the same data set. Concurrent access to the data from multiple nodes is synchronized through a private cluster interconnect network.
PAGE 3
Figure 1. Example high level system diagram Protocols used by Oracle RAC in an InfiniBand environment There are 3 major components in an Oracle RAC cluster that make use of the cluster interconnect: • Oracle Cluster Ready Services (CRS) • Database or ASM instances • Oracle Cluster File System (OCFS) On Linux, each of these components is designed to communicate across a standard Ethernet network using the UDP protocol.
PAGE 4
Figure 2. Protocols used in an Oracle RAC environment with InfiniBand Internode synchronization in an Oracle RAC environment Because RAC is a shared-disk clustering architecture, the database instances on each node must cooperate in order to maintain the integrity of the data. Database users could potentially attempt to update the same records from different nodes. The database instances must coordinate user access to the data to ensure the database does not become corrupted.
PAGE 5
write/write). In both cases, the node that holds the initially updated data block ships the block to the requesting node across the high speed cluster interconnect. In the case of the write/read case scenario, a node wants to update a block that’s already read and cached by a remote instance. An update operation typically involves reading the relevant block into memory and then writing the updated block back to disk.
PAGE 6
Figure 3.
PAGE 7
Quantity Part Number Description Shared Storage 4 321835B21 StorageWorks 2 Gb, Dual Port, 64-Bit/133 MHz PCI-X-to-Fibre Channel Host Bus Adapter for W2K, Microsoft® Windows® Server 2003 and Linux 8 221692B22 5 Meter FC LC-LC Multi-Mode cable 2 288247B21 MSA SAN Switch 2/8 (embedded 8 port FC Switch) 1 201723B22 Modular Smart Array 1000 with 256 MB cache 1 218231B22 MSA1000 Controller with 256 MB cache 1 302970B21 HP StorageWorks Modular Smart Array 30 Dual Bus Enclosure 1 400982002 SC
PAGE 8
the HP predefined cluster solutions. For cluster solutions built from other hardware configurations or for those that scale beyond the boundaries of this service, HP offers other cluster configuration services to meet your requirements.
PAGE 9
• Set the host_mode for each connection to Linux. In a redundant configuration, each node in the cluster should have 2 connections, one for each port on the HBA. • If Selective Storage Presentation (SSP) is used to control access to the LUNs, make sure to present the LUNs used for storage by the database to ALL of the nodes in the cluster. Installing HP PDC for Oracle10g RAC on Linux Follow the procedure described in the Installing HP PDC for Oracle 10g RAC on Linux IA32 (PDC Installation) document.
PAGE 10
6. Verify that the ib device started successfully by examining the output of the command ifconfig. Output from the command should contain something similar to the following: ib0 Link encap:Ethernet HWaddr DD:8B:86:C8:4D:0B inet addr:192.168.0.1 Bcast:192.168.0.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:2044 Metric:1 RX packets:967 errors:0 dropped:0 overruns:0 frame:0 TX packets:1264 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:128 RX bytes:65612 (64.
PAGE 11
Oracle installation Once the ib interface has been configured on all of the nodes, Oracle10g can be installed using the procedure outlined in the Oracle® Real Application Clusters Installation and Configuration Guide 10g Release 1 (10.1) for AIX-Based Systems, Apple Mac OS X, hp HP-UX, hp Tru64 UNIX®, Linux, Solaris Operating System, and Windows Platforms. When installing Oracle’s Cluster Ready Services (CRS), the ib device should be configured for use as the private cluster interconnect.
PAGE 12
representative in order to gather the needed patches and information necessary to enable uDAPL and OCFS support. Verifying that the uDAPL protocol is being used for inter-node communication Once all of the Oracle software components have been successfully installed and a database has been created, the alert log can be examined to determine if the instance is using uDAPL for interinstance communication.
PAGE 13
For more information For additional information, refer to the resources detailed below. Resource description Web address HP ProLiant InfiniBand options http://h18004.www1.hp.com/products/servers/networking/index -ib.html HP StorageWorks Modular Smart Arrays (MSA) http://h18006.www1.hp.com/storage/arraysystems.html HP ProLiant servers http://h18004.www1.hp.com/products/servers/platforms/ HP Parallel Database Clusters (PDC) for Oracle RAC http://h18004.www1.hp.