Support of Oracle RAC ASM with SGeRAC, January 2008

ManualsBrandsHP ManualsSoftwareHP Serviceguard Extension for RAC (SGeRAC)

Support of Oracle RAC ASM with SGeRAC

Jan 2008

Introduction......................................................................................................................................... 2

ASM Background ................................................................................................................................ 2

What is Automatic Storage Management (ASM)? ................................................................................ 2

Single Node (Non-Clustered) ASM Environment................................................................................... 3

Clustered ASM Environment .............................................................................................................. 5

SG/SGeRAC Support for ASM on HP-UX 11i v2..................................................................................... 6

Overview ........................................................................................................................................ 6

Why ASM over SLVM? ..................................................................................................................... 7

Configuring SLVM Volume Groups for ASM Disk Groups...................................................................... 7

SG/SGeRAC Support for ASM on HP-UX 11i v3................................................................................... 11

Overview ...................................................................................................................................... 11

ASM over SLVM ............................................................................................................................ 12

Configuring SLVM Volume Groups for ASM Disk Groups.................................................................... 12

Sample Command Sequence for Configuring SLVM Volume Groups ................................................ 14

ASM over Raw disk........................................................................................................................ 15

Configure Raw Disks/Disk Array Logical Units for ASM Disk Group..................................................... 16

Additional Hints on ASM Integration with SGeRAC ............................................................................... 16

Consider using the MNP/Simple Dependency-based SGeRAC Toolkit’s Framework ............................... 16

ASM Halt is needed to ensure disconnect of ASM from SLVM Volume Groups ...................................... 16

ASM may require Modified Backup/Restore Procedures ..................................................................... 17

Installation, Configuration, Support and Troubleshooting .................................................................... 17

Related Documentation and Scripts .................................................................................................. 18

Summary of content (18 pages)

PAGE 1
Support of Oracle RAC ASM with SGeRAC Jan 2008 1 Introduction......................................................................................................................................... 2 2 3 4 5 ASM Background ................................................................................................................................ 2 What is Automatic Storage Management (ASM)? ................................................................................
PAGE 2
Introduction This document discusses the use of the Oracle 10g Release 2 (10g R2) and11g Release 1 (11g R1) database server feature called Automatic Storage Management (ASM) in configurations of HP Serviceguard Extension for Real Application Clusters (SGeRAC). We begin with a brief review of ASM - its functionality, pros and cons, and method of operation. Then we look in detail at how we configure ASM with SGeRAC (version A.11.17 or later is required).
PAGE 3
Figure 1. Oracle database storage hierarchy without and with ASM Without ASM With ASM Tables Tablespaces Files 0111000 0111000 00100… 00100… .. .. VxFS, CFS,.. File Systems Logical Volumes Volume Groups (S)LVM, VxVM, CVM Files and Disk Groups Managed by ASM, displayed via Oracle utilities Disks/Disk Array Logical Units Please note that the 'with ASM' stack shows raw disks/disk array logical units (LUs) being used by ASM.
PAGE 4
• It has a set of foreground (asm1.. asmN) and background processes accessing a shared memory segment. The background processes include processes that resemble those of the database instance such as pmon, dbwr, .. as well as some additional ASM-specific processes such as rbal to perform disk group rebalancing. • It uses its foreground processes to interact with clients such as database instances and db utilities such as sqlplus.
PAGE 5
Clustered ASM Environment In an Oracle Clusterware environment, the individual ASM instances, one per node, cooperate to offer shared disk group access across the cluster. One or more RAC databases may be configured in the cluster, using the shared disk groups. Single instance databases may also be configured to use the shared disk groups but Oracle Clusterware does not provide built-in management and monitoring for them. The ASM instances are under the control of Oracle Clusterware.
PAGE 6
Oracle Clusterware maintains a resource dependency on the ASM instance running on a node from every RAC database instance on that node that uses ASM services; this ensures that the RAC database instances using the services of a failed ASM instance will be halted. The other ASM instances in the cluster are responsible to address the issue of operations only partially completed by the failed ASM instance and to preserve the integrity of the ASM disk group metadata.
PAGE 7
size of 16 Why ASM over SLVM? As mentioned above, we require ASM disk group members in SGeRAC A.11.17 (or later) configurations to be raw logical volumes managed by SLVM. The main reason for this requirement is to expedite ASM support with SGeRAC. We leverage existing HP-UX capabilities to provide multipathing for SLVM logical volumes, using either the PV Links feature, or separate products such as HP StorageWorks Secure Path that provide multipathing for specific types of disk arrays.
PAGE 8
The idea is that ASM provides the mirroring, striping, slicing and dicing functionality as needed and SLVM supplies the multipathing functionality not provided by ASM. Figure 4 indicates this 1-1 mapping between SLVM PVs and LVs used as ASM disk group members. Further, the default retry behavior of SLVM could result in an I/O operation on an SLVM LV taking an indefinitely long period of time.
PAGE 9
The SLVM volume groups are marked as shared volume groups and exported across the SGeRAC cluster using standard SGeRAC procedures. As noted above, multiple physical paths to each physical volume should be configured using the LVM PV Links feature or a separate multipathing product such as HP StorageWorks Secure Path. Please note that, for the case in which the SLVM PVs being used by ASM are disk array LUs, the requirements in this section do not place any constraints on the configuration of the LUs.
PAGE 10
# lvcreate -n lvol1 vgora_asm # lvcreate -n lvol2 vgora_asm # lvchange -C y /dev/vgora_asm/lvol1 # lvchange -C y /dev/vgora_asm/lvol2 # Assume vgdisplay shows each PV has 2900 extents in our example # lvextend -l 2900 /dev/vgora_asm/lvol1 /dev/dsk/c9t0d1 # lvextend -l 2900 /dev/vgora_asm/lvol2 /dev/dsk/c9t0d2 # # # # Assume a PV timeout of 30 seconds.
PAGE 11
In step 3, switch the volume group back to shared mode, using SNOR, and export the VG across the cluster, ensuring that the right ownership and access rights are assigned to the raw logical volumes. Activate the volume group, and restart ASM and the database(s) using ASM-managed storage on all nodes (they are already active on node A). SG/SGeRAC Support for ASM on HP-UX 11i v3 Overview Support for ASM with SG/SGeRAC commences with SGeRAC A.11.17.01 on HP-UX 11i v3.
PAGE 12
Maximum SGeRAC cluster size for ASM configurations is 16 SGeRAC Toolkit3 framework based on Multi-Node Package (MNP) /Simple Package Dependency to manage ASM configurations on SGeRAC up to a maximum cluster size of 16 Raw disks/disk array LUs as ASM Disk Group Members. ASM over SLVM SGeRAC support for ASM-over-SLVM continues in HPUX 11i v3. SGeRAC 11.17.01 or later is required for ASM-over-SLVM configurations on HP-UX 11i v3.
PAGE 13
The idea is that ASM provides the mirroring, striping, slicing and dicing functionality as needed and SLVM supplies the multipathing functionality not provided by ASM. Figure 5 indicates this 1-1 mapping between SLVM PVs and LVs used as ASM disk group members. Further, the default retry behavior of SLVM could result in an I/O operation on an SLVM LV taking an indefinitely long period of time.
PAGE 14
Sample Command Sequence for Configuring SLVM Volume Groups In this section, we provide an example of a command sequence that can be used to prepare SLVM Logical Volumes for use by ASM. The example below demonstrates how volume group is created using new DSF format. HP-UX will automatically use the redundant path for the volume group in the background. The scenario for our example is that we are preparing a new volume group named vgora_asm.
PAGE 15
1. Export the volume group across the SGeRAC cluster and mark it as shared, as specified by SGeRAC documentation.6 Assign the right set of ownerships and access rights to the raw logical volumes on each node as required by Oracle (oracle:dba and 0660, respectively). We can now use the raw logical volume device names as disk group members when configuring ASM disk groups using the Oracle database management utilities.
PAGE 16
Configure Raw Disks/Disk Array Logical Units for ASM Disk Group Oracle provides instructions on how to configure disks for ASM where the member disks are raw logical volume. The instructions to configure raw disks/disk LUs are: For Oracle 10g R2, please refer to Oracle Database Installation Guide 10g Release 2 for hpux Itanium , Chapter 2, Preinstallation Tasks, section “Preparing Disk Group for an Automatic Storage Management Installation”.
PAGE 17
Many SGeRAC customers use SGeRAC packages to start and shut down Oracle RAC instances. In the startup and shutdown sequences, the package scripts activate and deactivate the SLVM volume groups used by the instance. For the ASM environment, it is not appropriate to include SLVM volume group activation and deactivation in the database instance package control script, since the deactivation may fail.
PAGE 18
Related Documentation and Scripts • Using Serviceguard Extension for RAC, June 2007 at http://docs.hp.com/en/ha.html → Serviceguard Extension for Real Application Cluster (Serviceguard OPS Edition) • Whitepaper “Use of Serviceguard Extension for RAC Toolkit with Oracle 10g RAC” http://docs.hp.com under High Availability, Serviceguard Extension for RAC December 2006 • Serviceguard Extension for RAC Toolkit README is available when SGeRAC 11.18 is installed or download from http://software.hp.