HP Serviceguard Toolkit for Oracle Version A.05.01.
© Copyright 2012 Hewlett-Packard Development Company, L.P. Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license. The information contained herein is subject to change without notice.
Contents 1 Overview..................................................................................................5 Advantages.............................................................................................................................5 Prerequisites.............................................................................................................................5 Hardware support....................................................................................................
A Sample package configuration file for Oracle database legacy package without ASM..........................................................................................................59 B Sample package configuration file for Oracle database modular package without ASM..........................................................................................................61 C Sample package configuration file for Oracle ASM database legacy package.......................................................
1 Overview HP Serviceguard toolkit for Oracle on Linux enables you to configure an Oracle single instance in a Serviceguard cluster on Linux and integrate an Oracle database server with HP Serviceguard for Linux (SG/LX). This toolkit consists of a set of scripts that are used to start, stop, and monitor an Oracle database instance, ASM instance, and the configured listeners.
2 Installing and uninstalling HP Serviceguard toolkit for Oracle NOTE: • The steps described here are applicable to both legacy and modular style packages. • Oracle toolkit RPM must be installed on all the nodes of the cluster that is configured to run the Oracle toolkit package. • For a fresh installation of the Oracle toolkit, run the following command: rpm -ivh serviceguard-oracle-toolkit-..noarch.rpm For example : rpm -ivh serviceguard-oracle-toolkit-A.05.01.01-00.
Table 1 Scripts for HP Serviceguard toolkit for Oracle File name Description Available in directory haoracle.conf This is the toolkit configuration file and $SGROOT/oracletoolkit is specific to the legacy style packaging. This script contains a list of pre-defined variables that the user must customize for use with a particular database instance. This configuration file is read by the toolkit script, haoracle.sh. haoracle.
Table 1 Scripts for HP Serviceguard toolkit for Oracle (continued) File name Description Available in directory hadbhang.mon The haoracle.sh script invokes the hadbhang.mon script to monitor the Oracle instance for possible “hung” state. hadbhang.mon script in turn uses hagetdbstatus.sh and hatimeoutdbhang.sh hagetdbstatus.sh It is used to check the database status. hatimeoutdbhang.sh This script checks whether a process with the specified PID exists.
3 Supported configuration This section explains the following supported configurations for Oracle database toolkit packages: • Without Automatic Storage Management (ASM) • With Automatic Storage Management Without Automatic Storage Management This section explains how to configure an Oracle database instance package without ASM in a Serviceguard environment.
MNP (Multi-node Package) is used to encapsulate the per node ASM instances, with one or more Oracle single instance failover packages dependent on this MNP. This configuration enables the database instance to start up in the right order in relation to the ASM instance, and in the event of a failover, to relocate only to a node where an ASM instance is available. To configure Oracle database using ASM in a Serviceguard environment: 1.
4 Using HP Serviceguard toolkit for Oracle After installing the HP Serviceguard toolkit for Oracle, you can use this toolkit to: • Deploy Oracle ASM, Oracle database instances in Serviceguard cluster environment. • Manage the packages. Configuring legacy and modular Oracle ASM instance and Database instance toolkit packages This section describes the steps to configure Oracle ASM instance and Database instance toolkit packages.
Table 2 Lists the attributes that you can edit (continued) Attribute Name Description Example NOTE: If you modify this attribute after issuing the # cmmakepkg -i -m -t command, you must reconfigure the MONITOR_PROCESSES attribute defined in this file to reflect the same. Also, reconfigure PFILE attribute defined in this file to reflect this change, if PFILE is using this attribute. ASM This parameter determines whether ASM is being used to manage the storage for the database instance.
Table 2 Lists the attributes that you can edit (continued) Attribute Name Description Example LISTENER_RESTART number of times to restart the listener. This parameter is commented by default. The Oracle database parameter file. ${ORACLE_HOME}/dbs/ If this attribute is not specified, init${SID_NAME}.ora Oracle picks this up from the Oracle configuration directory $ORACLE_HOME/dbs. PFILE When this parameter is configured, it overrides the default file location.
Table 2 Lists the attributes that you can edit (continued) Attribute Name Description Example NOTE: If the Maintenance flag is set to no, then the maintenance feature is not available which means the toolkit cannot be brought into maintenance mode. MONITOR_INTERVAL The time interval, in seconds, this script waits between checks to ensure that Oracle instance is running. The default value is 30 seconds.
Table 2 Lists the attributes that you can edit (continued) Attribute Name Description Example executed and the variables set in this file are available to the new shell. CLEANUP_BEFORE_STARTUP This parameter indicates whether shutdown abort must be executed before the startup of the Oracle database or ASM instance. no You can set this parameter to yes or no. The default value is no.
5. Generate the database package configuration file and control script using the following commands: • Database package configuration file: #cmmakepkg -p db1pkg.conf • Package control script: #cmmakepkg -s db1pkg.cntl NOTE: 6. 7. Edit the following parameters in the package configuration file, db1pkg.conf. Attribute Name Description Example PACKAGE_NAME The package name must be unique in the cluster. oracle_legacy RUN_SCRIPT Script to start up the service. $SGCONF/oracle_legacy/ db1pkg.
8. Edit the package control script db1pkg.cntl. Attribute Name Example SERVICE_NAME[0] "oracle_service" SERVICE_CMD[0] "$SGCONF/oracle_legacy/toolkit.sh monitor" SERVICE_RESTART[0] "" If LISTENER parameter has been set to yes in the package configuration file haoracle.conf then initialize the following parameters: Attribute Name Example SERVICE_NAME[1] "oracle_listener_service" SERVICE_CMD[1] "$SGCONF/oracle_legacy/toolkit.
Table 3 Lists the attributes that you can edit Attribute Description Example package_name The package name must be unique in the cluster. oracle_pkg TKIT_DIR When you apply a package using $SGCONF/oracle_pkg the cmapplyconf command, TKIT_DIR contains the toolkit configuration file, haoracle.conf on all the configured nodes. NOTE: The TKIT_DIR for this package must be different from the TKIT_DIR configured for other database packages.
Table 3 Lists the attributes that you can edit (continued) Attribute Description Example LISTERNER_RESTART This parameter defines the number 2 of attempts to be made to restart the listeners when a listener failure is detected. If the listener does not restart after LISTENER_RESTART number of consecutive attempts, the package fails over. If the listener restart is successful, then the next time listener fails, the toolkit again tries LISTENER_RESTART number of times to restart the listener.
Table 3 Lists the attributes that you can edit (continued) Attribute Description Example After the maintenance work, ensure that the instance is running. Then delete the file oracle.debug in the package directory. This enables the toolkit to continue monitoring the database server application or the ASM instance. The default value is yes. NOTE: If the Maintenance flag is set to no, then the maintenance feature is not available which means the toolkit cannot be brought into maintenance mode.
Table 3 Lists the attributes that you can edit (continued) Attribute Description Example a shutdown is initiated by the user and not because of the failure of a service. You can set this parameter to abort or immediate. If abort is specified, the instance is shutdown using the abort option. If immediate is specified, the instance is shutdown using the immediate option. The default value is abort.
Table 3 Lists the attributes that you can edit (continued) Attribute Description Example You can list multiple recipients by repeating the email_id field and address. Toolkit sends an alert email on listener and package failure. If configured, the alert mail is sent for Oracle database hang detection. For example: $SGCONF/scripts/ tkit/oracle/tkit_module.sh oracle_hang_monitor 30 alert. By default, email_id is commented. 5.
6. Apply the package configuration using the command: # cmapplyconf -P pkg2.conf 7. Start the "oracle_pkg" package using the command: #cmrunpkg oracle_pkg Configuring legacy ASM instance and Database instance packages To configure a legacy ASM instance package: 1. Setting up and configuring ASM multi-node package. a. Create an ASM package directory under $SGCONF/. b. Copy the toolkit scripts from $SGROOT/oracletoolkit to the directory created in step 1a. c. cd $SGCONF/. d.
Table 4 Lists the attributes that you can edit (continued) Attribute Name Description Example NOTE: If a change is made to this attribute after the # cmmakepkg -i -m -t command is run, reconfigure the MONITOR_PROCESSES attribute defined in this file to reflect the same. Also, reconfigure PFILE attribute defined in this file to reflect the change if PFILE is using this attribute. The default value is +ASM. PFILE The Oracle database parameter ${ASM_HOME}/dbs/ file.
Table 4 Lists the attributes that you can edit (continued) Attribute Name Description Example Serviceguard toolkit for Oracle files are located. If the file exists and if maintenance feature is enabled, then monitoring is paused, the ASM or the database instance might be brought down for maintenance, and package does not fail over to the adoptive node even though the instance is not running. After the maintenance work, ensure that the instance is running. Then delete the oracle.
Table 4 Lists the attributes that you can edit (continued) Attribute Name Description Example this file are available to the new shell. CLEANUP_BEFORE_STARTUP This parameter indicates whether no shutdown abort must be executed before the startup of the Oracle or ASM instance. You can set this parameter to yes or no. The default value is no.
g. h. i. Edit the following parameters in the package control script, asmpkg.cntl Attribute Name Example SERVICE_NAME[0] "oracle_service" SERVICE_CMD[0] "$SGCONF/oracle_asm_legacy/toolkit.sh monitor" SERVICE_RESTART[0] "" Distribute the package configuration file, package control script, and the framework scripts to all the nodes. Apply the package configuration using the following command: # cmapplyconf -P asmpkg.conf j.
Attribute Name Description Example You can set the value to yes or no. Set this parameter to yes if ASM is used, otherwise set to no. The default value is no. START_MODE This parameter defines the Oracle open database startup mode. Valid options are no mount, mount, and open. The default mode is open. ASM_DISKGROUP If ASM is being used to manage diskgroup1 the storage for the database instance, configure this parameter to indicate the ASM Diskgroups used by the database instance.
Attribute Name Description Example Set to yes if you want this toolkit to start and stop the Oracle listener. If set to no, comment the LISTENER_NAME and oracle_listener_service attributes. NOTE: If this attribute is set to no, you must comment the LISTENER_NAME and oracle_listener_service attributes. For example: #LISTENER_NAMEThe default value is yes. LISTENER_NAME This parameter contains the names LISTENER of the Oracle listener processes.
Attribute Name Description Example MONITOR_PROCESSES This is the list of all the critical Oracle processes for an instance that must be executing to assume that the instance is up and running. You can add more instance processes. The processes mentioned will be monitored by default.
Attribute Name Description Example KILL_ASM_FOREGROUNDS If ASM is being used, this yes parameter specifies whether any ASM foreground processes having file descriptors open on the dismounted disk group volumes must be killed or not. You can set this parameter to yes or no. The default value is yes. If this parameter is set to yes, ASM foreground processes of the form oracle having file descriptors open will be killed.
Attribute Name Description Example DB_SERVICE This parameter is used to start and all stop the DB service through the DB package. It is commented by default. You can set the attribute to all or specify the DB service names that must be started and stopped by the DB package. DISABLE_BACKUP_MODE This parameter is used to disable no the Oracle database from the backup mode. The valid values for this parameter are yes and no. By default, the value is set to no.
g. Edit the following service parameters: • • • h.
If service is configured for database hang detection in the package configuration file, set the following parameters: i. j. Attribute Name Example SERVICE_NAME[2] "oracle_hang_service" SERVICE_CMD[2] “$SGCONF/oracle_asmdb_legacy/toolkit.sh hang_monitor 30 failure” SERVICE_RESTART[2] "" Distribute the package configuration file, package control script, and the framework scripts to all the nodes. Apply the package configuration using the following command: # cmapplyconf -P db1pkg.conf k.
Attribute Description Example INSTANCE_TYPE This parameter determines whether ASM the instance is an ASM instance or a database instance. Set this parameter to ASM. The default value is ASM. ORACLE_ADMIN This is the user name of the Oracle oracle database administrator who manages the database. The default value is oracle. ORACLE_HOME This is the base directory where Oracle is installed. SID_NAME The Oracle session name. It is also ORACLE_TEST0 called the session ID (SID).
Attribute Description Example ASM_USER The user name of the Oracle ASM oracle administrator. It is used to start or stop the ASM instance and mounting or dismounting the ASM disk groups. The default value is oracle. ASM_SID The ASM session name that uniquely identifies an ASM instance. The default value is +ASM. PFILE It is the Oracle database ${ASM_HOME}/dbs/ parameter file. If this attribute is init${ASM_SID}.
Attribute Description Example monitoring the database server application or the ASM instance. The default value is yes. NOTE: If Maintenance flag is set to no, then the maintenance feature is not available and toolkit cannot be brought into maintenance mode. MONITOR_INTERVAL The time interval, in seconds, this script waits between checks to ensure that ASM instance is running. The default value is 30 seconds.
Attribute Description Example If abort is specified, the instance is shutdown using the abort option. If immediate is specified, the instance is shutdown using the immediate option. The default value is abort. KILL_ASM_FOREGROUNDS If ASM is being used, this yes parameter specifies whether any ASM foreground processes having file descriptors open on the dismounted disk group volumes must be killed or not. You can set this parameter to yes or no. The default value is yes.
c. Create the modular package configuration file, pkg2.conf using the following command: #cmmakepkg -m tkit/oracle/oracle pkg2.conf d. Edit the following attributes manually in the package configuration file: Attribute Description Example package_name The package name must be unique oracle_asmdb_pkg in the cluster. TKIT_DIR When you apply a package using $SGCONF/oracle_asmdb_pkg the cmapplyconf command, TKIT_DIR contains the toolkit configuration file, haoracle.conf on all configured nodes.
Attribute Description Example to indicate the ASM diskgroups used by the database instance. You can add more than one disk group by adding to the array variable ASM_DISKGROUP. ASM_VOLUME_GROUP Defines the volume groups used in vgora_asm1 the ASM disk groups for the database instance. You can add more than one volume group by adding to this array variable. ASM_HOME The home directory where ASM is /ASM_TEST0 installed. ASM_USER User name of the Oracle ASM oracle administrator.
Attribute Description Example LISTENER_RESTART number of times to restart the listener. This parameter is commented by default. PFILE The Oracle database parameter ${ORACLE_HOME}/dbs/ file. If this attribute is not specified, init${SID_NAME}.ora Oracle picks this up from the Oracle configuration directory $ORACLE_HOME/dbs. When this parameter is configured, it overrides the default file location.
Attribute Description Example MONITOR_INTERVAL The time interval, in seconds, this script waits between checks to ensure that Oracle instance is running. The default value is 30 seconds. 30 TIME_OUT The time period, in seconds, this 30 toolkit waits for a completion of a normal shutdown before initiating forceful halt of the application.
Attribute Description Example ASM foreground processes having file descriptors open on the dismounted disk group volumes need to be killed or not. You can set this parameter to yes or no. The default value is yes. If this parameter is set to yes, ASM foreground processes of the form oracle having file descriptors open will be killed. If this parameter is set to no, and if the ASM foreground processes have file descriptors open, it leads to package halt and hence, fails the package failover.
Attribute Description Example $SGCONF/scripts/tkit/ oracle/tkit_module.sh oracle_hang_monitor 30 alert. By default, email_id is commented. e. Edit the following service parameters: • • • 44 To start monitoring Oracle processes: Attribute Name Example service_name oracle_db_service service_cmd "$SGCONF/scripts/tkit/oracle/ tkit_module.
f. g. To configure the dependency for the modular Oracle ASM database package, you must set the following parameters in the package configuration file: Attribute Name Example dependency_name asm_dependency dependency_condition =up dependency_location same_node Apply the package configuration using the command: # cmapplyconf -P pkg2.conf Managing a package You can manage a package by adding, running, maintaining, halting, and deleting it.
is up and running. Table 5 (page 46) lists the attributes that are supported for online toolkit package modification. Table 5 List of Attributes Supported for Online Toolkit Package Modification 46 Attribute Description TKIT_DIR Oracle Toolkit supports online modification of this attribute. START_MODE Oracle Toolkit supports online modification of this attribute. This attribute denotes the database startup. The attribute value takes effect during the next startup of the database.
Table 5 List of Attributes Supported for Online Toolkit Package Modification (continued) Attribute Description during online modification. Database service must be configured and running during online modification, so that toolkit package can use the start or stop control on user defined database services. DISABLE_BACKUP_MODE Oracle Toolkit supports the online modification of this attribute. This parameter is used to disable the Oracle database from the backup mode.
Halting a package To halt a package, run the following command: # cmhaltpkg Deleting a package To delete a package from the cluster, run the following command: # cmdeleteconf -p This command prompts for a confirmation, before removing the package configuration from the cluster binary file, unless you use the -f option.
5 Migrating a database package You can migrate from an existing database package that uses the older version of HP Serviceguard toolkit for Oracle to use the new Linux HP Serviceguard toolkit for Oracle scripts provided for ASM support. The new legacy or modular database package may or may not use ASM. Migrating legacy Oracle database package without ASM to modular To migrate a legacy Oracle database package without ASM to modular Oracle database package without ASM: 1. Halt the database package. 2.
6 Oracle toolkit features Alert mail notification This feature facilitates you to configure email address to get the alert notification from toolkit package. Alert mails are sent to configured email addresses in case of failure scenarios. It can be a resource failure or altogether database failure that results in package failure. Toolkit sends an alert email with failure messages in the following scenarios: • Listener failure.
7 Supporting Oracle features Support for multiple listeners This section explains support for multiple listeners with the HP Serviceguard toolkit for Oracle.
NOTE: In a single package, it is not possible to configure the services using both the approaches. You must either configure all the listeners using a single service, or use a separate service for each listener. Ensure that the elements in the LISTENER_RESTART array and the LISTENER_PASS array correspond to those in the LISTENER_NAME array in the package configuration file.
the listener name to the service command. The service_cmd in the package configuration file appears as follows: Attribute name Description service_name oracle_listener_monitor service_cmd “$SGCONF/scripts/tkit/oracle/tkit_module.sh oracle_monitor_listener ” service_restart none service_fail_fast_enabled no service_halt_timeout 300 A separate service to monitor each listener This service is recommended if listeners are critical.
NOTE: • This feature is available only on systems running Oracle Clusterware. • If user defined DB services are configured to start with database package startup, then database must be started in open mode. Set the package configuration file attribute START_MODE to open. The attribute, DB_SERVICE is used to start and stop the database service through the database package. This attribute is commented by default.
Support for Oracle database backup mode During the oracle database instance package start or package failover on the other cluster node, if the database is in the online backup mode based on your input through the package attribute DISABLE_BACKUP_MODE, the toolkit gets the database out of the backup mode. The valid values for this parameter are yes and no. By default, the value is set to no. For example: tkit/oracle/oracle/DISABLE_BACKUP_MODE no.
8 Troubleshooting This section explains some of the problem scenarios that you might encounter while using the HP Serviceguard toolkit for Oracle in a Serviceguard cluster. Problem scenario Possible cause The cmcheckconf or cmapplyconf The TKIT_DIR specified in the command logs the following message: package configuration file is not created. WARNING! Directory $SGCONF/ is not present.
9 Support and other resources Information to collect before contacting HP Ensure that the following information is available before you contact HP: • Software product name • Hardware product model number • Operating system type and version • Applicable error message • Third-party hardware or software • Technical support registration number (if applicable) How to contact HP Use the following methods to contact HP technical support: • In the United States, see the Customer Service / Contact HP Un
Ctrl+x or Ctrl-x A key sequence that indicates you must hold down the keyboard key labeled Ctrl while you press the letter x. ENVIRONMENT VARIABLE The name of an environment variable, for example, PATH. Key The name of a keyboard key. Return and Enter both refer to the same key. Term A term or phrase that is defined in the body text of the document, not in a glossary. User input Indicates commands and text that you type exactly as shown.
A Sample package configuration file for Oracle database legacy package without ASM The toolkit configuration file (haoracle.conf) INSTANCE_TYPE=database ORACLE_HOME=/oracle/app/oracle/product/11.2.0/dbhome_1 ORACLE_ADMIN=oracle SID_NAME=oranew START_MODE=open ASM=no LISTENER=yes LISTENER_NAME[0]=LISTENER PFILE=${ORACLE_HOME}/dbs/init${SID_NAME}.
SERVICE_CMD[0]="$SGCONF/oracle_legacy/toolkit.sh monitor" SERVICE_RESTART[0]="" SERVICE_NAME[1]="oracle_listener_service" SERVICE_CMD[1]="$SGCONF/oracle_legacy/toolkit.sh monitor_listener" SERVICE_RESTART[1]="" SERVICE_NAME[2]="oracle_hang_service" SERVICE_CMD[2]="$SGCONF/oracle_legacy/toolkit.sh hang_monitor 30 failure" SERVICE_RESTART[2]="" function customer_defined_run_cmds { # ADD customer defined run commands. : # do nothing instruction, because a function must contain some command.
B Sample package configuration file for Oracle database modular package without ASM # ********************************************************************** # ****** HIGH AVAILABILITY PACKAGE CONFIGURATION FILE (template) ******* # ********************************************************************** # ******* Note: This file MUST be edited before it can be used. ******** # * For complete details about package parameters and how to set them, * # * consult the Serviceguard manual.
C Sample package configuration file for Oracle ASM database legacy package Sample package configuration file for Oracle ASM MNP legacy package The toolkit configuration file (haoracle.conf) INSTANCE_TYPE=ASM ORACLE_ADMIN=oracle ASM=no PFILE=${ASM_HOME}/dbs/init${ASM_SID}.
function customer_defined_halt_cmds { # ADD customer defined halt commands. : # do nothing instruction, because a function must contain some command. if [ $SG_HALT_REASON = "user_halt" ]; then reason="user" else reason="auto" fi $SGCONF/oracle_asm_legacy/toolkit.sh stop $reason test_return 52 } Sample package configuration file for Oracle ASM database legacy package The toolkit configuration file (haoracle.conf) INSTANCE_TYPE=database ORACLE_HOME=/oracle/app/oracle/product/11.2.
SERVICE_NAME SERVICE_FAIL_FAST_ENABLED SERVICE_HALT_TIMEOUT oracle_hang_service no 300 Package control script # ********************************************************************** # *HIGH AVAILABILITY PACKAGE CONTROL SCRIPT (template)* # *Note: This file MUST be edited before it can be used.* # *You must have bash version 2 installed for this script to work* # *properly. Also required is the arping utility available in the* # *iputils package.
D Sample package configuration file for Oracle ASM database modular package ASM multi-node package configuration file # ********************************************************************** # ****** HIGH AVAILABILITY PACKAGE CONFIGURATION FILE (template) ******* # ********************************************************************** # ******* Note: This file MUST be edited before it can be used.
tkit/oracle/oracle/MONITOR_PROCESSES ora_lgwr_${SID_NAME} tkit/oracle/oracle/MONITOR_PROCESSES ora_reco_${SID_NAME} tkit/oracle/oracle/MAINTENANCE_FLAG yes tkit/oracle/oracle/MONITOR_INTERVAL 30 tkit/oracle/oracle/TIME_OUT 30 tkit/oracle/oracle/PARENT_ENVIRONMENT no tkit/oracle/oracle/CLEANUP_BEFORE_STARTUP no tkit/oracle/oracle/USER_SHUTDOWN_MODE abort tkit/oracle/oracle/KILL_ASM_FOREGROUNDS yes tkit/oracle/oracle/DB_SERVICE all tkit/oracle/oracle/DISABLE_BACKUP_MODE no email_id abc@hp.
Glossary $SGCONF /usr/local/cmcluster/conf for RHEL /opt/cmcluster/conf for SLES $SGROOT /usr/local/cmcluster in RedHat /opt/cmcluster in SLES ADF Attribute definition file ASM Automatic Storage Management Configuration directory mode of operation It allows you to maintain your own directory and copy all the toolkits scripts in this directory.
Index A U Alert mail notification, 50 Using HP Serviceguard toolkit for Oracle, 11 Configuring legacy and modular Oracle database toolkit packages, 11 C Configuring legacy ASM instance and Database instance packages, 23 Configuring legacy Oracle database package without ASM, 11 Configuring modular ASM database package, 34 Configuring modular Oracle database package without ASM, 17 H Hardware Support, 5 I Installation uninstallation, 6 M Managing a package, 45 adding the package to the cluster, 45 Man
