Designing High-Availability for Xen Virtual Machines with HP Serviceguard for Linux
Executive Summary .............................................................................................................................. 3 Introduction ......................................................................................................................................... 3 Scope ................................................................................................................................................ 3 Support requirements ......................................
Executive Summary This white paper describes how to integrate Xen Dom0 hosts into a Serviceguard for Linux (SG/LX) cluster, and how to configure Xen Virtual Machines (VMs) as SG/LX packages. It also makes recommendations for eliminating single points of failure and provides pointers to other useful documents. Introduction Virtual machines on Xen are increasingly being deployed for server consolidation and flexibility.
SG/LX 11.19 now provides support for monitoring an application running on a Xen Virtual Machine. The application monitoring feature ensures that applications running on a Xen VM are controlled and monitored so that in case of a failure, the application is restarted or an entire VM is brought down and failed over to other node. A reasonable expertise in the installation and configuration of Xen server on x86_64 platforms and familiarity with its capabilities and limitations is assumed.
You can use the command ‘yast2’ (on SLES10) or ‘rpm’ (on RHEL5) to install additional rpms required for Xen. These rpms are : • • • • • • • • • ‘python’ ‘python-virtinst’ ‘libvirt’ ‘bridge-utils’ ‘libvirt-python’ ‘dnsmasq’ ‘xen-libs’ ‘xen’ ‘kernel-xen’ Xen is added to the GRUB configuration. The installation process places an entry in ‘/boot/grub/menu.lst’. This entry should be similar to the following: title Xen2 kernel (hd0,0)/boot/xen.
Serviceguard for Linux Configuration Storage Configuration When you install the OS (xen-dom0), the fibre channel drivers ‘qla1280.ko’ and ‘qla2xxx/qla2xxx.ko’ do not get loaded by default. A modprobe on the driver module name results in ‘Invalid module format’. The procedure for loading the SCSI QLogic drivers is as follows: 1. Edit ‘/etc/modprobe.conf’, remove any ‘qla*’ and ‘scsi_adapter*’ alias lines.
Secure Linux Settings On RHEL5, for successful creation of virtual machines, and also for creation of volume groups, it is required that appropriate role-based ‘selinux’ rules be placed on the files and directories related to Xen and Serviceguard for Linux. On SLES10, ‘apparmor’ provides equivalent role-based security for files and directories. For additional details on configuring ‘selinux’ or ‘apparmor’, refer to RHEL5 [8] Deployment Guide or SLES10 Apparmor Administration Guide[9].
Note: Citrix XenSource is currently not supported. Packaging the Xen virtual machine with Serviceguard for Linux Creation of Xen VM and dependencies A Xen VM guest can be created using ‘virt-install’ (Redhat) or ‘vm-install’ (Novell). The VM install script prompts for the following information at the time of installation: • • • • VM types: fully-virtualized (FV) or para-virtualized (PV).
The two main resources used by a Xen VM are as follows: • • ‘Xen virtual disk file’ located in a directory specified by the user at time of VM creation ‘Xen VM configuration file’ located in a sub-directory of ‘/etc/xen/’. Follow these steps to consolidate the VM resources: 1. Create a logical volume large enough to hold the virtual disk file and the guest configuration file. If the Xen VM uses a physical shared disk, then the logical volume would hold the configuration file. 2.
# This function is a place holder for customer define functions. # You should define all actions you want to happen here, after the service is # halted. # function customer_defined_halt_cmds { # ADD customer defined halt commands. : # do nothing instruction, because a function must contain some command. ${XENPATH}/xenhost.sh stop } Where ${XENPATH} is an absolute path to the Xen Legacy Scripts. Refer to Appendix III for details on the monitor script (cmxenvmd).
When a Xen VM guest is started, it first goes into a running state, which is then followed by a booted up state. It is necessary to wait for the guest to get into the booted up state before using the Xen virtual machine. This is usually significant when the Xen guest is started up using SG/LX package control script. The package control script expects the VM to get into a booted up state before it can perform further user-defined operations on the virtual machine.
A return of '0' from the command indicates that the machine is in the 'booting up' state (represented by the ‘r’ in the grep command) or in the 'booted up' (represented by the ‘b’) state. The command can be called periodically to check the status of the virtual machine. Virtualization-mode dependent approach • Fully-Virtualized mode For every Fully-Virtualized VM, a 'qemu-dm' process is spawned.
1. To get the virtual machine ID of the virtual machine “s102vm1” , use the command xm list | grep “s102vm1” | awk ‘{print $2}’ 2. Now use the ‘vmid’ to get the name of the virtual interface that is mapped to the virtual machine on the dom0 host. vif < vmid >. so, the derived interface is ‘vif2.0’ 3. Probe the interface using the command ‘ifconfig vif2.0 | grep “UP”’. vif2.
for every guest VM application that needs to be monitored. A template for the `service_cmd` for application monitoring is as follows: service_name cmappmon_srv1 service_cmd $SGLBIN/cmappmgr -node cmappserver_timeout 30 -service service_restart none service_fail_fast_enabled no service_halt_timeout 300 The parameters in the service template are similar to any SG/LX packaged service except for the `service_cmd`.
For more information: 1. Serviceguard for Linux http://www.hp.com/go/sglx 2. Serviceguard for Linux Certification Matrix www.hp.com/go/sglx/info Æ under Solution Planning 3. Managing HP Serviceguard for Linux, Eighth Edition, March 2008 http://docs.hp.com/en/B9903-90060/B9903-90060.pdf 4. Xen User Administration Guide http://bits.xensource.com/Xen/docs/user.pdf 5. Xen Installation section of the SUSE Linux 10 Reference Guide http://www.novell.com/documentation/suse10/index.
Appendix Section I Xen Legacy Configuration File (xenhost.cnf) The file “xenhost.cnf” defines a set of configuration parameters for the Xen Legacy Control Script and Xen Legacy Monitor Script. The file “xenhost.cnf” along with the control script “xenhost.sh” and the monitor script “cmxenvmd” is used to package Xen VMs as SG/LX packages in Legacy Mode. #################################################################### # (C) Copyright 2008 Hewlett-Packard Development Company, L.P.
Appendix Section II Xen Legacy Control Script (xenhost.sh) The script was designed for the purpose of Xen VM Legacy Packages. The script functions may be called from a Legacy Package Control Script through “customer_defined_run_cmds” and “customer_defined_halt_cmds”. The script depends on the “xenhost.cnf” file for Xen environment setup. Template for “xenhost.sh” script #################################################################### # (C) Copyright 2008 Hewlett-Packard Development Company, L.P.
echo "sgx_validate_vm" typeset -i retval=0 # validate if the package is run on xen host if [ -d /proc/xen ]; then grep -q control_d /proc/xen/capabilities case $? in 0) echo "Xen Dom0 Host found !" retval=0 ;; 1) echo "ERROR:: Xen Package Module cannot be configured on a Xen VM Guest" retval=1 ;; *) echo "ERROR:: Xen Environment Detection Failed" retval=255 ;; esac fi if (( retval != 0 )) then echo "ERROR:" $retval " Failed to validate Xen VM Package" to_exit=1 fi } ########################################
1) *) xm esac else fi retval=0 ;; # An error occured while running xm echo "ERROR: Xen VM startup failed" retval=1 ;; # xm command failed with an undocumented error or an error occured while starting echo "Unrecoverable error occured " retval=255 ;; echo "ERROR: 255 Xen VM configuration file not found !!!" exit 255 if (( retval != 0 )) then echo "ERROR:" $retval " Function sgx_start_vm" echo "ERROR:" $retval " Failed to start xen vm" to_exit=1 fi } #####################################################
# An error occured while stopping vm echo "ERROR: Xen VM shutdown failed" retval=0 ;; *) # xm command failed with an undocumented error echo "Unrecoverable error occured " retval=255 ;; esac if (( retval != 0 )) then echo "ERROR:" $retval " Function sgx_stop_vm" echo "ERROR:" $retval " Failed to halt xen vm ${SG_XEN_VM_NAME}" to_exit=1 fi } ####################################################################### ### # # sgx_probe_vm_status() is called to probe the status of a running vm; # it exits only wh
*) done } esac # exit only when Vm is completely halted return 1 ;; echo "ERROR: 255 Unrecoverable error occured " return 255 ;; fi ################ # main routine ################ # # # # # # # Module script must be specified with three required entry points: start, stop, and validate. The variable to_exit indicates the success or failure of the entry point.
Appendix Section III Xen Legacy Monitor Script (cmxenvmd) The script is designed to monitor Xen VMs which are started using the Xen control script. The script depends on the Xen Configuration file to source Xen specific environment variables. The script must be registered as a SERVICE_CMD in the package control script. #################################################################### # (C) Copyright 2008 Hewlett-Packard Development Company, L.P.
Appendix IV Table of Acronyms and Abbreviations Product Name Serviceguard Serviceguard for Linux Quorum Server Business Continuity and Availability High Availability Integrity Virtual Machines SUSE Linux Enterprise SUSE Linux Enterprise Server Red Hat Enterprise Linux Xen Domain0 Xen DomainU Virtual Machine Abbreviation SG or SG/UX SG/LX QS BC&A HA Integrity VM SUSE SLES RH or RHEL Dom0 DomU VM © 2009 Hewlett-Packard Development Company, L.P.
