HP StorageWorks X5500 Network Storage Gateway for Linux setup guide This guide provides information about setting up the HP StorageWorks X5500 Network Storage Gateway for Linux (formerly known as HP Scalable NAS Clustered Gateway). This guide is intended for administrators who will be performing the setup.
Legal and notice information © Copyright 2006, 2009 Hewlett-Packard Development Company, L.P. Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license. The information contained herein is subject to change without notice.
Contents 1 Configuration information ................................................................... 5 Hardware configuration limits ....................................................................................................... Supported HBA drivers .......................................................................................................... Server requirements ....................................................................................................................
9. Configure HP X5500 Storage Gateway for Linux for public versus private network ................... Other configuration tasks ........................................................................................................... Configure HP X5500 Storage Gateway for Linux features ........................................................ Configure firewalls (optional) ............................................................................................... External network port numbers ........
1 Configuration information HP is continually expanding its supported hardware and operating system configurations. Check the compatibility guide on www.hp.com for the latest compatibility information: http://h18006.www1.hp.com/products/storage/software/polyserve/support/, then click the Compatibility Guide link on the right side.
NOTE: QLogic and Emulex cards should not be placed in the same fabric zone unless both HBA manufacturers have approved this configuration. Server requirements NOTE: To ensure that file times are consistent across the cluster, it is important that all cluster servers operate with synchronized time-of-day clocks. An NTP server is one commonly used mechanism for synchronizing system clocks. When configuring servers, you should be aware of the following: • All servers in the cluster must be on the same subnet.
IMPORTANT: If the cluster configuration includes an HP Fibre Channel Virtual Connect Module, the guideline above must be implemented. If this is not done, the cluster can experience excessive timeouts and a loss of stability. • The Fibre Channel switches managed by the cluster must be from the same vendor. Switches that will not be managed by HP X5500 Storage Gateway for Linux can be from other vendors only if all of the vendors have approved the configuration.
Configuration information
2 Setting up physical connectivity This chapter describes how to physically configure a new HP StorageWorks X5500 Network Storage Gateway for Linux (HP X5500 Storage Gateway for Linux) server. Setting up the HP X5500 Storage Gateway for Linux Before attempting to power on the nodes on an HP X5500 Storage Gateway for Linux solution, there is some required hardware setup. The HP support specialist who installs your system performs most of these tasks.
The illustration below identifies the network and other connections on the HP DL360 G6. 1. Quad Port NIC port 1 7. iLO port 2. Quad Port NIC port 2 8. Monitor connection 3. Quad Port NIC port 3 9. Mouse connection 4. Quad Port NIC port 4 10. Keyboard connection 5. Fibre HBA port 2 11. Built-in NIC port 2 6. Fibre HBA port 1 12.
NOTE: It is important to configure the software so it does not use the public network connections for intra-cluster network traffic. See 9. Configure HP X5500 Storage Gateway for Linux for public versus private network for details. iLO card network connections The DL360 servers include a built-in remote management card called the Integrated Lights Out (iLO) card. The iLO plays an integral part in the HP X5500 Storage Gateway for Linux solution.
Setting up physical connectivity
3 Set up HP X5500 Storage Gateway for Linux solutions This chapter describes how to set up HP X5500 Storage Gateway for Linux solutions. Contents of the HP X5500 Storage Gateway for Linux Quick Restore DVD • SLES 10 SP2 • The HP X5500 Storage Gateway for Linux software installed with the FS option. Debug kernels are also included.
Operating System Kernel SuSE Linux Enterprise Server 10, Service Pack 2, 64-bit 2.6.16.60-0.21 SuSE Linux Enterprise Server 10, Service Pack 2, 64-bit 2.6.16.60-0.37_f594963d HP X5500 Storage Gateway for Linux provides binary kernels for SLES10 SP2. These kernels include the vendor patches and HP X5500 Storage Gateway for Linux and FS Option for Linux patches. The installation procedure, described later in this chapter, specifies when the binary kernel should be installed.
For SLES10, the following binary and source kernel RPMs are provided: • kernel-HPPS-2.6.16.60-0.21.370.1349.x86_64.rpm • kernel-HPPS-2.6.16.60-0.37_f594963d.370.1349.x86_64.rpm • kernel-HPPS_debug-2.6.16.60-0.21.370.1349.x86_64.rpm • kernel-HPPS_debug-2.6.16.60-0.37_f594963d.370.1349.x86_64.rpm • kernel-HPPS_kdump-2.6.16.60-0.21.370.1349.x86_64.rpm • kernel-HPPS_kdump-2.6.16.60-0.37_f594963d.370.1349.x86_64.rpm • kernel-source-2.6.16.60-0.21.370.1349.src.rpm • kernel-source-2.6.16.60-0.21.370.1349.x86_64.
Action Description Configure basic node settings (password, networking, display). As part of the initial boot process, you must set up some initial settings for each node that include a root password, basic networking settings, and display subsystem setup. Set up the SAN configuration. Set up the storage array, create LUNs for use as HP X5500 Storage Gateway for Linux membership partitions, and configure the Fibre Channel switches that will be under cluster control.
1. Power on each node and configure the iLO card interface One by one, power on each node in the cluster. You will need information from the HP Pre-Planning Worksheet for this task. Complete the following steps for each node: 1. 2. 3. Attach a keyboard, mouse, and monitor to the system. Power on the system. Configure the iLO card interface. The iLO card interface is an integral part of the solution.
discussed in step 9. Configure HP X5500 Storage Gateway for Linux for public versus private network. NOTE: The IP address for these cards is not used directly by the clients. Rather, virtual IP addresses are used and assigned later during the configuration of the HP X5500 Storage Gateway for Linux software.
• Proxy—This section can be ignored. It sets web proxy information used by some tools. However, the default browser (Firefox) does not use the proxy information set here. • Test Internet Connection—Select No, Skip This Test. • Installation Settings—Select Next. • User Authentication Method—Select the authentication method: • Local (/etc/passwd) (This is the default.) • LDAP • NIS • Windows Domain • New Local User—You can select the source of the user database (for example, NIS, LDAP, /etc/hosts).
http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?lang=en&cc=us&taskid=115& prodSeriesId=3435148&prodTypeId=18964&prodSeriesId=3435148&objectID=c01060345 Create LUNs or disk partitions for membership partitions HP X5500 Storage Gateway for Linux uses a set of membership partitions to control access to the SAN and to store the device naming database, which includes the global device names that HP X5500 Storage Gateway for Linux assigns to the SAN disks placed under its control.
• Fibre Channel switch-based fencing (also called “fabric fencing”): When a server needs to be fenced, HP X5500 Storage Gateway for Linux disables the server’s access in the Fibre Channel fabric. • Web management-based fencing (also called “server-based fencing”): HP X5500 Storage Gateway for Linux uses remote management hardware on the server to reset or power off the server. (HP ProLiant servers (except the 1xx series) support this kind of fencing using iLO remote management functionality.
NOTE: Note the HBA driver is not loaded as part of the initial system boot sequence. Use the above commands to load/unload the driver and not the modprobe command usually used in a Linux environment. • Record the WWID values of each HBA for use in configuring the storage subsystem. The /opt/ hpcfsmgmt/bin/cfshbainfo tool can be used to display the WWIDs of each HBA port.
6. Run the mxcheck utility This utility should be run on each server. It verifies that the server’s configuration meets the requirements for running HP X5500 Storage Gateway for Linux. Issue the following command: # /opt/hpcfs/bin/mxcheck -l -p -t - install The utility performs checks such as the following: operating system version and configuration; available physical memory and disk space; HBA driver versions; Fibre Channel switch versions; network addresses.
Connect to the cluster using the Management Console To start the Management Console, execute the following command: $ mxconsole On the HP X5500 Storage Gateway for Linux Connect window, type the host name or IP address of the server in the “Connect to” field. Then click on the down-arrow on the Connect button. You will see two options: Connect and Configure. Click Configure. The mxconsole asks for a user/password pair when you connect the first time, then notes that the cluster is not configured.
License. The license file is installed at /etc/opt/hpcfs/licenses/license. No action is required here. Secret Network Key. This password is required. It provides additional security for network communications among the cluster servers. To set this key, click Set Secret Network Key. You can enter anything you want for this password. Cluster Administrative Traffic Protocol. Select either multicast or unicast mode. Multicast mode is recommended.
Select your storage type and, for Fibre Channel, specify the fencing method that you want to use. Fibre Channel switch-based fencing Complete the following on the SAN & Fencing tab: SAN Switches. Specify the hostnames or IP addresses of the Fibre Channel switches that are directly connected to the nodes in the cluster. Click Add, and then specify the hostname or IP address of the first Fibre Channel switch. Repeat this procedure to specify the remaining Fibre Channel switches, including cascading switches.
NOTE: Later in this procedure the complete cluster configuration will be exported to the other servers that will be in the cluster. During the export, HP X5500 Storage Gateway for Linux will, where possible, use the information that you specified on the SAN & Fencing tab to configure Web management-based fencing on the other servers. If a particular entry on the SAN & Fencing tab did not apply to all servers, HP X5500 Storage Gateway for Linux will ask for that information for each of the other servers.
and server2 and their Remote Management Controllers are server1-iLO and server2-iLO, enter -iLO as the suffix. • IP Delta. Specify the delta to add to each server’s IP address to determine the IP addresses of the associated Remote Management Controllers. For example, if your servers are 1.255.200.12 and 1.255.200.15 and their Remote Management Controllers are 1.255.201.112 and 1.255.201.115, enter 0.0.1.100 as the delta. • Remote Management Controller Access.
SAN Switches. This is used to specify the hostnames or IP addresses of the Fibre Channel switches that are directly connected to the nodes in the cluster. Click Add, and then specify the hostname or IP address of the first Fibre Channel switch. Repeat this procedure to specify the remaining Fibre Channel switches, including cascading switches. SNMP Community String. The default SNMP community string for HP X5500 Storage Gateway for Linux is private.
Membership Partitions. HP X5500 Storage Gateway for Linux uses a set of membership partitions to control access to the SAN and to store the device naming database, which includes the global device names that HP X5500 Storage Gateway for Linux assigns to the SAN disks placed under its control. You will need to select the LUNs or disk partitions that should be used as membership partitions.
All of the available partitions on that disk or LUN then appear in the bottom of the window. Select one of these partitions and click Add. (The minimum size for a membership partition is 1 GB.) Repeat this procedure to select two more membership partitions. We recommend that the partitions be on different disks. NOTE: When selecting partitions for use as membership partitions, be sure that they do not contain any needed data. When the membership partitions are created, any existing data will be erased.
For XP arrays, you will be asked for the local and remote instance numbers of your XP RAID Manager configuration. For Engenio arrays, you will be asked for the hostnames or IP addresses and password for your storage array controllers.
Apply the configuration. When you have completed the entries on the Storage Settings tab, click Apply (at the bottom of the Cluster Configuration window). You will then see a message stating the operation will erase all of the data on the membership partitions. Click Yes to continue. The configuration is then saved to the server you specified for the connection at the start of step 8. You will then be asked whether you want to start the HP X5500 Storage Gateway for Linux software on that server.
If you are using Web Management-based fencing, you may be asked for additional information about each server. Based on the entries you made on the SAN & Fencing tab, the information can include any or all of the following: the vendor/type for the Remote Management Controller, the IP address/hostname of the Remote Management Controller, the username and password for the Remote Management Controller.
connection is the “private” network. You must indicate which network is considered to be the private one. Failure to perform this step could allow the public network to be selected, which could adversely affect performance. It is important to correctly complete this task. To do so, use one of the following methods to discourage administration traffic off the public network: 1. Using the Management Console: a. Select the Servers tab. b.
External network port numbers The following port numbers are used for external connections to HP X5500 Storage Gateway for Linux. If HP X5500 Storage Gateway for Linux is behind a firewall, it may be necessary to change the firewall rules to allow traffic on these ports.
NFS network port numbers The NFS network port numbers need to be specified explicitly in the /etc/sysconfig/nfs file to ensure that they are consistent across reboots. HP recommends that you use the following ports. Port Transport Type Description 111 TCP Portmapper 111 UDP Portmapper 861 TCP mountd 861 UDP mountd 863 TCP rquotad 863 UDP rquotad 865 TCP statd 865 UDP statd 866 TCP statd 866 UDP statd 892 TCP rpc.mountd 892 UDP rpc.mountd 2049 TCP rpc.
Port Transport Type Description 445 UDP Microsoft Directory Service Enable NFS file locking if necessary When FS Option for Linux is installed in a software-only configuration, NFS file locking is disabled by default. See the HP StorageWorks X5500 Network Storage Gateway for Linux File Serving Software administration guide (“Using the NLM Protocol” in the chapter “Configure FS Option for Linux” ) for information about why file locking is disabled and how to enable it if necessary.
4 Disaster recovery procedures This chapter discusses tasks to perform after you complete the initial setup. The following procedures are discussed: • Disaster recovery Disaster recovery HP X5500 Storage Gateway for Linux servers in the cluster are highly reliable. The hardware has redundant components that should keep a server running, in most scenarios. If you must replace a server or re-image the operating system and software, a Quick Restore DVD is included with the product for this purpose.
Disaster recovery procedures
5 Support and other resources HP technical support For worldwide technical support information, see the HP support website: http://www.hp.
Support and other resources
A Building a custom kernel This appendix contains the following procedures for SLES10 installations: • Compile a third-party kernel module • Extract HP X5500 Storage Gateway for Linux kernel patches • Rebuild the entire kernel from source SLES10 Compile a third-party kernel module In general, third-party kernel modules are compiled using a skeletal kernel source tree and pre-computed symbols for the specific kernel configuration they will be loaded into.
To begin, open the architecture-independent kernel source RPM: mkdir tmp rpm2cpio kernel-source-2.6.16.60–0.21.370..src.rpm | (cd tmp; cpio -id) The HP X5500 Storage Gateway for Linux patches are all in tmp/patches.hpps.tar.bz2. When you integrate the patches into your kernel build source, be sure to add their names to the series.conf file. hppsversion.
B Configure Linux Device Mapper MPIO NOTE: HP X5500 Storage Gateway for Linux customers: The Device Mapper Multipath is installed and configured by default on HP X5500 Storage Gateway for Linux systems. This appendix serves as reference information for HP X5500 Storage Gateway for Linux customers. In addition, fc_pcitable entries are updated with the required parameters Linux Device Mapper Multipath can be used to provide multipath support for HP X5500 Storage Gateway for Linux.
Complete the following steps on each server: 1. Download the HP Device Mapper Multipath Enablement Kit for HP StorageWorks Disk Arrays 4.2.0 package. The package is available at the following location: http://www.hp.com/go/devicemapper 2. Login as root. 3. Copy the tar package to a temporary directory such as /tmp/HPDMmultipath. 4. Unbundle the package: # cd /tmp/HPDMmultipath # tar –xvzf HPDMmultipath-4.2.0.tar.gz # cd HPDMmultipath-4.2.0 5. Install the package: # ./INSTALL 6.
• Path. If the path begins with “/”, it is considered to be an absolute path. Otherwise, it is considered to be relative to the /opt/hpcfs/lib/modules/current directory. • Options, enclosed in double quotes, to pass to insmod when it loads the driver. • A text description of the driver.
Configure Linux Device Mapper MPIO
C Configure the cluster from the command line HP X5500 Storage Gateway for Linux provides mx commands that can be used to create the initial cluster configuration. These commands are equivalent to the Configure Cluster graphical user interface described in Chapter 2 and can be used in configuration scripts. Be sure to review the description of the Configure Cluster window (see “8. Configure the cluster” on page 23) to become familiar with the actions performed by the mx commands.
mx --matrix nodeA config fcfencing mx --matrix nodeA config mp set [ ] mx --matrix nodeA config snapshot add --method [--options ] (This command is needed only if you will be using the snapshot feature.) mx --matrix nodeA server add nodeA mx --matrix nodeA server start nodeA mx --matrix nodeA config description The cluster is now configured and HP X5500 Storage Gateway for Linux is running on nodeA.
The cluster is now configured and HP X5500 Storage Gateway for Linux is running on nodeA. Repeat the following set of commands to add the remaining nodes to the cluster and export the configuration to them. mx --matrix nodeA server add mx --matrix nodeA config export Now configure web management-based fencing on the nodes just added to the cluster. For each node, specify the appropriate fencing information.
# Cluster Details # Start Node MATRIX="nodeA" # Other Nodes NODES="nodeB nodeC nodeD" # Fibre Channel Switch Information SWITCHES="99.10.180.253" # Membership Partitions MP="6-6005-08B3-0090-A860-22BE-8098-CC0A-0041/1 6-6005-08B3-0090-A860-22BE -8098-CC0A-0041/2 6-6005-08B3-0090-A860-22BE-8098-CC0A-0041/3" # License File LICENSE="permanent.
HP X5500 Storage Gateway for Linux assigns to the SAN disks placed under its control. Specify either one or three LUNs or disk partitions be to used as membership partitions. Note the following: • LUNs must already be partitioned as described under “Create LUNs or Disk Partitions for Membership Partitions.” • The minimum size for a membership partition is 1GB. mx server add Add the specified node to the cluster. mx server start Start HP X5500 Storage Gateway for Linux on the specified node.
When a server needs to be restricted from the SAN, HP X5500 Storage Gateway for Linux can either power-cycle the server or shut it down. Select the method that you want to use (either reset or shutdown). mx config snapshot add --method [--options ] Configure a snapshot method. The options are: --method The supported types are hpmsa2000, hpxp, hpeva, and engenio.
