HP StorageWorks HP Scalable NAS File Serving Software administration guide HP Scalable NAS 3.
Legal and notice information © Copyright 2004, 2009 Hewlett-Packard Development Company, L.P. Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license. The information contained herein is subject to change without notice.
Contents About this guide ................................................................. 20 Intended audience ............................................................................................. HP technical support .......................................................................................... Subscription service ........................................................................................... HP websites ....................................................................
3 Cluster Administration ...................................................... 41 Administrative considerations .............................................................................. Network operations .................................................................................... Servers ...................................................................................................... SAN .....................................................................................................
Back up and restore membership partitions .................................................... Quotas information for PSFS filesystems .......................................................... Firewalls and network port numbers ..................................................................... Configure firewalls (optional) ........................................................................ External network port numbers ...............................................................
Configure the SAN .......................................................... 85 Overview ......................................................................................................... 85 SAN configuration requirements ................................................................... 85 Storage Control Layer module ....................................................................... 85 Device names .............................................................................................
Dynamic volume names ............................................................................. Volume signature ...................................................................................... Configuration limits ................................................................................... Guidelines for creating dynamic volumes ..................................................... Create a dynamic volume .................................................................................
Note for Oracle Real Application Clusters users ...................................... Advanced mount options ..................................................................... Mount a filesystem from the command line ................................................... The HP Scalable NAS mx fs command .................................................. The Linux mount command ................................................................... Create the administrative filesystem .........................
Configure FS Option for Linux ......................................... 161 Overview ....................................................................................................... FS Option concepts and definitions ............................................................. Export Groups ................................................................................... Export records ................................................................................... Virtual NFS Services ......
From the Servers or Virtual Hosts tab ..................................................... From the Applications tab .................................................................... From the command line ....................................................................... Other Virtual NFS Service procedures .......................................................... Update a Virtual NFS Service .............................................................. Disable a Virtual NFS Service .............
Supported arrays ............................................................................................. HP MSA2000 storage arrays ..................................................................... HP EVA storage arrays .............................................................................. HP XP storage arrays ................................................................................. Engenio storage arrays ..............................................................................
Cluster event viewer ......................................................................................... View event details ..................................................................................... Filter the event output ................................................................................. View events from the command prompt ........................................................ View the Cluster Log ..........................................................................
Determine when replication is complete ....................................................... Restore replicated files ............................................................................... Start or stop replication ............................................................................. Start replication .................................................................................. Stop all replication programs ...............................................................
Create the administrative filesystem .................................................................... Using the Performance Dashboard ..................................................................... Performance views .................................................................................... Display locations on the dashboard ............................................................ Cluster Report ..........................................................................................
Service monitors and failover ...................................................................... Types of service monitors ........................................................................... FTP service monitor ............................................................................. HTTP service monitor .......................................................................... SMTP service monitor ..........................................................................
Servers for device monitors ......................................................................... Set a global event delay ................................................................................... Other configuration procedures ......................................................................... Delete a device monitor ............................................................................. Disable a device monitor .....................................................................
Active and inactive membership partitions ............................................. Export configuration changes ............................................................... mprepair options ................................................................................ Increase the membership partition timeout .................................................... Servers ..........................................................................................................
Browser accesses inactive server ................................................................. Troubleshoot monitor problems .......................................................................... Monitor status .......................................................................................... “Down” status .................................................................................... “Undefined” status ..............................................................................
Changes for ADS level security ............................................................. 415 Other windbind information ....................................................................... 415 C Initial configuration for HP 4400 Scalable NAS systems ..... 417 Networks ....................................................................................................... Storage ..........................................................................................................
About this guide This guide provides information about configuring and managing a cluster running HP Scalable NAS File Serving Software. Intended audience This guide is intended for system administrators managing HP Scalable NAS clusters. HP technical support For worldwide technical support information, see the HP support website: http://www.hp.
HP websites For additional information, see the following HP websites: • http://www.hp.com • http://www.hp.com/go/scalablenas • http://www.hp.com/go/storage • http://www.hp.com/service_locator • http://www.hp.com/support/manuals Documentation feedback HP welcomes your feedback. To make comments and suggestions about product documentation, please send a message to storagedocsFeedback@hp.com. All submissions become the property of HP.
About this guide
1 Quick start checklist The following information is intended for new HP Scalable NAS installations and includes typical steps to configure the cluster. It assumes that the installation and configuration steps described in the HP Scalable NAS File Serving Software installation guide or HP Scalable NAS File Serving Software setup guide have been completed. NOTE: For HP StorageWorks 4400 Scalable NAS systems, see Appendix C, page 417 for a description of the initial configuration.
Action Description Create dynamic volumes. Dynamic volumes can include multiple disks and are used for PSFS filesystems. See Create a dynamic volume, page 105. Create PSFS filesystems. Select the dynamic volumes to be used for each filesystem and configure the appropriate options such as block size and disk quotas. See Create a filesystem, page 125. Mount filesystems. Mount the new filesystems with the appropriate mount options. See Mount a filesystem, page 132.
Action Description Prepare for user access: Configure NFS clients. Configure NFS clients to access the exported filesystems. See NFS clients, page 192. Configure Samba. Configure Samba for failover support. See Appendix B, page 399 Prepare for monitoring the cluster: Configure event notifier services. The event notifier services (SMTP, email, and scripts) can be configured to send notifications or run scripts when specific cluster events occur. See Configure event notifier services, page 238.
Quick start checklist
2 Introduction HP Scalable NAS provides a cluster structure for managing a group of network servers and a Storage Area Network (SAN) as a single entity. It includes a cluster filesystem for accessing shared data stored on a SAN and provides failover capabilities for applications. The Cluster Volume Manager can be used to create and manage dynamic volumes, which allow large filesystems to span multiple disks, LUNs, or storage arrays.
• Cluster-wide administration. The Management Console (a Java-based graphical user interface) and the corresponding command-line interface enable you to configure and manage the entire cluster either remotely or from any server in the cluster. • Failover support for network applications. HP Scalable NAS uses virtual hosts to provide highly available client access to mission-critical data for Web, e-mail, file transfer, and other TCP/IP-based applications.
• NFS client lock re-acquisition on server failover. When a group of NFS clients fail over to another NFS server, any file or byte-range locks held by those clients are released and then automatically re-acquired after the clients have successfully transitioned to another server. • Group migration of NFS clients.
Servers. Each server must be running HP Scalable NAS. Public LANs. A cluster can include up to four network interfaces per server. Each network interface can be configured to support multiple virtual hosts, which provide failover protection for Web, e-mail, file transfer, and other TCP/IP-based applications. Administrative Network. HP Scalable NAS components communicate with each other over a common LAN. The network used for this traffic is called the administrative network.
Following are some of the daemons provided with HP Scalable NAS: ClusterPulse. Monitors the cluster, controls failover of virtual hosts and devices, handles communications with the Management Console, and manages device monitors, service monitors, and event notification. Distributed Lock Manager (DLM). Provides a locking mechanism to coordinate server access to shared resources in the cluster.
Shared SAN devices Before a SAN disk can be used, you will need to import it into the cluster. This step gives HP Scalable NAS complete and exclusive control over access to the disk. During the import, the disk is given a unique global device name. The servers in the cluster use this name when they need to access the disk. Cluster Volume Manager The Cluster Volume Manager can be used to create dynamic volumes consisting of disk partitions that have been imported into the cluster.
in progress at the time of the crash are completed. Users on other servers will experience only a slight delay in filesystem operations during the recovery. HP Scalable NAS databases HP Scalable NAS uses the following databases to store cluster information: • Shared Memory Data Store (SMDS). The SANPulse daemon stores filesystem status information in this database. The database contains cp_status and sp_status files that are located in the directory /var/opt/hpcfs/run on each server.
Virtual hosts and failover protection HP Scalable NAS uses virtual hosts to provide failover protection for servers and network applications. A virtual host is a hostname/IP address configured on one or more servers. The network interfaces selected on those servers to participate in the virtual host must be on the same subnet. One server is selected as the primary server for the virtual host. The remaining servers are backups.
When a device monitor is assigned to a server, you can select the virtual hosts that will be dependent on the device monitor. If a device monitor indicates that a device is not functioning properly on the primary server, HP Scalable NAS transfers the dependent virtual host addresses from the primary server to a backup server. Event notifier services HP Scalable NAS provides event notifier services that can be configured to respond when certain cluster events occur.
diagrams, the FC fabric cloud can include additional FC switches that are not managed by HP Scalable NAS. Single FC port, single FC switch, single FC fabric This is the simplest configuration. Each server has a single FC port connected to an FC switch managed by the cluster. The SAN includes two RAID arrays. In this configuration, multiported SAN disks can protect against a port failure, but not a switch failure.
Single FC port, dual FC switches, single FC fabric In this example, the fabric includes two FC switches managed by the cluster. Servers 1–3 are connected to the first FC switch; servers 4–6 are connected to the second switch. The cluster also includes two RAID arrays, which contain multiported disks. If a managed FC switch fails, the servers connected to the other switch will survive and access to storage will be maintained.
Dual FC ports, dual FC switches, single FC fabric This example uses multipath I/O to eliminate single points of failure. The fabric includes two FC switches managed by the cluster. Each server has two FC ports; the first FC port connects to the first FC switch and the second FC port connects to the second FC switch. The cluster also includes two RAID arrays containing multiported disks.
Dual FC ports, dual FC switches, dual FC fabrics This example is similar to the previous example, but also includes dual FC fabrics, with a cluster-managed FC switch in each fabric. If one of the fabrics should fail, the servers can access the storage via the other fabric.
iSCSI configuration This example shows an iSCSI configuration. The iSCSI initiator is installed on each server. Ideally, a separate network should be used for connections to the iSCSI storage arrays.
3 Cluster Administration HP Scalable NAS can be administered either with the Management Console or from the command line. Administrative considerations You should be aware of the following when managing HP Scalable NAS. Network operations • Normal operation of the cluster depends on a reliable network hostname resolution service. If the hostname lookup facility becomes unreliable, this can cause reliability problems for the running cluster.
Servers • For best performance, HP recommends that you monitor the cluster from a separate administrative station rather than from a server in the cluster. The Management Console can be installed on Linux or Windows systems. • HP Scalable NAS components can fail if the /var filesystem fills up. To avoid this situation, move /var/opt/hpcfs to a separate partition containing at least 400 MB of free space.
Cluster software requirements The operating system and HP software must be installed consistently across the servers in the cluster. When configuring your servers, you should be aware of the following requirements: • All servers must be running versions of the operating system and kernel that are supported by HP Scalable NAS. • The servers in the cluster can be running different supported versions of the same operating system type as long as the above rule is met.
FS Option for Linux FS Option for Linux has been tested to the following configuration limits: • 128 Virtual NFS Services per cluster. • 64 Export Groups per cluster. • 1024 export records per Export Group. • 512 PSFS filesystems exported via NFS per cluster. Dynamic volumes The configuration limits for dynamic volumes are as follows: • • • • A maximum size of 128 TB for a dynamic volume. A maximum of 512 dynamic volumes per cluster. A maximum of 128 subdevices per dynamic volume.
the software from the cache whenever possible. If for some reason the software version running on the server cannot be identified, the applications use the latest version in their cache. When you invoke mxconsole or mx from a local machine, by default the application checks the current software version on the server to which it is being connected and then downloads the software only if that version is not already in the local cache.
Connect to: Type a cluster or server name or select a name from the drop-down list. When you connect to a server or cluster, it is added to the drop-down list. Click Clear History to delete the list. (Saved bookmarks will remain.) By default, the Connect window logs you onto the cluster using your OS user credentials. If you want to log on as another user, click the “As User” button: You can then specify the user name and password for that user as specified under “Authentication Parameters and Bookmarks.
Password: Type the user’s password. If you do not want to be prompted for the password again, click Remember this password. (For the password to be saved, you will also need to create a bookmark.) Add to bookmarks: Click this checkbox to create a bookmark for this connection. After clicking OK on the Authentication Parameters dialog, you can configure the bookmark on the Add Bookmark dialog. Specify a name for the cluster or select an existing name from the drop-down list.
The bookmark options are: • Add. This option opens the Add Bookmark dialog, allowing you to configure a new bookmark. • Delete. If a cluster is selected, this option removes the bookmark for that cluster. If a server is selected, the option removes just that server from the bookmark. • Rename. If a cluster is selected, this option allows you to rename that cluster. If a server is selected, you can replace that server with a different server in the cluster. After typing the new name, press Enter.
NOTE: If you are using a wildcard to match the servers in the cluster, the wildcard entry should appear after any server entries. You can use the move buttons to reorder the entries as necessary. (For information about wildcards, see the description of the .matrixrc file in the HP Scalable NAS File Serving Software command reference guide.) • Reset. This option is useful if you have checked Remember this password on the Authentication Parameters dialog.
• Remove the server entries from the .matrixrc file. Then specify one of the servers on the HP Scalable NAS Connect window. When the Authentication Parameters dialog appears, check Add to Bookmarks. Including passwords in the .matrixrc file is now optional. You can remove the passwords from your file if desired, or select the bookmark entry on the Connect window and click Reset. For more information about the .matrixrc file, see the HP Scalable NAS File Serving Software command reference guide.
Servers tab The Servers tab lists the entire configuration of each server configured in the cluster, including the network interfaces on the server, any virtual hosts associated with those interfaces, any device monitors created on the server, and any PSFS filesystems mounted on the server.
Virtual Hosts tab The Virtual Hosts tab shows all virtual hosts in the cluster. For each virtual host, the window lists the network interfaces on which the virtual host is configured, any service monitors configured on that virtual host, and any device monitors associated with that virtual host.
Applications tab This view shows the HP Scalable NAS applications, virtual hosts, service monitors, and device monitors configured in the cluster and provides the ability to manage and monitor them from a single screen. The applications, virtual hosts, and monitors appear in the rows of the table. The servers on which they are configured appear in the columns.
Filesystems tab The Filesystems tab shows all PSFS filesystems in the cluster.
Cluster alerts The Alerts section at the bottom of the Management Console window lists errors that have occurred in cluster operations. Double click an alert message to see all of the information about the alert. For alerts affecting cluster components such as servers or monitors, you can double-click in the Source column to highlight the source of the error on the main Management Console window.
To view the current Alerts from the command line, use the mx alert status command. View installed software The Installed Software window lists the operating system and HP Scalable NAS software that are currently installed on each server in the cluster. To see this window, select Help > Installed Software.
the HP Scalable NAS SNMP sub-agent, add the following line to the /etc/snmp/ snmpd.conf file and then restart snmpd. dlmod mxsnmp /opt/hpcfs/lib/snmp/mxsnmp.so HP Scalable NAS also provides two MIBs that can be loaded into the MIB browser tool provided with your Network Management Station (NMS). The MIBs are named mxsnmp.mib and mxtrap.mib and are located in the /opt/hpcfs/lib/snmp directory. Start or stop snmpd Use the following commands to start or stop snmpd: /etc/init.d/snmpd start /etc/init.
Process monitoring The mxinit utility is started automatically as a daemon on each server and monitors all HP Scalable NAS processes running there. (You can start another instance of mxinit to perform other tasks provided by the utility.) If a HP Scalable NAS process dies unexpectedly on a server, mxinit will restart it if possible. However, certain process failures may force a restart of that particular server.
FibreChannel adapter module status: intermodule - Intermodule communications driver, is Loaded qla2xxx_conf - QLogic conf module, is Loaded qla2xxx - QLogic 2432 Fibre Channel Adapter, is Loaded The PID is displayed for running processes; “Stopped” is displayed for processes that are not running. For modules, the status specifies whether the module is loaded. The “Fibre Channel adapter module status” section displays status for the Fibre Channel adapter modules installed on the system.
Start or stop HP Scalable NAS with mxinit Typically, the pmxs script is used to start or stop HP Scalable NAS. However, if you want to see verbose output during the start or stop operation, you can run mxinit manually with the --verbose option. You can use the following mxinit options to start or stop HP Scalable NAS: -s, --start Start the HP Scalable NAS processes. -x, --stop Gently stop the HP Scalable NAS processes. mxinit first attempts to unmount PSFS filesystems.
Administer init.d scripts When services such as NFS or Samba are configured on PSFS filesystems, HP Scalable NAS must be started before the service is started. To ensure that HP Scalable NAS starts before the service, modify the /etc/init.d script for the desired service and add pmxs as a dependency. For example, on SLES10, locate the line in the /etc/init.d/nfs script beginning with “# Required-Start:” and add pmxs to the end of the line.
Also be sure to back up your custom scripts and notifiers. It is possible to save much of the cluster configuration with the mx matrix dump command. See the HP Scalable NAS File Serving Software command reference guide for more information. FS Option for Linux configuration The FS Option configuration is stored in an online database. To back up the information in the database, you will need to save the Export Group configuration and the NLM state.
Back up and restore membership partitions The membership partitions contain three databases that need to be backed up: • The device database, which contains information about imported disks. • The volume database, which contains information about dynamic volumes. • The mxds datastore, which contains cluster configuration information. The device database and the volume database are automatically backed up to the file /var/opt/hpcfs/run/MP.
restoring the filesystem, run the psfrq command to restore the quota information. See Back up and restore quotas, page 211. Firewalls and network port numbers HP Scalable NAS uses a set of network ports for external connections, such as connections from the Management Console and the mx utility. It also uses a different set of network ports for internal, or server-to-server communications. The external ports numbers are fixed, and cannot be changed.
Port Transport Type Description 2381 UDP Array Configuration Utility 6771 TCP HTTPS connection from the HP Scalable NAS Management Console (fixed, IANA registration has been applied for) Internal network port numbers The following network port numbers are used for internal, server-to-server communication. Should you have HP Scalable NAS nodes firewalled from each other, you will need to configure the firewall to allow communication on these ports.
Port Transport Type Description 9065 TCP MSM control and statistics connections 9065 UDP MSM point-to-point messages NFS network port numbers The NFS network port numbers need to be specified explicitly in the /etc/ sysconfig/nfs file to ensure that they are consistent across reboots. HP recommends that you use the following ports.
Samba/CIFS network port numbers Samba/CIFS uses the following port numbers.
pmxs-mxconsole pmxs-dlm pmxs-dlm pmxs-msm pmxs-msm 9050/tcp 9060/tcp 9060/udp 9065/tcp 9065/udp HP Scalable NAS man pages HP Scalable NAS includes man pages for the commands and utilities provided with it. To enable the man command to locate these man pages, prepend /opt/hpcfs/ man to your MANPATH. The following example is for the Bash shell.
Man Page Description mx(8) Manipulate a cluster mxinit(8) Start, stop, or monitor HP Scalable NAS processes mxfence(8) Verify the fencing module configuration mxfsreplycachebasename(8) Enable or disable virtualized NFSD reply caches (provided with FS Option) mxlogger(8) Add a log message to the cluster log mxmpio(8) Monitor or manage MPIO devices mxnlmconfig(8) Enable or disable NLM locking mxsanlk(8) Display status of SAN ownership locks netif(8) mx command to manipulate network interfac
Man Page Description rplwatch(8) Watch PSFS filesystems for changes and schedule replication rpl_create_hr(8) Convert binary replication log files into a human-readable format role(8) mx command to configure management roles sandiskinfo(8) Display information about PSFS filesystems server(8) mx command to manipulate servers service(8) mx command to manipulate service monitors sleep(8) Pause between mx commands snapshot(8) mx command to manage hardware snapshots vhost(8) mx command to
4 Configure servers Before adding a server to a cluster, verify the following: • The server is connected to the SAN if it will be accessing PSFS filesystems. • The server is configured as a fully networked host supporting the services to be monitored. For example, if you want HP Scalable NAS to provide failover protection for your Web service, the appropriate Web server software must be installed and configured on the servers. • If the /etc/hosts file has been modified, it should be consistent with the DNS.
6. Restart HP Scalable NAS on the existing nodes. Add a new server To add a new server to a cluster, complete the following steps: 72 1. Install or set up HP Scalable NAS on the server. Perform the installation procedure in the HP Scalable NAS File Serving Software installation guide up to the step to configure the cluster. 2. Import the cluster configuration to the server. Issue the mxconfig command on the new server and open the Configure Cluster window.
Modify server properties To modify the properties for a server in the cluster, select that server on the Servers window, right-click, and select Properties. Then make your changes on the Server Properties window. Server: Enter the name or IP address of the server.
NOTE: For improved performance, the Management Console caches hostname lookups. If your DNS changes, you may need to restart the console so that it will show the new hostname. To update a server from the command line, use this command: mx server update [--serverSeverity autorecover|noautorecover] ...
Enable a server Select the server to be enabled from the Servers window on the Management Console, right-click, and select Enable. To enable servers from the command line, use this command: mx server enable ... Change the IP address for a server A server’s IP address can be changed without affecting the other servers in the cluster. The following procedure uses this scenario: three servers, S1, S2, and S3, with a single NIC. You want to change the IP address of server S2. 1.
• On the command line, enter the following: mx server read_license Until the new license file is read on all servers, you may see alerts on the Management Console reporting that the license file does not match on all servers. Supported HP Scalable NAS features The license agreement for each server determines which HP Scalable NAS features are supported on that server. You can use the View Features option on the Management Console to determine the supported features for a particular server.
name of the server to the virtual host, which is generally the better solution, or keep the published name associated with the server. • Migrate the existing server name to a virtual hostname. In this case, you will need to assign a new name to the server. External sites that then access the original server name will actually be served by the virtual host—basically, a failover-protected version of the server, with no difference in appearance to the clients. • Keep the existing name on the server.
The addresses on the domain name server are virtual_acmd1 and virtual_acmd2. Two virtual hosts have also been created with those names. The first virtual host uses acmd1 as the primary server and acmd2 as the backup. The second virtual host uses acmd2 as the primary and acmd1 as the backup. The DNS server is configured for round robin using the following A records: Address Time to Live Service Record Type IP Address www.acmd.com. 60 IN A 10.1.1.1 www.acmd.com. 60 IN A 10.1.1.
5 Configure network interfaces When you add a server to the cluster, HP Scalable NAS determines whether each network interface on that server meets the following conditions: • The network interface is up and running. • Broadcast and multicast are enabled on the network interface. • Each network interface card (NIC) is on a separate network. Network interfaces meeting these conditions are automatically configured into the cluster.
will attempt to communicate with the other servers over the main interface (the hostname interface) if possible; however, it may drop out of the cluster. Network topology The cluster should have a symmetrical network configuration; each server should be connected to the same set of networks. This topology helps to ensure that the administrative network can fail over to another network and maintain communications with all servers in the cluster.
Network interfaces and the Management Console When HP Scalable NAS is started, the ClusterPulse daemon probes the servers in the cluster to locate network interfaces and then adds the interfaces to its configuration file. The Servers window on the Management Console shows the network interfaces for each server as defined in this file. (Because there can be stale information in the configuration file, the Servers window may not match your current network configuration exactly.
If HP Scalable NAS must use a network that was configured to discourage administrative traffic, it will fail over to a network that allows the traffic as soon as that network becomes available to all servers in the cluster. If multiple interface failures occur on a server and there is not another network available for the administrative network, the server may drop out of the cluster. The remaining servers will continue to use the existing administrative network.
When you configure a network interface to allow, discourage, or exclude administrative traffic, the setting applies to all interfaces within the same subnet on all servers of the cluster. Enable or disable a network interface for virtual hosting By default, all network interfaces are enabled for virtual hosting. To enable or disable a network interface for virtual hosting, select that interface on the Servers window, right-click, and then select either Enable Hosting or Disable Hosting as appropriate.
Server: The name or IP address of the server to include the new network interface. IP: Type the IP address for the network interface. Netmask: Type the net mask for the network interface. Allow Administrative Traffic: Specify whether the network interface can host administrative traffic for HP Scalable NAS. The options are: • Allow. The default is to allow the traffic. • Discourage. This network will be used for administrative traffic only if no other network is available.
6 Configure the SAN SAN configuration includes the following: • Import or deport SAN disks. After a disk is imported, it can be used for PSFS filesystems. • Change the partitioning on SAN disks. • Display information about SAN disks. • Manage multipath I/O. Overview SAN configuration requirements Be sure that your SAN configuration meets the requirements specified in the HP Scalable NAS File Serving Software installation guide or HP StorageWorks X5500 Network Storage Gateway for Linux setup guide.
Device names The SCL uses unique device names to control access to shared SAN devices. These names form the pathnames that servers use to access shared data. When you import a SAN disk, the SCL gives it a global device name that represents the entire disk. The name consists of psd followed by an integer. For example, psd1, psd2, and psd25 are all valid names. Individual disk partitions also have a global device name. It consists of the name assigned to the disk followed by p and the partition number.
Disk partition alignment Many disk arrays are sensitive to misaligned I/O operations and require I/O to be aligned on block boundaries for performance reasons. For example, an array may “prefer” to have I/O operations occur on natural block boundaries, such as 4-KB blocks. If a partition starts at a non-block boundary, a single logical write can span more physical blocks that it would on a partition starting on a block boundary, resulting in additional I/O operations.
one operation, either by selecting multiple disks on the Import Disks window or by specifying multiple disks on the mx disk import command line. When you import a disk, the SCL gives it a global device name such as psd25. It also assigns global device names to all of the partitions on the disk. The individual partitions are identified by the disk name followed by p and the partition number, such as psd25p4.
Deport SAN disks Deporting a disk removes it from cluster control. The /dev/psd device nodes are removed and the original /dev entries are re-enabled. You cannot deport a disk that contains a mounted filesystem or a membership partition. Also, disks configured in a dynamic volume cannot be deported. (You will need to destroy the dynamic volume and then deport the disk.) To deport a disk from the Management Console, select Storage > Disk > Deport or click Deport on the toolbar.
Change the partitioning on a disk The Linux fdisk utility can be used to change the partition layout on a SAN disk. If the disk is currently imported into the cluster, you must first deport the disk. When you use fdisk, the changes made to the partition table are visible only to the server where you made the changes. When you reimport the disk, the other servers in the cluster will see the updated partition table.
When you select a disk, the window displays information about the partitions on the disk. Select a partition to display the corresponding Linux mount path for the PSFS filesystem. To import or deport a disk, select that disk and then click Import or Deport as appropriate. Storage Summary window The Storage Summary window shows information about the PSFS filesystems configured on the cluster and also lists the LUNs that are currently unused and available for filesystems.
• The volume used for the filesystem. Click in the cell to see the properties for the filesystem. • The number of exports for the filesystem multiplied by the number of servers exporting the filesystem. Click in the cell to see the names of the servers and the export paths. • The number of servers that are configured to host the filesystem. Click in the cell to see the names of the servers. • The total space, free space, and percent of space free on the filesystem.
-f Display PSFS filesystem volumes. -a Display all information; for -v, display all known volumes. -l Additionally display host-local device name. -r Additionally display local device route information. -U Display output in the format used by the Management Console. This option is used internally by HP Scalable NAS and does not produce human-readable output. -q Suppress output of all log messages. Following are some examples of these options.
# sandiskinfo -f Volume: /dev/psv/psv1 Size: 2439M (PSFS Filesystem) Stripesize=0K Local Mount Point=/mnt Volume: /dev/psd/psd1p6 Size: 490M (PSFS Filesystem) Disk=20:00:00:04:cf:13:38:18::0 partition=06 type=Linux (83) Local Mount Point=(not mounted) Show available volumes The -v option lists available volumes on imported disks. These volumes are not currently being used for a PSFS filesystem or a membership partition.
Dynamic Volume: psv2 Dynamic Volume: psv3 Size: Size: 490M 490M Stripe=32K Stripe=8K Show properties for dynamic volumes The --dynvol_properties [volname] option lists detailed properties for the specified dynamic volumes. volname is the psv name, such as psv2. If this option is omitted, the properties for all dynamic volumes are displayed.
HP Scalable NAS mxmpio utility HP Scalable NAS uses multipath I/O (MPIO) to eliminate single points of failure. A cluster can include multiple Fibre Channel switches, multiple FC ports per server, and multiported SAN disks. This configuration provides multiple paths for I/O between cluster servers and SAN disks. When you start HP Scalable NAS, it automatically configures all paths from each server to the storage devices. On each server, it then uses the first path it discovered for I/O with the SAN devices.
Shows the load for each target (SCSI command I/Os) and total for the PSD device (block layer I/Os), number of failovers, and fatal errors for each device. iostat [-u] [interval [count]] [ ...] Show general I/O statistics for each device. rawstat [interval [count]] [ ...] Dump general I/O statistics for each device in a raw format. Enable or disable failover for a server A cluster server can use multiple FC ports to connect to the Fibre Channel fabric.
target can be one of the following values: I A numerical index on the PSD device target array (0..). M,m A decimal major/minor number identifying the host adapter. M:m A hexadecimal major/minor number identifying the host adapter. scsiN/C A scsi bus ID. N is the scsi host adapter ID (0..) and C is the scsi host channel number (0..) as defined in /proc/scsi/scsi and /proc/scsi/*/[09]. If C is omitted, zero is assumed. sdNsP A device node in /dev.
# /opt/hpcfs/sbin/mxmpio status -l MPIO Failover is globally enabled Failover Timeout Targets psd1 enabled 30000 0. (41:50) 1. (08:90) psd1p1 enabled 10000 0. (41:51) 1. (08:91) psd1p2 enabled 30000 0. (41:52) 1. (08:92) psd2 enabled 30000 0. (41:10) 1. (08:50) psd2p1 enabled 10000 0. (41:11) 1. (08:51) psd2p2 enabled 30000 0. (41:12) 1.
Display MPIO statistics mxmpio also includes the following commands, which display various MPIO statistics. See the mxmpio(8) man page and the HP Scalable NAS File Serving Software command reference guide for details about these commands. mpiostat Display the number of transient errors for each target, and show the number of failovers and fatal errors for each device. mpioload Display load information for each target and the PSD device, and show the number of failovers and fatal errors for each device.
The fields contain the following information: • • • • Vendor ID. Device ID. Driver name. Do not include .o in the name. Path. If the path begins with “/”, it is considered to be an absolute path. Otherwise, it is considered to be relative to the /opt/hpcfs/lib/modules/current directory. • Options, enclosed in double quotes, to pass to insmod when it loads the driver. If no options are required, type a pair of double quotes (““) in the field. • A text description of the driver.
When HP Scalable NAS is started later in the boot process, the FC switch ports are enabled and HP Scalable NAS attempts to reinitiate the device scanning process in the hope that the third-party MPIO software will discover and manage the devices. If the process succeeds, HP Scalable NAS will continue to start as normal. If the third-party software is unable to discover the target devices during this process, HP Scalable NAS causes the node to reboot a second time.
7 Configure dynamic volumes HP Scalable NAS includes a Cluster Volume Manager that you can use to create, extend, recreate, or delete dynamic volumes. Dynamic volumes allow large filesystems to span multiple disks, LUNs, or storage arrays. Dynamic volumes can be deported from the cluster and later imported back into the original cluster or into another cluster. Overview Basic and dynamic volumes Volumes are used to store PSFS filesystems. There are two types of volumes: dynamic and basic.
second subdevice, and then to the third subdevice. This method fills the subdevices at the same rate and may provide better performance. • Concatenation. When a dynamic volume is created without striping, data is written to the first subdevice until it is full. Then data is written to the second subdevice, and so on. Dynamic volume names The Cluster Volume Manager uses unique device names to control access to dynamic volumes. These names form the pathnames that servers use to access shared data.
Guidelines for creating dynamic volumes When creating striped dynamic volumes, follow these guidelines: • The subdevices used for a striped dynamic volume should be the same size. The Cluster Volume Manager uses the same amount of space on each subdevice in the stripeset. When a striped dynamic volume is created, the Cluster Volume Manager determines the size of the smallest specified subdevice and then uses only that amount of space on each subdevice.
Filesystem: If you want HP Scalable NAS to create a filesystem that will be placed on the dynamic volume, enter a label to identify the filesystem. If you do not want a filesystem to be created, remove the checkmark from Create filesystem after volume creation. If you are creating a filesystem, you can also select the options to apply to the filesystem. Click Options to see the Filesystem Option dialog. The General tab allows you to select the block size, which can be 4KB, 8KB, 16KB, or 32KB.
The Quotas tab allows you to specify whether disk quotas should be enabled on this filesystem. If you enable quotas, you can set the default quota for users and groups on that filesystem. If you do not want a default limit, click Unlimited. To assign a limit, click Limit and then specify the appropriate size in either kilobytes, megabytes, gigabytes, or terabytes. The defaults are rounded down to the nearest filesystem block.
Available Subdevices: The display includes all imported subdevices that are not currently in use by another volume and that do not have a filesystem in place. Subdevices belonging to unimported volumes that are on imported disks are also included in the display. The subdevices that you select are used in the order in which they appear on the list. Use the arrow keys to reorder, one-at-a-time, the appropriate subdevices and then highlight those subdevices.
Dynamic volume properties To see the configuration for a dynamic volume, select Storage > Dynamic Volume > Volume Properties and then select the volume that you want to view. If a filesystem is associated with the volume, the Volume Properties window shows information for both the dynamic volume and the filesystem. The Stripe State reported in the “Dynamic Volume Properties” section will be one of the following: • Unstriped. The volume is concatenated and striping is not in effect. • Optimal.
To display the properties from the command line, use the following command: mx dynvolume properties Extend a dynamic volume The Extend Volume option allows you to add subdevices to an existing dynamic volume. When you extend the volume on which a filesystem is mounted, you can optionally increase the size of the filesystem to fill the size of the volume. The subdevices used for a striped dynamic volume are called a stripeset.
Dynamic Volume Properties: The current properties of this dynamic volume. Filesystem Properties: The properties for the filesystem located on this dynamic volume. Available Subdevices: Select the additional subdevices to be added to the dynamic volume. The list includes all available subdevices on imported disks, including subdevices belonging to unimported volumes. Use the arrow keys to reorder those subdevices if necessary.
NOTE: If you selected a subdevice that is associated with an unimported volume, you will see a message reporting that the subdevice contains a volume signature. The message asks whether you want to destroy the affected unimported dynamic volume and reuse this subdevice for the volume you are extending. Be sure that you do not need the unimported dynamic volume before doing this. If the unimported volume associated with the subdevice is in a “duplicate” state, the attempt to destroy the volume will fail.
To delete a dynamic volume from the command line, use the following command: mx dynvolume destroy Recreate a dynamic volume Occasionally you may want to recreate a dynamic volume. For example, you might want to implement striping on a concatenated volume or, if a striped dynamic volume has been extended, you might want to recreate the volume to place all of the subdevices in the same stripe set.
You can change or reorder the subdevices used for the volume and enable striping if desired. To recreate a volume from the command line, run the dynvolume destroy command and then run the dynvolume create command. Convert a basic volume to a dynamic volume If you have PSFS filesystems that were created directly on an imported disk partition or LUN (a basic volume), you can convert the basic volume to a dynamic volume.
NOTE: The new dynamic volume is unstriped. It is not possible to add striping to a converted dynamic volume. If you want to use striping, you will need to recreate the volume. To convert a basic volume, select the associated PSFS filesystem on the Management Console, right-click, and select Convert to Dynamic Volume. The filesystem must be unmounted.
Deport a dynamic volume The volume to be deported must contain a PSFS filesystem. During the deport operation, the filesystem is unmounted and the psv binding (which HP Scalable NAS uses to control access to the dynamic volume) is removed. If the filesystem has persistent mounts, the mounts should also be deleted. Select the filesystem on the Filesystems tab of the Management Console, right-click, and select Edit Persistent Mounts.
Import a dynamic volume When a dynamic volume is imported, the unimported LUNs associated with the volume will be imported and the psv binding, which HP Scalable NAS uses to control access to the dynamic volume, will be created. NOTE: When you reimport a dynamic volume, the psv name assigned to the volume may be different from the psv name used by the volume before it was deported. HP Scalable NAS does not reserve the psv name originally used by the volume, and may instead assign it to another volume.
To import dynamic volumes from the command line, first use the following command to list the dynamic volumes that can be imported: mx dynvolume list --importable Locate the entry for the that you want to import. (The appears in the first column of the output.) Then use the following command to import the volume, specifying the . mx dynvolume import ... Unimported volumes An unimported volume is a dynamic volume that does not have a psv binding.
Truncated. The volume cannot be reassembled because one of the physical devices is smaller than its logical subdevice specification. Potential causes of this problem are: • The disk partition table was changed. • The size of the LUN was reduced on the array. Incomplete. The volume cannot be reassembled because one or more of its subdevices were not found. Potential causes of this problem are: • The missing subdevices are on LUNs that are not accessible from the cluster.
Configure dynamic volumes
8 Configure PSFS filesystems HP Scalable NAS provides the PSFS filesystem. This direct-access shared filesystem enables multiple servers to concurrently read and write data stored on shared SAN storage devices. A journaling filesystem, PSFS provides live crash recovery.
Journaling filesystem When you initiate certain filesystem operations such as creating, opening, or moving a file or modifying its size, the filesystem writes the metadata, or structural information, for that event to a transaction journal. The filesystem then performs the operation. If a server using the filesystem should crash during an operation, the journal is replayed and any journaled operations in progress at the time of the crash are completed.
recovery operation) and the status of the filesystem on each server (for example, whether the filesystem is mounted or unmounted). Filesystem synchronization and device locking The Distributed Lock Manager (DLM) daemon coordinates access to shared resources in the cluster. To do this, it provides the locking mechanisms used to implement the necessary synchronization for various filesystem operations.
Quotas are enabled or disabled at the filesystem level. When quotas are enabled, the filesystem performs quota accounting to track the disk use of each user or group having an assigned disk quota. When a file owner reaches the hard limit, the filesystem will not allow the owner to create files or increase the size of an existing file; any attempts to allocate more space will fail. The file owner will need to remove files or reduce their size until the disk usage falls below the hard limit.
Create a filesystem A PSFS filesystem can be created on a basic volume (a psd device) or a dynamic volume (a psv device). PSFS filesystems can use 4KB, 8KB, 16KB, or 32KB as the block size. The maximum filesystem size is dependent on the block size used for the filesystem. The block size cannot be changed after the filesystem is created.
Label: Type a label that identifies the filesystem. Available Volumes: This part of the window lists the basic or dynamic volumes that are currently unused. Subdevices belonging to unimported volumes that are on imported disks are also included. Select one of these volumes for the filesystem. IMPORTANT: If you selected a subdevice that is associated with an unimported volume, when you click OK to create the filesystem you will see a message reporting that the subdevice contains a volume signature.
Filesystem options Click Options to see the Filesystem Options window, which lists the available options for the filesystem. The General tab describes how the filesystem block size is used. The supported block sizes are 4K, 8K, 16K, and 32K. Be sure to select the appropriate block size for your filesystem. The Quotas tab allows you to specify whether disk quotas should be enabled on this filesystem.
If you enable quotas, you can set the default quota for users and groups on that filesystem. If you do not want a default limit, click Unlimited. To assign a limit, click Limit and then specify the appropriate size in either kilobytes, megabytes, gigabytes, or terabytes. The defaults are rounded down to the nearest filesystem block. The default user and group quotas apply to all users and groups, respectively; however, you can change the quota for a specific user or group.
Create a filesystem from the command line To create a filesystem, use one of the following HP Scalable NAS commands. The mx command Use this syntax: mx fs create A storage device is a basic or dynamic volume. To locate an available volume, use the fs showcreateopt command. The optional arguments are: --label
The default quota for all users on the filesystem. unlimited specifies that there is no default. The optional size modifiers specify that the size is in kilobytes (K), megabytes (M), gigabytes (G), or terabytes (T). If a modifier is not specified, the size will be calculated in bytes. (The default is rounded down to the nearest filesystem block.) --defaultGroupHardLimit The default quota for all groups on the filesystem. unlimited specifies that there is no default.
-q | ––quiet Quiet. Do not display progress messages. -y | ––yes Answer “yes” to all questions. -reuse Reuse a psd device. If you will be creating a filesystem on a psd device that was once part of a dynamic volume, the -reuse option must be used to tell the command to reuse the device. Without this option, the attempt to create the filesystem will fail because the device contains a volume signature. The -o option has the following parameters: blocksize=#[K] The block size for the filesystem.
Mount a filesystem You can mount a PSFS filesystem on any server that can access the storage device over the SAN. The directory mountpoint for the filesystem must exist before the filesystem is mounted. The filesystem must be mounted either read-write or read-only across the cluster. It cannot be mounted read-only on one server and read-write on another server. To change the way the filesystem is mounted, first unmount it on all servers, then remount it.
On Servers: Select the servers where the filesystem is to be mounted. Shared mount options This option applies to all servers on which the filesystem is mounted. Read/Write or Read Only. Mount the filesystem read/write or read-only. Read/Write is the default. Server mount options These mount options can be different on each server. Mount point: Type the directory mount point for the filesystem. Activate: To mount the filesystem now, click Activate. (This option is enabled by default.
Persist: This option causes the filesystem to be remounted automatically when the server is rebooted and is enabled by default. If you do not want the filesystem to be remounted automatically, remove the checkmark. Create Directory: If you want HP Scalable NAS to create the directory mountpoint on each server where the filesystem is to be mounted, click Create Directory. Select any other mount options for the filesystem. Async or Sync.
Unaligned I/Os from applications that support Database Operations are still allowed, but will be slower. Examples of Database Operations applications are tools used to back up, copy, and compress database files. These tools may sometimes issue I/O operations that are not aligned properly for this performance mount option, but will nonetheless operate correctly. The DB Optimized option controls only the way in which database files are accessed for reads and writes; it does not affect the data itself.
Advanced mount options Click Advanced on the Mount Filesystem window to see the additional options. The advanced mount options are as follows: Shared or Exclusive. Either allow all servers having physical access to the filesystem to mount it or allow only one server. Shared is the default. Ordered or Unordered. The Ordered option provides additional security for writes to the filesystem.
Mount a filesystem from the command line To mount a PSFS filesystem from the command line, use one of the following commands. The HP Scalable NAS mx fs command Use this syntax: mx fs mount --path [--persist] [--activate] [--options
The Linux mount command Use the following syntax to mount a filesystem. The directory mountpoint must already exist. mount -t psfs -o options device mountpoint The device can be either a psd or psv device. Use this syntax to specify the device: • For a psd device partition, the device is specified as /dev/psd/psdXXXpYY, where XXX is the drive number and YY is the partition number. For example, / dev/psd/psd6p4 specifies partition 4 on disk psd6.
Create the administrative filesystem The replication feature and the Performance Dashboard require that an administrative filesystem exist on the cluster. This filesystem can also be used for the mxfs reply cache, which FS Option uses when the virtualized NFSD RPC reply cache feature is enabled. The administrative filesystem should be used only for HP Scalable NAS operations. NOTE: The administrative filesystem is preconfigured on HP StorageWorks 4400 Scalable NAS systems.
For example, if there is no backlog (B=1) and 1000 files change per node per interval on a 16-node cluster, the formula would be: S = (4KB * 1000) * 17 = 68000KB = 68MB If 100,000 files changed, the size would be 6.8GB. Similarly, if 750,000 files changed, the size would be 51GB. Add 1GB to the total if the Performance Dashboard will be used and 1GB if the mxfs reply cache will be configured on the administrative filesystem. Create the filesystem The administrative filesystem must be created manually.
Allow disks that contain existing volume information to be reused. (The existing data is destroyed.) The command creates a dynamic volume containing one or more subdevices. You can extend the volume later with additional subdevices if needed. Striping is not configured on the volume. The administrative filesystem is mounted at /_adminfs on all nodes existing in the cluster at the time the filesystem is created. The filesystem appears on the Management Console in the same manner as other PSFS filesystems.
Mount the administrative filesystem on new nodes When a new node is added to the cluster, you will need to mount the administrative filesystem on the node. Select the filesystem on the Management Console, right-click, and select Add Mount, or click Mount on the toolbar. On the Mount Filesystem window, select the new server from the On Servers pane and click OK. (Do not change the default mount options.) The filesystem will be mounted immediately at the mount point specified on the Mount Filesystem window.
Unmount from the command line To unmount a filesystem from the command line, use one of the following commands. PSFS filesystems cannot be forcibly unmounted. The HP Scalable NAS mx command. The --persistent argument removes the persistent status from the filesystem mount; the --active argument unmounts the filesystem now. mx fs unmount [--persistent] [--active] ALL_SERVERS| ... The Linux umount command. Be sure to specify the mountpoint, such as /mnt/ data1, not the partition.
The Edit Persistent Mounts window lists all filesystems having a persistent mount on this server. • To remove the “persistent” mount status for one or more filesystems, select those filesystems and then click Delete. • To mount a filesystem with the options specified for the persistent mount, select that filesystem and then click Activate.
The Edit Persistent Mounts window lists all servers that are configured to have a persistent mount for the filesystem. • To remove the “persistent” mount status on a particular server, select that server and then click Delete. • To mount the filesystem with the options specified for the persistent mount, select the appropriate servers and click Activate. View or change filesystem properties To see information about a specific filesystem, select that filesystem, right-click, and select Properties.
volume, click Volume Properties to see more information about the volume, including the subdevices used for the dynamic volume and the stripe size and state. Extend a mounted filesystem If the Volume allocation display shows that there is space remaining on the volume, you can use Extend Filesystem on the Properties window to increase the size of the PSFS filesystem to the maximum size of the volume. When you click Extend Filesystem, you will see a warning such as the following.
Features tab The Features tab shows whether Full Zone Bit Maps (FZBM), quotas, or small files support are enabled on the filesystem. The HP Scalable NAS 3.7 release includes a performance enhancement for small files on PSFS filesystems. This feature is enabled by default in PSFS filesystems created on 3.7. PSFS filesystems created on earlier releases must be upgraded to enable the small files performance enhancement. (See the HP Scalable NAS File Serving Software upgrade guide for the upgrade procedure.
If you enable quotas, you can set the default quota for users and groups on that filesystem. If you do not want a default limit, click Unlimited. To assign a limit, click Limit and then specify the appropriate size in either kilobytes, megabytes, gigabytes, or terabytes. The defaults are rounded down to the nearest filesystem block. The Manage Quotas option allows you to view or change quotas for individual users or groups. See Manage quotas, page 204 for more information.
To see information about a filesystem mount on a particular server, select the filesystem entry for that server, right-click, and select Properties. You can also use Linux utilities such as df or the HP Scalable NAS mx command to display status information. mx fs status [--mounted|--unmounted] [--persistent] [--verbose] [--standard|--snapshots] The --verbose option displays the FS type (always PSFS), the size of the filesystem in KB, and the UUID of the parent disk.
To run the utility from the Management Console, select the filesystem, which must be unmounted, and then right-click and select Check for Errors. The utility runs in read-only mode and does not replay the journal or fix any corruption. HP recommends that you let the check run to completion. Canceling the check can potentially leave your filesystem in a corrupted state. For more information about the check, click Details.
If psfsck locates errors that need to be repaired, it will display a message telling you to run the utility from the command line. For more information, see the HP Scalable NAS File Serving Software command reference guide or the psfsck(8) man page. CAUTION: We strongly recommend that you make a backup copy of the entire partition before you attempt to run psfsck with the --rebuild-tree option.
By default, the atime for a file is updated whenever the file is accessed; however, you may want an update to occur only after a specific number of seconds have passed since the file was last updated. This can minimize the performance impact of atime updates for files that are accessed frequently (at the cost of sometimes seeing stale access times). To specify an update period, add the following parameter to the mxinit.
To suspend a filesystem, issue the following command on one server that has mounted the filesystem. You must be user root. # /opt/hpcfs/tools/psfssuspend directory_mountpoint The psfssuspend command prevents modifications to the filesystem and forces any changed blocks associated with the filesystem to disk. The command performs these actions on all servers that have mounted the filesystem and then returns successfully.
The utility can be used with either psd or psv devices. Use this syntax to specify the device: • For a psd device partition, the device is specified as /dev/psd/psdXXXpYY, where XXX is the drive number and YY is the partition number. For example, / dev/psd/psd6p4 specifies partition 4 on disk psd6. • For a non-partitioned psd device, the device is specified as /dev/psd/psdXXX, where XXX is the drive number. For example, /dev/psd/psd5.
Destroy a filesystem If necessary, you can remove a filesystem from the volume on which it is located. Select the filesystem on the Management Console, right-click, and select Delete. All persistent mounts for that filesystem will also be destroyed. The filesystem must be unmounted when you use this option. NOTE: Any snapshots of the filesystem must be destroyed before the filesystem is destroyed.
HP Scalable NAS supports the following keywords, which must be enclosed in curly braces. HOSTNAME Output from uname -n (the node’s network hostname). MACH Output from uname -m (the machine type). OS Output from uname -s (the operating system name). SYS Concatenated output from uname -m and uname -s, separated by an underscore (for example, i386_Linux). UID The effective UID of the process accessing the link. The UID refers to the numerical UID, not the user name.
2. Populate the directories with the data from the /oracle/etc directory: cp -r /oracle/etc/* /oracle/serv1/etc/ cp -r /oracle/etc/* /oracle/serv2/etc/ cp -r /oracle/etc/* /oracle/serv3/etc/ 3. Edit the files in the server-specific /etc directories as appropriate. 4.
ls -l /oracle/*bin lrwxrwxrwx 1 root root 10 Jun 16 16:40 /oracle/bin -> {MACH}/bin lrwxrwxrwx 1 root root 11 Jun 16 16:40 /oracle/sbin -> {MACH}/sbin When you log in on serv1, the /oracle/bin and /oracle/sbin symbolic links resolve to /oracle//bin and /oracle// sbin. On serv2, the links resolve to /oracle//bin and / oracle//sbin.
Create a semaphore To create a PSFS command-line semaphore, use this command: $ psfssema -i This command creates the PSFS command-line semaphore file and the supplemental file if either file does not already exist. It then initializes the files to create the PSFS command-line semaphore. must be a path on a PSFS filesystem.
This command deletes the PSFS command-line semaphore associated with , which is the name of a semaphore file created by psfssema -i. It also deletes and .
9 Configure FS Option for Linux HP Scalable NAS and FS Option for Linux provide scalability and high availability for the Network File System (NFS), which is commonly used on UNIX and Linux systems to share files remotely. Overview FS Option concepts and definitions FS Option uses and manages the following objects to provide scalable and highly available file service across the cluster: Export Groups, export records, and Virtual NFS Services.
Export records An export record is equivalent, indeed exactly equal to, the individual records contained in the /etc/exports file of a traditional NFS server. The format of the record and the options available are precisely the same in form and function. Each record identifies a filesystem and directory sub-tree to be exported via NFS, the list of clients authorized to mount and access the specified filesystem sub-tree, and the options that modify how the filesystem sub-tree will be shared with clients (e.
robin configuration such that a single IP address (or DNS name) can be used by all clients, yet the client connections will be evenly distributed among the several Virtual NFS Services running on separate nodes of the cluster. Other configurations are possible. Supported NFS versions FS Option supports versions 2 and 3 of the NFS file sharing protocols based on UDP and TCP.
Export records and groups The PSFS filesystems to be exported are described in export records, which are similar to the entries in an /etc/exports file. They specify the directory to be made available for mounting by NFS clients, the clients that can mount the directory, and the permissions on the directory. When you configure an Export Group, you will need to create an export record for each filesystem or directory to be exported.
If a failure occurs on the server to which NFS clients are connected or the server loses access to the PSFS filesystem (for example, because of a SAN problem), the appropriate monitor probe will report the failure. The Virtual NFS Services active on that server are then failed over (redirected) to a backup server. Failover is transparent to the NFS clients; their access to the exported filesystems continues after a brief pause.
Considerations for ensuring data safety NFS write performance and the risk of data loss during NFS write operations are affected by the use of the sync and async options. In your configuration, you will need to determine whether the following should be synchronous or asynchronous: • The PSFS filesystem mount • The server export of the PSFS filesystem • The client mount of the exported data For the least risk of data loss during NFS operations, use the sync option for all of these components.
Name: Type a name for this Export Group. The name must be unique among all of the Export Groups in the cluster and cannot include spaces. Application name: The application is used to group the Export Group and the Virtual NFS Services associated with the Export Group. Enter a new name for the application, or select an existing application. When you assign the Export Group to one or more Virtual NFS Services later in this procedure, the Virtual NFS Services will be added to this application.
NFS Exports tab This tab is used to configure the export records for the Export Group. An export record specifies a PSFS filesystem or directory to be exported, the NFS clients allowed to mount it, and the export options for the client. You can type the export records directly on the form, optionally using the Export Record Details window to construct the record, or you can import an existing exports file.
An export record uses the following format, which is documented in the Linux exports(5) man page: [] [(
that shows the export record as you construct it. The following Export Record Details dialog shows an example of a completed export record. Exported Path: Specify the filesystem or directory to be exported. Client Names: Click Add to specify a client to be allowed to mount the exported filesystem. Then, on the Add NFS Client dialog, identify the client by either its Netmask, IP address, or FQDN. Optionally, you can enter an asterisk (*) to specify all clients.
Options: Select any options that apply to this record. The default options are selected initially and do not appear on the Preview line. After you have created the export records, you can use the arrow buttons on the NFS Exports tab to reorder the entries as you prefer them. NOTE: When the Export Group is created, FS Option validates each export record and reports any syntax errors.
If the appropriate Virtual NFS Service does not currently exist, you can create it by clicking the Create a single Virtual NFS Service button. (See Add a Virtual NFS Service, page 187 for more information.) You can also create multiple Virtual NFS Services in one operation by clicking Create a spanning set of Virtual NFS Services. This step creates and configures a set of Virtual NFS Services, one on each available server with the other servers as backups.
Create a spanning set of Virtual NFS Services The “Create a spanning set of Virtual NFS Services” button on the Virtual NFS Services tab enables you to create multiple Virtual NFS Services with a single operation and assign them to the Export Group that you are creating. When you click this button, the dialog “Create a spanning set of Virtual NFS Services” appears and you can configure the Virtual NFS Services.
You can enter the DNS name or IP address for each Virtual NFS Service directly on the dialog, or you can specify a range of virtual IP addresses to be used for the Virtual NFS Services. To specify a range, click Automatically create range of VNFS addresses. Then specify the range on the dialog that appears next. When you click OK, the IP addresses will be filled in on the “Create a Spanning Set of Virtual NFS Services” dialog.
High-availability monitors FS Option provides two types of monitors that ensure high-availability for exported NFS filesystems: a global NFS monitor and Export Group monitors. Global NFS monitor The global NFS monitor periodically checks the status of the NFS service and the virtual FS Option processes on each node. If the monitor detects that the NFS service or a FS Option process is not available on a particular node, any Virtual NFS Services currently active on that node will fail over to a backup.
The Export Group monitor also appears on the Virtual Hosts tab. This example shows Export Group group128 being exported by Virtual NFS Service 99.11.14.101. Advanced options for Export Group monitors The advanced options allow you to tailor the configuration of the high-availability monitor associated with an Export Group. Probe configuration The Probe Configuration tab can be used to configure the frequency of the probe initiated by the Export Group monitor.
Frequency: The interval of time, in seconds, at which the monitor checks that each exported path in the Export Group is available and is mounted on the PSFS filesystem. The default is 30 seconds. To set the frequency from the command line, use this option: --frequency Probe severity The Probe Severity tab lets you specify the failover behavior of the Export Group monitor.
The Probe Severity setting works with the Virtual NFS Service policy (either AUTOFAILBACK or NOFAILBACK) to determine what happens when a monitor probe fails. The default policies (AUTOFAILBACK for the Virtual NFS Service and AUTORECOVER for the Export Group monitor) cause HP Scalable NAS to fail over the associated Virtual NFS Services to a backup network interface on another node when the monitor probe fails.
FS Option with a custom application where certain application-specific actions must be taken before failback can occur. To set the Probe Severity from the command line, use this option: --probeSeverity nofailover|autorecover|noautorecover Scripts The Scripts tab lets you configure custom Recovery, Start, and Stop scripts for the Export Group monitor. Monitors can optionally be configured with scripts that are run at various points during cluster operation. The script types are as follows: Recovery script.
nodes configured for the monitor to ensure that the service is not active on those nodes. Start scripts must be robust enough to run when the service is already started, without considering this to be an error. Similarly, Stop scripts must be robust enough to run when the service is already stopped, without considering this to be an error. In both of these cases, the script should exit with a zero exit status.
IGNORE. Events are ignored and Start or Stop script failures will not cause failover. This is useful when the action performed by the Start and Stop scripts is not critical, but is important enough that you want to keep a record of it. To configure event severity from the command line, use this option: --scriptSeverity consider|ignore Script ordering Script ordering determines the order in which the Start and Stop scripts are run when a Virtual NFS Service moves from one node to another.
View Export Group properties To view all of the Export Groups configured with FS Option, select View > View All Export Groups, or select an Export Group on the Virtual Hosts tab or a Virtual NFS Service on the Applications tab, right-click, and select View All Export Groups. The All Export Groups window then appears. This window lists the names of all Export Groups that have been configured. To see the properties for a particular Export Group, highlight that Export Group and click OK.
To view status from the command line, use the following command: mx exportgroup status [--up|--down] [--enabled|--disabled] [--primary|--backup] [--active|--inactive] [ ...] Other Export Group procedures Modify an Export Group To modify an existing Export Group, go to the Export Group Properties window, as described above. To change an export record, go to the NFS Exports tab.
Disable an Export Group Select the high-availability monitor associated with the Export Group on the Servers or Application tab, right-click, and select Disable. To disable the high-availability monitor from the command line, use this command. If no servers are specified, the action takes place on all servers. mx exportgroup disable ([ALL_SERVERS]| [ ...
Configure the global NFS probe settings The global NFS probe periodically checks the following on each node: • The health of the NFS Server. The probe does this by issuing a NULL RPC call to the NFS Server on the local node. • The general health of the FS Option high-availability service. The probe does this by checking for critical FS Option processes. (If the node is initializing or shutting down, the processes may not be running.
The --noHeaders option does not include the column headings in the output. The --csv option prints the report in a comma-separated format. Configure Virtual NFS Services A Virtual NFS Service exports the PSFS filesystems specified in its associated Export Group. Clients access the filesystems via the hostname/IP address of the Virtual NFS Service instead of using the hostname/IP address of the physical node.
the active servers and they will not be able to absorb the workload from another server if it should fail over. Guidelines for creating Virtual NFS Services When creating Virtual NFS Services, follow these guidelines: • When planning the Virtual NFS Services needed for your cluster, first determine the network services that will be available to your clients. Then determine the IP addresses for those services. You will need to create a Virtual NFS Service for each IP address.
IP or DNS: Enter a hostname or IP address for this Virtual NFS Service. Export Group: Select the Export Group that will be exported by this Virtual NFS Service. If the Export Group has not yet been created, select NONE. Always active: If you check this box, upon server failure, the Virtual NFS Service will move to an active server even if the associated Export Group monitor is inactive or down.
is equal to or greater than the backup node where the Virtual NFS Service currently resides, the Virtual NFS Service will automatically attempt to fail over to that node. • NOFAILBACK. This policy is intended to minimize failovers. The Virtual NFS Service remains active on the backup node, even after the primary node is returned to service, until a “healthier” node becomes available, at which point the Virtual NFS Service fails over to that node.
Migrate a Virtual NFS Service The Virtual NFS Service Rehost option allows you to move a Virtual NFS Service to another node. For example, you might want to move the Virtual NFS Service from the primary node before taking that node down for maintenance. You can use this option in the following ways. From the Servers or Virtual Hosts tab Select the Virtual NFS Service on either the Servers or Virtual Hosts tab, right-click, and select Rehost.
From the Applications tab Select the Virtual NFS Service on the Applications tab, right-click, and then select Rehost. The Virtual NFS Service Rehost window shown above then appears. From the command line Issue the following command, where is the Virtual NFS Service to be rehosted. You will need to specify all network interfaces on which the Virtual NFS Service should be configured (the primary and all backups). mx vnfs move ...
Delete a Virtual NFS Service Select the Virtual NFS Service on the Management Console, right-click, and select Delete. To delete the Virtual NFS Service from the command line, use this command: mx vnfs delete ... NFS clients After FS Option is configured, your NFS clients can begin accessing the exported PSFS filesystems. Timeout configuration It is recommended that NFS clients have a minimum timeout value of 120 seconds.
Client mount options Soft and hard mounts can be characterized by how they handle errors. • Hard: When an NFS client with a hard-mounted share requests a file, it keeps trying until either it succeeds or someone interrupts its attempts. You can use the intr and nointr options to specify whether you want users to be able to interrupt an NFS request.
The following example shows an /etc/fstab entry for a hard mount on the client system: 10.0.0.1:/share /mnt/share nfs hard,intr 0 0 The next example shows an /etc/fstab entry for a soft mount on the client system: 10.0.0.1:/share /mnt/share nfs soft,timeo=7,retrans=4 0 0 If you are receiving I/O errors with a soft mount, you may want to consider either switching to a hard mount or raising your timeio and/or retrans parameters to compensate.
sync and async mount options See Considerations for ensuring data safety, page 166 for information about these mount options. Requirements for using Windows NFS clients To use a Windows client with FS Option, complete the following configuration steps. 1. Configure locking. Use one of the following methods: • On the server side, enable locking with the mxnlmconfig -e command. • Use the mount -o nolock option on the Microsoft NFS client command line. 2.
If necessary, NLM can be enabled; however, you should be aware of the following caveat: File locks granted by the NFS server are cluster-coherent. When a failover occurs, the locks are released by the original server and the client automatically reclaims them on the new server (the backup node). However, during the period after the lock is released, another client or application may compete for and win the lock.
of the NFS servers that once held locks for the client. If the power fails, these files are used by the statd process when it restarts to send an SM Notify command to each of these servers, releasing all locks previously held by this NFS client. If an NFS client is powered down while holding a lock and cannot come back up, the locks for this client will continue to be held on the server. This is expected behavior for the NFS/NLM/SM protocols.
that can move between nodes, clients are again exposed to spurious error returns. The virtualized NFSD RPC reply cache feature allows the state of the NFSD RPC reply cache to be associated with the relevant virtualized NFS server (vhost) and transferred together with the virtualized NFS server. NOTE: This feature is enabled by default on HP 4400 Scalable NAS systems. The mxfsreplycachebasename command can be used to enable or disable virtualized NFSD RPC reply caches.
• Disable Ethernet flow control. The TCP/IP protocol has its own flow-control mechanism and can better handle overrun conditions. (Check your NIC documentation to determine whether this feature can be disabled.) If you have made the NIC changes described above and suspect that your systems still have networking issues, make the following additional network changes: • TCP Segmentation Offload (TSO).
Adjust operating system parameters HP Scalable NAS includes a script called SizingActions that configures certain operating system parameters to improve system performance, particularly in a file serving environment. The changes improve network throughput and make better use system memory.
10 Manage filesystem quotas The PSFS filesystem supports disk quotas, including both hard and soft limits. After quotas are enabled on a filesystem, you can use the Quotas editor provided with the Management Console to view or set quotas for specific users and groups. Hard and soft filesystem limits The PSFS filesystem supports both hard and soft filesystem quotas for users and groups.
Enable quotas. Check or clear the box as appropriate. If you are enabling quotas, you can set the default hard limit for users and groups on that filesystem. To do this, click Limit and then specify the appropriate size in either kilobytes, megabytes, gigabytes, or terabytes. The default is rounded down to the nearest filesystem block. (If you do not want a default limit, click Unlimited.) The default quotas apply to all users and groups; however, you can change the quota for a specific user or group.
If you are enabling quotas, you can also set the default quota for users or groups on that filesystem: --set-udq [T|G|M|K] Set the default quota for users on the specified filesystem. (The default is rounded down to the nearest filesystem block.) The optional modifiers specify that the size is in terabytes (T), gigabytes (G), megabytes (M), or kilobytes (K). If a modifier is not specified, the size will be calculated in bytes.
Manage quotas This section describes the following: • The quota editor • Quota searches • Viewing or changing limits for a user or group • Adding quotas for a user or group • Removing quotas for a user or group • Exporting quota information to a file Quota editor The Management Console includes a quota editor that you can use to view quota information and to set or change the hard and soft limits for specific users or groups on a filesystem.
The default display includes columns for the name and common name of the user or group, the hard limit, the disk space currently used, and the percent of the hard limit that has been reached. NOTE: When an asterisk (*) appears in the “Hard Limit” column of the quota report, it means that the hard limit is set to the default for the filesystem. You can add columns to the display for the user or group ID and the soft limit. To do this, use View > Show IDs or View > Show Soft Limit.
information must be in a database (such as a password file or LDAP database) that can be accessed from the server where the filesystem is mounted. The search locates the name in the database and matches it with the ID, which is the value stored on the filesystem. If you are searching by ID, the search locates that ID (if it exists) in the quota information stored on the filesystem. The basic search procedure is as follows: • Enter a search pattern if desired.
Add quotas for a user or group To assign quotas to a user or group, click Add on the Quota editor toolbar and then search for the user or group in the database. The search pattern works in the same manner as on the Quotas window. When you are searching for a group, you can check the Members option, which selects all members of the group. You can then assign quotas to all of the group members. When the user or group information appears, select one or more users or groups and then click Add.
If you know the user or group ID and want to skip the search (or if the LDAP or password file is missing), click Advanced and enter the ID on the Advanced User/Group Add dialog. You can specify ranges of IDs or a list of IDs separated by commas. Then select the type of search (User or Group) and click Add.
NOTE: If the user or group has been assigned quotas on another filesystem, you can highlight the entry for that user or group on that filesystem and then select Edit > Insert to open the Add Quota dialog. When the Add Quota dialog appears, select the appropriate filesystem and set the quota limits. Any existing quota limits on the filesystem will be overwritten. Quota type: When Static is selected, you can set the hard limit for this user or group.
You will need to do another search to see the information that you added. If you added the user or group by name, do a search by name. If you added the user or group by ID, do a search by ID. Remove quotas for a user or group If a particular user or group no longer owns files on a filesystem, you can remove the quotas for that user or group. Select the user (or group) on the Quotas dialog and then click Delete on the toolbar.
For the HP Scalable NAS software-only product, these commands are provided in a separate RPM as described in the HP Scalable NAS File Serving Software installation guide. The commands are installed by default on HP Scalable NAS 4440 and HP X5500 Storage Gateway for Linux/HP Clustered Gateway systems. The administrator commands are located in the /opt/hpcfs/sbin directory and the quota command is located in the /opt/hpcfs/bin directory.
psfsrq command This command restores the quota data generated by the psfsdq command to the specified PSFS filesystem. The command has this syntax: /opt/hpcfs/sbin/psfsrq [-f ] The -f option specifies the file that psfsrq should read to obtain the quota data. If this option is not specified, psfsrq reads from stdin. filesystem is the psd or psv device used for the filesystem. Examples The following command saves the quota information for the filesystem located on device psd1p5.
11 Manage hardware snapshots HP Scalable NAS provides support for taking hardware snapshots of PSFS filesystems. The subdevices on which the filesystems are located must reside on one or more storage arrays that are supported for snapshots. Snapshot support can be configured either on the Management Console “Configure Cluster” window or with the mx config snapshot command. (See the HP Scalable NAS File Serving Software installation guide for more information.
HP EVA storage arrays To take hardware snapshots on HP StorageWorks Enterprise Virtual Array (EVA) storage arrays, the latest version of the HP StorageWorks Scripting System Utility (SSSU) must be installed on all servers in the cluster. Also, the latest version of CommandView EVA software must be installed on your Management Appliance. Be sure that your versions of SSSU and CommandView EVA are consistent. The SSSU utility must be renamed, or linked, to /usr/sbin/sssu, and must be executable by all users.
Snapshots can be both readable and writable and there can often be multiple snapshots of a filesystem existing at the same time. A snapclone is similar to a snapshot, except that it completely copies the source filesystem data at a particular point in time. When you create a snapshot, you can specify whether you want a fully-allocated snapshot, a demand-allocated snapshot, or a complete-copy snapclone. NOTE: Snapclones are not supported on Engenio storage arrays.
In the following example, the first filesystem entry is a snapclone. The second entry is a regular filesystem, and is followed by a snapshot. When creating a snapshot from the command line, you will need to determine the options available for the snapshot. Each type of array (for example, HP EVA or XP) provides its own set of snapshot options. To see the options available for the array containing the volume that you want to snapshot, run the following command.
To delete a snapshot, select the snapshot on the Management Console, right-click, and select Delete. When deleting a snapshot volume from the command line, you will need to determine which psd or psv volume contains the snapshot. The following command reports the volumes used by the existing snapshots. mx fs status ––snapshots To delete the snapshot, type the following command: mx snapshot destroy Snapclone devices, like regular filesystem LUNs, cannot be deleted from the Management Console.
Manage hardware snapshots
12 Configure security features HP Scalable NAS provides the following security features: • Role-Based Security. By default, the root account has full cluster rights and can perform all HP Scalable NAS operations. You can use the Role-Based Security feature to create roles that allow or deny other users and groups the ability to perform specific cluster operations.
belong. It then assigns cluster permissions, or rights, to you based on the roles to which your user account and groups belong. For example, if you belong to a role that allows filesystem operations and also belong to another role that allows you to configure servers, you will have both sets of permissions. A role denying an operation takes precedence over a role that allows the operation.
Add a new role To define a new role, click Add to display the Role Properties window.
Name: Type a name for the new role. Role names cannot include the forward slash character (/). Enabled: By default, the role will be enabled when it is created. To disable the role, remove the checkmark. Resource: Use this pane to specify the rights that will apply to the new role. You can allow or deny the right to create, modify, or delete the cluster resources listed on the pane. The resources are as follows. • HP Scalable NAS.
• Event Notification. Configure event notification settings. Create affects the ability to enable or disable notifiers. Modify affects the ability to configure the event notifiers (SNMP, email, script) and to select the events that will trigger notifications. Delete has no effect. • Security. Manipulate the roles and role memberships in the cluster. Create affects the ability to add or import new roles. Modify affects the ability to change existing roles or memberships.
Typically, the deny operation is used to exclude a user or group from a particular right granted to a role to which the user or group belongs. For example, a group might belong to a role that allows filesystem creation, modification, and deletion; however, you do not want certain members of the group to be able to delete filesystems. You would create a role just for those users and deny the right to delete filesystems. You can assign or deny rights for a role either manually or by using a template.
When you select a template, the rights appropriate to that role will be marked with a checkmark to allow the right or an X to deny the right. You can adjust the rights as necessary. Then go to the Members tab to assign group or user accounts to the role. Assign accounts to a role The Members tab on the Role Properties window shows the user and group accounts that belong to the role.
Click Add to assign accounts to the role. The Enter an Account dialog then asks for the user or group to be added. Enter an account to add. Type the name or ID for the user or group. Type. Specify whether you are adding a user account or a group account.
Form. Specify whether you entered a name or an ID for the account. Tips for specifying accounts When specifying accounts for a role, you should be aware of the following: • To simplify Role-Based Security administration, specify groups instead of users wherever possible. • Specify groups that are valid for all servers in the cluster. NOTE: HP Scalable NAS will not prevent you from adding users or groups that are not valid on all nodes.
the user’s account is a member of the role and there is not an entry in the Assigned Group column. Other role-based security procedures Export or import roles The import and export features can used if you will be configuring a new cluster and want to use the Role-Based Security settings that you have configured on the existing cluster. Click Export to save the current settings to the file of your choice. (The default location is your home directory.) The file is written in XML format.
you want to export both the role definitions and the membership lists, or just the role definitions. When configuring the new cluster, click Import to import the file containing the Role-Based Security settings. The imported settings will replace any current Role-Based Security settings.
Rename a role When you rename a role, accounts belonging to the role automatically become members of the new role. Roles are renamed on the Role Properties window. Select the role on the Role-Based Security Control Panel and click Edit to display the Role Properties window. Delete a role When a role is deleted from the cluster configuration, the accounts belonging to the role will automatically lose their membership in that role. Roles are deleted on the Role Properties window.
Remove roles from an account Use this command: mx account removerole --form --type ... The --form option specifies whether you are entering the name or ID of the account (NAME is the default). The --type option specifies whether the account is for a user or group or is unknown (GROUP is the default).
Configure security features
13 Configure event notifiers and view events HP Scalable NAS generates an event message when an error condition or failure occurs or when the status of the cluster changes. To provide an audit trail of cluster operations, a message is also generated when a user requests and is granted or denied authorization to perform a task. Event messages are logged and can be viewed either with the Cluster Event Viewer provided with the Management Console or with command-line tools.
View event logs The cluster log is not intended to be read directly. Instead, use the Cluster Event Viewer or the command-line tools provided with HP Scalable NAS. The Cluster Event Viewer, available on the HP Scalable NAS Management Console, shows the messages in the cluster log on a particular server. The command-line tools can display the entire cluster log for a server or only selected events.
The title bar shows the last time that the Event Viewer was updated. Click Refresh to update the display. By default, the Event Viewer shows the last 1000 messages in the cluster log. To display a different number of messages, select Viewer > Max Events to Display. Then, on the following dialog, specify the maximum number of events to display on the Event Viewer. You can save the event listing to a file by clicking Save As on the toolbar or by selecting Viewer > Save As.
View event details To view all of the information for a particular event, double-click that event on the Event Viewer. The Event Properties window shows the information. Filter the event output The Event Viewer includes three filters that can be used to limit the events that are displayed: • Search All. This filter allows you to enter text to be matched. The Event Viewer will show only those events that include the text in any of the event fields. • Severity.
View events from the command prompt HP Scalable NAS provides commands that can be used to view the cluster log on a particular server and to view outstanding alerts.
The severity values are INFO, WARNING, ERROR, CRITICAL. If you specify more than one alert level, use commas to separate the levels. The remaining options determine the format of the output. --noHeaders omits column headers, --csv prints output in comma-separated format, and --showborder displays borders in the output. Insert a message into the cluster log When necessary, you can insert your own messages into the cluster event log.
The Control Panel opens on the Event Definition tab, which lists all events defined in the HP Scalable NAS event catalog. The Event Definition tab provides a Search All filter that lists messages matching the specified term. You can also select one or more severity levels to be matched.
• Select a message row, right-click, and then set or clear that message for the appropriate notifier services. To add or remove notifier events from the command line, use these commands. If you do not specify a service, the events will be added or removed from all services. You can specify individual event IDs or a range of IDs to be added. Use commas to separate the values, for example: 100, 300-400,555.
To define a new SNMP trap forwarding target, click Add New Target. Target. Enter either the hostname or the IP address of the SNMP trap forwarding target. The trap-forwarding destination port is the IANA registered port for snmptrap (162/udp). Community. Enter the community string that is used to access the target. The default is public. Disable the SNMP trap forwarding service. This checkbox can be used to enable or disable the service as necessary.
Traps are sent as SNMP v1 traps by the server that logged the event. The trap ID is the HP Scalable NAS message ID. You can change the information for an existing target by selecting it from the SNMP Trap Forwarding Table and then clicking Edit Target. To remove a target, select it from the table and click Remove Target.
From Email address. Type the email address that will be specified as the sender of the notification emails. If this option is not included, the server name will be used as the sender. To Email address. Type the email addresses to which event notifier email should be sent. If multiple addresses will be specified, use semicolons to separate the addresses. Subject line. Select the amount of information that will appear in the Subject line of the email.
[--subject ] [--omitdesc] [--smtpport ] [--smtpuser ] [--smtppass ] Configure the script notifier service To configure the script notifier service, select the Script Notification Settings tab. This service runs a script when an event configured for the service occurs. You can specify only one script. This service is similar to the Notifiers provided in earlier releases of HP Scalable NAS. Script.
To configure the service from the command line, use this command: mx eventnotifier configurescript --script For more information about notifier scripts, see “Using custom notifier scripts” on page 246. View configurations from the command line The following command can be used to view the events configured for one or more notifier services: mx eventnotifier viewconfig [--snmp] [--email] [--script] With no options, the command displays the configured events for all of the notifier services.
To restore event settings from the command line, use this command: mx eventnotifier restoreevents [--snmp] [--email] [--script] Import or export the notifier event settings The import and export features and can used if you will be configuring a new cluster and want to use the notifier event settings that you have configured on the existing cluster. Click Export Definitions to save the current settings to the file of your choice. (The default is your home directory.
• Event details, formatted in XML, are passed to the standard input (stdin) of the script or program. Script requirements For the script to work properly, the following requirements must be met: • The script or program must be accessible from each node in the cluster. It is recommended that an identical copy of the script or program be placed on local storage on each node to ensure that it will always be available. • The script must be able to be executed on each node.
Following is an example of the input XML (as with the above, not all elements are required for each event): PAGE 24914 Configure and manage replication Overview HP Scalable NAS provides tools and processes to enable automatic replication of the data in your PSFS filesystems. During replication, data in specific filesystems or directories on the source cluster is transferred to another cluster called the destination cluster. Before using replication, you will need to create an administrative filesystem and install a matching ssh key pair on both the source and destination clusters.
The sentinel node The nodes on the source cluster that are involved with replication are called the replication group. One node from the replication group coordinates replication activities such as performing the file transfers. This node is called the sentinel node. When you configure replication, you can specify the node that you prefer to use as the sentinel. Occasionally when the replication group is formed and replication starts, a different node may actually become the sentinel node.
• Replication increases the use of physical memory and swap space. If your system is running well with 8GB of main memory and you add replication to it at near the configuration limits, there is a substantial chance that the system will be unable to service its normal workload in addition to the replication workload. • Replication must have sufficient bandwidth between sites to allow data transfers to complete in a timely fashion.
Best practices for replication To tune your system for best replication performance (and lowest load on the replicating systems), HP recommends the following: 1. 2. 3. 4. 5. 6. 7. 8. 9. 252 The replication destination filesystem must be larger than the filesystem being replicated on the source. HP recommends that the destination be 15% larger than the source. This is necessary to accommodate the temporary files that replication uses while ensuring replication is complete.
10. Reducing the number of changes that are being made to files being replicated will reduce the load on both source and destination systems. If the number of changes being made cannot be reduced or predicted (say, in a home directory environment), increasing the size of the changes will tend to make replication more efficient in its use of network bandwidth. 11. Reducing the depth of directories being replicated will improve the performance of replication on the destination nodes.
system’s physical memory. For example, for a 16GB system, this would mean setting the parameter to 800000. HP has seen some cases where a slightly higher value (6 or 7% of physical memory) further improves replication performance. • vm.Vfs_cache_pressure controls the rate at which filesystem inode structures are freed from the system and should be set to a value of 10000 when replicating large sets of files.
NOTE: Because the location of the private key is specific for replication and is specified on the command line used for rsync, any other applications using ssh should not be affected. Configure replication Replication can transmit files to an IP address for either a node in the destination cluster or an HP Scalable NAS virtual host configured on that cluster. Using a virtual host is preferred so that replication can fail over on the destination cluster, but it is not required.
6. On node 1 of the source cluster, restart HP Scalable NAS, which starts replication. /etc/init.d/pmxs start 7. Run the following command on the node to verify that replication is running. rplstatus 8. Run rplkeys -d on the remaining nodes on the source cluster to verify the ssh keys. If ssh connections to the destination cluster were previously made for the virtual host IP address, remove those entries from /root/.ssh/ known_hosts on the nodes. See Correct the known.
Offending key in /root/.ssh/known_hosts:12 RSA host key for 10.30.12.4 has changed and you have requested strict checking. Host key verification failed. To correct this problem, edit the /root/.ssh/known_hosts file, remove the offending line, and attempt the ssh again. Configure replication for a node IP on the destination cluster Complete these steps: 1. Create the administrative filesystem on both the source and destination clusters if it does not already exist.
# rplconfig -e If you specified a path for the filename, such as /tmp/rplconfig, the file will be written to the specified location. If you specified only a filename, the file will be written to the current directory. Use any text editor to modify the file. You will need to specify the replication directory entries, the snapshot interval, and the node that should be used to run the replications (the sentinel node).
NOTE: The cluster administrative traffic and the replication traffic should be on separate interfaces. During high replication load, the administrative traffic may be overwhelmed if it shares the interface with replication. • is a PSFS filesystem that will hold the replicas on the destination cluster. You can specify only one and in the file. All replications directed to the specified IP address are placed under the on the destination cluster.
#interval interval 120 #Preferred Sentinel #sentinel sentinel 99.30.35.6 #Note: In current version, only one preferred sentinel should be specified After creating the file and entering the replication information, run the following command to omport the configuration file into the mxds datastore. # rplconfig -i NOTE: An error will be reported if ssh connections to the destination cluster were previously made for the virtual host IP address. Remove those entries from /root/.
and to maintain communication among the nodes in the source cluster and synchronize replication intervals. The rplwatch process watches the PSFS filesystems and directories that are included in the replication set specified in the configuration file. (There is a separate instance of the process for each directory.) For each replication interval, rplwatch verifies that the filesystem is mounted and then creates log files noting any files that have been added, changed, or deleted during the interval.
The replication state Check the replication state Replication is always in one of the states listed below. The state information is stored in the mxds datastore. The replication state can be different for the cluster and individual nodes. For example, replication can be running on the cluster but not be running on a particular node. State Description 1 Configured. A replication configuration file has been imported and replication can be started. 2 Running. The replication program rplmonitor is running.
If replication is not running, the output will specify the following in place of the “current sentinel” line: replication is not running and the current sentinel is not defined The rplstatus command includes the following options that can be included in scripts. Only one option can be specified at a time. If an option is specified, the script runs silently without output to stdout. -status If replication is running on the node on which the script is run, the exit value is 0.
Command Current replication state New replication state start or 1 3 (waiting) 2 (running) 2 (running) 4 (stopped) 4 (stopped) 2 (running) exit or 4 2 (running) 5 (exited) Force rplmonitor to exit, which stops worker threads if they are running. 3 (waiting) enable or 1 5 (exited) 1 (configured) Force replication to terminate the ongoing replication interval and start a new interval.
Command Current replication state New replication state configchange or 3 2 (running) No state change This command is issued automatically when a new configuration is imported by the rplconfig command. 3 (waiting) 4 (stopped) The following diagram shows how the rplcontrol actions affect the replication state.
The rplcontrol commands described above can be specified by name (with the -s option) or by a number (with the -c option). The -h ip|hostname option can be used to run the command on a specific node. The remainder of this chapter shows some examples of using the rplcontrol commands. Determine when replication is complete To determine whether a particular replication cycle has completed, check for the changed files on the destination cluster.
Start replication Replication must be enabled before it can be restarted. You may need to enable replication on one or more nodes and on the cluster itself, depending on the replication state. Use the rplstatus command to determine the cluster replication status and the node replication status. • If replication is running on the cluster (cluster replication status 2), but is stopped or exited on one or more nodes, enable replication on the affected nodes. You can then restart replication on those nodes.
# rplcontrol -s exit This command sets the replication state to 5 (exited) on the cluster and on all nodes. To restart replication, see Restart replication when it is not running on the cluster, above.
NOTE: When the replication configuration file is imported, the replication service validates only that the source path exists, not that it is a PSFS directory or mount point. When replication starts, the replication service verifies that the source path is a PSFS directory or mount point. If this verification fails, an error is generated.
OR: • Use the ssh-keygen command to create the custom key pair. Then run the rplkeys -i command to import the key pair into the mxds datastore and publish it for replication use. The rplkeys -c and -i commands attempt to publish the key on the destination cluster. If the command is not successful (for example, because of ssh connection issues), you can use rplkeys -d to install the key on the destination cluster after the issues are fixed.
• Change logs. These log files contain a history of the watched files and changes to those files during a single replication interval. There is a change log for each node. At the end of a replication interval, these logs are merged into one change log that specifies all of the files that are to be transferred to the destination cluster. (If a file already exists on the destination cluster, only the changes in the file will be transferred.) • Delete logs.
Debug log Replication errors and warning messages are logged in /var/opt/hpcfs/debug/ rpl.log and also appear in the HP Scalable NAS event viewer on the Management Console. Repair damaged or deleted ssh keys The current key pair is stored in the mxds datastore. If the key pair installed on the source or destination cluster becomes damaged or is deleted, run the following command to reinstall the key pair from the mxds datastore. # rplkeys Use rplkeys -D to reset the keys to the shipped defaults.
Pause replication when network connection interruptions occur If the network connection between the source and destination clusters is interrupted, you can use the rplcontrol -s pausetransport command to stop attempts to send replication changes to the destination cluster until you are able to fix the issue. When you run the command, logging of changes to the cluster (or to the specified node) will continue but data will not be transported to the destination cluster.
-t or --verbosetransport Pass the verbose flag to the transport process created by rplwatch.
15 Cluster operations on the Applications tab The Applications tab on the Management Console shows all HP Scalable NAS applications and resources configured in the cluster and enables you to manage and monitor them from a single screen. Applications overview An application provides a way to group associated cluster resources (Virtual NFS Services and Export Groups, HP Scalable NAS virtual hosts, service monitors, and device monitors) so that they can be treated as a unit.
The Applications tab The Management Console lists applications and their associated resources (Virtual NFS Services and Export Groups, virtual hosts, service and device monitors) on the Applications tab. The applications and resources appear in the rows of the table. (Double-click on a resource to see its properties.) The servers on which the resources are configured appear in the columns. You can change the order of the server columns by dragging a column to another location.
The icons used on the Applications tab report the status of the servers, applications, and resources. The following icons are used in the server columns to indicate the status of applications and resources. Servers also use these icons when they are in a state other than Up/Okay. Up/Okay Starting Stopping Down Unknown Virtual hosts and single-active monitors use the following icons to indicate the primary and backups.
The possible states for the application are: Icon Status Meaning OK Clients can access the application. For example, an Export Group is healthy. All of its high-availability monitors are currently running and active on the associated nodes. All of the Virtual NFS Services associated with the Export Group are currently running and active on their primary interfaces. Warning Clients can access the application but not from the primary node.
Filter the applications display You can use filters to limit the information appearing on the Application tab. For example, you may want to see only a certain type of monitor, or only monitors that are down or disabled. You can use filters to do this. To add a filter, click Add Filter and then configure the filter.
Name: Specify a name for this filter. On the Type tab shown above, select the types of virtual hosts, service monitors, device monitors, and solution-specific devices that you want to see. Click the State tab to select specific states that you are interested in viewing. (The Applications tab will be updated immediately.
Click OK to close the filter. The filter then appears as a separate tab and will be available to you when you connect to any cluster. To modify an existing filter, select that filter, right-click, and select Edit Filter. To remove a filter, select the filter, right-click, and select Delete Filter.
current mouse location does not allow drops. When you reach a cell that accepts drops, the cursor will change to an arrow. The following drag and drop operations are allowed. Applications These operations are allowed only for applications that include at most only one virtual host. • Assign an application to a server. Drag the application from the Name column to the empty cell for the server. • Move an application to a new server and make that server the primary for the associated virtual host.
the virtual host will also be removed. If the virtual host was primary on the server, it will fail over to a backup server. If the virtual host was a backup, any other backups will be reordered as necessary. (The virtual host cannot be removed via drag and drop if it is configured on only one server.) Device monitors • Add a device monitor to a server. Drag the device monitor from the Name column to the empty server cell.
Enable an application on a server: Left-click on the cell corresponding to the application and server. Then right-click and select Enable. Disable an application on a server: Left-click on the cell corresponding to the application and server. Then right-click and select Disable. Servers To add a resource such as a service or device monitor to a server, left-click in the cell for that server and resource, right-click, and select Add To Resource.
• • • • • Enable or disable the Virtual NFS Service. Delete the Virtual NFS Service. Add a new Virtual NFS Service. Add a new Export Group. View all Export Groups. On the Applications tab, right-click on an Export Group to see these options: • • • • • • View properties for this Export Group, including status. Delete this Export Group. Add a new Export Group. Rehost the Export Group. Enable or disable the Export Group on all servers. View all Export Groups.
To perform these procedures on the Applications tab, left-click on the cell corresponding to the monitor and server. Then right-click and select the appropriate operation from the menu. For more information about using these procedures on service monitors, see Add or modify a service monitor, page 318 and Other configuration procedures, page 326. For more information about using these procedures on device monitors, see Add or modify a device monitor, page 332 and Other configuration procedures, page 343.
16 Performance monitoring The MxS_Perfmon*.rpm package provided with HP Clustered Filesystem provides a Performance Dashboard that displays information for the cluster, individual servers, and PSFS filesystems. (See the HP Scalable NAS File Serving Software installation guide for information about installing this package.) The Performance Dashboard provides performance data over a period of time (the last hour, day, week, month, or year). You can view data for individual servers or the entire cluster.
Performance views The Performance Dashboard provides the following metrics for the entire cluster: • Cluster Report. Reports processor, memory, network, and filesystem metrics for the entire cluster over the selected time period (the default is one hour). • Physical View. Reports hardware configuration information. • Filesystem Aggregate View. Reports total filesystem throughput and total filesystem I/O operations for selected filesystems. • Filesystem Detail View.
To use the location feature, you will need to configure the appropriate rack location for the nodes in the cluster. On each node, edit the /etc/opt/hpcfs/perfmon/ gmond_host.conf file. /* The host section describes attributes of the host, like the location */ host { location = "unspecified" } Replace unspecified with the location of the node. There are three parameters: • Rack number. This is the number of the rack where the node is mounted. • Rank number. This is the position of the node in the rack.
Cluster Report The Cluster Report includes processor, memory, network, and filesystem metrics for the entire cluster over the selected time period (last hour, day, week month, or year). Use the Hosts menu to select how you want the nodes to be identified (by local names, full names, or IP addresses, or locations). Physical View Click Physical View on the Cluster Report to see details about the configuration of the nodes in the cluster.
Verbosity level. This selection controls the amount of information that is reported for each node. Columns. If locations are configured for the hosts, this selection controls the number of columns that are used to display the racks. If locations are not configured, only one column will be displayed regardless of the value selected here. Click Full View to return to the Cluster Report. Filesystem Aggregate View Click Filesystem - Aggregate View on the Cluster Report to open the display.
Filesystem Detail View Click Filesystem - Detail View on the Filesystems View to see the metrics for a particular filesystem across the cluster and on each individual node. On the Filesystem Detail View, select the filesystem that you want to monitor. By default, the view reports filesystem metrics for the last hour. Select a different interval from the “Last” drag-down menu, or click Get Fresh Data to update the display.
Metrics View The Metrics View reports processor, memory, network, and other system metrics on each node in the cluster. Click Metrics View on the Cluster Report to open the display. Use the drag-down menus at the top of the display to select the metric to be viewed, the time interval, and the order in which the graphs should be displayed.
Hosts menu to select how you want the nodes to be identified (by local names, full names, IP addresses, or locations). Host View The Host View provides details about the configuration of the selected node and reports performance metrics for the selected time period. Use the Hosts menu to select how you want the node to be identified (by local name, full name, IP address, or location). Last. Select the time interval to be displayed. The choices are the last hour, day, week, month, or year.
Node View The Node View lists the hardware and operating system used on the node and reports when the node was last booted and how long it has been up. Use the Hosts menu to select how you want the node to be identified (by local name, full name, IP address, or location). In the following example, the node is identified by its location.
Click Physical View to go to the Physical View for the entire cluster. Host-Specific Filesystem View This view shows filesystem throughput and filesystem I/O on the node. To open the view, click Host-specific Filesystem View on the Host Report. You can select up to five filesystems to monitor. If you need to monitor more than five filesystems, open another browser window.
Performance Dashboard metrics The metrics provided with the Performance Dashboard are as follows: Metric Description bytes_in Number of bytes in per second bytes_out Number of bytes out per second cpu_aidle Percentage of time CPU was idle since system boot cpu_idle Percentage of time CPU is spending in the idle state cpu_nice Percentage of time CPU is spending executing processes with a modified scheduling priority cpu_system Percentage of time CPU is spending executing processes in kernel mode
Metric Description disk_total Total available disk space fs_ops Filesystem I/O operations per second fs_throughput Filesystem throughput load_fifteen Average system load (fifteen minutes) load_five Average system load (five minutes) load_one Average system load (one minute) mem_buffers Kernel disk block buffer size mem_cached Kernel file cache size mem_free Amount of available memory mem_shared Total size of shared memory pages nfsd thread usage The percentage of time a certain fracti
of time during which from 0% to 60% of available threads were busy; the second metric is for 60% to 70% of threads busy; and so on. When the load shifts towards the last few metrics, you should increase the number of available threads for the server. To do this, run the following command, where NUM is an integer representing the number of threads to set.
Edit Persistent Mounts. If the filesystem is not mounted on a particular server, select that server and click Activate. fsockopen error: Connection refused You may see the following error when you attempt to connect to the Dashboard: There was an error collecting perfmon data (IP:8652):fsockopen error: Connection refused Typically this error occurs because either HP Scalable NAS or mxpersrv is not running, or the administrative filesystem is not mounted.
17 Configure virtual hosts HP Scalable NAS uses virtual hosts to provide failover protection for servers and network applications. Overview A virtual host is a hostname/IP address configured on a set of network interfaces. Each interface must be located on a different server. The first network interface configured is the primary interface for the virtual host. The server providing this interface is the primary server.
The following example shows the virtual host failover mechanism. Servers A and B provide the http service, which is accessed with the virtual host www.xvz.com. Each server is configured with an http service monitor and a custom device monitor to ping the router. The failover operation to another network interface has minimal impact on clients. For example, if clients were downloading Web pages during the failover, they would receive a “transfer interrupted” message and could simply reload the Web page.
• Update the DNS name service or the /etc/hosts file with the virtual hostnames and IP addresses. (For improved performance, the Management Console caches hostname lookups. If your DNS changes, you may need to restart the console so that it will reflect the new hostname.) • Do not use ifconfig or another tool to configure the IP address in the operating system or a system startup script. HP Scalable NAS configures the operating system appropriately to support the virtual host.
Virtual Host: Type a hostname or an IP address for the virtual host. Application name: An “application” provides a way to group virtual hosts and related service and device monitors on the Applications tab. All of the HP Scalable NAS resources associated with the application can then be treated as a unit. You can specify a new application name, select an existing application name, or leave this field blank.
a virtual host. The status and enablement of each monitor contributes to the health of a server with respect to the virtual host. When a server is completely “healthy,” all of the services associated with the virtual host are up and enabled. • AUTOFAILBACK. This is the default policy. It is intended to return the virtual host to its original configuration, or as close to it as possible.
To add or update a virtual host from the command line, use the following command. The first network interface specified is the primary interface and the additional interfaces are backups. mx vhost add|update [--application ] [--policy autofailback|nofailback] [--activitytype single|always] ([ ...
Change the virtual IP address for a virtual host When you change the virtual IP address of a virtual host, you will also need to update your name server and to configure applications to recognize the new virtual IP address. The order in which you perform these tasks is dependent on your application and the requirements of your site. You can use mx commands to change the virtual IP address of a virtual host. Complete these steps: 1. At the command prompt, run the following command: mx matrix dump > mxdump.
When you make your changes and click OK, you will see a message warning that this action may cause a disruption of service. Your changes will occur when you confirm the update. To move a virtual host from the command line, use this command: mx vhost move [--policy autofailback|nofailback] [--application ] [--activitytype single|always] ([...
Virtual hosts and failover When you create a virtual host, you specify a list of network interfaces on which the virtual host can be located. The interfaces are placed in order: primary, backup #1, backup #2, and so on. The ClusterPulse process considers the “health” of the servers providing those interfaces when determining where to place a virtual host. The status and enablement of the service and device monitors associated with the virtual host also contribute to a server’s health calculation.
2. 3. 310 ClusterPulse considers the list of servers that are both up and enabled and that are configured for the virtual host. The network interface that the virtual host is associated with must also be both up and enabled for hosting. Note the following: • A server that has not finished joining the cluster (see Server access to the SAN, page 355) is not considered up for the purpose of activating the device monitor.
Customize service and device monitors for failover By default, when a service or device monitor probe fails, indicating that the watched service is down or the monitored device cannot be accessed, ClusterPulse will fail over the associated virtual host to another server where the monitored service or device is up. You can customize this behavior using the Advanced monitor settings.
selects a network interface for failover. If the events are considered, the network interface for the affected server becomes less desirable. • For service monitors, you can assign a priority to each monitor (the Service Priority setting). If ClusterPulse cannot locate an interface where all services are “up” on the underlying server, it selects an interface where the highest priority service is “up” on the underlying server.
because the servers are equally healthy. If the failed service is then restored on node 1, that node will now be healthier than node 2 and failback will occur. (Note that if the virtual host policy was AUTOFAILBACK, failback would occur when the probe failed on node 2 because both servers were equally healthy.) • After the virtual host fails over to node 2, all service monitor probes fail on that node.
Configure virtual hosts
18 Configure service monitors Service monitors are typically used to monitor a network service such as HTTP or FTP. If a service monitor indicates that a network service is not functioning properly on the primary server, HP Scalable NAS can transfer the network traffic to a backup server that also provides that network service. Overview Before creating a service monitor for a particular service, you will need to configure that service on your servers.
to move the virtual host. The virtual host may then remain active on a server with a failed monitored service. HP Scalable NAS will avoid this situation if there is a healthy server configured and available, but sometimes it may not be possible in a multiple-failure scenario. Types of service monitors HP Scalable NAS supports the following service monitors. When configuring a monitor, you will need to supply the following information.
HTTP service monitor By default, the HTTP service monitor probes TCP port 80 of the virtual host address. You can change this port number to the port number configured for your HTTP server. The default frequency for the probe is every 30 seconds; the default time that the service monitor waits for a probe to complete is five seconds. You can also supply a URL (either a directory or a full URL) that will be read as part of the probe function. The probe attempts to connect to the HTTP server and read the URL.
NOTE: Service monitors using TCP port numbers must be assigned distinct port numbers. For example, the built-in FTP service monitor and a generic TCP monitor cannot both attempt to use TCP port 21. You do not need to specify an IP address because a service monitor is associated with a virtual host. HP Scalable NAS uses that virtual host IP address in combination with the TCP port number to create network TCP connections for service monitors.
Virtual Host: The service monitor is assigned to this virtual host. If the virtual host is associated with a HP Scalable NAS application, the service monitor will also be associated with that application. Port: HP Scalable NAS supplies the default port number for the service you select. If your service uses a port other than the default, type that port number here. Monitor Type: Select the type of service that you want to monitor.
your server. Set domain to the NIS domain that should be monitored by the NIS probe. The rpcinfo command can be useful for determining the correct values for proto, rpcname, and rpcvers. • Custom service monitor. You will be asked for the location of the user probe script. Enter the pathname for the probe script to be used with the monitor. See Custom scripts, page 345 for information about writing probe scripts.
Timeout and failure severity This setting works with the virtual host policy (either AUTOFAILBACK or NOFAILBACK) to determine what happens when a probe of a monitored service fails. The default policies (AUTOFAILBACK for the virtual host and AUTORECOVERY for the monitor) cause ClusterPulse to fail over the associated virtual host to a backup network interface on another server. When the service is recovered on the original node, the virtual host fails back to that node.
where certain application-specific actions must be taken before the failback can occur. For more information on the interaction between the Timeout and Failure Severity attribute and the virtual host failback policy, see Virtual hosts and failover, page 309. To set the Timeout and Failure Severity attribute from the command line, use this option: --probeSeverity nofailover|autorecover|noautorecover Service priority The service priority is used by HP Scalable NAS when it fails over services.
Custom scripts The Scripts tab lets you customize a service monitor with Start, Stop, and Recovery scripts. You can also configure the event severity and script ordering for the service monitor. Scripts Service monitors can optionally be configured with scripts that are run at various points during cluster operation. The script types are as follows: Recovery script. Runs after a monitor probe failure is detected, in an attempt to restore the service. Start script.
when the service is already stopped, without considering this to be an error. In both of these cases, the script should exit with a zero exit status. This behavior is necessary because HP Scalable NAS runs the Start and Stop scripts to establish the desired start/stop activity, even though the service may actually have been started by something other than HP Scalable NAS before ClusterPulse was started.
Such a failure or timeout creates an event associated with the monitor on the server where the failure or timeout occurred. You can view these events on the Management Console and clear them from the Console or command line after you have fixed the problems that caused them. You can configure the failover behavior with the Event Severity attribute. There are two settings: CONSIDER. This is the default value. Events are considered when HP Scalable NAS makes failover decisions. IGNORE.
To configure script ordering from the command line, use this option: --ordering serial|parallel Other configuration procedures These procedures can be performed from the Servers, Virtual Hosts, or Applications tab on the Management Console. Locate the appropriate service monitor and then right-click to see the available procedures. Delete a service monitor Select the service monitor to be deleted, right-click, and select Delete.
Clear service monitor errors Select the service monitor where the error occurred, right-click, and select Clear Last Event. To clear a monitor error from the command line, use this command: mx service clear ...
Configure service monitors
19 Configure device monitors HP Scalable NAS provides built-in device monitors that can be used to watch disk devices or to monitor the status of PSFS filesystems. You can also create custom device monitors. Overview HP Scalable NAS provides the following types of device monitors. To configure a device monitor, you will need to specify the probe timeout and frequency and a monitor-specific value.
monitors are those monitors that are associated with the same virtual host as this device monitor.) • Multi-Active. The monitor is active simultaneously on all servers configured for the monitor. By default, DISK and SHARED_FILESYSTEM monitors are multi-active. This configuration allows the device monitor to probe a shared resource such as a SAN partition containing a PSFS filesystem. CUSTOM monitors can be either single-active or multi-active, depending on the service or device being monitored.
Custom device monitor A CUSTOM device monitor can be particularly useful when integrating HP Scalable NAS with a custom application. Custom monitors can be configured to be either single-active or multi-active. You must supply the probe script for the custom monitor. In the script, probe commands should determine the health of the device as necessary. If the device is operating normally, the probe script should exit with exit status zero.
was shut down on the server, the server failed to schedule a cluster group communication process for an extended period of time). 3. 4. 5. 6. If the device monitor is multi-active, it will be active on all servers passing evaluation for steps 1 and 2. If the device monitor is single-active, then the list of servers is filtered by the following policy items to determine the one server where the device monitor will be active.
Device name: Type the name of the device monitor. You can use up to 32 alphanumeric characters. Application name: Specify the name of the HP Scalable NAS application to be associated with this device monitor. HP Scalable NAS applications are used to group related virtual hosts, service monitors, and device monitors on the Applications tab. If you leave this field blank, HP Scalable NAS will use the name of the device monitor as the application name.
you do not enter a filename, HP Scalable NAS will create a file that will be used by the monitor probe. The file will be named MxS//, where is the dotted decimal IP address of the hostname for the server, and is the name assigned to the SHARED_FILESYSTEM device monitor. The following example shows a monitor created on the server r18. To add a device monitor from the command line, use this command: mx device add --type --servers ,,..
Probe severity The Probe Severity tab lets you specify the failover behavior of the device monitor. The Probe Severity setting works with the virtual host policy (either AUTOFAILBACK or NOFAILBACK) to determine what happens when a monitored device fails. The default policies (AUTOFAILBACK for the virtual host and AUTORECOVERY for the device monitor) cause ClusterPulse to fail over the associated virtual hosts to a backup network interface on another server when the monitor probe fails.
NOAUTORECOVER. The virtual host fails over when a monitor probe fails and the monitor is disabled on the original node, preventing automatic failback. When the monitor is reenabled, failback occurs according to the virtual host’s failback policy. This option is useful when integrating HP Scalable NAS with a custom application where certain application-specific actions must be taken before failback can occur.
When a monitor is instantiated for a device (because the ClusterPulse daemon is starting or the configuration has changed), HP Scalable NAS selects the best server to make the device active. The Start script is run on this server. The Stop script is run on all other servers configured for the monitor to ensure that the device is not active on those servers. Start scripts must be robust enough to run when the device is already started, without considering this to be an error.
If you want to reverse this order, preface the Stop script with the prefix [post] on the Scripts tab. Event severity By default, HP Scalable NAS treats the failure or timeout of a Start or Stop script as a failure of the associated monitored device and may initiate failover of the associated virtual hosts. Configuration errors can also cause this behavior. Such a failure or timeout creates an event associated with the monitor on the server where the failure or timeout occurred.
The SERIAL setting considers events and takes precedence over the setting for Event Severity. PARALLEL. HP Scalable NAS does not enforce the strict ordering sequence for Stop and Start scripts. The scripts run in parallel across the cluster as a shared device or virtual host is in transition. The PARALLEL configuration can speed up failover time for services and devices that do not depend on strict ordering of Start and Stop scripts.
on the servers used with the device monitor are dependent on the device monitor. However, you can specify that only certain virtual hosts be dependent on the device monitor. For example, you might have a DISK monitor for a disk containing Web and FTP files. If the disk fails, you want HP Scalable NAS to fail over the virtual hosts for these services. The server might also provide mail service; however, the virtual host for this service is not dependent on the disk and should not fail over if the disk fails.
Activity Type. Where the monitor can be active. The options are: • Single-Active. The monitor is active on only one of the selected servers. Upon server failure, the monitor will fail over to an active server unless all associated service and device monitors are down. (“Associated” service and device monitors are those monitors that are associated with the same virtual host as this device monitor.) • Single-Always-Active. The monitor is active on only one of the selected servers.
Set a global event delay A device monitor that is configured to be multi-active or to probe on multiple servers can experience a global event, in which the shared resource being monitored is reported to be down on all servers. When the shared resource becomes active again, the monitor probe on each server will report that the resource is up. Typically a monitor instance on one server will report that the resource is up before the instances on all of the other servers have completed their probe operations.
To enable a Global Event Delay from the command line, use this command: mx device globalsettings [--eventdelay ] If you do not specify the --eventdelay option, the command will return the current value of the Global Event Delay. Other configuration procedures These procedures can be performed from the Servers or Applications tab on the Management Console. Locate the appropriate device monitor and then right-click to see the available procedures.
Configure device monitors
20 Advanced monitor topics The topics described here provide technical details about HP Scalable NAS monitor operations. This information is not required to use HP Scalable NAS in typical configurations; however, it may be useful if you want to design custom scripts and monitors, to integrate HP Scalable NAS with custom applications, or to diagnose complex configuration problems.
Types of custom scripts You can develop the following types of custom scripts for use with service and device monitors. Probe scripts Probe scripts run periodically and check the health of a service or device. You will need to determine what the probe should look at and how it should exit. For example, you might want to create a custom service monitor for an application that creates a pid file when it is running.
NAS will then take the action configured for the service monitor, which is typically to fail over the virtual host associated with the monitor. When you create the custom service or device monitor for the probe script, you can set both the frequency at which the probe script should be executed and the timeout period, which is the maximum amount of time that the monitor_agent daemon will wait for the probe to complete. You can create more elaborate probe scripts as necessary.
• Ensure the availability of non-shared resources. For example, a Start script can start an auxiliary process needed by the monitored application if it is not already running. • Perform cleanup tasks such as killing any unreaped children of a failed application process. Stop scripts can be used for this purpose. In some cases the monitored service or device is actually started by something other than HP Scalable NAS before ClusterPulse is started.
The port or name of the service monitor. (Applies only to service monitors.) MX_NAME=name The name of the device monitor. (Applies only to device monitors.) HP Scalable NAS does not set any other variables. If a script requires a variable such as a pathname, the script will need to set it. The effect of monitors on virtual host failover Typically a virtual host has a primary network interface and one or more backup network interfaces.
When a failure occurs on the Primary, the virtual host needs to fail over to a backup. HP Scalable NAS now looks for the best location for the virtual host. Because the probe status on the first backup is “down,” HP Scalable NAS selects the second backup, where the probe status is “up.” At i5 in the following example, the probe fails on the Primary. At i6, the virtual host is deconfigured on the Primary. At i7, the monitor stop script begins on the Primary.
Custom device monitors A custom device monitor is associated with a list of servers and a list of virtual hosts configured on those servers. A custom device monitor can be active on only one server at a time. On each server, the monitor uses a probe mechanism to determine whether the service is active. The probe mechanism is in one of the following states on each server: Up, Down, Unknown, Timeout. A custom device monitor also has an activity status on each server.
autorecovery, priority, and serial script ordering and has Start and Stop scripts defined. The example begins at initial startup from an unknown state and the virtual host is then located on the primary interface. At time t1, the service monitor probe on the Primary reports a Down status. The virtual host then becomes inactive all servers. The virtual host cannot be active on the Primary server because the custom service monitor is Down.
Integrate custom applications There are many ways to integrate custom applications with HP Scalable NAS: • Use service monitors or device monitors to monitor the application • Use a predefined monitor or your own user-defined monitor • Use Start, Stop, and Recovery scripts Following are some examples of these strategies.
script that connects to the port and then tests how the script responds to various commands. NOTE: The user-defined monitor dialog prompts you for a service monitor name and not a port because you may be writing a monitor for an application that does not provide network services and therefore needs no port. A sample custom monitor This example uses service monitors with a custom application called myservice. This application provides some facilities to clients who connect to port 2468 and speak a protocol.
21 SAN maintenance The following information and procedures apply to SANs used with HP Scalable NAS. Server access to the SAN When a server is either added to the cluster or rebooted, HP Scalable NAS needs to take some administrative actions to make the server a full member of the cluster with access to the shared filesystems on the SAN. During this time, the Management Console reports the message “Joining cluster” for the server.
Membership partitions HP Scalable NAS uses a set of membership partitions to control access to the SAN and to store the mxds datastore and the device naming database. (The mxds datastore contains cluster configuration information and the device naming database includes the global device names for SAN disks imported into the cluster.) Typically the membership partitions are created when you install HP Scalable NAS. You can add, replace, or repair membership partitions as necessary.
Any of these messages can appear in the “SANlock State” column. held by SDMP administrator The SANlock was most recently held by the SDMP administrator of the cluster to which the host where mxsanlk was run belongs. trying to lock, last held by host X.X.X.X The SANlock was most recently held by host X.X.X.X and may still be held by that host. The host on which mxsanlk was run is trying to acquire the SANlock. cannot access The host on which mxsanlk was run is unable to access the SANlock.
lock is corrupt, will repair This transitional state occurs after the SDMP has detected that the SANlock has been corrupted but before it has repaired the SANlock. trying to lock (lock is corrupt, will repair) The host on which mxsanlk was run is trying to acquire the SANlock. The SANlock was corrupted but will be repaired. locked (lock is corrupt, will repair) The host on which mxsanlk was run holds the lock. The SANlock was corrupted but will be repaired.
• The mprepair command can be used to display information about membership partitions and to perform all operations affecting membership partitions. HP Scalable NAS must be offline on all nodes when this command is used. Online operations When HP Scalable NAS is running, the Add, Repair, and Replace options on the Storage Settings tab and the mx config mp set and repair commands can be used only in the following circumstances: • A disk containing a membership partition is out-of-service.
Membership partition states The Storage Settings tab reports the state of each membership partition. The possible states are: • OK. The membership partition is functioning correctly. • FENCED. The server has been fenced and cannot access the SAN. Start HP Scalable NAS if it is not running or reboot the server. • NOT_FOUND. HP Scalable NAS cannot find the device containing the membership partition. Check the device for hardware problems.
• CORRUPT. The membership partition is not valid. Resilver the partition. • CID_MISMATCH. The Cluster-ID is out-of-sync among the membership partitions and must be reset. Repair a membership partition When HP Scalable NAS is started on a server, it checks the status of the membership partitions and reports any invalid partitions. If a single membership partition is found to be corrupt, you can repair that partition while HP Scalable NAS is running.
When you select a partition and click Replace, you will see a confirmation message describing the replace operation. A message also appears when the replace operation is complete. NOTE: The Replace option on the Storage Settings tab is available only when HP Scalable NAS is running. Add membership partitions When you initially configure the cluster, you can add either one or three membership partitions, as described in the HP Scalable NAS File Serving Software installation guide.
All of the available partitions on that disk or LUN then appear in the bottom of the window. Select one of these partitions and click Add. (The minimum size for a membership partition is 1 GB.) Repeat this procedure to select one more membership partition. HP recommends that the partitions be on different disks. When selecting partitions for use as membership partitions, be sure that they do not contain any needed data. When the membership partitions are created, any existing data will be erased.
Determine disks/partitions available for membership partitions To determine the disks that can be used for membership partitions, run this command: mx config mp list_avail_disks [--noHeaders] [--csv] [--showBorder] Next, run the following command to see a list of the available partitions on one of the available disks.
Replace a membership partition To replace a membership partition, use this command: mx config mp set Specify the UUIDs for the new membership partition and for the two existing membership partitions that will be retained. The mprepair utility The /opt/hpcfs/lib/mprepair utility is invoked from the command line. IMPORTANT: HP Scalable NAS cannot be running when you use mprepair. To stop the cluster, issue the command # /etc/init.d/pmxs stop on each node.
This command lists the current membership partitions according to the membership file maintained on the server where you are running the utility. Each server in the cluster has a membership partition file, which is called the “local MP list.” Each SAN disk containing a membership partition also has its own list of the membership partitions. Under normal operations, these lists should all match. The output from --get_current_mps contains a record for each membership partition. Following is a sample record.
CID_MISMATCH. The Cluster-ID is out-of-sync among the membership partitions and must be reset. See “Reset the Cluster-ID” in the section mprepair options, page 367. Active and inactive membership partitions A membership partition can be either active or inactive. (This attribute is reported in the last field of the record displayed by the mprepair --get_current_mps command.) The current membership partitions should all be active.
Display membership partitions. To display information about membership partitions, type the following command: mprepair --display_mplists The output shows the local membership partition list on the server where you are running mprepair. It then compares this list with the lists located on the disks containing the membership partitions. The output also includes the device database records for the disks containing the membership partitions. Following is an example.
is the UUID for the device and is the number of the partition on the device. NOTE: If you resilver from a partition that has a status of RESILVER, the operation may initialize partitions that are not currently membership partitions; any existing data on those partitions will be overwritten. Use the --display_mplists option to see the membership partition lists for the current membership partitions.
unlikely; however, if HP Scalable NAS cannot be started on any server in the cluster, you can use the following command to determine whether all membership partitions have a valid Cluster-ID. mprepair --sync-clusterids The command displays the Cluster-IDs found in each membership partition and flags those partitions containing an invalid ID. You can then specify whether you want the command to repair the partitions having a mismatched Cluster-ID.
value is too large, more time may be needed for filesystem recovery after a node failure. The following example sets the timeout to 10,000ms (10 seconds): sanpulse_start_options = { "--mxinit","-o sdmp_io_timeout=10000" }; After adding the parameter, you can restart HP Scalable NAS. A setting of 10,000ms will be adequate for many storage arrays. If you are using DDN storage, you may need to increase the parameter beyond 10,000ms.
7. Verify the fencing configuration as described in the HP Scalable NAS File Serving Software installation guide. Server cannot be located If the cluster reports that it cannot locate a server on the SAN but you know that the server is connected, there may be an FC switch problem. For example, on a Brocade FC switch, log into the switch and verify that all F-Port and L-Port IDs specified in switchshow also appear in the local nameserver, nsshow. If the lists of ports are different, reboot the switch.
this utility could result in filesystem corruption. Do you wish to continue? y SUCCESS 99.10.20.4 has been marked as down. CAUTION: Be sure to verify that the server is physically down before running the mx server markdown command. Filesystem corruption can occur is the server is not actually down. If mx server markdown is run for a server that is up and in communication with the cluster, the following message will be displayed: Do you wish to continue? y ERROR 99.10.20.
the Management Console or mx utility, and can be used normally from that point forward. If OLI is not possible with your hardware combination, you will need to restart HP Scalable NAS after inserting a new disk. The disk will then be visible to the cluster. Changing the number of LUNs can cause cluster failure HP Scalable NAS uses worldwide names and LUN numbers to associate PSD device names with SAN devices.
Membership partition timeout The membership partition timeout should be increased to 120 seconds (120000ms). The timeout is specified in the mxinit.conf file, which is located in the /etc/ opt/hpcfs directory. Locate the following line in the file: # sanpulse_start_options = { “--mxinit” }; Change the line as follows: # sanpulse_start_options = { “mxinit”, “-o sdmp_io_timeout=120000” }; PSFS/psd timeout This timeout affects the psd devices that make up the abstracted block layer to PSFS filesystems.
Host Bus Adapters (HBAs) Reduce the HBA queue depth The HBA queue depth is the maximum number of outstanding I/O requests that the HBA can hold while it is waiting for responses from the LUNs on the storage array.
Change the HBA queue depth To change the HBA queue depth, you will need to edit the /etc/opt/hpcfs/ fc_pcitable file. In the file, locate the line for your HBA driver. The following examples are for QLogic 2300 and Emulex 9000 drivers. #0x1077 0x2300 qla2300 scsi QLogic 2300 Adapter w/v6.01 driver #0x10df 0xf900 lpfcdd scsi Emulex Lightpulse 9000 Fibre Channel Adapter Edit the line as follows: • Remove the comment character (#) from the beginning of the line.
3. Shut down the server: # init 0 4. Swap out the HBA card. 5. Power up the server. 6. Configure HP Scalable NAS's default HBA driver version for the hardware installed on your system: # /opt/hpcfs/lib/chhbadriver default 7. Import the cluster configuration to the server: # mxconfig -import 8. Enable the HP Scalable NAS startup script. Run this command: # /sbin/chkconfig --add pmxs 9. Start HP Scalable NAS: # /etc/init.
6. Enable the HP Scalable NAS startup script. Run this command: # /sbin/chkconfig --add pmxs 7. Start HP Scalable NAS: # /etc/init.d/pmxs start NOTE: The /etc/hba.conf file must point to the correct hbaapi library. Install a driver not provided with HP Scalable NAS If your system configuration requires that you use an HBA driver version not provided with HP Scalable NAS, you can install that driver version on the system.
5. Update the fc_pcitable file with information about the driver that you installed. The beginning of the file describes the syntax of the entries in the file. Be sure to review this information. The end of the file contains uncommented lines that specify the hardware installed on your system. Following is an example: # Adapters found on this system: 0x1077 0x2300 qla2300 scsi/qla2x00-6.06.10 "" QLogic 2300 Fibre Channel Adapter 6. Make sure that the third field is the name of the driver (without the .
0x1077 0x2200 qla2x00 scsi "" QLogic 2200 Adapter The fields contain the following information: • • • • Vendor ID. Device ID. Driver name. Do not include .o in the name. Path. If the path begins with “/”, it is considered to be an absolute path. Otherwise, it is considered to be relative to the /opt/hpcfs/lib/modules/current directory. • Options, enclosed in double quotes, to pass to insmod when it loads the driver. If no options are required, type a pair of double quotes ("") in the field.
Fibre Channel switches Changes to switch modules Changing the hardware configuration, such as adding or replacing switch modules, is not supported while HP Scalable NAS is running. This restriction applies only to Fibre Channel switches that are under the control of HP Scalable NAS. Add a new Fibre Channel switch If you are using Fibre Channel switch-based fencing and need to add a new FC switch, complete these steps: 1. Shut down HP Scalable NAS on all nodes. # /etc/init.d/pmxs stop 2.
If these conditions are not met, you will not be able to perform online replacement of the switch. Instead, you will need to stop the cluster, replace the switch, and use mxconfig to reconfigure the new switch into the cluster. Consult your switch documentation for the appropriate replacement procedure, keeping in mind that the above requirements must be met. However, if this documentation is not available, the following procedures describe a method to replace a switch.
13. Connect the FC connectors to the new switch. Be sure to plug them into the same ports as on the original switch. 14. Set the Ethernet IP address on the new switch to the IP address of the original switch. Use the ipAddrSet command. If you had connected to the switch over the Ethernet interface, the session will be disconnected and you will need to log back into the switch with the IP address you just configured. 15. Enable the switch using the switchEnable command.
2. If possible, save configuration information from the original switch. Some items such as the zone configuration are not needed and are just insurance against further failures. Other items such as the IP address are available elsewhere but might conveniently be captured here. One way to record the information is to capture the output of a CLI session. The following commands show types of data that might be useful: • show ip ethernet for the IP address. • show switch for the fabric operating mode.
SAN maintenance
22 Other cluster maintenance This chapter describes how to perform the following activities: • Collect HP Scalable NAS log files with mxcollect • • • • Check the server configuration Disable a server for maintenance Troubleshoot a cluster Troubleshoot service and device monitors Collect log files with mxcollect The mxcollect utility collects error event logs that can be useful for diagnosing technical issues with HP Scalable NAS.
Following is a sample script that lets you specify the nodes on which mxcollect should be run. The script names the zip files with the node names specified on the command line. Example log collection script 'logs.sh': #! /bin/sh -x for i in $* do curl -ksu : https://$i:6771/cgi-bin/pmxs/mxcollect -o $i.zip & done wait for i in $* do unzip $i.zip & done wait Usage: logs.sh ...
• Network check: IP network and interface assignments, forward and reverse hostname lookup. • Storage check: Host Bus Adapters, drivers, and settings. • Miscellaneous check: other checks. Output from the utility appears on the screen and is also written to the Application Log section of the Event Viewer.
A running service is considered down If HP Scalable NAS indicates that a service is down even though it is running, the service might not be sending any information to HP Scalable NAS. HP Scalable NAS will conclude that the service is down even if the TCP connection succeeds. In particular, if an HTTP server is monitored by a FTP monitor, the HTTP server is considered down. Also check the following: 1. Verify that the server is connected to the network. 2.
Troubleshoot monitor problems You may encounter the following problems with service and device monitors. Monitor status If the monitor status is not reported as Up, check the last error message string and the last event message string that monitor_agent returned to HP Scalable NAS for any service or device monitor on any server in the cluster. The error or event message provides more status information.
could occur if the Management Console is out of date and does not support the version of HP Scalable NAS running on the server. “Event” status The “Event” status is displayed when monitor_agent encounters an error while executing the probe, Start, Stop, or Recovery scripts. The status of the monitor may be “Up” even though an event has been reported. This is the case if the error was not serious enough to stop the execution of the script. The events are: CONFIG_ERROR.
Clear an error After you have determined the cause for a script event, be sure to correct the script on all servers that reported the event and then clear the error. On the Management Console, select the monitor, right-click, and select Clear Last Error. To clear an error from the command line, use these commands: mx service clear ... mx device clear ...
Typically, the status is Active on all appropriate servers, as it is important to continue to probe the service on backup servers as well as on the primary servers. “Activity Unknown” status For a brief period while the monitor_agent daemon checks the monitor script configuration and creates a thread to serve the monitor, the activity may be displayed as “activity unknown.
Monitor recovery If a Recovery script is configured, a probe failure will not immediately cause the monitor to become “Down.” The Recovery script is attempted only once after the probe fails. However, when the monitor again reaches the “Up” status, the Recovery script is again enabled to run if the probe fails again later. A Recovery script is a valuable tool if you expect the service or device to fail periodically and you do not want to take the failover action for a single probe failure.
Other cluster maintenance
A HP Scalable NAS Management Console icons The HP Scalable NAS Management Console uses the following icons. HP Scalable NAS entities The following icons represent the HP Scalable NAS entities. If an entity is disabled, the color of the icon becomes less intense. Additional icons are added to the entity icon to indicate the status of the entity. The following example shows the status icons for the server entity. The status icons are the same for all entities and have the following meanings.
Monitor probe status The following icons indicate the status of service monitor and device monitor probes. If the monitor is disabled, the color of the icons is less intense. On the Applications tab, virtual hosts and single-active monitors use the following icons to indicate the primary and backups. Multi-active monitors use the same icons but do not include the primary or backup indication.
B Samba configuration This information describes how to configure Samba and HP Scalable NAS to provide failover support for virtual Samba servers. The configuration steps in this appendix apply only if you want to create a failover Samba configuration. If you do not need a failover configuration, you can configure Samba using traditional methods such as via the /etc/samba/smb.conf file.
When using Samba with HP Scalable NAS, you will need to use a active/passive configuration, with one node hosting the share and other nodes serving as backups. The data, or shares, to be exported are stored on PSFS filesystems. Clients access a share via a virtual Samba server. The virtual server is controlled by an HP Scalable NAS virtual host, which is configured with one node as the primary and one or more additional nodes as backups.
When a virtual server is started on a node via the solution, the Samba configuration file smb.conf is modified to include the virtual server. As an example, we will start the virtual server “cifs1” on a cluster node. Before the virtual server is started, the smb.
• Samba configuration files. You will need to configure two files: /etc/samba/ smb.default and smb.conf.. The /etc/samba/ smb.default file is used on servers that are not currently hosting a virtual Samba server and contains Samba configuration parameters that are not specific to a Samba share. smb.conf. is the configuration file for a particular virtual Samba server. • HP Scalable NAS virtual host.
Upgrade Samba HP recommends that you upgrade your servers to the Samba 3.0.30 release. Testing has shown that this release provides a better environment for the Samba solution than the Samba release included with the operating system distributions. NOTE: Samba releases later than 3.0.32 are not supported with HP Scalable NAS. Considerations for deploying Samba with HP Scalable NAS Be sure to review the following considerations before configuring Samba.
• The Samba options described in this appendix have been tested by HP. Samba has many other configuration options; those options have not been thoroughly tested by HP in the clustered Samba environment. If they are suited to your Samba environment, they can be used; however, if you use configuration parameters other than those included in this appendix, HP recommends that you test your configuration carefully before deploying it in a production environment.
Edit Samba configuration files Samba is configured via the smb.conf file. When Samba is used with HP Scalable NAS, you will need to configure two versions of the file: • /etc/samba/smb.default. This file is used on servers that are not currently hosting a virtual Samba server. For example, a node is being used as a standby node for Samba or the virtual host has been shut down for some reason. The file is copied over any smb.conf file in the /etc/samba directory. • /etc/samba/smb.conf..
local master = yes preferred master = yes browseable = yes wins support = no The parameters are: netbios name. The %h returns the Internet name of the server from NIS, DNS, or /etc/hosts. workgroup. The name of the Samba workgroup (the default is WORKGROUP). smb ports. Required for high availability; do not change or remove. Used by %L expansion. encrypt passwords. Optional. Specifies plaintext or encrypted passwords. Most Windows operating systems require encryption. socket options.
null passwords = yes [data1] comment = /mnt/data1 directory on %h path = /mnt/data1 guest ok = yes writeable = yes The parameters included in the file are: interfaces. Required for Samba high availability. Specify the actual network IP address of the Virtual Samba server that is to be used to access this Samba share. The network address must include the IP address and the netmask. The netmask can be specified in either the full dot format or CIDR format, as in the sample file.
smb.conf. files, and the smbpasswd file to all nodes in the cluster. The files are placed in the /etc/samba directory. Although only a specific file will be used on each node, placing all of the files on each node ensures that the proper file will always be available. When you need to modify the Samba configuration, you can rerun the smbcfg_dist utility. Before running the utility, ensure that your cluster has been configured and that all nodes are up and active.
smb.conf.virtual-server smb.default smbpasswd smbusers 100% 100% 100% 100% 804 1205 392 389 0.8KB/s 1.2KB/s 0.4KB/s 0.4KB/s 00:00 00:00 00:00 00:00 ...... Operations completed. Configure HP Scalable NAS to provide HA for Samba When the configuration files are in place, you can create a virtual host and a CUSTOM service monitor for each share. The following example shows this process. NOTE: Samba shares must be mounted when setting up the CUSTOM service monitor.
Now create the CUSTOM service monitor. On the Management Console, select the virtual host that you just created (99.12.10.205), right-click, and select Add Service Monitor. Name: Give the service monitor the same name as the virtual Samba server (cifs1 in our example).
Monitor type: Select CUSTOM. Timeout: Specify 120 seconds. Frequency: Specify 60 seconds. User probe script: Specify the /opt/hpcfs/methods/smb_meth script followed by the name of the virtual Samba server. For example: /opt/hpcfs/methods/smb_meth cifs1 After completing your entries on the Add Service Monitor window, click Advanced and then go to the Scripts tab on the Advanced Service Configuration window. You will need to specify Start and Stop scripts for the monitor.
Windows clients can access the newly created share via the virtual Samba server name (\\cifs1\data1 in our example). Clients must be able to resolve the virtual Samba server name (cifs1) via DNS or a local hosts file. The Samba share cannot be accessed via the IP address; it must be accessed via the CIFS virtual server name. Log files for Samba service monitor operations HP Scalable NAS generates log files for Samba service monitor operations.
Integrate Samba into ADS Using the Samba solution Samba has been designed to integrate with Windows domains using several models. The model chosen depends on the problem the customer is trying to solve.
and groups. These ranges are configurable. There are several winbind models as well. The default and most common involves assigning uids and gids in the order they are read from the domain. Another common model uses the RID portion of the Windows SID along with a formula to determine the uid or gid values. Permissions to Linux resources are granted through the group memberships defined in ADS using the IDs assigned through winbind. Winbind users cannot be mixed with Linux users in common groups.
Changes for ADS level security Set these parameters: • • • • Workgroup = [short Domain Name] security = ads password server = [IP of the KDC, then other DCs] realm = [WINDOWS DOMAIN] See the Samba Official HOWTO for more information about using net ads join to join Samba to the domain using ADS level security. Use net ads testjoin to test the join after the join has completed successfully. NOTE: All VHOSTs need to be kept active and running even if they are not serving any virtual servers.
NOTE: Ensure that a name service caching utility such as nscd is not running.
C Initial configuration for HP 4400 Scalable NAS systems The HP 4400 Scalable NAS system is shipped with the cluster already configured. Networks HP Scalable NAS uses a local network for its administrative traffic. This network has been configured with IP addresses in the 172.31.0.n network using VLAN #2 on the ProCurve 2910 network switch. VLAN #2 includes the first eight ports on the switch. This private network must not be connected to your network.
Command View on ABM Command View runs on the Array Based Management Module (ABM), which is part of the EVA 4400 controller. The ABM is configured at the factory with IP address 172.31.0.20 and can be accessed from a browser or one of the cluster servers at: htttps://172.31.0.20:2372/ Login as Administrator and use hpinvent as the password. For convenience, the ABM can be transferred from the private VLAN2 to public VLAN1. To do this, complete the following steps: 1. 2. 3.
NOTE: If you will be using PSFS filesystems for Oracle, see Appendix B in the HP Scalable NAS File Serving Software provisioning guide for Oracle for configuration information, including the necessary mount options. Administrative filesystem The HP Scalable NAS administrative filesystem has been created on a 50GB vdisk that is mounted on /_adminfs. For more information about the administrative filesystem, see Create the administrative filesystem, page 139.
Membership partitions can be added, replaced, or repaired as necessary. For more information, see Manage membership partitions, page 358. Fencing When certain problems occur on a server (for example, hardware problems or the loss of cluster network communications), and the server ceases to effectively coordinate and communicate with other servers in the cluster, HP Scalable NAS must remove the server’s access to filesystems to preserve data integrity. This step is called fencing.
/etc/opt/hpcfs/fc_pcitable file The following options were added to the driver load line for the QLogic HBA driver runtime execution: "ql2xmaxqdepth=0x80 qlport_down_retry=10 ql2xloginretrycount=30 ql2xfailover=0 ql2xlbType=0 displayConfig=1 ql2xautorestore=0x20 ql2xqfullrampup=3" The options are: ql2xmaxqdepth Maximum queue depth to report for target devices. qlport_down_retry Maximum number of command retries to a port that returns a PORT-DOWN status.
no_path_retry When a numeric value is entered for this attribute, it specifies the number of times the system should attempt to use a failed path before disabling queueing. "n=XX" indicates the number of retries until queuing is disabled. "fail" indicates immediate failure. "queue" indicates never stop queuing. The value was changed from 12 to fail.
node. For more information about this feature, see Virtualized NFSD RPC reply caches, page 197. NLM locking protocol The NLM locking protocol is enabled by default. For more information about this protocol, see Using the NLM protocol, page 195 NOTE: If Oracle over NFS is used, be sure to use the default setting for the NLM locking protocol. This setting is valid for Oracle when accessing filesystems from within the cluster or over NFS.
• Pentium-compatible processor or later. • 64 MB of memory. • SuSE Linux Enterprise Server Version 7, 8, 9, or 10; Red Hat Linux 7.2 or 7.3 (Server or Workstation installation); Red Hat Advanced Server 2.1; Red Hat Enterprise Linux AS/ES 2.1; Red Hat Enterprise Linux AS/ES 3.0; Red Hat Enterprise Linux AS/ES 4.0, Red Hat Enterprise Linux 5. • A windowing environment must be installed and configured.
Index A accounts assign to role, 225 administrative filesystem, 139 enlarge, 142 mount new nodes, 142 administrative network allow, discourage or exclude traffic, 82 defined, 30 failover, 81 network topology, 80 requirements for, 79 select, 79 alerts Alerts pane on Management Console, 55 icons shown on Management Console, 398 Applications tab drag and drop operations, 281 filter applications, 279 format, 279 icons, 277 manage application monitors, 285 manage applications, 281 menu operations, 283 modify dis
C CDSL (context dependent symbolic link), 155 checklist, quick start, 23 cluster features supported on server, 76 software requirements, 43 cluster configuration back up, 61 iSCSI, 40 limits, 43 SAN restrictions, 42 server recommendations, 42 cluster configurations Fibre Channel, 35 cluster log insert messages, 238 view with mx server command, 237 Cluster-ID, reset, 369 ClusterPulse defined, 31 failover, 309 command-line administration, 50 configuration virtual host , 303 back up, 61 device monitor, 332 fil
disks, SAN access, 86 deport, 89 display disk information, 92 display local disk names, 90 import, 87 online insertion, 373 repartition, 90 storage summary, 91 DLM (Distributed Lock Manager) defined, 31 filesystem synchronization, 123 documentation providing feedback, 21 dynamic volumes concatenated, 104 configuration limits, 44 convert from basic, 114 create, 105 defined; , 103 delete, 112 deport, 116 display unimported volumes, 95 extend, 110 guidelines, 105 import, 117 list subdevices with sandiskinfo, 9
Export Group Virtual NFS Service, associate with, 171 add, 166 advanced options for monitor event severity, 180 probe configuration, 176 probe severity, 177 script ordering, 181 scripts, 179 caveats, 165 clear monitor errors, 184 data safety and mount options, 166 defined, 161 delete, 184 disable, 184 enable, 184 filesystem mount options, 165 high availability monitor, 164 overview, 163 spanning Virtual NFS Services, create, 173 status, 182 view properties, 182 Export Group monitor probe actions; , 175 view
filesystem, PSFS access, 121 atime updates, 151 check with psfsck, 149 context dependent symbolic link (CDSL), 155 crash recovery, 123 create, 125 create with mkpsfs command, 130 create with mx fs command, 129 destroy, 155 extend, 146 features, 121 features, configured, 147 file locking, 158 journal, 122 mount, 132 mount errors on server, 149 mount information, 149 mount options on Management Console, 132 mount with Linux mount command, 138 mount with mx fs command, 137 persistent mounts, 143 properties, 14
I init.
mxds datastore backup and restore, 63 mxfsreplycachebasename command, 197 mxinit utility process monitoring, 58 start or stop software, 60 mxlogd daemon, 31 mxmpio command, 96 mxnlmconfig command, 195 NFS tuning client mount options, 193 NFS version, supported, 163 NFSD RPC reply caches, virtualized enable or disable, 197 NLM locking protocol, 195 NOFAILBACK, virtual host, 312 notifiers, See event notifier services, 233 O operating system parameters, adjust with SizingActions, 200 N network interface add
PSFS filesystem.
role-based security roles modify, 229 account commands, 230 Control Panel, 220 overview, 219 resources, 222 rights, allow or deny, 223 roles add, 221 assign accounts, 225 assign rights manually, 224 assign rights with a template, 224 delete, 230 enable or disable, 229 export or import, 228 rename, 230 view from command line, 230 view rights, 227 RPC program usage, 163 S Samba components in failover solution, 401 configuration, 404 configuration files, 402 copy to other nodes, 407 edit, 405 configure custom
server configuration delete, 74 disable, 74 DNS load balancing, 77 enable, 75 modify, 73 server registry defined, 122 server, configuration check with mxcheck, 388 servers mount administrative filesystem, 142 service monitor activity status, 393 custom starting/stopping actions, 324 defined, 34 errors, view, 326 failover, 315 remove from server, 326 troubleshooting, 391 service monitor configuration advanced settings probe type, 322 scripts, 323 service priority, 322 timeout and failure severity, 321 add or
U users, authentication, 46 V virtual host activeness policy, 309 active and inactive interfaces, 301 applications, configure to recognize, 306 change virtual IP address, 307 defined, 301 device monitors, dependency on, 339 enable or disable, 306 enable or disable network interface, 83 failover, 301, 309 guidelines, 302 monitors and failover, 349 policy for failback, 304 primary and backup servers, 301 rehost via Applications tab, 307 virtual host configuration add or update, 303 change IP address, 307 del
