X9320 Network Storage System Administrator Guide Abstract This guide describes tasks related to cluster configuration and monitoring, system upgrade and recovery, hardware component replacement, and troubleshooting. It does not document X9000 file system features or standard Linux administrative tools and commands. For information about configuring and using X9000 Software file system features, see the HP X9000 File Serving Software File System User Guide.
© Copyright 2010 Hewlett-Packard Development Company, L.P. Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor's standard commercial license. The information contained herein is subject to change without notice.
Contents 1 Product description...................................................................................10 X9320 Network Storage System features...................................................................................10 System components.................................................................................................................10 HP X9000 Software features....................................................................................................
Identifying standby-paired HBA ports...............................................................................34 Turning HBA monitoring on or off....................................................................................34 Deleting standby port pairings........................................................................................34 Deleting HBAs from the configuration database.................................................................34 Displaying HBA information...............
Installing the Statistics tool...................................................................................................56 Enabling collection and synchronization................................................................................56 Upgrading the Statistics tool from X9000 Software 5.6................................................................57 Using the X9000 Management Console Historical Reports GUI....................................................57 Generating reports........
Performing the upgrade......................................................................................................83 After the upgrade..............................................................................................................84 Offline upgrades for X9000 Software 5.5.x to 6.0......................................................................84 Automatic upgrade procedure.............................................................................................
Configuring a file serving node using the original template........................................................106 Completing the restore on a file serving node...........................................................................108 Configuring a file serving node manually.................................................................................110 18 Support and other resources...................................................................118 Contacting HP.............................
Korean notices.....................................................................................................................159 Class A equipment...........................................................................................................159 Class B equipment...........................................................................................................160 Taiwanese notices........................................................................................................
Slovak recycling notice.....................................................................................................172 Spanish recycling notice...................................................................................................172 Swedish recycling notice...................................................................................................173 Battery replacement notices...................................................................................................
1 Product description The HP X9320 Network Storage System is a highly available, scale-out storage solution for file data workloads. The system combines HP X9000 File Serving Software with HP server and storage hardware to create an expansible cluster of file serving nodes.
• • Optional HP X9300 Network Storage System Base Rack.
• Network interface. The IP address of a failed network interface is transferred to a standby network interface until the original network interface is operational again. • Storage connection. For servers with HBA-protected Fibre Channel access, failure of the HBA triggers failover of the node to a designated standby server.
2 Getting started IMPORTANT: Follow these guidelines when using your system: • Do not modify any parameters of the operating system or kernel, or update any part of the X9320 Network Storage System unless instructed to do so by HP; otherwise, the system could fail to operate properly. • File serving nodes are tuned for file serving operations. With the exception of supported backup programs, do not run other applications directly on the nodes.
• Data replication and validation. Use this feature to manage WORM and retained files. • X9000 software snapshots. This feature is included in the X9000 software and can be used to take scheduled or on-demand software snapshots of a file system. • Block Snapshots. This feature uses the array snapshot facility to take scheduled or on-demand snapshots of a file system. • Block Snapshots. This feature uses the array snapshot facility to take scheduled or on-demand snapshots of a file system.
The GUI prompts for your user name and password. The default administrative user is ibrix. Enter the password that was assigned to this user when the system was installed. (You can change the password using the Linux passwd command.) To allow other users to access the GUI, see “Adding user accounts for GUI access” (page 18). The GUI dashboard opens in the same browser window. You can open multiple GUI windows as necessary. See the online help for information about all GUI displays and operations.
System Status The System Status section lists the number of cluster events that have occurred in the last 24 hours. There are three types of events: Alerts. Disruptive events that can result in loss of access to file system data. Examples are a segment that is unavailable or a server that cannot be accessed. Warnings. Potentially disruptive conditions where file system access is not lost, but if the situation is not addressed, it can escalate to an alert condition.
Navigator The Navigator appears on the left side of the window and displays the cluster hierarchy. You can use the Navigator to drill down in the cluster configuration to add, view, or change cluster objects such as file systems or storage, and to initiate or view tasks such as snapshots or replication. When you select an object, a details page shows a summary for that object. The lower Navigator allows you to view details for the selected object, or to initiate a task.
Adding user accounts for GUI access X9000 Software supports administrative and user roles. When users log in under the administrative role, they can configure the cluster and initiate operations such as remote replication or snapshots. When users log in under the user role, they can view the cluster configuration and status, but cannot make configuration changes or initiate operations. The default administrative user name is ibrix. The default regular username is ibrixuser.
Configuring ports for a firewall IMPORTANT: To avoid unintended consequences, HP recommends that you configure the firewall during scheduled maintenance times. When configuring a firewall, you should be aware of the following: • SELinux should be disabled. • By default, NFS uses random port numbers for operations such as mounting and locking. These ports must be fixed so that they can be listed as exceptions in a firewall configuration file. For example, you will need to lock specific ports for rpc.
Port Description 9000:9200/udp 20/tcp, 20/udp Between file serving nodes and FTP clients (user network) 21/tcp, 21/udp 7777/tcp Between X9000 management console GUI and clients that need to access the GUI 8080/tcp 5555/tcp, 5555/udp Dataprotector 631/tcp, 631/udp Internet Printing Protocol (IPP) Configuring NTP servers When the cluster is initially set up, primary and secondary NTP servers are configured to provide time synchronization with an external time source.
http://www.hp.com/go/insightremoteadvanced-docs For IRSS documentation, see the following page: http://www.hp.com/go/insightremotestandard-docs Limitations Note the following: • For X9000 systems, the HP Insight Remote Support implementation is limited to hardware events. • Some manual configurations require that the X9320 and X9300 nodes be recognized as a X9000 solution. They are described under “Configure Entitlements” (page 22).
IMPORTANT: The /opt/hp/hp-snmp-agents/cma.conf file controls certain actions of the SNMP agents. You can add a trapIf entry to the file to configure the IP address used by the SNMP daemon when sending traps. For example, to send traps using the IP address of the eth1 interface, add the following: trapIf eth1 Then restart the HP SNMP agents: service hp-snmp-agents restart For more information about the cma.conf file, see Section 3.
• ◦ Select the System Country Code ◦ Enter the appropriate Customer Contact and Site Information details Custom field settings for MSA Storage Management Utility Configure SNMP settings on the MSA Storage Management Utility. section. (For more information, see “Configuring SNMP event notification in SMU” in the 2300 Modular Smart Array Reference Guide This document is available at http://www.hp.com/support/manuals.
3 Configuring virtual interfaces for client access X9000 Software uses a cluster network interface to carry management console traffic and traffic between file serving nodes. This network is configured as bond0 when the cluster is installed. For clusters with an agile management console configuration, a virtual interface is also created for the cluster network interface to provide failover support for the console.
# # # # ibrix_nic ibrix_nic ibrix_nic ibrix_nic –b –b –b –b –H –H –H –H node1/bond1:1,node2/bond1:2 node2/bond1:1,node1/bond1:2 node3/bond1:1,node4/bond1:2 node4/bond1:1,node3/bond1:2 Configuring NIC failover NIC monitoring should be configured on VIFs that will be used by NFS, CIFS, FTP, or HTTP. Use the same backup pairs that you used when configuring standby servers.
HTTP. When you create a virtual host on the Create Vhost dialog box or with the ibrix_httpvhost command, specify the VIF as the IP address that clients should use to access shares associated with the Vhost. X9000 clients. Use the following command to prefer the appropriate user network. Execute the command once for each destination host that the client should contact using the specified interface. ibrix_client -n -h SRCHOST -A DESTNOST/IFNAME For example: ibrix_client -n -h client12.mycompany.
4 Configuring failover This chapter describes how to configure failover for agile management consoles, file serving nodes, network interfaces, and HBAs. Agile management consoles The management console maintains the cluster configuration and provides graphical and command-line user interfaces for managing and monitoring the cluster. The agile management console is installed on all file serving nodes when the cluster is installed.
Failing over the management console manually To fail over the active management console manually, place the console into maintenance mode. Enter the following command on the node hosting the console: ibrix_fm -m maintenance The command takes effect immediately. The failed-over management console remains in maintenance mode until it is moved to passive mode using the following command: ibrix_fm -m passive A management console cannot be moved from maintenance mode to active mode.
1. 2. 3. The management console verifies that the standby is powered on and accessible. The management console migrates ownership of the node’s segments to the standby and notifies all file serving nodes and X9000 clients about the migration. This is a persistent change. If network interface monitoring has been set up, the management console activates the standby user network interface and transfers the IP address of the node’s user network interface to it.
Preliminary configuration The following configuration steps are required when setting up integrated power sources: • If you plan to implement automated failover, ensure that the management console has LAN access to the power sources. • Install the environment and any drivers and utilities, as specified by the vendor documentation. If you plan to protect access to the power sources, set up the UID and password to be used.
Manual failover does not require the use of programmable power supplies. However, if you have installed and identified power supplies for file serving nodes, you can power down a server before manually failing it over. You can fail over a file serving node manually, even when automated failover is turned on. A file serving node can be failed over from the GUI or the CLI. On the CLI, complete the following steps: 1. Run ibrix_server -f, specifying the node to be failed over in the HOSTNAME option.
problems if the cluster interface fails.) There is no difference in the way that monitoring is set up for the cluster interface and a user network interface. In both cases, you set up file serving nodes to monitor each other over the interface. Sample scenario The following diagram illustrates a monitoring and failover scenario in which a 1:1 standby relationship is configured. Each standby pair is also a network interface monitoring pair.
Setting up a monitor File serving node failover pairs can be identified as network interface monitors for each other. Because the monitoring must be declared in both directions, this is a two-pass process for each failover pair. To set up a network interface monitor, use the following command: /bin/ibrix_nic -m -h MONHOST -A DESTHOST/IFNAME For example, to set up file serving node s2.hp.com to monitor file serving node s1.hp.
Discovering HBAs You must discover HBAs before you set up HBA monitoring, when you replace an HBA, and when you add a new HBA to the cluster. Discovery informs the configuration database of only a port’s WWPN. You must identify ports that are teamed as standby pairs.
The following table describes the fields in the output. Field Description Host Server on which the HBA is installed. Node WWN This HBA’s WWNN. Port WWN This HBA’s WWPN. Port State Operational state of the port. Backup Port WWN WWPN of the standby port for this port (standby-paired HBAs only). Monitoring Whether HBA monitoring is enabled for this port.
Viewing a detailed report Execute the ibrix_haconfig -i command to view the detailed report: /bin/ibrix_haconfig -i [-h HOSTLIST] [-f] [-b] [-s] [-v] The -h HOSTLIST option lists the nodes to check. To also check standbys, include the -b option. To view results only for file serving nodes that failed a check, include the -f argument. The -s option expands the report to include information about the file system and its segments.
5 Configuring cluster event notification Cluster events You can be notified of cluster events by email or SNMP traps. To view the list of supported events, use the command ibrix_event –q. There are three major categories for events, depending on ttheir severity. Alerts. Disruptive events that can result in loss of access to file system data. Warnings. Potentially disruptive conditions where file system access is not lost, but if the situation is not addressed, it can escalate to an alert condition.
The notification threshold for Alert events is 90% of capacity. Threshold-triggered notifications are sent when a monitored system resource exceeds the threshold and are reset when the resource utilization dips 10% below the threshold. For example, a notification is sent the first time usage reaches 90% or more. The next notice is sent only if the usage declines to 80% or less (event is reset), and subsequently rises again to 90% or above.
Viewing email notification settings The ibrix_event command provides comprehensive information about email settings and configured notifications. /bin/ibrix_event -L Sample output follows: Email Notification SMTP Server From Reply To : : : : Enabled mail.hp.com FM@hp.com MIS@hp.com EVENT ------------------------------------asyncrep.completed asyncrep.failed LEVEL ----ALERT ALERT TYPE ----EMAIL EMAIL DESTINATION ----------admin@hp.com admin@hp.
Some SNMP parameters and the SNMP default port are the same, regardless of SNMP version. The agent port is 5061 by default. SYSCONTACT, SYSNAME, and SYSLOCATION are optional MIB-II agent parameters that have no default values. The -c and -s options are also common to all SNMP versions. The -c option turns the encryption of community names and passwords on or off. There is no encryption by default.
Associating events and trapsinks Associating events with trapsinks is similar to associating events with email recipients, except that you specify the host name or IP address of the trapsink instead of an email address. Use the ibrix_event command to associate SNMP events with trapsinks. The format is: /bin/ibrix_event -c -y SNMP [-e ALERT|INFO|EVENTLIST] -m TRAPSINK For example, to associate all Alert events and two Info events with a trapsink at IP address 192.168.2.
For example, to create the group group2 to require authorization, no encryption, and read access to the hp view, enter: ibrix_snmpgroup -c -g group2 -s authNoPriv -r hp The format to create a user and add that user to a group follows: ibrix_snmpuser -c -n USERNAME -g GROUPNAME [-j {MD5|SHA}] [-k AUTHORIZATION_PASSWORD] [-y {DES|AES}] [-z PRIVACY_PASSWORD] Authentication and privacy settings are optional.
6 Configuring system backups Backing up the management console configuration The management console configuration is automatically backed up whenever the cluster configuration changes. The backup takes place on the node hosting the active management console. The backup file is stored at /tmp/fmbackup.zip on the machine where it was created. In an agile configuration, the active management console notifies the passive management console when a new backup file is available.
Configuring NDMP parameters on the cluster Certain NDMP parameters must be configured to enable communications between the DMA and the NDMP Servers in the cluster. To configure the parameters on the management console GUI, select Cluster Configuration from the Navigator, and then select NDMP Backup. The NDMP Configuration Summary shows the default values for the parameters. Click Modify to configure the parameters for your cluster on the Configure NDMP dialog box.
To cancel a session, select that session and click Cancel Session. Canceling a session kills all spawned sessions processes and frees their resources if necessary. To see similar information for completed sessions, select NDMP Backup > Session History. To view active sessions from the CLI, use the following command: ibrix_ndmpsession –l To view completed sessions, use the following command. The -t option restricts the history to sessions occurring on or before the specified date.
NDMP events An NDMP Server can generate three types of events: INFO, WARN, and ALERT. These events are displayed on the management console GUI and can be viewed with the ibrix_event command. INFO events. These events specify when major NDMP operations start and finish, and also report progress. For example: 7012:Level 3 backup of /mnt/ibfs7 finished at Sat Nov 7 21:20:58 PST 2009 7013:Total Bytes = 38274665923, Average throughput = 236600391 bytes/sec. WARN events.
7 Creating hostgroups for X9000 clients A hostgroup is a named set of X9000 clients. Hostgroups provide a convenient way to centrally manage clients using the management console. You can put different sets of clients into hostgroups and then perform the following operations on all members of the group: • Create and delete mountpoints • Mount file systems • Prefer a network interface • Tune host parameters • Set allocation policies Hostgroups are optional.
To set up one level of hostgroups beneath the root, simply create the new hostgroups. You do not need to declare that the root node is the parent. To set up lower levels of hostgroups, declare a parent element for hostgroups. Optionally, you can specify a domain rule for a hostgroup. Use only alphanumeric characters and the underscore character (_) in hostgroup names. Do not use a host name as a group name. To create a hostgroup tree using the CLI: 1.
Deleting hostgroups When you delete a hostgroup, its members are assigned to the parent of the deleted group.
8 Monitoring cluster operations This chapter describes how to monitor the operational state of the cluster and how to monitor cluster health. Monitoring the status of file serving nodes The dashboard on the management console GUI displays information about the operational status of file serving nodes, including CPU, I/O, and network performance information. To view status from the CLI, use the ibrix_server -l command.
Events are written to an events table in the configuration database as they are generated. To maintain the size of the file, HP recommends that you periodically remove the oldest events. See “Removing events from the events database table” (page 51) for more information. You can set up event notifications through email (see “Setting up email notification of cluster events” (page 37)) or SNMP traps (see “Setting up SNMP notifications” (page 39)).
Health check reports The summary report provides an overall health check result for all tested file serving nodes and X9000 clients, followed by individual results. If you include the -b option, the standby servers for all tested file serving nodes are included when the overall result is determined. The results will be one of the following: • Passed. All tested hosts and standby servers passed every health check. • Failed. One or more tested hosts failed a health check.
lab15-62 Report =============== Overall Result ============== Result Type State Network Thread Protocol ------ ------ ----------------------- ------ -------PASSED Server Up, HBAsDown 99.126.39.72 16 true CPU Information =============== Cpu(System,User,Util,Nice) -------------------------0, 1, 1, 0 Memory Information ================== Mem Total Mem Free --------- -------1944532 1841548 Module Up time Last Update ------ --------- ---------------------------- Loaded 3267210.
Segment owner for segment 2 filesystem ifs2 matches on Iad and Fusion Manager PASSED ifs1 file system uuid matches on Iad and Fusion Manager PASSED ifs1 file system generation matches on Iad and Fusion Manager PASSED ifs1 file system number segments matches on Iad and Fusion Manager PASSED ifs1 file system mounted state matches on Iad and Fusion Manager PASSED Segment owner for segment 1 filesystem ifs1 matches on Iad and Fusion Manager PASSED Superblock owner for segment 1 of filesystem ifs2 on
---------CPU----------HOST User lab12-10.hp.com 0 ---------NFS v3-------HOST Null lab12-10.hp.com 0 System Nice Idle IoWait Irq SoftIrq 0 0 0 97 1 0 Getattr Setattr Lookup Access Readlink Read Write 0 0 0 0 0 0 0 HOST lab12-10.hp.com Create Mkdir Symlink Mknod Remove Rmdir Rename 0 0 0 0 0 0 0 HOST lab12-10.hp.
9 Using the Statistics tool The Statistics tool reports historical performance data for the cluster or for an individual file serving node. You can view data for the network, the operating system, file systems, memory, and block devices. Statistics data is transmitted from each file serving node to the management console, which controls processing and report generation. Installing and configuring the Statistics tool The Statistics tool has two parts: • Manager process.
NOTE: Do not run the command on individual nodes. All nodes must be specified in the same command and can be specified in any order. Be sure to use node names, not IP addresses. To test the rsync mechanism, see “Testing rsync access” (page 64). 4. 5. On the active node, edit the /etc/ibrix/stats.conf file to add the age.retain.files=24h parameter. See “Changing the Statistics tool configuration” (page 61). Create a symbolic link from /var/lib/ibrix/histstats to the /local/statstool/ histstats directory.
The Time View lists the reports in chronological order, and the Table View lists the reports by cluster or server. Click a report to view it. Generating reports To generate a new report, click Request New Report on the X9000 Management Console Historical Reports GUI.
Enter the specifications for your report and click Submit. The management console will then generate the report. The completed report will appear in the list of reports on the statistics home page. When generating reports, you should be aware of the following: • A report can be generated only from statistics that have been gathered. For example, if you start the tool at 9:40am and ask for a report from 9:00am to 9:30am, the report cannot be generated because data was not gathered for that period.
You should also configure how long to retain staging data and data stored in the statistics database. The retention periods are controlled by parameters in the /etc/ibrix/stats.conf file. • Aging configuration for staging data: The age.retain.files parameter specifies the number of hours, starting from the current time, to retain collected data. If the parameter is not set, data is retained for 30 days by default.
Changing the Statistics tool configuration The configuration can be changed only on the management node. To change the configuration, add a configuration parameter and its value to the /etc/ibrix/stats.conf file on the currently active node. The supported configuration changes are: • Interval for data collection. The default value is 15 seconds. To change the interval to 30 seconds, add the following line to the stats.conf file: collector.time=30 • Aging configuration for staging statistics data.
Management console failover and the Statistics tool configuration In an X9000 Software High Availability scenario, migrate the Stats active management console and the collected data (including reports) to the current active management console. On the current active node: 1. Stop statstool: # /etc/init.d/ibrix_statsagent stop ––agilefm_passive 2.
10. Run the passive migrator script: # /usr/local/ibrix/stats/bin/stats_passive_migrator 11. Start statstool: # /etc/init.d/ibrix_statsagent start ––agilefm_passive NOTE: Passwordless authentication is required for the migration to succeed. To determine whether passwordless authentication is enabled, see “Configuring shared ssh keys” (page 65) and “Enabling collection and synchronization” (page 56). Migrate reports from the old active node to the current active node.
#/etc/init.d/ibrix_statsmanager status ibrix_statsmanager (pid 25322) is running... In the output, the pid is the process id of the “master” process. Controlling Statistics tool processes Statistics tool processes on all file serving nodes connected to the active management console can be controlled remotely from the active management console. Use the ibrix_statscontrol tool to start or stop the processes on all connected file serving nodes or on specified hostnames only.
Log files See /var/log/stats.log for detailed logging for the Statistics tool. (The information includes detailed exceptions and traceback messages). The logs are rolled over at midnight every day and only seven days of compressed statistics logs are retained. The default /var/log/messages log file also includes logging for the Statistics tool, but the messages are short. Configuring shared ssh keys To configure one-way shared ssh keys on the cluster, complete the following steps: 1.
10 Maintaining the system Shutting down the system To shut down the system completely, first shut down the X9000 software, and then power off the system hardware. Shutting down the X9000 Software Use the following procedure to shut down the X9000 Software. Unless noted otherwise, run the commands from the dedicated Management Console or from the node hosting the active agile management console. 1. Disable HA for all file serving nodes: ibrix_server -m -U 2.
3. Monitor the nodes on the management console and wait for them all to report UP in the output from the following command: ibrix_server -l 4. Mount file systems and verify their content. Run the following command on the Management Console or file serving node hosting the active agile management console: ibrix_mount -f fs_name -m On Linux X9000 clients, run the following command: ibrix_lwmount -f fsname -m 5. Enable HA on the file serving nodes.
Starting and stopping processes You can start, stop, and restart processes and can display status for the processes that perform internal X9000 Software functions. The following commands also control the operation of PostgreSQL on the machine. The PostgreSQL service is available at /usr/local/ibrix/init/. To start and stop processes and view process status on the management console, use the following command: /etc/init.
The values that are restored depend on the values specified for the -h HOSTLIST command: • ◦ File serving nodes. The default file serving node host tunings are restored. ◦ X9000 clients. The host tunings that are in effect for the default clients hostgroup are restored. ◦ Hostgroups. The host tunings that are in effect for the parent of the specified hostgroups are restored. To list host tuning settings on file serving nodes, X9000 clients, and hostgroups, use the following command.
/bin/ibrix_fs -m -f FSNAME -H HOSTNAME1,HOSTNAME2 [-M] [-F] [-N] For example, to migrate ownership of all segments in file system ifs1 that reside on s1.hp.com to s2.hp.com: /bin/ibrix_fs -m -f ifs1 -H s1.hp.com,s2.hp.com Removing a node from the cluster Use the following procedure to remove a node from the cluster: 1. If the node is hosting a passive management console, go to step 2.
to receive the data. (If you do not select destination segments, the data is spread among the available segments.) The Task Summary window displays the progress of the rebalance operation and reports any errors. If you need to stop the operation, click Stop. 5. When the rebalance operation completes, remove the storage from the cluster: ibrix_fs -B -f FSNAME -n BADSEGNUMLIST The segment number associated with the storage is not reused. 6.
Adding user network interfaces Although the cluster network can carry traffic between file serving nodes and either NFS/CIFS or X9000 clients, you may want to create user network interfaces to carry this traffic. If your cluster must accommodate a mix of NFS/CIFS clients and X9000 clients, or if you need to segregate client traffic to different networks, you will need one or more user networks.
If you identify a VIF, the management console does not automatically query the node. If the VIF will be used only as a standby network interface in an automated failover setup, the management console will query the node the first time a network is failed over to the VIF. Otherwise, you must enter the VIF’s IP address and netmask manually in the configuration database (see “Setting network interface options in the configuration database” (page 73)).
Unpreferring network interfaces To return file serving nodes or X9000 clients to the cluster interface, unprefer their preferred network interface. The first command unprefers a network interface for a file serving node; the second command unprefers a network interface for a client.
/bin/ibrix_nic -t -n IFNAME -h HOSTNAME To specify a new virtual cluster interface for a cluster with an agile management console configuration, use the following command: /bin/ibrix_fm -c –d -n -v cluster [–I ] Managing routing table entries X9000 Software supports one route for each network interface in the system routing table.
Field Description MAC_ADDR MAC address of this NIC. ROUTE IP address in routing table used by this NIC. STATE Network interface state. TYPE Network type (cluster or user). When ibrix_nic is used with the -i option, it reports detailed information about the interfaces. Use the -h option to limit the output to specific hosts. Use the -n option to view information for a specific interface.
11 Migrating to an agile management console The agile management console configuration provides one active management console and one passive management console installed on different nodes in the cluster. The migration procedure configures the current Management Server machine as a host for an agile management console and installs another instance of the agile management console on a file serving node.
/etc/init.d/network restart service network restart Verify that you can ping the new local IP address. 4. Configure the agile management console: ibrix_fm -c -d –n -v cluster -I In the command, is the old cluster IP address for the original management console and is the new IP address you acquired. For example: [root@x109s1 ~]# ibrix_fm -c 172.16.3.1 -d bond0:1 -n 255.255.248.
================================================ Command succeeded! 11. Verify that there is only one management console in this cluster: ibrix_fm -f For example: [root@x109s1 ~]# ibrix_fm -f NAME IP ADDRESS ------ ---------X109s1 172.16.3.100 Command succeeded! 12. Install a passive agile management console on a second file serving node. In the command, the -F option forces the overwrite of the new_lvm2_uuid file that was installed with the X9000 Software.
Testing failover and failback of the agile management console Complete the following steps: 1. On the node hosting the active management console, place the management console into maintenance mode. This step fails over the active management console role to the node currently hosting the passive agile management console. /bin/ibrix_fm –m maintenance 2.
x109s3 172.16.3.3 Command succeeded! 3. Uninstall the management console from the Management Server machine: /ibrix/ibrixinit -tm -U 4. Verify that the uninstalled management console is no longer registered. Run the following command from the file serving node hosting the newly active management console: ibrix_fm -f The command should now report only the agile management console on the file serving node. [root@x109s3 ibrix]# ibrix_fm -f NAME IP ADDRESS ------ ---------x109s3 172.16.3.
12 Upgrading the X9000 Software This chapter describes how to upgrade to the latest X9000 File Serving Software release. The management console and all file serving nodes must be upgraded to the new release at the same time. Note the following: • Upgrades to the X9000 Software 6.0 release are supported for systems currently running X9000 Software 5.5.x and 5.6.x. If your system is running an earlier release, first upgrade to the latest 5.5 release, and then upgrade to 6.0. • The upgrade to 6.
7. On all nodes hosting the passive management console, place the management console into maintenance mode: /bin/ibrix_fm –m maintenance 8. On the active management console node, disable automated failover on all file serving nodes: /bin/ibrix_server -m -U 9. Run the following command to verify that automated failover is off. In the output, the HA column should display off. /bin/ibrix_server -l 10.
nodes. The management console is in active mode on the node where the upgrade was run, and is in passive mode on the other file serving nodes. If the cluster includes a dedicated Management Server, the management console is installed in passive mode on that server. 5. 6. Upgrade Linux X9000 clients. See “Upgrading Linux X9000 clients” (page 90). If you received a new license from HP, install it as described in the “Licensing” chapter in this guide. After the upgrade Complete the following steps: 1.
/stage, and /alt, your system was quick-restored and you can use the automated upgrade procedure. If you do not see these mount points, proceed with the manual upgrade process. • Automatic upgrades. This process uses separate partitioned space on the local disk to save node-specific configuration information. After each node is upgraded, its configuration is automatically reapplied. • Manual upgrades.
If you are using NFS, verify that all NFS processes are stopped: ps –ef | grep nfs If necessary, use the following command to stop NFS services: /etc/init.d/nfs stop Use kill -9 to stop any NFS processes that are still running. If necessary, run the following command on all nodes to find any open file handles for the mounted file systems: lsof Use kill -9 to stop any processes that still have open file handles on the file systems. 6.
6. If you are using CIFS, set the following parameters to synchronize the CIFS software and the management console database: • smb signing enabled • smb signing required • ignore_writethru Use ibrix_cifsconfig to set the parameters, specifying the value appropriate for your cluster (1=enabled, 0=disabled). The following examples set the parameters to the default values for the 6.
7. On all nodes hosting the passive management console, place the management console into maintenance mode: /bin/ibrix_fm –m maintenance 8. On the active management console node, disable automated failover on all file serving nodes: /bin/ibrix_server -m -U 9. Run the following command to verify that automated failover is off. In the output, the HA column should display off. /bin/ibrix_server -l 10. Unmount file systems on Linux X9000 clients. 11.
Performing the upgrade Complete the following steps on each node: 1. Obtain the latest HP X9000 Quick Restore DVD version 6.0 ISO image from the HP kiosk at http://www.software.hp.com/kiosk (you will need your HP-provided login credentials). 2. Burn the ISO image to a DVD. 3. Insert the Quick Restore DVD into the server DVD-ROM drive. 4. Restart the server to boot from the drive. 5. When the following screen appears, enter qr to install the X9000 software on the file serving node.
1. 2. Apply any custom tuning parameters, such as mount options. Remount all X9000 Software file systems: ibrix_mount -f -m 3. From the node hosting the active management console, run the following upgrade script: /usr/local/ibrix/setup/post_manualupgrade This script turns automated failover back on, performs a backup of the upgraded configuration, verifies versions, and verifies the health of the cluster.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.
The IAD service should be running, as shown in the previous sample output. If it is not, contact HP Support. Installing a minor kernel update on Linux clients The X9000 client software is upgraded automatically when you install a compatible Linux minor kernel update.