Red Hat Enterprise Linux 5 DM Multipath DM Multipath Configuration and Administration
DM Multipath Red Hat Enterprise Linux 5 DM Multipath DM Multipath Configuration and Administration Edition 3 Copyright © 2011 Red Hat, Inc. The text of and illustrations in this document are licensed by Red Hat under a Creative Commons Attribution–Share Alike 3.0 Unported license ("CC-BY-SA"). An explanation of CC-BY-SA is available at http://creativecommons.org/licenses/by-sa/3.0/.
Preface v 1. Audience ........................................................................................................................ v 2. Related Documentation ................................................................................................... v 3. Feedback ........................................................................................................................ v 4. Document Conventions .................................................................................
iv
Preface This book describes the Device Mapper Multipath (DM-Multipath) feature of Red Hat Enterprise Linux for the Red Hat Enterprise Linux 5 release. 1. Audience This book is intended to be used by system administrators managing systems running the Linux operating system. It requires familiarity with Red Hat Enterprise Linux. 2.
Preface Be sure to mention the manual's identifier: Bugzilla component: Documentation-cluster Book identifier: DM_Multipath(EN)-5 (2011-07-2115:35) By mentioning this manual's identifier, we know exactly which version of the guide you have. If you have a suggestion for improving the documentation, try to be as specific as possible. If you have found an error, please include the section number and some of the surrounding text so we can find it easily. 4.
Pull-quote Conventions File-related classes include filesystem for file systems, file for files, and dir for directories. Each class has its own associated set of permissions. Proportional Bold This denotes words or phrases encountered on a system, including application names; dialog box text; labeled buttons; check-box and radio button labels; menu titles and sub-menu titles. For example: Choose System → Preferences → Mouse from the main menu bar to launch Mouse Preferences.
Preface books books_tests Desktop Desktop1 documentation downloads drafts images mss notes photos scripts stuff svgs svn Source-code listings are also set in mono-spaced roman but add syntax highlighting as follows: package org.jboss.book.jca.ex1; import javax.naming.InitialContext; public class ExClient { public static void main(String args[]) throws Exception { InitialContext iniCtx = new InitialContext(); Object ref = iniCtx.lookup("EchoBean"); EchoHome home = (EchoHome) ref; Echo echo = home.
Chapter 1. Device Mapper Multipathing Device Mapper Multipathing (DM-Multipath) allows you to configure multiple I/O paths between server nodes and storage arrays into a single device. These I/O paths are physical SAN connections that can include separate cables, switches, and controllers. Multipathing aggregates the I/O paths, creating a new device that consists of the aggregated paths. 1.1. New and Changed Features for Red Hat Enterprise Linux 5.7 Red Hat Enterprise Linux 5.
Chapter 1. Device Mapper Multipathing Figure 1.1. Active/Passive Multipath Configuration with One RAID Device In this configuration, there is one I/O path that goes through hba1, SAN1, and controller 1 and a second I/O path that goes through hba2, SAN2, and controller2.
Overview of DM-Multipath Figure 1.2. Active/Passive Multipath Configuration with Two RAID Devices In the example shown in Figure 1.2, “Active/Passive Multipath Configuration with Two RAID Devices”, there are two I/O paths to each RAID device (just as there are in the example shown in Figure 1.1, “Active/Passive Multipath Configuration with One RAID Device”).
Chapter 1. Device Mapper Multipathing Figure 1.3. Active/Active Multipath Configuration with One RAID Device 1.3. Storage Array Support By default, DM-Multipath includes support for the most common storage arrays that support DMMultipath. The supported devices can be found in the multipath.conf.defaults file. If your storage array supports DM-Multipath and is not configured by default in this file, you may need to add them to the DM-Multipath configuration file, multipath.conf.
DM-Multipath Setup Overview 1.5. DM-Multipath Setup Overview DM-Multipath includes compiled-in default settings that are suitable for common multipath configurations. Setting up DM-multipath is often a simple procedure. The basic procedure for configuring your system with DM-Multipath is as follows: 1. Install device-mapper-multipath rpm. 2. Edit the multipath.conf configuration file: • comment out the default blacklist • change any of the existing defaults as needed • save the configuration file 3.
6
Chapter 2. Multipath Devices Without DM-Multipath, each path from a server node to a storage controller is treated by the system as a separate device, even when the I/O path connects the same server node to the same storage controller. DM-Multipath provides a way of organizing the I/O paths logically, by creating a single multipath device on top of the underlying devices. 2.1.
Chapter 2. Multipath Devices If you want the system-defined user-friendly names to be consistent across all nodes in the cluster, however, you can follow this procedure: 1. Set up all of the multipath devices on one machine. 2. Disable all of your multipath devices on your other machines by running the following commands: # service multipathd stop # multipath -F 3. Copy the bindings file from the first machine to all the other machines in the cluster.
Multipath Device Attributes 2.3. Multipath Device Attributes In addition to the user_friendly_names and alias options, a multipath device has numerous attributes. You can modify these attributes for a specific multipath device by creating an entry for that device in the multipaths section of the multipath configuration file. For information on the multipaths section of the multipath configuration file, see see Section 4.4, “Multipaths Device Configuration Attributes”. 2.4.
10
Chapter 3. Setting Up DM-Multipath This chapter provides step-by-step example procedures for configuring DM-Multipath. It includes the following procedures: • Basic DM-Multipath setup • Ignoring local disks • Adding more devices to the configuration file 3.1. Setting Up DM-Multipath Before setting up DM-Multipath on your system, ensure that your system has been updated and includes the device-mapper-multipath package. Use the following procedure to set up DM-Multipath for a basic failover configuration.
Chapter 3. Setting Up DM-Multipath The multipath -v2 command prints out multipathed paths that show which devices are multipathed. If the command does not print anything out, ensure that all SAN connections are set up properly and the system is multipathed. For further information on the multipath command output, see Section 5.5, “Multipath Command Output”. 5.
Ignoring Local Disks when Generating Multipath Devices create: 3600a0b80001327510000009b4362163e [size=12 GB][features="0"][hwhandler="0"] \_ round-robin 0 \_ 2:0:0:3 sde 8:64 \_ 3:0:0:3 sdi 8:128 2. In order to prevent the device mapper from mapping /dev/sda in its multipath maps, edit the blacklist section of the /etc/multipath.conf file to include this device.
Chapter 3. Setting Up DM-Multipath create: 3600a0b80001327510000009b4362163e [size=12 GB][features="0"][hwhandler="0"] \_ round-robin 0 \_ 2:0:0:3 sde 8:64 \_ 3:0:0:3 sdi 8:128 3.3. Adding Devices to the Multipathing Database By default, DM-Multipath includes support for the most common storage arrays that support DMMultipath. The default configuration values, including supported devices, can be found in the multipath.conf.defaults file.
Chapter 4. The DM-Multipath Configuration File By default, DM-Multipath provides configuration values for the most common uses of multipathing. In addition, DM-Multipath includes support for the most common storage arrays that support DMMultipath. The default configuration values and the supported devices can be found in the /usr/ share/doc/device-mapper-multipath-0.4.7/multipath.conf.defaults file. You can override the default configuration values for DM-Multipath by editing the /etc/ multipath.
Chapter 4. The DM-Multipath Configuration File When the system determines the attributes of a multipath device, first it checks the multipath settings, then the per devices settings, then the multipath system defaults. 4.2. Configuration File Blacklist The blacklist section of the multipath configuration file specifies the devices that will not be used when the system configures multipath devices. Devices that are blacklisted will not be grouped into a multipath device.
Blacklisting By Device Type } You can use a devnode entry in the blacklist section of the configuration file to specify individual devices to blacklist rather than all devices of specific type; this is not recommended, however. Unless it is statically mapped by udev rules, there is no guarantee that a specific device will have the same name on reboot. For example, a device name could change from /dev/sda to /dev/sdb on reboot.
Chapter 4. The DM-Multipath Configuration File 4.3. Configuration File Defaults The /etc/multipath.conf configuration file includes a defaults section that sets the user_friendly_names parameter to yes, as follows. defaults { user_friendly_names yes } This overwrites the default value of the user_friendly_names parameter. The configuration file includes a template of configuration defaults. This section is commented out, as follows.
Configuration File Defaults Attribute Description polling_interval Specifies the interval between two path checks in seconds. The default value is 5. selector Specifies the default algorithm to use in determining what path to use for the next I/O operation. The default value is round-robin 0. path_grouping_policy Specifies the default path grouping policy to apply to unspecified multipaths.
Chapter 4. The DM-Multipath Configuration File Attribute Description A value of 0 or immediate specifies that as soon as there is a path group with a higher priority than the current path group the system switches to that path group. A numeric value greater than zero specifies deferred failback, expressed in seconds. A value of manual specifies that failback can happen only with operator intervention. The default value is manual.
Multipaths Device Configuration Attributes Attribute Description the sum of the priorities of the included paths. The default value is sum. log_checker_err (Red Hat Enterprise Linux 5.6 and later) If set to once, a path checker error is logged once at logging level 2 and any later errors are logged at level 3 until the device is restored. If set to always, every path checker error is logged at logging level 2. The default value is always. fast_io_fail_tmo (Red Hat Enterprise Linux 5.
Chapter 4. The DM-Multipath Configuration File Attribute Description A value of 0 or immediate specifies that as soon as there is a path group with a higher priority than the current path group the system switches to that path group. A numeric value greater than zero specifies deferred failback, expressed in seconds. A value of manual specifies that failback can happen only with operator intervention.
Configuration File Devices rr_weight no_path_retry priorities 5 } multipath { wwid alias rr_weight } 1DEC_____321816758474 red priorities } 4.5. Configuration File Devices Table 4.3, “Device Attributes” shows the attributes that you can set for each individual storage device in the devices section of the multipath.conf configuration file. These attributes are used by DMMultipath unless they are overwritten by the attributes specified in the multipaths section of the multipath.
Chapter 4. The DM-Multipath Configuration File Attribute Description product Specifies the product name of the storage device to which the device attributes apply, for example HSV110 (C)COMPAQ. path_grouping_policy Specifies the default path grouping policy to apply to unspecified multipaths.
Configuration File Devices Attribute Description A numeric value greater than zero specifies deferred failback, expressed in seconds. A value of manual specifies that failback can happen only with operator intervention. rr_min_io Specifies the number of I/O requests to route to a path before switching to the next path in the current path group. fast_io_fail_tmo (Red Hat Enterprise Linux release 5.
26
Chapter 5. DM-Multipath Administration and Troubleshooting This chapter will provide information on administering DM-Multipath on a running system.
Chapter 5. DM-Multipath Administration and Troubleshooting 5.2. The Multipath Daemon If you find you have trouble implementing a multipath configuration, you should ensure that the multipath daemon is running, as described in Chapter 3, Setting Up DM-Multipath. The multipathd daemon must be running in order to use multipathed devices. 5.3.
Multipath Queries with multipath Command For each multipath device: action_if_any: alias (wwid_if_different_from_alias) [size][features][hardware_handler] For each path group: \_ scheduling_policy [path_group_priority_if_known] [path_group_status_if_known] For each path: \_ host:channel:id:lun devnode major:minor [path_status] [dm_status_if_known] For example, the output of a multipath command might appear as follows: mpath1 (3600d0230003228bc000339414edb8101) [size=10 GB][features="0"][hwhandler="0"
Chapter 5. DM-Multipath Administration and Troubleshooting mpath1 (3600d0230003228bc000339414edb8101) [size=10 GB][features="0"][hwhandler="0"] \_ round-robin 0 [prio=1][active] \_ 2:0:0:6 sdb 8:16 [active][ready] \_ round-robin 0 [prio=1][enabled] \_ 3:0:0:6 sdc 8:64 [active][ready] 5.7. Multipath Command Options Table 5.1, “Useful multipath Command Options” describes some options of the multipath command that you may find useful. Table 5.1.
Troubleshooting with the multipathd Interactive Console help to get a list of available commands, you can enter a interactive command, or you can enter CTRL-D to quit. The multipathd interactive console can be used to troubleshoot problems you may be having with your system. For example, the following command sequence displays the multipath configuration, including the defaults, before exiting the console.
32
Appendix A. Revision History Revision 5.7-1 Thu Jul 21 2011 Resolves: #703610 Documents new configuration file parameters. Steven Levine slevine@redhat.com Resolves: #510333 Clarifies procedure for maintaining consistent device names across a cluster. Revision 5.6-1 Thu Dec 23 2010 Steven Levine slevine@redhat.com Resolves: #661532 Updates configuration file documentation to include Red Hat Enterprise Linux 5.6 options and parameters. Resolves: #654525 Corrects small typos in document. Revision 5.
34
Index Symbols /etc/multipath.conf package, 11 A active/active configuration definition, 1 illustration, 3 active/passive configuration definition, 1 illustration, 1 alias parameter , 21 configuration file, 7 B bindings_file parameter, 18 blacklist configuration file, 16 default devices, 17 device name, 16 device type, 17 in configuration file, 11 WWID, 16 blacklist_exceptions section multipath.
Index feedback, v, v file_timeout parameter, 18 flush_on_last_del parameter, 18, 21, 23 getuid_callout parameter, 18, 23 gid parameter, 18, 21 path_grouping_policy parameter, 18, 21, 23 path_selector parameter, 21, 23 pg_prio_calc parameter, 18 polling_interval parameter, 18 prio_callout parameter, 18, 21, 23 product parameter, 23 product_blacklist parameter, 23 H Q G hardware_handler parameter, 23 queue_without_daemon parameter, 18 K R kpartx command , 4 resizing a multipath device, 27 rr_min_io
