019-0192_Cvr 8/17/04 4:49 PM Page 1 Xsan Administrator’s Guide Includes information for managing Xsan volumes in a storage area network using Xsan Admin or the command line
LL0192.book Page 2 Thursday, July 29, 2004 5:20 PM Apple Computer, Inc. © 2004 Apple Computer, Inc. All rights reserved. The owner or authorized user of a valid copy of Xsan software may reproduce this publication for the purpose of learning to use such software. No part of this publication may be reproduced or transmitted for commercial purposes, such as selling copies of this publication or for providing paid for support services. The Apple logo is a trademark of Apple Computer, Inc.
LL0192.
LL0192.
LL0192.
LL0192.
About This Book Preface LL0192.book Page 7 Thursday, July 29, 2004 5:20 PM Use this guide to learn how to set up and manage Xsan volumes on a storage area network. This guide shows how to use Xsan to combine Xserve RAID arrays and slices into large, easy-to-expand volumes of storage that clients use like local disks but are actually shared over a high-speed Fibre Channel fabric.
LL0192.book Page 8 Thursday, July 29, 2004 5:20 PM Notation Conventions The following conventions are used in this book wherever shell commands or other command-line items are described. 8 Notation Indicates monospaced font A command or other terminal text $ A shell prompt [text_in_brackets] An optional parameter (one|other) Alternative parameters (type one or the other) underlined A parameter you must replace with a value [...
LL0192.book Page 9 Thursday, July 29, 2004 5:20 PM 1 1 Overview of Xsan This chapter gives you an overview of Xsan and storage area networks. Read this chapter for an overview of Xsan and how you can use it to set up a storage area network (SAN) to provide fast, shared storage. What Is Xsan? Xsan is a storage area network file system and a management application (Xsan Admin) you can use to provide users or applications on client computers with shared highspeed access to expandable storage.
LL0192.book Page 10 Thursday, July 29, 2004 5:20 PM Xsan Storage Area Networks A storage area network is a way of connecting computers to storage devices that gives users very fast access to files and gives administrators the ability to expand storage capacity as needed without interrupting users.
LL0192.book Page 11 Thursday, July 29, 2004 5:20 PM Shared SAN Volumes Users and applications see shared SAN storage as local volumes. Xsan volumes are logical disks made up of groups of RAID arrays. The elements you combine to create an Xsan volume are described under “How Xsan Storage Is Organized” on page 12. Controllers and Clients When you add a computer to an Xsan SAN, you specify whether it will play the role of client, controller, or both.
LL0192.book Page 12 Thursday, July 29, 2004 5:20 PM How Xsan Storage Is Organized Users use an Xsan volume the same way they use a local disk. What they don’t see is that the SAN volume actually consists of numerous physical disks combined on several levels using RAID techniques. The following illustration shows an example of how disk space provided by the individual drive modules in Xserve RAID systems is combined into a volume that users see as a large local disk.
LL0192.book Page 13 Thursday, July 29, 2004 5:20 PM One of your first tasks when you set up a SAN volume is to prepare LUNs. If the two RAID 5 arrays on a new Xserve RAID are not right for your application, you can use RAID Admin to create arrays based on other RAID schemes. For help choosing schemes for your LUNs, see “Choosing RAID Schemes for LUNs” on page 23. The illustration on page 12 shows four Xserve RAID systems hosting two arrays each.
LL0192.book Page 14 Thursday, July 29, 2004 5:20 PM The following screen image shows how LUNs, storage pools, and volumes appear as you organize them in the Xsan Admin application. This example shows a SAN named “Test SAN” with a single shared volume named “SanVol.” Storage for the volume is provided by two storage pools, “Pool1” and “Pool2,” each based on a single LUN. Each of the LUNs is a 3-disk RAID 5 array on an Xserve RAID using 115 GB drive modules.
LL0192.book Page 15 Thursday, July 29, 2004 5:20 PM How Xsan Utilizes Available Storage Xsan stores both user files and file system data on SAN volumes, and stripes data across the LUNs in a volume for better performance. Metadata and Journal Data Xsan records information about the files in an Xsan volume using metadata files and file system journals.
LL0192.book Page 16 Thursday, July 29, 2004 5:20 PM Expanding Storage You can add free space to an Xsan volume without interrupting the users or applications that rely on the volume for access to files. There are two ways to add storage to a volume: • Add Xserve RAID systems (new LUNs) to existing storage pools • Add entire new storage pools to volumes The first method requires you to unmount and remount the volume on clients.
LL0192.book Page 17 Thursday, July 29, 2004 5:20 PM 2 Setting Up a Storage Area Network 2 This chapter lists requirements, suggests planning tips, and gives instructions for setting up an Xsan SAN.
LL0192.
LL0192.book Page 19 Thursday, July 29, 2004 5:20 PM Fibre Channel Fabric Unlike file system metadata, which is transferred over Ethernet, actual file content in an Xsan SAN is transferred over Fibre Channel connections.
LL0192.book Page 20 Thursday, July 29, 2004 5:20 PM IP Addresses and Domain Names For best results, assign fixed, non-routed IP addresses to all clients, controllers, and storage devices connected to the SAN Ethernet network. You can use the following ranges of IP addresses in your private (non-routed) subnet: Private address range Associated subnet mask Comments 10.0.0.0 – 10.255.255.255 255.0.0.0 10/8 172.16.0.0 – 172.31.255.255 255.240.0.0 172.16/12 192.168.0.0 – 192.168.255.255 255.255.0.
LL0192.book Page 21 Thursday, July 29, 2004 5:20 PM Planning Your SAN It’s easy to add storage to an existing Xsan SAN without interrupting users, but reorganizing a SAN after you set it up is not so simple. So, it’s important to plan the layout and organization of your SAN and its storage before you set it up.
LL0192.book Page 22 Thursday, July 29, 2004 5:20 PM Planning Considerations and Guidelines The following paragraphs might help you make some of your SAN design decisions. How Much Storage? Because it’s easy to add storage to an Xsan SAN without interrupting user work, you only need to decide on an adequate starting point. You can then add storage as needed. Note that the number of Xserve RAID systems you use affects not only available space but also SAN performance. See “Performance Considerations,” below.
LL0192.book Page 23 Thursday, July 29, 2004 5:20 PM • Store user files, file system metadata, and journal data on separate storage pools, and create these storage pools using LUNs from different RAID controllers. • Use a router to isolate the Ethernet network used by the SAN from a company intranet or the Internet, or better, use a second Ethernet network (including a second Ethernet card in each SAN computer) for the SAN.
LL0192.book Page 24 Thursday, July 29, 2004 5:20 PM Xserve RAID supports all popular RAID levels. Each RAID scheme offers a different balance of performance, data protection, and storage efficiency, as summarized in the following table.
LL0192.book Page 25 Thursday, July 29, 2004 5:20 PM To provide high performance, Xsan uses the RAID 0 scheme to stripe data across the LUNs in a storage pool. This requires that the LUNs in the pool be the same size. If you set up a storage pool using LUNs of different sizes, Xsan uses available space on each LUN equal to the capacity of the smallest LUN. If the LUNs vary in size, this can result in wasted capacity.
LL0192.book Page 26 Thursday, July 29, 2004 5:20 PM If high availability is important, you should use at least two controllers, one as the primary controller and one as a standby. You can specify additional controllers as needed, and set their failover priorities to determine the order in which they are tried if the primary controller stops responding.
LL0192.book Page 27 Thursday, July 29, 2004 5:20 PM Connecting Computers and Storage Devices Before you set up your Xsan SAN, connect client computers, controller computers, and storage devices to the SAN’s Fibre Channel and Ethernet networks. Make sure your networks meet the requirements summarized under “Fibre Channel Fabric” on page 19 and “Ethernet TCP/IP Network” on page 19.
LL0192.book Page 28 Thursday, July 29, 2004 5:20 PM Using the Xsan Admin Application You use the Xsan Admin application to set up and manage SANs. You can use Xsan Admin to manage a SAN from any computer that has access to the SAN’s TCP/IP subnet. Xsan Admin is installed in /Applications/Server. Installing Xsan Admin Separately Xsan Admin is included when you install the Xsan software on SAN controller and client computers.
LL0192.book Page 29 Thursday, July 29, 2004 5:20 PM SAN Setup Summary You’ll perform the following tasks to set up your first Xsan storage area network. Details for each task are on the indicated pages.
LL0192.book Page 30 Thursday, July 29, 2004 5:20 PM Setting Up an Xsan Storage Area Network m Step 1: Set Up the Fibre Channel Network Connect controller computers, client computers, and Xserve RAID storage systems to a Fibre Channel network. Be sure to configure the switch and make the connections so that you create a Fibre Channel fabric. For more information, see the guidelines and requirements under “Fibre Channel Fabric” on page 19.
LL0192.book Page 31 Thursday, July 29, 2004 5:20 PM Step 5: Install Xsan Software on Clients and Controllers Take the Xsan installer disc to each controller and client computer connected to the SAN and install the Xsan software. m To install the Xsan file system and Xsan Admin application: At each computer, insert the disc and double-click the Xsan installer icon. m To install just the file system without Xsan Admin: Click Customize on the final installer window and deselect Xsan Admin.
LL0192.book Page 32 Thursday, July 29, 2004 5:20 PM Step 7: Set Up Controllers and Clients All computers on the local network with Xsan installed are listed in the Computers pane. Your next step is to choose at least one computer to act as metadata controller and set up the others as clients. 1 Still in the Computers pane, select a computer in the list.
LL0192.book Page 33 Thursday, July 29, 2004 5:20 PM Step 8: Label and Initialize LUNs (RAID Arrays) Next, prepare (label and initialize) available LUNs for use with the Xsan file system. Each LUN represents a RAID array or slice you set up using RAID Admin. 1 In Xsan Admin, click LUNs on the Setup pane. Responding storage devices are listed by name and size. You should see one LUN for each RAID array or slice you created on Xserve RAID systems attached to the SAN’s Fibre Channel network.
LL0192.book Page 34 Thursday, July 29, 2004 5:20 PM Step 9: Create Volumes Next, you’ll create the volumes your users will see. 1 In Xsan Admin, click Storage on the Setup pane. 2 Click the New Volume button (next to the empty list). 3 Type a name for the volume. Use only uppercase letters (A-Z), lowercase letters (a-z), and numbers (0-9). Don’t include spaces, underscores (_), or hyphens (-). Maximum length: 70 characters. This is the name users will see in the Finder when the volume is mounted.
LL0192.book Page 35 Thursday, July 29, 2004 5:20 PM Step 10: Add Storage Pools to the Volumes 1 Select a volume in the list and click the New Storage Pool button (next to the list). 2 In the storage pool dialog, provide the following information. Storage Pool Name: Type a name for the storage pool. If the OK button is disabled after you type a name, the name is reserved. For a list of reserved names, see “You Can’t Add a Storage Pool” on page 76.
LL0192.book Page 36 Thursday, July 29, 2004 5:20 PM Step 12: (Optional) Set Up SAN Status Notifications Xsan can notify you by email or pager when the condition of the SAN changes. If you don’t want to set up notifications now, you can do it later. 1 On the Notification pane, click the Add button (+) next to the list to add a contact. 2 Double-click the new entry, type an email address or the address of a pager textmessaging account, and press Return.
LL0192.book Page 37 Thursday, July 29, 2004 5:20 PM Step 14: (Optional) Set User and Group Quotas You can set up quotas to control how much space in each SAN volume is used by each user or group. 1 If the volume is not already started, select it in the SAN Components list and click Start Volume. 2 With the volume still selected, click Quotas. 3 Click the Add (+) button, then drag a user or group from the drawer to the quotas list.
LL0192.book Page 38 Thursday, July 29, 2004 5:20 PM Renaming a SAN SANs are listed by name in the SAN Components list in Xsan Admin. By default, a SAN is named for the controller you connect to when you set up the SAN. You can change this name using Xsan Admin. To change the name of a SAN: 1 Open Xsan Admin (in /Applications/Server). 2 Select the SAN in the SAN Components list. If the SAN is not listed under SAN Components, click Connect and log in to the SAN’s controller.
LL0192.book Page 39 Thursday, July 29, 2004 5:20 PM Setting Up Additional SANs You can use Xsan Admin to set up and manage more than one SAN. To add a new SAN, you connect computers and storage devices to Fibre Channel and Ethernet networks, identify computers that will act as controllers or clients, organize available storage by combining Xserve RAID arrays and slices (LUNs) into storage pools and storage pools into volumes, and mount the resulting volumes on client computers.
LL0192.
LL0192.book Page 41 Thursday, July 29, 2004 5:20 PM 3 Managing SAN Storage 3 This chapter shows how to expand, modify, check, and repair SAN storage.
LL0192.book Page 42 Thursday, July 29, 2004 5:20 PM Adding Storage There are several ways you can add storage to a SAN: • Add RAID arrays (LUNs) to existing storage pools • Add storage pools to existing volumes • Add new volumes The first method requires you to unmount and remount the volume on clients. The second method lets you increase the available space in an existing SAN volume without interrupting users who are using the volume.
LL0192.book Page 43 Thursday, July 29, 2004 5:20 PM Adding LUNs to a Storage Pool You can increase the capacity of a SAN volume by adding LUNs (RAID arrays or array slices) to a storage pool that belongs to the volume. Choosing Compatible LUNs Add LUNs that are similar in performance and capacity to the LUNs already in the storage pool. Mixing LUNs of different sizes or speeds in the same storage pool can degrade performance and waste capacity.
LL0192.book Page 44 Thursday, July 29, 2004 5:20 PM From the Command Line You can also add LUNs to a storage pool by modifying the associated volume configuration file and using the cvlabel command in Terminal. For more information, see the cvfs_config and cvlabel man pages or “The Volume Configuration File” on page 96 and “Labeling LUNs (cvlabel)” on page 90. Adding a Storage Pool to a Volume You can add free space to a SAN volume by adding a storage pool to the volume.
LL0192.book Page 45 Thursday, July 29, 2004 5:20 PM Adding a Volume to a SAN A single Xsan SAN can provide multiple shared volumes. 1 Open Xsan Admin and select the SAN in the SAN Components list. 2 Click Setup, then click Storage. 3 Click the Add Volume button next to the list. 4 In the dialog that appears, provide the following information, then click OK. Volume Name: Type a name for the volume. Use only uppercase letters (A-Z), lowercase letters (a-z), and numbers (0-9).
LL0192.book Page 46 Thursday, July 29, 2004 5:20 PM From the Command Line You can also add a volume by setting up a configuration file. For more information, see the cvfs_config man page or “The Volume Configuration File” on page 96. Assigning a Storage Pool Affinity to a Folder You can set up an affinity between a folder and a storage pool so that files in the folder are stored only on the specified storage pool.
LL0192.book Page 47 Thursday, July 29, 2004 5:20 PM Assigning an Affinity to a Folder Within a Folder You can use Xsan Admin to assign an affinity to a folder at the top level of a volume, but to assign an affinity to a folder that is inside another folder you need to use the cvmkdir command-line tool. To assign a storage pool affinity to a folder within a folder: 1 Open Terminal (in /Applications/Utilities).
LL0192.book Page 48 Thursday, July 29, 2004 5:20 PM Changing Storage Pool Settings The best way to set up a SAN is to plan its organization carefully before you set it up, including settings for the storage pools that make up its volumes. You can change some storage pool settings in an existing SAN, but other changes require the associated volume to be re-initialized.
LL0192.book Page 49 Thursday, July 29, 2004 5:20 PM Setting Storage Pool Access Permissions You can set up storage pools for read-write or read-only access. Changing this setting doesn’t damage existing data on the volume. To set storage pool access permissions: 1 In Xsan Admin, select the SAN the storage pool belongs to, click Setup, and click Storage. 2 Double-click the storage pool in the list. 3 Choose Read & Write or Read Only from the Permissions pop-up menu. 4 Click OK, then click Save.
LL0192.book Page 50 Thursday, July 29, 2004 5:20 PM Setting Storage Pool Stripe Breadth The default stripe breadth value (64 file system blocks) is adequate for storage pools in most SAN volumes. However, you can adjust this value along with the file system block allocation size to tune SAN performance for special applications. For help choosing a stripe breadth, see “Choosing Stripe Breadth for a Storage Pool,” below.
LL0192.book Page 51 Thursday, July 29, 2004 5:20 PM The Fibre Channel interface in an Xserve RAID caches up to 1 MB of data before passing the data on to the RAID controller to be written to individual drive modules. So, to minimize the time data spends waiting in the cache, you want to send data to the Xserve RAID 1 MB at a time.
LL0192.book Page 52 Thursday, July 29, 2004 5:20 PM Changing Volume Settings You can change some settings for an existing volume, but other changes require the volume to be re-initialized (you’ll see a warning for these cases). Renaming a Volume You can use Xsan Admin to change the volume name users see when the volume is mounted on their computers. Warning: Renaming a volume causes all data on the volume to be lost. To rename a volume: 1 Unmount the volume from clients.
LL0192.book Page 53 Thursday, July 29, 2004 5:20 PM Setting the Volume Allocation Strategy You can set the allocation strategy for a volume to determine the order in which data is written to the volume’s storage pools. Warning: When you change a volume’s allocation strategy, all data on the volume and its storage pools is lost. To set the allocation strategy: 1 In Xsan Admin, select the SAN the volume belongs to, click Setup, and click Storage. 2 Double-click the volume in the list.
LL0192.book Page 54 Thursday, July 29, 2004 5:20 PM Setting the Block Allocation Size The default file system block allocation size (4KB) is adequate for most volumes. However, you can adjust this value along with the stripe breadth of the volume’s storage pools to tune performance for special applications. For help choosing a block allocation size, see “Choosing Block Allocation Size for a Volume,” below. Warning: When you change a volume’s block allocation size, all data on the volume is lost.
LL0192.book Page 55 Thursday, July 29, 2004 5:20 PM Defragmenting a Volume When you create a file, Xsan breaks the file into pieces and distributes these pieces efficiently over the LUNs that make up one of the volume’s storage pools. Over time, as the file is modified, its pieces can become scattered in less efficient arrangements. Defragmenting the file reassembles its pieces into the most efficient arrangement.
LL0192.book Page 56 Thursday, July 29, 2004 5:20 PM Checking the Integrity of a Volume If SAN users are having trouble accessing files, you can use the cvfsck command to check the integrity of a volume, its metadata, and files. To check a volume: 1 Open Terminal (in /Applications/Utilities).
LL0192.book Page 57 Thursday, July 29, 2004 5:20 PM 4 Managing Clients and Users 4 This chapter shows how to add, control, and remove client computers and their users. Xsan clients are computers that have access to a SAN. Xsan volumes are mounted on client computers without regard to which user is logged in to the computer.
LL0192.book Page 58 Thursday, July 29, 2004 5:20 PM Adding a Client Before a computer can access a SAN volume, you need to set up that computer as a SAN client. To add a client computer to a SAN: 1 If you haven’t already, connect the client computer to the SAN’s Fibre Channel and Ethernet networks and install the Xsan software. 2 Open Xsan Admin, select the SAN in the SAN Components list, and click Setup. 3 Click Computers, select the new computer in the list, and click Edit.
LL0192.book Page 59 Thursday, July 29, 2004 5:20 PM Mounting a Volume on a Client A user who is logged in to a client computer can’t mount a SAN volume. A SAN administrator must use Xsan Admin to mount the volume on the client computer. To mount an Xsan volume on a client: 1 Open Xsan Admin, select the volume in the SAN Components list, and click Clients. 2 Select the client computer in the list. 3 Click Mount Read & Write.
LL0192.book Page 60 Thursday, July 29, 2004 5:20 PM Unmounting a Volume From a Client To prevent a client from accessing a volume, you can unmount the volume from the client. Clients can’t mount SAN volumes themselves; only an administrator can mount a SAN volume on a client. To unmount a volume: 1 Open Xsan Admin, select the volume, and click Clients. 2 Select the client computer in the list and click Unmount.
LL0192.book Page 61 Thursday, July 29, 2004 5:20 PM Setting User and Group Quotas You can set quotas to manage the amount of storage available to a user or group. To set storage quotas for users and groups: 1 If the volume is not already started, start it. Open Xsan Admin, select the volume, and click Start Volume. 2 In Xsan Admin, select the volume and click Quotas. 3 If the user or group is not in the list, click the Add (+) button next to the list and drag users or groups from the drawer to the list.
LL0192.book Page 62 Thursday, July 29, 2004 5:20 PM Soft Quotas Change to Hard Quotas If a user or group exceeds their soft quota for a time longer than their grace period, the soft quota is changed to a hard quota, and they will not be able to save additional data on the volume until they delete old files and return below the soft quota. Example Suppose you assign Aldo a soft quota of 75 GB, a hard quota of 100 GB, and a grace period of 48 hours.
LL0192.book Page 63 Thursday, July 29, 2004 5:20 PM 5 Managing Metadata Controllers 5 This chapter shows how to add, switch, monitor, and upgrade Xsan metadata controllers. Every SAN volume you set up is managed by a metadata controller. To be sure that the volume is available to clients even if the active controller becomes unresponsive, you can set up standby controllers, one of which will assume control of the volume if the primary controller fails.
LL0192.book Page 64 Thursday, July 29, 2004 5:20 PM Adding a Controller You can add one or more standby controllers to a SAN so that volumes are still available if the primary controller fails. Any computer set to act as a controller can also function as a client. So, if you don’t want to dedicate a computer to act solely as a standby controller, you can use an existing client.
LL0192.book Page 65 Thursday, July 29, 2004 5:20 PM Switching to a Standby Controller You can force an active metadata controller to turn over control of a volume to a standby controller using the cvadmin command or by stopping the file system manager process for the volume. To switch a volume to a standby controller: 1 Open Terminal (in /Applications/Utilities).
LL0192.book Page 66 Thursday, July 29, 2004 5:20 PM Finding Out Which Controller Is Hosting a Volume Control of a particular volume can move from one controller to another as a result of controller failover. You can use Xsan Admin to find out which controller is currently hosting a particular volume. m To view a volume’s controller: Open Xsan Admin, select the volume in the SAN Components list, and click Overview.
LL0192.book Page 67 Thursday, July 29, 2004 5:20 PM Changing a Controller’s IP Address Follow these instructions if you need to change the IP address of an Xsan controller. To change a controller’s IP address: 1 If you have a standby controller in the SAN, switch the active controller’s volumes to the standby controller. For help, see “Switching to a Standby Controller” on page 65.
LL0192.book Page 68 Thursday, July 29, 2004 5:20 PM Upgrading Controller Software If your configuration includes a standby controller, you can upgrade the Xsan software without interrupting the SAN. Xsan controller software is always compatible with the preceding version of the client software (controllers can be one version ahead of clients). So, you can upgrade your controllers first and your client computers will continue to work until it is convenient to upgrade them to the same version.
LL0192.book Page 69 Thursday, July 29, 2004 5:20 PM 6 Monitoring SAN Status 6 This chapter shows how to check the condition of a SAN and its components. This chapter shows how you can check on or automatically report the condition of a SAN and its components.
LL0192.book Page 70 Thursday, July 29, 2004 5:20 PM Checking Free Space on a SAN Volume There are several ways to see how much space is available on a SAN volume. m m To see how much free space is available on a volume: If you’re using a computer on which the volume is mounted, select the volume in a Finder window and look at the size information at the bottom of the window (in column or list view) or choose File > Get Info.
LL0192.book Page 71 Thursday, July 29, 2004 5:20 PM Checking User Quota Use You can use Xsan Admin to check file system quotas to see how much of their allotment a user or group is using. m To view quota utilization: Open Xsan Admin, select a volume in the SAN Components list, and click Quotas. To be sure you’re seeing the latest information, click Refresh at the top of the window. Xsan Admin lists the following information for each user or group: Quota: The soft and hard quotas.
LL0192.book Page 72 Thursday, July 29, 2004 5:20 PM Setting Up Status Notifications Xsan can send an email or dial a pager to notify you or other administrators when: • A server or Fibre Channel connection fails • Free space on a volume falls below a specific percentage • A user or group exceeds a specific percentage of their hard quota To send email notifications outside the local network, the controller needs access to an SMTP server.
LL0192.book Page 73 Thursday, July 29, 2004 5:20 PM Viewing Logs You can use Xsan Admin to view diagnostic and informational messages that Xsan has written to a computer’s system and console logs. m m To view the system logs: Open Xsan Admin, select a SAN in the SAN Components list, and click Logs. Use the Show pop-up menu to change logs and the On pop-up menu to switch to another computer. To find entries containing specific text, type the text in the Filter field and press Return.
LL0192.book Page 74 Thursday, July 29, 2004 5:20 PM Checking for Fibre Channel Connection Failures Fibre Channel connection failures or errors are recorded in the system log. m To view the system log: Open Xsan Admin, select a SAN in the SAN Components list, and click Logs. Choose System Log from the Show pop-up menu and the computer you’re interested in from the On pop-up menu.
LL0192.book Page 75 Thursday, July 29, 2004 5:20 PM 7 Solving SAN Problems 7 This chapter lists solutions to common problems you might encounter while working with a SAN. Look here for solutions to common problems you might encounter while setting up, managing, or using an Xsan SAN. You Can’t Install the Xsan Software If the installer says you can’t install the Xsan software on a particular computer, make sure: • The computer has Mac OS X v10.3 version 10.3.5 or Mac OS X Server v10.3 version 10.3.
LL0192.book Page 76 Thursday, July 29, 2004 5:20 PM You Can’t Add a Storage Pool There are some reserved names you can’t use to name a storage pool. If you type one of these names, the OK button in the storage pool dialog is disabled. Reserved names are listed in the following table.
LL0192.book Page 77 Thursday, July 29, 2004 5:20 PM After Slicing, Some LUNs Aren’t Listed in Xsan Admin • If you slice an array that was previously labeled for use with Xsan, you might need to remove the old label from the first slice. To see if the LUN is mounted using its old label, open Terminal and type: $ sudo cvlabel -l -s (The cvlabel tool is in /Library/Filesystems/xsan/bin.) This will show the old label. In the following sample output, the label is sanvol1. /dev/rdisk4/ [APPLE Xserve RAID 1.
LL0192.
A Combining Xsan and StorNext Clients and Controllers A Appendix LL0192.book Page 79 Thursday, July 29, 2004 5:20 PM This appendix shows how to add Macintosh clients to an existing StorNext File System SAN or connect Windows, Solaris, AIX, Irix, or Linux clients to an Xsan SAN.
LL0192.book Page 80 Thursday, July 29, 2004 5:20 PM Adding Macintosh Clients to a StorNext SAN If you already have a StorNext File System SAN, you can add a Macintosh client using Xsan. To add a Macintosh Xsan client to a StorNext SAN: 1 Connect the Macintosh computer to the SAN’s Ethernet and Fibre Channel networks. 2 Install the Xsan software on the Macintosh computer. 3 License the Xsan software on the Macintosh client.
LL0192.book Page 81 Thursday, July 29, 2004 5:20 PM Using Xsan Controllers With StorNext Clients You can use ADIC’s StorNext software to access an Xsan SAN from a Windows, UNIX, AIX, Irix, or Linux computer. 1 Connect the non-Macintosh client to the SAN’s Fibre Channel and Ethernet networks. 2 Install the StorNext File System software on the non-Macintosh client following the instructions that ADIC provides in the StorNext package.
LL0192.
B Using the Command Line B Appendix LL0192.book Page 83 Thursday, July 29, 2004 5:20 PM This appendix describes Xsan shell commands and configuration files you can use to work with a SAN from the command line. You can use the shell commands and configuration files described here to access, set up, and manage Xsan SANs, LUNs, storage pools, and volumes from the command line.
LL0192.book Page 84 Thursday, July 29, 2004 5:20 PM If you get the response “No manual entry for...” when you try to view a man page, add the Xsan man pages directory /Library/Filesystems/Xsan/man to your man search path. One way to do this is to add the following line to the file /etc/manpath.config on your computer: OPTIONAL_MANPATH /Library/Filesystems/Xsan/man Notation Conventions The following conventions are used throughout the command descriptions.
LL0192.book Page 85 Thursday, July 29, 2004 5:20 PM Viewing or Changing Volume and Storage Pool Settings (cvadmin) You can use the cvadmin tool to perform a variety of information and setup tasks related to Xsan volumes.
LL0192.book Page 86 Thursday, July 29, 2004 5:20 PM cvadmin Commands 86 cvadmin command Description activate [volume|index] Choose the “active” volume that you want to work with interactively. volume – the name of the volume index – the numeric ID of the volume (to see a list of these, use the cvadmin select command without any parameters) disable pool [read|write] Prevent read or write access to a storage pool. pool – the name of a storage pool in the currently active volume.
LL0192.book Page 87 Thursday, July 29, 2004 5:20 PM cvadmin command Description repquota Generate the following quota report files in /Library/Filesystems/Xsan/data/: quota_report.txt – text file quota_report.csv – comma-delimited file quota_regen.in – cvadmin commands that will set up identical quotas on another controller. You can use cvadmin -f to execute the commands.
LL0192.book Page 88 Thursday, July 29, 2004 5:20 PM Copying Files or Folders (cvcp) You can use the cvcp command to perform high-speed file copies to or from an Xsan volume. You can use this command to: • Copy files or directories • Copy tar-formatted data to a directory • Copy a file or directory to a tar-formatted data stream $ cvcp [options] source destination Parameter Description options See “cvcp Command Options,” below. source The file or folder (directory) to be copied.
LL0192.book Page 89 Thursday, July 29, 2004 5:20 PM Perform a similar copy as above, but only copy files with names that begin “jul”: $ cvcp -vxy -p jul data1 /datasets/data1/july Copy directory /datasets to a tar file named /transfers/data.tar: $ cvcp . - > /transfers/data Checking or Repairing a Volume (cvfsck) You can use the cvfsck command to check or repair an Xsan volume. $ cvfsck [options] volume Parameter Description options See “cvfsck Command Options,” below.
LL0192.book Page 90 Thursday, July 29, 2004 5:20 PM Labeling LUNs (cvlabel) You can use the cvlabel command to initialize LUNs so they can be added to storage pools. To list available LUNs: $ cvlabel -l [-s] [-v] To prepare label template file: $ cvlabel -c To label a LUN: $ cvlabel [-v] [-f] [labelfile] To remove the existing label from a LUN: $ cvlabel -u lun 90 Parameter Description -l List available LUNs. -s Display device serial numbers. -v Show progress display.
LL0192.book Page 91 Thursday, July 29, 2004 5:20 PM Creating a Folder With an Affinity (cvmkdir) You can use the cvmkdir command to create a folder with an affinity for a particular storage pool in an Xsan volume. $ cvmkdir [-k affinity] directory Parameter Description -k Optional parameter for specifying the name of the storage pool to be associated with the directory (folder). If you omit this parameter, Xsan creates a folder without an affinity. affinity The name of a storage pool’s affinity key.
LL0192.book Page 92 Thursday, July 29, 2004 5:20 PM Initializing a Volume (cvmkfs) You can use the cvmkfs command to initialize an Xsan volume based on the information in the corresponding configuration file for the volume (in /Library/Filesystems/Xsan/config/.cfg). Warning: Initializing a volume destroys all existing data on the volume. $ cvmkfs [-G] [-F] [volume] Parameter Description -G Don’t display “Press return to continue” prompts. -F Don’t display warning and verification prompts.
LL0192.book Page 93 Thursday, July 29, 2004 5:20 PM Starting a Volume Controller (fsm) You can use the fsm command to start a file system manager process on a controller. The fsm process manages the volume’s name space, file allocations, and metadata. $ fsm [volume] [controller] Parameter Description volume The volume that the process will manage. controller The computer on which the process is started, and which therefore becomes the volume’s metadata controller.
LL0192.book Page 94 Thursday, July 29, 2004 5:20 PM Parameter Description -K affinity Only operate on files with the specified storage pool affinity. affinity – the affinity key (in Xsan, the affinity key is the same as the name of the storage pool) You can use the cvadmin show long command to see a storage pool’s affinity key. -k affinity Allocate new extents on the storage pool with this affinity. -l List files that might benefit from defragmentation.
LL0192.book Page 95 Thursday, July 29, 2004 5:20 PM Mounting an Xsan Volume You can use the mount command to mount an Xsan volume on a computer. 1 Either go to the computer and open Terminal, or use SSH to log in to the computer remotely: $ ssh user@computer where user is a user account on the remote computer and computer is its IP address or DNS name.
LL0192.book Page 96 Thursday, July 29, 2004 5:20 PM The Configuration Files Xsan stores its configuration information in the following files: File Contents /Library/Filesystems/Xsan/config/.cfg Volume settings /Library/Filesystems/Xsan/config/fsmlist Volume auto-start list /Library/Filesystems/Xsan/config/fsnameservers Controller list The Volume Configuration File The parameters that describe an Xsan volume are stored in the file /Library/Filesystems/Xsan/config/.
LL0192.book Page 97 Thursday, July 29, 2004 5:20 PM # # The maximum number in file system blocks a file should expand by. # InodeExpandMax 2048 # 8M per expansion (big!) # # Align the file system to start on a stripe breadth boundary # ForceStripeAlignment Yes # # Debug flags # # See the debug flags legend in cvadmin (1M) # Debug 0 # # File system block size # # - Must be power of two in the range of 4K to 512K.
LL0192.book Page 98 Thursday, July 29, 2004 5:20 PM # # Maximum log file size # MaxLogSize 1M # # Client Pool Threads. Number of threads in the message handling pool. # ThreadPoolSize 16 #*************************************************************************** # A disktype section for defining disk hardware parameters.
LL0192.
LL0192.book Page 100 Thursday, July 29, 2004 5:20 PM # # Set up a stripe group for exclusive 8-bit NTSC 525 video use.
LL0192.book Page 101 Thursday, July 29, 2004 5:20 PM [StripeGroup AudioGroup2] Status UP Exclusive Yes Affinity 6100_p8a Read Enabled Write Enabled StripeBreadth 15 Node CvfsDisk11 0 # # Set up a 2 drive stripe group with no exclusive properties.
LL0192.book Page 102 Thursday, July 29, 2004 5:20 PM The Volume Auto-Start List You can use the fsmlist configuration file to control which file system processes are automatically started. For more information, see the fsmlist man page. Example Auto-Start File # # # # # # # # # # # # # # # # # # # # # Autostart file List the primary and backup Stornext File System Services that should be started at boot time.
LL0192.book Page 103 Thursday, July 29, 2004 5:20 PM # # # Below are some example file system service configuration lines. # # Example #1. # # This file system is named "default". The host name defaults to the same # one the fsm port mapper uses. It is the only service for this file # system. # #default # # Example #2. # # This file system service is named "example" and uses an alternate path # host name on host name "rock". # #example rock # # Example #3.
LL0192.book Page 104 Thursday, July 29, 2004 5:20 PM The Controller List Xsan stores a list of SAN controllers in the file /Library/Filesystems/Xsan/config/fsnameservers A copy of this file appears on every computer that belongs to the SAN. For information, see the fsnameservers man page.
LL0192.
LL0192.
Glossary Glossary LL0192.book Page 107 Thursday, July 29, 2004 5:20 PM ACFS Apple Cluster File System. The underlying Xsan file system. ACFS is a shared SAN file system that uses a controller to manage file system metadata. Derived from ADIC’s StorNext File System. affinity An association between a folder on an Xsan volume and one of the storage pools that make up the volume. The affinity guarantees that files placed in the folder are stored only on that storage pool.
LL0192.book Page 108 Thursday, July 29, 2004 5:20 PM fill An Xsan storage pool allocation strategy. In a volume consisting of more than one storage pool, Xsan fills up the first pool before writing to the next. format (verb) See “initialize.” FSS File system server. The StorNext File System term for an Xsan metadata controller. initialize In general, to prepare a disk for use by a particular file system. In Xsan, to prepare a LUN for use in a storage pool. label (noun) An identifying name for a LUN.
LL0192.book Page 109 Thursday, July 29, 2004 5:20 PM storage pool A group of logical disks that share common characteristics, such as throughput or latency, across which Xsan stripes user data. Storage pools are combined to form Xsan volumes. stripe breadth An Xsan storage pool property. The number of bytes of data, expressed as a number of file system blocks, that Xsan writes to a LUN in a storage pool before moving to the next LUN in the pool.
LL0192.
LL0192.book Page 111 Thursday, July 29, 2004 5:20 PM “command not found” message 77 “No manual entry for...
LL0192.book Page 112 Thursday, July 29, 2004 5:20 PM free space checking quota use 71 checking storage pool 70 checking volume 70 fsm command 93 fsmpm command 93 fsnameservers file 80, 104 FSS. See controllers G GID (group ID) assigning 30 grace period (quota) 61 graphs controller utilization 71 smoothing 28 group ID.
LL0192.
