Xsan Deployment and Tuning Guide Guidelines and examples for configuring Xsan storage area network volumes
K Apple Computer, Inc. © 2005 Apple Computer, Inc. All rights reserved. Under the copyright laws, this manual may not be copied, in whole or in part, without the written consent of Apple. The Apple logo is a trademark of Apple Computer, Inc., registered in the U.S. and other countries. Use of the “keyboard” Apple logo (Option-Shift-K) for commercial purposes without the prior written consent of Apple may constitute trademark infringement and unfair competition in violation of federal and state laws.
2 Preface 5 5 6 6 Contents About This Guide Using This Guide For More Information Notation Conventions Chapter 1 7 7 8 8 8 8 8 9 9 9 10 10 10 11 11 12 13 13 13 15 Setup and Tuning Guidelines Setting Up the Ethernet TCP/IP Network Set Up a Private Metadata Network Use Switches Instead of Hubs Setting Up the Fibre Channel Network Verify Base Fibre Channel Performance If Your Fibre Channel Fabric Is Running Slower Than Expected Configuring Xserve RAID Systems Install the Latest Firmware Connecting Xserv
Chapter 3 4 19 20 23 23 24 About the Test Files Performing a Test Interpreting Test Results Understanding UNIX Test Results Understanding Final Cut Pro Test Results 27 27 28 29 29 32 32 32 34 35 36 37 Deployment Examples Which Example Should You Follow? Example 1: Network Attached Storage Objectives of This Configuration Deployment Decisions Example 2: Storage for Computational Clusters Requirements for This Example Small Computational Cluster Setup A Larger Computational Cluster Volume Configuration E
Preface About This Guide This guide shows you how to choose the best Xsan setup and configuration options for your users and applications. The performance and availability of an Xsan volume depend on a variety of parameters, including how you organize available storage, how you configure the SAN’s Fibre Channel and Ethernet networks, and how you set basic file system parameters such as block size and stripe breadth.
For More Information The Xsan Administrator’s Guide contains basic instructions for setting up Xsan volumes along with information about managing Xsan volumes, including problem-solving tips and command-line alternatives for common tasks. You can find the guide:  On the Xsan Installer disc  In the folder /Library/Documentation/Xsan on any computer where Xsan is installed  At www.apple.com/server/documentation You can also check the Xsan web pages at www.apple.com/xsan.
1 Setup and Tuning Guidelines 1 This chapter offers guidelines for making configuration choices that can affect SAN performance.
Set Up a Private Metadata Network Non-SAN-related Ethernet traffic can interfere with the exchange of metadata among Xsan controllers and clients. For example, using the same connection for both Xsan metadata exchange and Internet access can slow file system performance. Similarly, using the same Ethernet network to connect client computers to directory services and SAN metadata can affect SAN performance.
Use Qualified Transceivers in Matching Pairs Check with the manufacturers of the devices you are connecting to your fabric to be sure that the transceivers (GBICs) you are using are qualified for use with their devices. Also, use identical transceivers (same manufacturer and model number) on both ends of each cable. Mismatched optical transceivers (even if they are both separately qualified for use with your devices) can cause Fibre Channel communication errors and degrade SAN performance.
Choosing RAID Levels for LUNs Use RAID 1 for metadata LUNs and RAID 5 for data LUNs. Use RAID 1 for Metadata LUNs RAID 1 (mirroring) can give slightly better performance than the default RAID 5 scheme for the small, two-drive metadata LUNs that Xsan uses to store volume information. A single drive is almost always adequate for storing the primary volume metadata (10 GB of metadata space is enough for approximately 10 million files). The second, mirror drive protects you against metadata loss.
Adjusting Xserve RAID Fibre Channel Settings There are several Xserve RAID settings that can affect the Fibre Channel performance of the device and the SAN as a whole. Fibre Channel Speed Be sure the Fibre Channel connection is set to operate at 2 GB/s. Fibre Channel Topology To add an Xserve RAID system to a Fibre Channel fabric, set the topology to Automatic. Disable Hard Loop ID Don’t enable hard loop IDs for Xserve RAID systems in a Fibre Channel fabric.
Although some large write requests might benefit from caching, often they do not. By placing a volume’s metadata storage pool on a controller separate from the data storage pools, you can configure the metadata controller to use caching and the data controller to run without caching. When the file system is relying on caching in this way, you must guarantee that data in the cache isn’t lost before it is actually written to disk.
Configuring the Xsan File System The following paragraphs summarize information you should consider when using Xsan Admin to configure your Xsan volumes.
If, however, the workflow supported by the volume consists mostly of sequential reads or writes, as is the case for audio or video streaming or capture, you can get better performance with a larger block size. Try a 64 KB block size in such cases. Choosing a Storage Pool Stripe Breadth The Mac OS X (or Mac OS X Server) operating system, which handles file data transfers for Xsan, performs 1 megabyte (MB) data transfers.
About SAN Write Latency The waiting period from the time of the write request until the notification that the request has been processed is called “latency,” and is an important measure of volume performance. Xsan constantly monitors file system write latency and records hourly summaries in the cvlog log file. In general, peak metadata throughput begins to suffer when the average latency exceeds 500 microseconds. Xsan writes hourly summaries of metadata write latency into each volume’s log file.
Chapter 1 Setup and Tuning Guidelines
2 Using the Xsan Tuner Application 2 This chapter shows how to use the Xsan Tuner application to test the performance of Xsan volumes. You can use Xsan Tuner to test the data and video transfer capabilities of your storage area network and its Xsan volumes. Xsan Tuner can simulate both standard UNIX reads and writes and Final Cut Pro video reads and writes for a variety of common video formats. Use Xsan Tuner to see if your SAN can handle planned workloads before you put it into production use.
Starting Xsan Tuner Double-click the Xsan Tuner icon to start the application. About the Tests Xsan Tuner can perform four basic types of tests:  Standard UNIX file reads  Standard UNIX file writes  Final Cut Pro video stream reads  Final Cut Pro video stream writes The UNIX Read and Write Tests The UNIX Read and UNIX Write tests perform low-level sequential I/O using the standard BSD read and write calls.
The following table lists the transfer rate that Xsan Tuner tries to sustain for each stream of the indicated video format. Video type Format Minimum data rate per stream Standard Definition MiniDV 3.43 MB/s DVCAM 3.43 MB/s DVCPRO 3.43 MB/s DVCPRO 50 6.87 MB/s Uncompressed SD 8-bit 20.02 MB/s Uncompressed SD 10-bit 26.7 MB/s Compressed High Definition DVCPRO HD 5.49 MB/s Uncompressed High Definition 720p 24 fps 42.19 MB/s 720p 30 fps 52.73 MB/s 720p 60 fps 105.
Performing a Test To test the performance of a SAN volume, you need to:  Prepare the client computers  Choose test type and related settings  Start the test Step 1: Prepare the Client Computer Mount the Xsan volume on the client . . . . . . and install the Xsan Tuner application. 1 Make sure that the Xsan volume you want to test is mounted on the client. If the volume is not mounted, use Xsan Admin to mount it. 2 Copy the Xsan Tuner application to the client you want to test.
Step 2: Choose Test Type and Settings 1 Open Xsan Tuner on the client computer. 2 Choose the volume you want to test from the Volume pop-up menu. Note: You can use Xsan Tuner to test any volume mounted on the client, not just Xsan volumes. 3 To test a specific storage pool in an Xsan volume, select the pool’s affinity name from the Affinity pop-up menu. 4 Choose a test type from the Task pop-up menu. 5 If you have chosen a Final Cut Pro test, choose a video format from the Size pop-up menu.
Step 3: Run the Test 1 When you have selected the volume to test and the test parameters, click Start Test. If you are performing a write test, the test begins immediately. If you are performing a read test, Xsan Tuner first creates readable test files. 2 If this is the first time you’ve requested a read test, wait until Xsan Tuner creates its test files, and then click Start Read Test.
The following screen captures show how the creation of the test file for a read test can influence the results of a separate test already in progress. Creating test files for a read test . . . . . . can have significant effect on another test in progress. Interpreting Test Results Xsan Tuner is designed to estimate SAN performance in terms of the number of streams a client should be able to read or write (for video) or the raw data transfer rate (for UNIX reads and writes).
Understanding Final Cut Pro Test Results The Final Cut Pro Read and Final Cut Pro Write tests performed by Xsan Tuner compare the data rates possible through a specific Final Cut Pro codec against the throughput needed to stream that data without dropping frames. The white line marks the minimum required data rate for the specified streams.
Adjusting the Result for Staggered Streams When you test a client for multiple streams, Xsan Tuner launches all of the streams simultaneously. Xsan Tuner does not simulate staggered streaming. If you configure Final Cut Pro to take advantage of staggered streams for your routing work, you might see a two- or three-stream improvement in actual use over the number of streams reported by Xsan Tuner.
Chapter 2 Using the Xsan Tuner Application
3 Deployment Examples 3 This chapter shows examples of how to set up Xsan volumes in a storage area network. Read this chapter to see how you can use Xsan to set up volumes on a storage area network to provide these different types of storage:  Network attached storage  Computational cluster  Video production Which Example Should You Follow? The three deployment examples address different performance needs.
Example 1: Network Attached Storage Network attached storage (NAS) is commonly used to provide centralized, manageable storage for clients on the Internet or on a private intranet. A typical NAS appliance contains storage devices and a controller that gives clients access to the storage using a network file system such as NFS, AFP, or SMB/CIFS.
In a NAS configuration, the real clients of the storage are not connected to the SAN, but instead connect to the storage server over a TCP/IP network. The storage server itself is the true Xsan client computer and, in the case of a single server computer, is also the SAN controller. The server provides file services using Mac OS X Server AFP, SMB, or NFS services to other computers on the network.
What levels of performance do your users require? The primary constraint on file access speed is the Ethernet network that connects users to the file servers. By comparison, the speed of the SAN and its storage devices is not an issue. However, if you host many users, the performance of the file servers is also important. So, this NAS setup includes multiple servers. How important is availability? Users expect access to their files at all times, so high availability is critical.
Which clients should be able to access each volume? The real SAN clients in this example are the file servers that users connect to. Each of the servers should have access to the SAN volume for maximum availability. Which computers will act as controllers? To avoid dedicated controllers, one of the file servers will act as the metadata controller and a second file server will be configured as a standby controller.
Example 2: Storage for Computational Clusters You can also use Xsan with Xserve and Xserve RAID systems to provide manageable storage for a high performance computational cluster.
Planning Decisions The following paragraphs show how the planning questions in Chapter 2 are answered for this small computational cluster example. How much storage is needed? Three Xserve RAID systems, one for metadata and the 1.8 TB of input data and two others for up to 8 TB of results, are adequate.
Which LUNs go in each storage pool? For metadata, we use a single 2-drive RAID 1 array. For the data pools, we can use the RAID 5 LUNs available straight out of the box on the Xserve RAID systems: one on the system that also supports the metadata and two on each of the other Xserve RAID systems for a total of 5.
To handle more than 64 devices, you can set up some of the processors to act as “head nodes” that are connected to the SAN and reshare the SAN data with the other processors over Ethernet. Ethernet - TCP/IP Metadata controller Xserve RAID storage Standby controller Fibre Channel switch Head nodes Computational nodes Volume Configuration In this example, input data and output data are stored on separate volumes to increase performance.
Example 3: Storage for Video or Film Production Group This example shows how to use Xsan to set up a storage area network to support a group of video editors.
The example is based on these assumptions: Â 16 editing workstations working with up to 4 streams of DVCPRO 50 Â 1 editing workstation working with 2 streams of 8-bit 1080i Deployment Decisions The following paragraphs answer the planning questions listed in the Xsan Administrator’s Guide in a way that satisfies the objectives of the video storage deployment example. How much storage is needed? In this example we are dealing with video and want to avoid dropped frames.
What levels of performance do your users require? Performance is critical in this example. The primary concern is avoiding dropped video frames. We were guided by this requirement when we selected the number of Xserve RAID systems for the configuration.
What allocation strategy? Data in each volume is stored in a single storage pool. Because there is only one user data storage pool, allocation strategy is not an issue, so we’ll accept the default allocation strategy: Round Robin. What block size and stripe breadth should we use? Because video streams involve mostly sequential reads and writes, the video storage pools can benefit from a larger file system block size of 64 KB together with a 16-block stripe breadth.
