Maximizing File Transfer Performance Using 10Gb Ethernet and Virtualization

ManualsBrandsIntel ManualsOtherIntel 10 Gigabit XF SR Server Adapter

Maximizing File Transfer Performance

Using 10Gb Ethernet and Virtualization

A FedEx Case Study

THE CHALLENGE

Spiraling data center network complexity, cable maintenance and troubleshooting costs,

and increasing bandwidth requirements led FedEx Corporation to investigate techniques



In response to this challenge, FedEx—in collaboration with Intel—conducted a case

study to determine the most effective approach to achieving near-native 10-gigabit





were equipped with Intel® 10 Gigabit AF DA Dual Port Server Adapters supporting





File transfer applications are widely used in production environments, including















The latest 10G solutions let users cost-effectively consolidate the many Ethernet and







THE PROCESS

In the process of building a new data center, FedEx Corporation, the largest express













CASE STUDY

Intel® Xeon® processor 5500 series

Virtualization

FedEx Corporation is a

worldwide information and

business solution company, with a

superior portfolio of transportation

and delivery services.

Summary of content (12 pages)

PAGE 1
case study Intel® Xeon® processor 5500 series Virtualization Maximizing File Transfer Performance Using 10Gb Ethernet and Virtualization A FedEx Case Study the Challenge Spiraling data center network complexity, cable maintenance and troubleshooting costs, and increasing bandwidth requirements led FedEx Corporation to investigate techniques for simplifying the network infrastructure and boosting file transfer throughput.
PAGE 2
Cost Factors How does 10G affect costs? Both 10GBASE-T and Direct Attach Twinax cabling (sometimes referred to as 10GSFP+Cu or SFP+ Direct Attach cost less than USD 400 per adapter port. In comparison, a 10G Short Reach (10GBASESR) fiber connection costs approximately USD 700 per adapter port.1 In existing VMware production environments, FedEx had used eight 1GbE connections implemented with two quad-port cards (plus one or two 100/1000 ports for management) in addition to two 4Gb FibreChannel links.
PAGE 3
destination Server Source Server bbcp scp rsync File Transfer Direction bbcp scp rsync netperf netperf SSH SSH hpnssh rhel 5.3 64-bit Intel® Xeon® Processor X5500 Series Intel Oplin 10GbE CX4 Directly connected back-to-back hpnssh rhel 5.3 64-bit Intel® Xeon® Processor X5500 Series Hardware Intel® Xeon® processor X5560 series @ 2.
PAGE 4
Synthetic Benchmarks versus Real File Transfer Workloads for Native Linux* PCIe slots can present a problem if the physical slot size does not match the actual connection to the chipset. To determine which slots are capable of full PCIe width and performance, check with the system vendor. The proper connection width can also be verified using system tools and log files. PCIe Gen1 x8 (or PCIe Gen2 x4, if supported) is necessary to achieve 10 Gbps throughput for one 10G port.
PAGE 5
According to a presentation created by SLAC titled “P2P Data Copy Program bbcp” (http://www.slac.stanford.edu/ grp/scs/paper/chep01-7-018.pdf), bbcp encrypts sensitive passwords and control information but does not encrypt the bulk data transfer. This design tradeoff sacrifices privacy for speed. Even without encrypting the bulk data, this can still be an effective trade-off for many environments where data privacy is not a critical concern.
PAGE 6
transfer throughput in Linux. These tradeoffs may or may not be applicable for a given network environment, but they do indicate an effective technique for gaining better performance with fewer trade-offs in the future. Intel is continuing to work with the Linux community to find solutions to increase file transfer performance.
PAGE 7
Most of the tools and utilities—including ssh and scp—are single threaded; the rsync utility is dual threaded. Using the HPN-SSH layer to replace the OpenSSH layer helps increase the throughput. In HPN-SSH, the cryptography operations are multi-threaded (four threads), which boosts performance significantly. The singlethreaded MAC layer, however, still creates a bottleneck. When HPN-SSH is run with cryptography disabled, the performance increases, but the benefits of encrypted data transfer are lost.
PAGE 8
Case 2: One Virtual Machine with Eight vCPUs and VMDirectPath Source Server The test bed setup and configuration in this case was similar to that of the previous case except that the 10G was direct assigned to the VM. Figure 10 shows the test configuration for this case. A number of features are not available for VM configured with VMDirectPath, including VMotion*, suspend/resume, record/replay, fault tolerance, high availability, DRS, and so on.
PAGE 9
Case 3: Eight Virtual Machines, Each with One vCPU and 2 GB RAM As shown in Figure 13, the aggregate throughput with netperf across eight VMs reaches the same level of throughput as achieved in the native case. With the file transfer tools, cryptography still imposes performance limitations, but the multi-threaded cryptography in HPN-SSH improves performance when compared to the standard utilities, such as ssh.
PAGE 10
Sub Case: Eight Virtual Machines, with and without VMDq The results of the first four copy operations in Figure 14 show that there is no difference in throughput, regardless of whether VMDq is enabled or disabled. This is primarily because of the limitations imposed by bulk cryptography operations. When running the file transfer applications tools by disabling the cryptography—as in last three cases— the advantages of the VMDq feature become clear.
PAGE 11
The Results This case study and the results obtained through the collaborative efforts of FedEx and Intel engineers suggest a number of ways in which file transfer performance in the data center can be maximized, depending on the tools and virtualization techniques used, as well as the hardware configuration. The most important performance-related considerations, based on the observations and data results obtained during testing, are as follows: • Be aware of potential PCIe slot hardware issues.
PAGE 12
For More Information Authors For answers to questions about server performance, data center architecture, or application optimization, check out the resources on The Server Room site: http://communities.intel.com/community/ openportit/server. Intel experts are available to help you improve performance within your network infrastructure and achieve your data center goals. Chris Greer, chief engineer, FedEx Services Chris Greer is a chief engineer of technical architecture at FedEx Services.