IBM Systems Group April 2005 AIX AK April 2005 Forschungszentrum Karlsruhe AIX-AK FZK 2005 © 2005 IBM Corporation
IBM Systems Group IBM Storage with Linux 2005 Alexander Warmuth ATS EMEA Storage AIX-AK FZK 2005 © 2005 IBM Corporation
IBM Systems Group Topics What is supported The Linux SCSI subsystem Linux Kernel version 2.
IBM Systems Group What Is Supported What is supported The Linux SCSI subsystem Linux Kernel version 2.
IBM Systems Group IBM Storage Support for Linux Enterprise Storage Server DS4000 Storage Servers Linear Tape Open Network Attached Storage Enterprise Tape Storage Virtualization AIX-AK FZK 2005 © 2005 IBM Corporation
IBM Systems Group pLinux Support Disk ESS: SLES8, SLES9, RH-EL 3 SDD available JS20, p5 and OP: SLES9, RH-EL 3 Remote boot supported DS6000, DS8000: SLES8, SLES9, RH-EL 3 SDD available JS20: SLES8, SLES9, RH-EL 3 p5 and OP: RH-EL 3 only Remote boot supported DS4000: SLES8, SLES9, RH-EL 3 Emulex Multipulse driver for multipathing SLES 9 single path only Remote boot with JS20, others require RPQ AIX-AK FZK 2005 © 2005 IBM Corporation
IBM Systems Group pLinux Support Tape LTO: SLES 8, SLES 9, RH-EL 3 Data Path (for 3584) and Media Changer failover supported 359x: SLES 8, SLES 9, RH-EL 3 Data path failover supported for 3592 Parallel SCSI attachment also supported Advanced IBMtape device driver IBMtapeutil AIX-AK FZK 2005 © 2005 IBM Corporation
IBM Systems Group The Linux SCSI Subsystem What is supported The Linux SCSI subsystem Linux Kernel version 2.
IBM Systems Group Linux Device Addressing Everything is a file! brw-rw---- 1 root disk 8, 0 2003-03-14 14:07 /dev/sda brw-rw---- 1 root disk 8, 1 2003-03-14 14:07 /dev/sda1 brw-rw---- 1 root disk 3, 0 2003-03-14 14:07 /dev/hda crw-rw---- 1 root disk 9, 0 2003-03-14 14:07 /dev/st0 crw-rw---- 1 root disk 9, 96 2003-03-14 14:07 /dev/st0a crw-rw---- 1 root disk 9, 32 2003-03-14 14:07 /dev/st0l crw-rw---- 1 root disk 9, 64 2003-03-14 14:07 /dev/st0m AIX-AK FZK 2005 © 2005 IB
IBM Systems Group Design User Space Kernel Space SD disks block device (sd_mod.o) upper level SR cdrom/dvd block device (sr_mod.o) ST tapes char device (st.o) SG pass-through char device (sg.o) IBMTape char device (IBMtape.o) SCSI unifying layer (scsi_mod.o, scsi*.[hc], hosts.[hc], constants.c) mid level SCSI / FC Host Bus Adapter drivers (e.g. qla2300.o) lower level SCSI / FC disks SCSI / FC disks AIX-AK FZK 2005 SCSI / FC disks Pseudo drivers for non SCSI buses (e.g. ide-scsi.
IBM Systems Group Linux Kernel Version 2.6 What is supported The Linux SCSI subsystem Linux Kernel version 2.
IBM Systems Group Storage Changes in Linux Kernel 2.
IBM Systems Group Wellknown Linux SCSI Limitations Limited number of devices Fixed with Kernel Up to 256 SCSI disk devices 2.6 Up to 256 SCSI generic devices Up to 32 tape drives Still there Gaps in LUN sequence Fixed with Kernel Limited “on-the-fly” attachment 2.6 Conditionally Device re-ordering fixed with Kernel 2.
IBM Systems Group Other Problems and Pitfalls Multiple LUN support of RH-EL DS4000 Specific QLogic failover driver configuration Potential LUN thrashing UTM (Access LUN) ESS, DS6000, DS8000 Specific SDD and LVM, ext3 Mounting PPRC targets DS6000 Preferred Path AIX-AK FZK 2005 © 2005 IBM Corporation
IBM Systems Group Large filesystems support AIX-AK FZK 2005 Linux Kernel Limits Max file size: 2 TB (2^41 bytes) Max file system size: 8 ZB (2^73 bytes) © 2005 IBM Corporation
IBM Systems Group Multipathing Scenarios What is supported The Linux SCSI subsystem Linux Kernel version 2.
IBM Systems Group Multipathing Concepts Linux host 1 HBA 0 Linux host 1 HBA 0 Linux host 1 HBA 0 HBA 1 HBA 0 HBA 1 Linux host 2 HBA 1 HBA 0 HBA 1 1 DS8000 2 Linux host 2 HBA 1 HBA 0 HBA 1 1 A Linux host 2 DS4000 B DS6000 2 Disk LUN 0 Disk LUN 0 Disk LUN 0 AIX-AK FZK 2005 © 2005 IBM Corporation
IBM Systems Group LUN Transfer to Alternate Controller DS4000 transfers LUNs to alternate controller Volumes are owned by one controller Volumes can be accessed through both controllers Volume ownership is always transferred to the controller that is used for volume access -> transfer time approx 1 s Two multipathing solutions available QLogic failover driver uses AVT Difficult to configure Potential LUN thrashing RDAC uses inband communication Self configuring Suitab
IBM Systems Group LUN Thrashing Scenario Linux host 1 HBA 0 Linux host 2 HBA 1 A DS4000 HBA 0 HBA 1 B Disk LUN 0 AIX-AK FZK 2005 © 2005 IBM Corporation
IBM Systems Group LUN Thrashing Scenario Linux host 1 HBA 0 Linux host 2 HBA 1 A DS4000 HBA 0 HBA 1 B Disk LUN 0 AIX-AK FZK 2005 © 2005 IBM Corporation
IBM Systems Group LUN Thrashing Scenario Linux host 1 HBA 0 Linux host 2 HBA 1 A DS4000 HBA 0 HBA 1 B Disk LUN 0 AIX-AK FZK 2005 © 2005 IBM Corporation
IBM Systems Group Multipathing with RDAC Must use QLogic non-failover driver Always uses current path (as reported by DS4000) RDAC installation FC HBA driver must be installed and loaded At least one LUN must be assigned and available Must use Host Type LNXCLS - AVT turned off Must update boot loader configuration Must run mppUpdate after each configuration change Updates RDAC configuration files Rebuilds Initial RAMDisk AIX-AK FZK 2005 © 2005 IBM Corporation
IBM Systems Group RDAC Shared Data Scenario Linux host 1 HBA 0 Linux host 2 HBA 1 A DS4000 HBA 0 HBA 1 B Disk LUN 0 AIX-AK FZK 2005 © 2005 IBM Corporation
IBM Systems Group RDAC Shared Data Scenario Linux host 1 HBA 0 Linux host 2 HBA 1 A DS4000 HBA 0 HBA 1 B Disk LUN 0 AIX-AK FZK 2005 © 2005 IBM Corporation
IBM Systems Group RDAC Shared Data Scenario Linux host 1 HBA 0 Linux host 2 HBA 1 A DS4000 HBA 0 HBA 1 B Disk LUN 0 AIX-AK FZK 2005 © 2005 IBM Corporation
IBM Systems Group Preferred Path DS6000 uses concept of preferred path Volumes are owned by one controller Volumes can be accessed through both controllers Data is transferred to and from owning controller to requesting controller internally -> performance penalty SDD knows preferred path automatically Access only through owning controller if possible Dynamic load balancing across ports of preferred controller Other multipathing solutions theoretically possible, but must (still
IBM Systems Group Preferred Path Shared Data Scenario Linux host 1 HBA 0 Linux host 2 HBA 1 1 DS6000 HBA 0 HBA 1 2 Disk LUN 0 AIX-AK FZK 2005 © 2005 IBM Corporation
IBM Systems Group Preferred Path Shared Data Scenario Linux host 1 HBA 0 Linux host 2 HBA 1 1 DS6000 HBA 0 HBA 1 2 Disk LUN 0 AIX-AK FZK 2005 © 2005 IBM Corporation
IBM Systems Group Preferred Path Shared Data Scenario Linux host 1 HBA 0 Linux host 2 HBA 1 A DS6000 HBA 0 HBA 1 B Disk LUN 0 AIX-AK FZK 2005 © 2005 IBM Corporation
IBM Systems Group Preferred Path Shared Data Scenario Linux host 1 HBA 0 Linux host 2 HBA 1 DS6000 HBA 0 HBA 1 2 Disk LUN 0 AIX-AK FZK 2005 © 2005 IBM Corporation
IBM Systems Group Host Ports Independent of Controller ESS and DS8000 have independent host ports Volumes are owned by one controller All host ports can communicate with both controllers Dynamic load balancing across all ports possible AIX-AK FZK 2005 © 2005 IBM Corporation
IBM Systems Group Independent Host Port Shared Data Scenario Linux host 1 HBA 0 Linux host 2 HBA 1 1 DS8000 HBA 0 HBA 1 2 Disk LUN 0 AIX-AK FZK 2005 © 2005 IBM Corporation
IBM Systems Group Independent Host Port Shared Data Scenario Linux host 1 HBA 0 Linux host 2 HBA 1 1 DS8000 HBA 0 HBA 1 2 Disk LUN 0 AIX-AK FZK 2005 © 2005 IBM Corporation
IBM Systems Group Independent Host Port Shared Data Scenario Linux host 1 HBA 0 Linux host 2 HBA 1 DS8000 HBA 0 HBA 1 2 Disk LUN 0 AIX-AK FZK 2005 © 2005 IBM Corporation
IBM Systems Group Tape Specifics What is supported The Linux SCSI subsystem Linux Kernel version 2.
IBM Systems Group IBMtape driver For download as binary rpm package Kernel module IBMtape.
IBM Systems Group IBMtapeUtil For download as source code Exerciser tool Software example Build and install using make Provides -------------------------- General Commands: ----------------------------1. Open a Device 7. Request Sense 2. Close a Device 8. Log Sense Page 3. Inquiry 9. Mode Sense Page 4. Test Unit Ready 10. Switch Tape/Changer Device 5. Reserve Device 11. Create Special Files 6. Release Device 12. Query Driver Version Q.
IBM Systems Group Use LTO Devices Native tools: mt, mtx, IBMtapeutil applications: cpio, tar, taper, afio 3rd party applications All major backup solutions available for Linux Attention: some are only tested with parallel SCSI attachment Check ISV Martrix for LTO AIX-AK FZK 2005 © 2005 IBM Corporation
IBM Systems Group Native Library Management Linux tool for media changers: mtx Media changer is addressed through SCSI generic device /dev/sg0 - internal SCSI disk, not relevant here Linux host HBA SCSI / FC tape SCSI / FC tape /dev/sg1 - 1st SCSI tape drive /dev/sg2 - tape robot (media changer) Robot /dev/sg3 – 2nd SCSI tape drive mtx -f /dev/sg1 inquiry mtx -f /dev/sg2 status mtx -f /dev/sg2 load [ ] AIX-AK FZK 2005 © 2005 IBM Corporation
IBM Systems Group Medium Changer Failover Automatitcally moves robot control to another drive in case of a failure Available for 2582, 3583, 3584 Enabled as an option for IBMtape driver Check the /proc/scsi/IBMchanger file Linux host HBA SCSI / FC tape SCSI / FC tape Robot AIX-AK FZK 2005 © 2005 IBM Corporation
IBM Systems Group Tape and Disk Connected to the Same HBA Possible, but not recommended Use separate switch zone, too One driver for all HBAs! Disk LUN 0 HBA 0 Disk LUN 0 Linux host HBA 0 Linux host HBA 1 Disk LUN 1 HBA 2 SCSI / FC tape SCSI / FC tape HBA 1 Disk LUN 1 SCSI / FC tape SCSI / FC tape Robot Robot AIX-AK FZK 2005 © 2005 IBM Corporation
IBM Systems Group Questions & Discussion More Questions? What are your customers needs? Contact: warmuth@de.ibm.
IBM Systems Group ESS / DS6000 / DS8000 Resources Enterprise Storage Server interoperability matrix Subsystem Device Driver (SDD) Fibre channel host bus adapter firmware and driver level Additional supported configurations ESS host systems attachment guide http://www.storage.ibm.com/disk/ess/ess800/supserver.htm http://www.storage.ibm.com/disk/ds8000/supserver.htm http://www.storage.ibm.com/disk/ds6000/supserver.
IBM Systems Group DS4000 Resources DS4000 Storage interoperability matrix Fibre channel host bus adapter firmware and driver level Additional supported configurations http://www.ibm.com/servers/storage/disk/ds4000/interop-matrix.html DS4000 Technical Support DS4000 Downloads http://www.ibm.
IBM Systems Group LTO Resources LTO Compatibility Information LTO ISV Matrix http://www.storage.ibm.com/tape/lto/compatibility.html LTO Downloads http://www.ibm.com/servers/storage/support/lto/ltodownloads.html ftp://ftp.software.ibm.
IBM Systems Group Redbooks Implementing Linux with IBM Disk Storage http://www.redbooks.ibm.com/redbooks/pdfs/sg246261.pdf Linux with xSeries and FAStT: Essentials http://www.redbooks.ibm.com/redbooks/pdfs/sg247026.pdf Implementing IBM LTO in Linux and Windows http://www.redbooks.ibm.com/redbooks/pdfs/sg246268.pdf Linux Clustering with CSM and GPFS http://www.redbooks.ibm.com/redbooks/pdfs/sg246601.
IBM Systems Group White Papers FAStT and Linux HowTo http://www.ibm.com/developerworks/eserver/articles/install_fibre/index.html FAStT and RH AS Cluster http://www.ibm.com/servers/esdd/articles/redhat/index.html ESS Attachment to United Linux 1 (IA-32) http://www.ibm.com/support/docview.wss?uid=tss1td101235 http://w3.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/TD101235 Addendum to the Solution Assurance Process http://ulrich.walter.de.userv.ibm.com/portal.
IBM Systems Group Legal Notices Both Linux and Storage are rapidly changing environments. This information is presented "as is" without any warranty of any kind. Customers are responsible for determining the suitability to their respective environments. Only a representative subset of the IBM offerings are presented here. Products not mentioned should not be interpreted as a lack or withdrawal of support of those products.
