IB Switch Module for IBM eServer BladeCenter User Guide Release 2.2.
Copyright © 2004 Topspin Communications, Inc. All rights reserved. The Topspin Switched Computing System, Topspin Host Channel Adapter, Topspin Element Manager, and collateral software programs and documentation are subject to and made available only pursuant to the license agreement signed by you and Topspin, Communications, Inc. (or if no signed license agreement exists, the license agreement included with your original media) and may only be used in accordance with the terms of that agreement.
Important Notice License information for IB Switch Module for IBM eServer BladeCenter IMPORTANT NOTICE: This hardware product was manufactured for IBM by Topspin Communications and contains software and/or firmware provided to you by Topspin Communications under provided license agreements. By using the enclosed hardware and/or software products, you are agreeing to the terms of these license agreements, and agree that the terms of such agreement are extended to you solely by Topspin Communications.
v Contents Important Notice ........................................................................................................................................ iii Statement of Limited Warranty Preamble ................................................................................................. iii Regulatory Notices......................................... ix FCC Statement ...........................................................................................................................
vi ASIA PACIFIC............................................................................................................................. xx EUROPE, MIDDLE EAST, AFRICA (EMEA)........................................................................ xxii EUROPEAN UNION (EU) ...................................................................................................... xxiv CONSUMERS ......................................................................................................................
vii 2: Getting Started ........................................... 5 Install the Server Switch Module.................................................................................................................5 Configure the Server Switch Module ..........................................................................................................5 Change the Default Password .........................................................................................................
viii
ix Regulatory Notices FCC Statement This equipment has been tested and found to comply with the limits for a Class A digital device, pursuant to Part 15 of the FCC Rules. These limits are designed to provide reasonable protection against harmful interference when the equipment is operated in a commercial environment.
x CAUTION: Observe and follow service markings. Do not service the IB Switch Module, except as explained in the Topspin documentation supplied with your IB Switch Module. Opening the chassis or removing the enclosure cover exposes you to electrical shock and may damage IB Switch Module components. If one or more of the following situations occurs, do not install the IB Switch Module in a chassis and contact your Topspin support representative: • The IB Switch Module has been exposed to water.
xi To obtain warranty service contact IBM or your IBM reseller. In Canada or the United States, call 1-800-IBM-SERV (426-7378). In the EU countries, see the telephone numbers in Table iii-1.
xii • (Linux only) Run the dmesg command on your Linux host and provide the output.
xiii Table iii-2: Contact Information (Continued) Telephone, Corporate Headquarters (866) TOPSPIN or (650) 316-3300 Fax (650) 316-3269 Telephone, Technical Support (800) 499-1473 Email, Technical Support support@ibm.com (contact IBM first) support@topspin.com Web site, Support http://www.ibm.com/support (contact IBM first) http://support.topspin.
xiv
xv IBM Statement of Limited Warranty Z125-4753-08 04/2004 Part 1 - General Terms Part 1 - General Terms This Statement of Limited Warranty includes Part 1 - General Terms, Part 2 - Country-unique Terms, and Part 3 - Warranty Information. The terms of Part 2 replace or modify those of Part 1. The warranties provided by IBM in this Statement of Limited Warranty apply only to Machines you purchase for your use, and not for resale.
xvi JURISDICTIONS DO NOT ALLOW THE EXCLUSION OF EXPRESS OR IMPLIED WARRANTIES, SO THE ABOVE EXCLUSION MAY NOT APPLY TO YOU. IN THAT EVENT, SUCH WARRANTIES ARE LIMITED IN DURATION TO THE WARRANTY PERIOD. NO WARRANTIES APPLY AFTER THAT PERIOD. SOME STATES OR JURISDICTIONS DO NOT ALLOW LIMITATIONS ON HOW LONG AN IMPLIED WARRANTY LASTS, SO THE ABOVE LIMITATION MAY NOT APPLY TO YOU.
xvii Exchange of a Machine or Part When the warranty service involves the exchange of a Machine or part, the item IBM or your reseller replaces becomes its property and the replacement becomes yours. You represent that all removed items are genuine and unaltered. The replacement may not be new, but will be in good working order and at least functionally equivalent to the item replaced. The replacement assumes the warranty service status of the replaced item.
xviii PARTY CLAIMS AGAINST YOU FOR DAMAGES (OTHER THAN THOSE UNDER THE FIRST ITEM LISTED ABOVE); 2) LOSS OF, OR DAMAGE TO, DATA; 3) SPECIAL, INCIDENTAL, OR INDIRECT DAMAGES OR FOR ANY ECONOMIC CONSEQUENTIAL DAMAGES; OR 4) LOST PROFITS, BUSINESS REVENUE, GOODWILL OR ANTICIPATED SAVINGS. SOME STATES OR JURISDICTIONS DO NOT ALLOW THE EXCLUSION OR LIMITATION OF INCIDENTAL OR CONSEQUENTIAL DAMAGES, SO THE ABOVE LIMITATION OR EXCLUSION MAY NOT APPLY TO YOU.
xix COLOMBIA Jurisdiction: The following is added after the first sentence: Any litigation arising from this Statement of Limited Warranty will be settled exclusively by the Judges of the Republic of Colombia. EQUADOR Jurisdiction: The following is added after the first sentence: Any litigation arising from this Statement of Limited Warranty will be settled exclusively by the Judges of Quito.
xx CANADA Limitation of Liability: The following replaces item 1 of this section: 1. damages for bodily injury (including death) or physical harm to real property and tangible personal property caused by IBM's negligence; and Governing Law: The following replaces “laws of the country in which you acquired the Machine” in the first sentence: laws in the Province of Ontario.
xxi If one of the parties refuses or otherwise fails to appoint an arbitrator within 30 days of the date the other party appoints its, the first appointed arbitrator shall be the sole arbitrator, provided that the arbitrator was validly and properly appointed. All proceedings shall be conducted, including all documents presented in such proceedings, in the English language. The English language version of this Statement of Limited Warranty prevails over any other language version. HONG KONG S.A.R.
xxii Guarantees Act 1993 will not apply in respect of any goods which IBM provides, if you require the goods for the purposes of a business as defined in that Act. Limitation of Liability: The following is added to this section: Where Machines are not acquired for the purposes of a business as defined in the Consumer Guarantees Act 1993, the limitations in this section are subject to the limitations in that Act.
xxiii The warranty for Machines acquired in Western Europe shall be valid and applicable in all Western Europe countries provided the Machines have been announced and made available in such countries.
xxiv Arbitration Court of Moscow; 6) in South Africa, Namibia, Lesotho and Swaziland, both of us agree to submit all disputes relating to this Statement of Limited Warranty to the jurisdiction of the High Court in Johannesburg; 7) in Turkey all disputes arising out of or in connection with this Statement of Limited Warranty shall be resolved by the Istanbul Central (Sultanahmet) Courts and Execution Directorates of Istanbul, the Republic of Turkey; 8) in each of the following specified countries, any legal
xxv Limitation of Liability: The following replaces the terms of this section in its entirety: Except as otherwise provided by mandatory law: 1.
xxvi The limitation period for consumers in action for breach of warranty is the statutory period as a minimum. In case IBM or your reseller is unable to repair an IBM Machine, you can alternatively ask for a partial refund as far as justified by the reduced value of the unrepaired Machine or ask for a cancellation of the respective agreement for such Machine and get your money refunded. The second paragraph does not apply.
xxvii The following sentence is added to the end of item 2: IBM's liability under this item is limited to the violation of essential contractual terms in cases of ordinary negligence. HUNGARY Limitation of Liability: The following is added at the end of this section: The limitation and exclusion specified herein shall not apply to liability for a breach of contract damaging life, physical well-being, or health that has been caused intentionally, by gross negligence, or by a criminal act.
xxviii SOUTH AFRICA, NAMIBIA, BOTSWANA, LESOTHO AND SWAZILAND Limitation of Liability: The following is added to this section: IBM's entire liability to you for actual damages arising in all situations involving nonperformance by IBM in respect of the subject matter of this Statement of Warranty will be limited to the charge paid by you for the individual Machine that is the subject of your claim from IBM.
xxix NOTE: “Region” means either Hong Kong or Macau Special Administrative Region of China. A warranty period of 3 years on parts and 1 year on labor means that IBM provides warranty service without charge for: 1. parts and labor during the first year of the warranty period; and 2. parts only, on an exchange basis, in the second and third years of the warranty period.
xxx to you at IBM’s expense, unless IBM specifies otherwise. You are responsible for the subsequent installation and verification of the Machine. 6. CRU and On-site Service This type of Warranty Service is a combination of Type 1 and Type 2 (see above). 7. CRU and Courier or Depot Service This type of Warranty Service is a combination of Type 1 and Type 3 (see above). 8. CRU and Customer Carry-In or Mail-In Service This type of Warranty Service is a combination of Type 1 and Type 4 (see above).
xxx Notices This information was developed for products and services offered in the U.S.A. IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used.
xxxii Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product, and use of those Web sites is at your own risk. IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you. Edition notice © Copyright International Business Machines Corporation 2004.
xxxiii Linux is a trademark of Linus Torvalds in the United States, other countries, or both. Red Hat, the Red Hat “Shadow Man” logo, and all Red Hat-based trademarks and logos are trademarks or registered trademarks of Red Hat, Inc., in the United States and other countries. Other company, product, or service names may be trademarks or service marks of others. Important notes Processor speeds indicate the internal clock speed of the microprocessor; other factors also affect application performance.
xxxiv equipment. For information on proper disposal of these batteries, contact IBM at 1-800-426-4333. Have the IBM part number listed on the battery available prior to your call. In the Netherlands, the following applies. Electronic emission notices Federal Communications Commission (FCC) statement NOTE: This equipment has been tested and found to comply with the limits for a Class A digital device, pursuant to Part 15 of the FCC Rules.
xxxv Australia and New Zealand Class A statement Attention: This is a Class A product. In a domestic environment this product may cause radio interference in which case the user may be required to take adequate measures. United Kingdom telecommunications safety requirement Notice to Customers This apparatus is approved under approval number NS/G/1234/J/100003 for indirect connection to public telecommunication systems in the United Kingdom.
xxxvi Japanese Voluntary Control Council for Interference (VCCI) statement
xxx WEEE Notice NOTE: This mark applies only to countries within the European Union (EU) and Norway. This appliance is labelled in accordance with European Directive 2002/96/EC concerning waste electrical and electronic equipment (WEEE). The Directive determines the framework for the return and recycling of used appliances as applicable throughout the European Union.
xxxviii
xxx Safety
xl
xli
xlii
xliii
xliv
1 1 About the Server Switch Module The following sections appear in this chapter: • “Introduction” on page 1 • “Topologies” on page 2 • “Management” on page 3 • “LEDs” on page 3 Introduction The Topspin IB Switch Module for IBM eServer BladeCenter (hereafter “Server Switch Module”) adds InfiniBand (IB) switching capability to hosts in your IBM BladeCenter chassis.
2 Each Server Switch Module includes 14 1x ports to the backplane and 4 4x ports (in the form of 1 4x connector and 1 12x connector) on the front panel. (In the case of the BladeCenter T, the internal ports beyond the first 8 appear in the CLI but do not function.) The Server Switch Module provides fully non-blocking switching for all 18 ports. All internal 1x ports provide 2.5 Gbps connections to the HCA expansion cards on server blades.
3 NOTE: The Server Switch Modules do not connect to each other within the BladeCenter chassis. To connect the modules to enable features such as subnet manager failover, connect the modules with an IB cable via the external connectors. Before you connect the two Server Switch Modules in your chassis, configure the priority of the subnet managers on the modules. For step by step instructions, refer to “Connect Beyond BladeCenter” on page 8.
4 NOTE: As of this release, the Server Switch Module does not support the 12x-port physical link and logical link LEDs. Server Switch Module Status LEDs Module Status LEDs provide an at-a-glance indication of the health of the Server Switch Module. Table 1-1 lists and describes the states of the Module Status LEDs. Table 1-1: Module Status LED Indications State Indication both LEDs off No system power or LED malfunction. yellow solid, green off Module error detected: operator intervention required.
5 2 Getting Started The following sections appear in this chapter: • “Install the Server Switch Module” on page 5 • “Configure the Server Switch Module” on page 5 • “Connect Beyond BladeCenter” on page 8 Install the Server Switch Module To install the Topspin Server Switch Module, perform the following steps: 1. Ground yourself appropriately. 2. Choose a bay into which you want to install the Server Switch Module. You must install the module in bay 3 or bay 4. 3.
6 4. Click Management, under I/O Module Tasks, in the left-hand navigation tree. An I/O Module Management heading appears in the display. 5. Click the link of the physical bay that holds the Server Switch Module that you want to configure. A Bay X heading appears, where X is the number of the bay. 6. Enter an IP address, in the IP address field of the New Static IP Configuration section, for the Server Switch Module.
7 2. Enter the configure terminal command to enter Global Configuration mode. Example IBM BladeCenter# configure terminal IBM BladeCenter(config)# 3. Enter the username command with • the user ID for which you need to change the password (in this case, USERID) • the password keyword • the new password that you want to apply to the user ID to change the password of the user ID. Example IBM BladeCenter(config)# username USERID password n0accE$$ 4.
8 Synchronize with NTP To configure primary, secondary, and tertiary NTP servers to synchronize the Server Switch with the network, perform the following steps: 1. Enter the enable command to enter Privileged Exec mode. Example IBM BladeCenter> enable IBM BladeCenter# 2. Enter the configure terminal command to enter Global Configuration mode. Example IBM BladeCenter# configure terminal IBM BladeCenter(config)# 3.
9 The device with the highest SM priority becomes the master SM in the IB network. To configure SM priority, perform the following steps: 1. Enter the enable command to enter Privileged Exec mode. Example IBM BladeCenter> enable IBM BladeCenter# 2. Enter the configure terminal command to enter Global Configuration mode. Example IBM BladeCenter# configure terminal IBM BladeCenter(config)# 3.
10 Connecting InfiniBand Cables To plug in an InfiniBand cable, push the connector into the interface until you hear/feel a click (Figure 2-1).
11 To remove a cable with a pull connector, grasp the connector with one hand and push it toward the port, then pull the latch away from the port with your other hand and gently wiggle the connector away from the port. 1. push connector toward port 2. pull latch away from port 3.
12
13 3 Specifications and Compliance This chapter details the Server Switch Module specifications and compliance certifications. Safety Certifications The Topspin Server Switch Module, model 99-00045-XX, is an UL Listed Accessory in accordance with the UL Standard for Safety for Information Technology and Telecommunications Equipment, UL 60950-1.
14 equipment in a residential area is likely to cause harmful interference in which case the user will be required to correct the interference at his own expense. Table 3-2: EMC/EMI Certifications Certification FCC: Title 47 CFR 47 Part 15, Subpart B, Class A ICES-003 Issue 2 CE: EN55022:1998 and EN55024:1998 VCCI: V3/2000.04, Class A C-Tick: AS/NZS 3548:1995, Am.1:1997, Am.2:1997 BSMI MIC Notice No.
Index A N about the server switch module ...............................1 new static IP configuration .....................................6 NTP, configuring ....................................................8 C cable connection ....................................................10 change the default password ...................................6 clock, setting ...........................................................7 configuration, initial ................................................
