SPARC® Enterprise M8000/M9000 Servers Product Notes For XCP version 1072 Order No. U41813-J-Z816-1-76 Part No.
Copyright 2008 Sun Microsystems, Inc., 4150 Network Circle, Santa Clara, California 95054, U.S.A. and FUJITSU LIMITED, 1-1, Kamikodanaka 4-chome, Nakahara-ku, Kawasaki-shi, Kanagawa-ken 211-8588, Japan. All rights reserved. Sun Microsystems, Inc.
Copyright 2008 Sun Microsystems, Inc., 4150 Network Circle, Santa Clara, California 95054, U.S.A. et FUJITSU LIMITED, 1-1, Kamikodanaka 4-chome, Nakahara-ku, Kawasaki-shi, Kanagawa-ken 211-8588, Japon. Tous droits réservés. Entrée et revue tecnical fournies par Fujitsu Limited sur des parties de ce matériel. Sun Microsystems, Inc. et Fujitsu Limited détiennent et contrôlent toutes deux des droits de propriété intellectuelle relatifs aux produits et technologies décrits dans ce document.
Contents Preface vii Technical Support vii Software Resources vii Accessing Documentation viii Fujitsu Siemens Computers Welcomes Your Comments General Information about XCP 1 Supported Firmware and Operating System Solaris OS Patch Information 1 2 Patches for SPARC64 VI Processors Patches for SPARC64 VII Processors Updating to XCP viii 2 2 3 Resetting the XSCF Firmware 3 Updating from a Version Earlier Than XCP 1050 3 Updating from a Version Earlier Than XCP 1070 4 Functionality Iss
Hardware Issues and Workarounds 7 Sun Crypto Accelerator 6000 Cards Information about Software XCP Issues and Workarounds 7 8 8 Solaris OS Issues and Workarounds Software Documentation Updates 12 27 Identifying Degraded Memory in a System 29 Identifying Different Memory Sizes in a System Board Using the showdevices Command 29 29 Using the prtdiag Command to Identify Memory Size Identifying Permanent Memory in a Target Board CPU Upgrade 30 31 32 Installation Notes 32 Updating the OpenBoot P
Preface These product notes contain late-breaking information about the SPARC® Enterprise M8000/M9000 server hardware, software, or documentation that became known after the documentation set was published. Technical Support If you have technical questions or issues that are not addressed in the SPARC Enterprise M8000/M9000 servers documentation, contact a sales representative or a certified service engineer.
Accessing Documentation Instructions for installing, administering, and using your SPARC Enterprise M8000/M9000 servers are provided in the SPARC Enterprise M8000/M9000 servers documentation set. The documentation set is available for download from the following website: http://manuals.fujitsu-siemens.com/ Note – Information in these product notes supersedes the information in the SPARC Enterprise M8000/M9000 servers documentation set. Solaris documentation is available at: http://www.sun.
General Information about XCP 1072 This section describes the general information about XCP 1072. ■ ■ ■ Supported Firmware and Operating System Updating to XCP Functionality Issues and Limitations Supported Firmware and Operating System The following firmware and operating system (OS) are supported in this release.
Note – By using the Solaris 10 8/07 installation DVD, you cannot boot the domain mounted with the SPARC64 VII processors. When you newly install Solaris OS to a domain mounted with the SPARC64 VII processors, use the Solaris 10 5/08 installation DVD to install Solaris 10 5/08. For XCP, you can download the latest files of firmware at the following websites. Global Site: http://www.fujitsu.com/sparcenterprise/firmware/ Japanese Site: http://primeserver.fujitsu.
■ ■ ■ ■ 119254-51 or later 125891-01 or later 127755-01 or later 127127-11 The patches are not required for servers running Solaris 10 5/08 OS or later. Note – See “Software Resources” on page vii for information on how to find the latest patches. Installation information and README files are included in the patch download.
Updating from a Version Earlier Than XCP 1050 ■ You cannot update to XCP 1071 or later directly. If you are currently running a version earlier than XCP 1050, you must first update to an interim version of XCP between 1050 and 1070 (inclusive) before updating to XCP 1071 or later. Refer to the product notes document for the interim version for instructions. ■ Delete any accounts named "admin". Any accounts named admin must be deleted prior to updating to XCP 1050 or later.
General Functionality Issues and Limitations Caution – For dynamic reconfiguration (DR) and hot-plug issues, see TABLE 4. ■ Domains using the ZFS file system cannot use Dynamic Reconfiguration. ■ The maximum number of IOUA (Base I/O Card) cards per domain is limited to six cards. ■ Do not use the internal CD-RW/DVD-RW drive unit and the TAPE drive unit at the same time. ■ For this XCP release, the XSCF browser user interface (XSCF Web) does not support the External I/O Expansion Unit Manager feature.
■ On the SPARC Enterprise M8000/M9000 servers with XCP 1050 or later, the dual XSCF Unit feature is working. Therefore, you can not downgrade SPARC Enterprise M8000/M9000 servers with XCP 1050 or later to XCP 1040 or XCP 1041, which does not support dual XSCF Unit feature. ■ You cannot use the following user account names, as they are reserved for system use: root, bin, daemon, adm, operator, nobody, sshd, rpc, rpcuser, ldap, apache, ntp, admin, and default.
Information about Hardware This section describes the special instructions and the issues about the SPARC Enterprise M8000/M9000 servers hardware. ■ Hardware Issues and Workarounds Hardware Issues and Workarounds TABLE 2 lists known hardware issues and possible workarounds. TABLE 2 Hardware Issues and Workarounds CR ID Description Workaround 6433420 The domain console might display a Mailbox timeout or IOCB interrupt timeout error during boot.
Information about Software This section describes the special instructions and the issues about the SPARC Enterprise M8000/M9000 servers software. ■ ■ ■ ■ ■ ■ ■ XCP Issues and Workarounds Solaris OS Issues and Workarounds Software Documentation Updates Identifying Degraded Memory in a System Identifying Different Memory Sizes in a System Board Identifying Permanent Memory in a Target Board CPU Upgrade XCP Issues and Workarounds TABLE 3 lists known XCP issues and possible workarounds.
TABLE 3 XCP Issues and Workarounds (Continued) ID Description Workaround RTIF1070823-001 Using the XSCF Web, when you selected SSH on the snapshot screen, the maximum number of character input for Host, Directory, ID, and Password doesn't correspond to the maximum number of character input on the XSCF Shell. To specify the value which exceeds the maximum number of character input for the XSCF Web, use XSCF Shell.
TABLE 3 XCP Issues and Workarounds (Continued) ID Description Workaround RTIF1070914-025 When you execute XCP Sync on the Firmware Update page, after 15 minutes, the error message "Another flashupdate is now processing" or "The page cannot be displayed" may appear. There is no workaround. However, the XCP Sync process has been continuously executed. Check the XSCF update completion message on the monitoring message to confirm the completion of Sync process.
TABLE 3 XCP Issues and Workarounds (Continued) ID Description Workaround RTIF1080512-001 When you specify "localhost" to the hostname of the sethostname(8) command and reset XSCF by using the applynetwork(8) and the rebootxscf(8) commands, a process goes down in XSCF. Do not specify "localhost" to the hostname of the sethostname(8) command. RTIF1080526-001 When the system is stressed with many faults, the fmd process on the service processor might hang.
Solaris OS Issues and Workarounds TABLE 4 lists known Solaris OS issues and possible workarounds TABLE 4 Solaris OS Issues and Workarounds CR ID Description Workaround 5076574 A PCIe error can lead to an invalid fault diagnosis on a large M8000/M9000 domain. This bug has been fixed in Solaris 10 5/08. For Solaris 10 8/07 or earlier, this has been fixed in patch 127127-11. [Workaround] Create a file /etc/fm/fmd/fmd.conf containing the following lines; setprop client. buflim 40m setprop client.
TABLE 4 Solaris OS Issues and Workarounds (Continued) CR ID Description Workaround 6449315 The Solaris OS cfgadm(1M) command does not unconfigure a DVD drive from a domain on a SPARC Enterprise M8000/M9000 server. Disable the Volume Management Daemon (vold) before unconfiguring a DVD drive with the cfgadm(1M) command. To disable vold, stop the daemon by issuing the command /etc/init.d/volmgt stop. After the device has been removed or inserted, restart the daemon by issuing the command /etc/init.
TABLE 4 Solaris OS Issues and Workarounds (Continued) CR ID Description Workaround 6481002 Installing the Solaris OS from the network using certain PCI-Express cards may cause a panic. If you are using a Sun PCI-E Dual Gigabit Ethernet Adapter MMF card or a Sun PCI-E Dual Gigabit Ethernet Adapter UTP card, do not install the Solaris using either of these cards. Instead, use other network devices, such as the onboard Gigabit Ethernet or another network device.
TABLE 4 Solaris OS Issues and Workarounds (Continued) CR ID Description Workaround 6498283 Using the DR deleteboard(8) command while psradm operations are running on a domain might cause a system panic. This bug has been fixed in Solaris 10 8/07. For Solaris 10 11/06, this has been fixed in patch 120011-07. There is no workaround. 6499304 6502204 6502750 CPU isn't offlined and unexpected message is displayed on console when many correctable error(CE) occurs.
TABLE 4 Solaris OS Issues and Workarounds (Continued) CR ID Description Workaround 6508432 Many correctable errors (CE) may occur, and despite these are the correctable errors, domain may panic. This bug has been fixed in Solaris 10 8/07. For Solaris 10 11/06, this has been fixed in patch 120011-08.
TABLE 4 Solaris OS Issues and Workarounds (Continued) CR ID Description Workaround 6515648 "Replumb Failed" error appears when dr@0:SB1::memory fails. Once the DR operation is complete, it can be plumbed up manually. Example steps to re-plumb the interface manually: # ifconfig interface plumb xxx.xxx.xxx.xxx netmask + broadcast + up # ifconfig interface group group-name # ifconfig interface addif xxx.xxx.xxx.xxx -failover deprecated up This workaround assumes that the /etc/hostname.
TABLE 4 Solaris OS Issues and Workarounds (Continued) CR ID Description Workaround 6522433 After the CPU hardware error occurred, the fmdump(1M) command on the domain may display an incorrect faulty component. This bug has been fixed in Solaris 10 5/08. For Solaris 10 8/07 or earlier, this has been fixed in patch 127127-11. [Workaround] Check system status on XSCF. 6527781 The cfgadm command fails while moving the DVD/DAT drive between two domains. This bug has been fixed in Solaris 10 8/07.
TABLE 4 Solaris OS Issues and Workarounds (Continued) CR ID Description Workaround 6531668 System hangs when executing parallel hot plug operation with SP DR in suspend phase. There is no workaround. 6532215 volfs or dscp service may fail when domain is booted. Restart the service if the failure is observed. To avoid the problem, issue the following commands. svc:/platform/sun4u/dscp:default: Method "/lib/svc/method/svc-dscp start" failed with exit status 95.
TABLE 4 Solaris OS Issues and Workarounds (Continued) CR ID Description Workaround 6536564 showlogs(8) and showstatus(8) command on XSCF might report wrong I/O component due to wrong diagnosis by Solaris Fault management Architecture when faults in I/O devices occur. This bug has been fixed in Solaris 10 5/08. For Solaris 10 8/07 or earlier, this has been fixed in patch 125369-05. [Workaround] To avoid this problem, issue the following commands on the domain.
TABLE 4 Solaris OS Issues and Workarounds (Continued) CR ID Description Workaround 6542632 Memory leak in PCIe module if driver attach fails. This bug has been fixed in Solaris 10 8/07. For Solaris 10 11/06, this has been fixed in patch 120011-09. There is no workaround. 6545143 6545685 When kcage thread is expanding the kcage area, if the user stack exists in the expanded area, its area is demapped and might cause a ptl_1 panic during the flushw handler execution.
TABLE 4 Solaris OS Issues and Workarounds (Continued) CR ID Description Workaround 6559504 Messages of the form nxge: NOTICE: nxge_ipp_eccue_valid_check: rd_ptr = nnn wr_ptr = nnn will be observed on the console with the following cards: • X4447A-Z, PCI-e Quad-port Gigabit Ethernet Adapter UTP • X1027A-Z1, PCI-e Dual 10 Gigabit Ethernet Fiber XFP Low profile Adapter This bug has been fixed in Solaris 10 5/08. For Solaris 10 8/07, this has been fixed in patch 127741-01.
TABLE 4 Solaris OS Issues and Workarounds (Continued) CR ID Description Workaround 6584984 On SPARC Enterprise M8000/M9000 servers, busstat(1M) command may cause rebooting of domains. This bug has been fixed in Solaris 10 5/08. For Solaris 10 8/07 or earlier, this has been fixed in patch 127127-11. There is no workaround. Do not use busstat(1M) command. Check for the availability of a patch for this defect. 6588555 XSCF failover during DR operation to the permanent memory might cause domain panic.
TABLE 4 Solaris OS Issues and Workarounds (Continued) CR ID Description Workaround 6614737 The DR deleteboard(8) and moveboard(8) operations might hang if any of the following conditions exist: • A DIMM has been degraded. • The domain contains system boards with different memory size. This has been fixed in patch 137111-01. [Workaround] Avoid performing DR operations if any of the listed conditions exist. To determine whether the system contains degraded memory, use the XSCF showstatus(8) command.
TABLE 4 Solaris OS Issues and Workarounds (Continued) CR ID Description Workaround 6632549 fmd service on domain might fail to maintenance mode after DR operations. This has been fixed in patch 138050-01. [Workaround] If fmd service fails, issue the following commands on the domain to recover: # svcadm clear fmd 6660168 If a ubc.piowbeue-cpu error occurs on a domain, the Solaris Fault Management cpumem-diagnosis module might fail, causing an interruption in FMA service.
TABLE 4 Solaris OS Issues and Workarounds (Continued) CR ID Description Workaround 6660197 DR might cause the domain to hang if either of the following conditions exist: • A domain contains 256 or more CPUs. • Memory error occurred and the DIMM has been degraded. This has been fixed in patch 138397-01. 6679370 The following message may be output on the console during the system booting, the External I/O Expansion Unit adding by hotplug, or the FMEMA operating by DR. [Workaround] 1.
Software Documentation Updates This section contains late-breaking software information that became known after the documentation set was published and corrections in the SPARC Enterprise M8000/M9000 servers software documentation. The corrections for SPARC Enterprise M4000/M5000/M8000/M9000 servers XSCF Reference Manual, if not otherwise specified, also apply to the man pages which XSCF provides. And they supersede the information on the man pages. TABLE 5 lists known documentation updates.
TABLE 5 Software Documentation Updates (Continued) Title Page Number Update SPARC Enterprise M4000/M5000/M8000/ M9000 Servers XSCF Reference Manual sendbreak(8) command The sendbreak(8) command will not work when the secure mode is set to on while the mode switch on the operator panel is set to locked. Refer to the setdomainmode(8) for more information. setdscp(8) commands The references to site planning guide of the caution in EXAMPLES are now referenced to as administration guide.
Identifying Degraded Memory in a System 1. Log in to XSCF. 2. Type the following command: XSCF> showstatus The following example identifies DIMM number 00A on CMU#3 has degraded memory..
2. Type the following command: XSCF> showdevices -d domain_id The following example displays 00-0 has 64GB of memory while the other system boards have 16GB.
Identifying Permanent Memory in a Target Board 1. Log in to XSCF. 2. Execute the following command: XSCF> showdevices -d domain_id The following example shows a display of the showdevices -d command where 0 is the domain_id. XSCF> showdevices -d 0 ... Memory: ------DID 00 00 00 XSB 00-0 00-2 00-3 board mem MB 8192 8192 8192 perm mem MB 0 1674 0 base address 0x0000000000000000 0x000003c000000000 0x0000034000000000 domain target deleted remaining mem MB XSB mem MB mem MB 24576 24576 24576 ...
CPU Upgrade This section describes the procedure to mount the SPARC64 VII processor on the SPARC Enterprise M8000/M9000 server.
Adding a New CMU Equipped with SPARC64 VII as a New Domain 1. Log in to the XSCF on an account with platform administrative privileges. 2. Use the showstatus(8) command to confirm that a component in Faulted or Deconfigured status doesn't exist. XSCF> showstatus If no failures found, the following message appears: "No failures found in System Initialization." In case of other messages, contact a service engineer before proceeding to the next step. 3.
Note – Do not fail to execute the diagnosis of newly mounted CMU in the maintenance menu of addfru(8) command. 10. Confirm that the mounted CPU module has been recognized by the server, and the error indicator asterisk (*) is not displayed. XSCF> showhardconf -M 11. Confirm no abnormality occurred by using showlogs error -v and showstatus(8) commands. XSCF> showlogs error -v XSCF> showstatus If you encounter any hardware abnormality of the XSCF contact a service engineer. 12.
18. Install Solaris 10 5/08. 19. Use the setdomainmode(8) command, enable the autoboot function of the domain. For detail, see the SPARC Enterprise M4000/M5000/M8000/M9000 Servers XSCF User’s Guide. The autoboot function is applied by a domain reboot. Upgrading an On-CMU SPARC64 VI to SPARC64 VII, or Adding SPARC64 VII to an Existing CMU, to an Existing Domain Configured with SPARC64 VI 1.
8. Collect an XSCF snapshot to archive system status prior to upgrade. This will be help in case any problem occurred in this procedure. XSCF> snapshot -t user@host:directory 9. Update the XCP version to 1072. Before updating the XCP, be sure to see “Updating to XCP” on page 3. For the XCP updating procedures, see the SPARC Enterprise M4000/M5000/M8000/M9000 Servers XSCF User’s Guide. 10. After updating the XCP, reset the XSCF. XSCF> rebootxscf 11. After resetting the XSCF, log in to the XSCF again. 12.
14. Turn off the power to the target domain. XSCF> poweroff -d domain_id 15. Upgrade an on-CMU SPARC64 VI processors to SPARC64 VII processors, or add SPARC64 VII processors to an existing CMU. ■ For upgrading the CPU, operate by hot replacement, referring to "6.2 Active Replacement and Hot Replacement" in SPARC Enterprise M8000/M9000 Servers Service Manual. ■ For adding the CPU, operate by hot replacement, referring to "6.
21. Power on the target domains. XSCF> poweron -d domain_id 22. Confirm that the target domain has been correctly started. XSCF> showlogs power 23. Confirm no abnormality occurred by using showlogs error -v and showstatus(8) commands. XSCF> showlogs error -v XSCF> showstatus If you encounter any hardware abnormality of the XSCF contact a service engineer. Adding a New CMU Equipped with SPARC64 VII to an Existing Domain Configured with SPARC64 VI 1.
7. Change the key position on the operator panel from Locked to Service. 8. Collect an XSCF snapshot to archive system status prior to upgrade. This will be help in case any problem occurred in this procedure. XSCF> snapshot -t user@host:directory 9. Update the XCP version to 1071 or later. Before updating the XCP, be sure to see “Updating to XCP” on page 3. For the XCP updating procedures, see the SPARC Enterprise M4000/M5000/M8000/M9000 Servers XSCF User’s Guide. 10.
In case that the OpenBoot PROM version of the XSB to which the resource of the target CMU has been assigned is not displayed as 02.03.0000, contact a service engineer. 14. Turn off the target domain. XSCF> poweroff -d domain_id 15. Mount the CPU module (CPUM) on the CMU for add-on. For the procedure, see the description about the CPU module installation in Section 6.4.1, "Replacing a CPU module" in the SPARC Enterprise M8000/M9000 Servers Service Manual. 16.
■ Set up the CPU operational mode of the domain. For each setting, see the SPARC Enterprise M4000/M5000/M8000/M9000 Servers XSCF User’s Guide. 21. Power on the target domains. XSCF> poweron -d domain_id 22. Confirm that the target domain has been correctly started. XSCF> showlogs power 23. Confirm no abnormality occurred by using showlogs error -v and showstatus(8) commands. XSCF> showlogs error -v XSCF> showstatus If you encounter any hardware abnormality of the XSCF contact a service engineer.
42 SPARC Enterprise M8000/M9000 Servers Product Notes for XCP Version 1072 • September 2008
