System Fault Management C.07.09.02.
© Copyright 2012 Hewlett-Packard Development Company, L.P Legal Notices ©Copyright 2012 Hewlett-Packard Development Company, L.P.Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor’s standard commercial license.
Contents 1 Introduction...............................................................................................7 Overview................................................................................................................................7 Features and benefits................................................................................................................7 Components of SFM......................................................................................................
Viewing FRU information.....................................................................................................47 Viewing information about Management Processor.................................................................48 Viewing information about Firmware information....................................................................48 Viewing information about Onboard Administrator.................................................................
Viewing List of Low Level Logs..............................................................................................67 Viewing List Of Low Level Logs using GUI.........................................................................67 Viewing List Of Low Level Logs using CLI...........................................................................67 Viewing Details of Low Level Logs.........................................................................................
Temperature instances...........................................................................................................107 Voltage instances..................................................................................................................108 FRU Information instances......................................................................................................109 Management Processor instances....................................................................................
1 Introduction The System Fault Management (SFM) supports HP Integrity Superdome 2 (HP Superdome 2), Intel Itanium 9500 Processor Series based servers, HP Integrity BL860c i2, BL870c i2 & BL890c i2 Server Blades, and rx2800 i2 in addition to other HP Integrity Servers. All the features supported on systems running the HP-UX 11i v3 operating system are available for HP Integrity Servers. This chapter introduces you to the System Fault Management (SFM) software and the tools that SFM includes.
• Enables you to view and administer WBEM indications. • Provides the same features and benefits as those found in the EMS hardware monitors.
Components of SFM This section discusses the following topics: • EVWEB • Error Management Technology (EMT) • CIMUtil • IPMI Event Viewer • providers EVWEB EVWEB is a component of SFM that enables you to administer and view WBEM indications generated on the local system on which SFM is installed. For more information on EVWEB, see “Evweb overview” (page 50). EMT EMT is a component of SFM that enables you to view and administer information about errors which can occur on the server.
Table 1 Instance providers (continued) Instance provider Description NOTE: The Blade instance provider is available on HP Integrity BL860c i2/BL870c i2/BL890c i2 Server Blades, HP Integrity Superdome 2.
Table 1 Instance providers (continued) Instance provider Description MPProvider The Management Processor provider retrieves information about the management processor on the system.
Table 1 Instance providers (continued) Instance provider Description Firmware Revision Management Processor Enclosure Temperature Sensor Indication providers SFM includes four indication providers, the EMS Wrapper provider, the Event Manager Common Information Model (EVM CIM) provider, SFMIndicationProvider and MCA indication provider. Table 2 describes the SFM indication providers.
Table 2 Indication providers (continued) Indication provider Description can use an SFM tool, called EVWEB, to view and administer events through the HP SMH interface. 3. Logs messages logged by EVMCimProvider in /var/opt/sfm/EvmCimProvider.log log file. SFMIndicationProvider The SFMIndicationProvider generates indications that are compliant with the WBEM standards.
In addition, the support for CPUIndicationProvider and MemoryIndicationProvider has been added with additional support for hardware events on HP Integrity BL860c i2, BL870c i2 & BL890c i2 Server Blades. The support for MemoryIndicationProvider has been added for Legacy memory events and that for MemoryIndicationProviderIA for HP Integrity BL860c i2, BL870c i2 & BL890c i2 Server Blades, HP Integrity Superdome 2 and rx2800 i2 servers.
Table 4 Mapping provider module and provider Provider Module Provider registered with CIMOM StateChangeIndicationProvider EvmCimProvider Provider name in Events StateChangeIndicationProvider EvmCimProvider SEL02_IndicationProvider FPL_IndicationProvider CMC_IndicationProviderIA CPUIndicationProvider MemoryIndicationProviderIA SFMProviderModule MemoryIndicationProvider SFMIndicationProvider CPE_IndicationProviderIA PCIeIndicationProvider MCAIndicationProvider LPMC_IndicationProviderPA ChassisIndicationPro
User interfaces You can use two types of interfaces to view SFM provider queries: HP SIM and HP SMH. This section describes these interfaces. HP Systems Insight Manager HP SIM is a WBEM-based user interface for controlling and monitoring resources within a large-scale system. You can use HP SIM to create subscriptions and to view indications and instances on a remote system. You must install HP SIM on the CMS. You can use HP SIM to launch HP SMH.
1. 2. 3. 4. EVWEB and CMS subscriptions are created. The EMS Wrapper provider receives events generated by the EMS monitors through the EMS framework. The provider converts these events into WBEM indications and reports these indications to the CIMOM. CIMOM directs these indications to the CMS that has created subscriptions for indications.
2 Installing the SFM software The System Fault Management (SFM) software is installed by default with the HP-UX 11i v3 Operating Environment (OE) media. However, at some point you may need to install the SFM software separately. This chapter describes how to install the SFM software as a standalone component on the HP-UX 11i v3 operating system.
NOTE: • The listed versions of the software are the minimum supported requirements. Subsequent versions are compatible with this version of SFM unless otherwise noted. • WBEM Services, Online Diagnostics, SysMgmtWeb, and HP SIM are available on the Operating Environment (OE) media and can be selected for install during the SFM installation. • HP System Management Homepage (SMH) – bundled in SysMgmtWeb.
Selecting these options automatically installs all the dependencies. NOTE: The system selects some options by default. However, you must select the two options mentioned in step 5 to automatically install the prerequisites. 7. 8. Click OK in the Note window to confirm the selection of dependencies. In the SD Install - Software Selection window, select Actions->Install, as shown in the following figure.
When the SFM software installs, the Install window appears indicating that the SFM software is installed successfully, as shown in the following figure: 9. Unmount the CD. To unmount, enter the following command at the HP-UX prompt: # unmount /tmp/cdrom 10.
3. To install the SFM software and all the dependencies, enter the following command at the HP-UX prompt: # swinstall -x autoselect_dependencies=true -x enforce_dependencies=true -s /tmp/cdrom SysFaultMgmt 4. Unmount the CD. To unmount, enter the following command at the HP-UX prompt: # unmount /tmp/cdrom 5.
Fix the error and reinstall the product. Verifying the installation This section describes how to verify the SFM software installation using the TUI and the CLI. Verifying the installation using the TUI To verify the SFM software installation, complete the following steps: 1. Log in to the system as a superuser. 2. Click Logfile in the Install window, as shown in the following figure: The Logfile, which includes details about the installation, is displayed.
3. For information about errors related to installation, enter the following command at the HP-UX prompt: # swjob -a log @ :/ The jobid is available in the Logfile, as underlined in the Logfile window, in the following figure: For example, enter the following command at the HP-UX prompt: # swjob -a log iemlhamia-0013 @ iemlhamia.india.hp.com:/ Verifying the installation using the CLI To verify your installation using the CLI, complete the following steps: 1.
3. For information about installation-related errors, enter the following command at the HP-UX prompt: # swjob -a log @ :/ For example, enter the following command at the HP-UX prompt: # swjob -a log iemlhamia-0005 @ iemlhamia.india.hp.com:/ NOTE: The logs to /var/opt/sfm/log/install.log are written when SFM is getting installed. Removing the SFM software This section describes how to remove of the SFM software using the TUI and the CLI.
5. Select Actions->Remove, as shown in the following figure: 6.
The following figure is a sample of the removal process in progress: 7.
8. To verify whether the SFM software is removed properly, enter the following command at the HP-UX prompt: # swlist | grep SysFaultMgmt If the SFM software is removed properly, SysFaultMgmt and the version number of the SFM software does not appear in the output. If the SFM software is not removed properly, you must repeat the removal procedure. For more information, see “Verifying removal of the SFM software” (page 28).
3. For information about errors related to the removal of SFM, enter the following command at the HP-UX prompt: # swjob -a log @ :/ The jobid is available in the Logfile. Verifying removal using the CLI To verify if the SFM software is removed successfully, complete the following steps: 1. Log in to the system as a superuser. 2. Enter the following command at the HP-UX prompt: # swjob If the output contains no errors, the SFM software is removed successfully.
3 Configuring indication providers This chapter describes how to configure indication filters, error logging, and the SFMIndicationProvider. Configuring indication filters You must configure the indication filters to view desired indications. You use the Filter Metadata provider (FMD) to configure indication filters that deliver important or desired indications, for example, indications with a certain severity.
Filter Filter Filter Filter Filter Filter Filter Unique Identifier Query Query Language Source Namespace Description State Last Operation : : : : : : : 10002 Select * from HP_AlertIndication where (PerceivedSeverity >= 4) WQL root/cimv2 Admin Filter Enabled Filter State Add Filter HP_AlertIndication is derived from CIM_AlertIndication and HP_DeviceIndication is derived from HP_HardwareIndication. HP_HardwareIndication is derived from HP_AlertIndication.
Sending test event for memory monitor. NOTE: You can also send test events for other devices that the SFMIndicationProvider monitors. For information on the devices monitored by the SFMIndicationProvider, see Table 2 (page 12). To view the list of events, enter the following command at the HP-UX prompt: # evweb eventviewer -L A list of events along with the details such as event archive number, severity, and event category are displayed by querying the Event Archive.
4 Administering indications and instances using HP SIM This chapter describes System Fault Management (SFM) administration on a remote system using HP Systems Insight Manager (HP SIM). NOTE: You can perform similar tasks using other management applications that are compliant with the Common Information Model (CIM) (2.8) schema (or later) of the Distributed Management Task Force (DMTF). The terms events and indications are used interchangeably in this document.
2. To create subscriptions, select Options-->Protocol Settings-->Global Protocol Settings in the HP SIM Home page, as shown in Figure 4-1. Figure 2 HP SIM Home Page The Global Protocol Settings window is displayed, as shown in Figure 4-2. Figure 3 Global protocol settings 3. 34 In Figure 4-2, under default WBEM settings, select Enable WBEM. Click OK to save your settings.
4. Select Configure->Configure or Repair Agents, as shown in Figure 4-3. Figure 4 Configuration The Configure or Repair agents window is displayed, as shown in Figure 4-4. Figure 5 Configure or Repair Agents 5. From the Add targets by selecting from: list in Figure 3-4, select All systems to view and select the systems.
the selected system. The list of systems is displayed in the Select Target Systems window, as shown in Figure 6. Figure 6 Select Target Systems 6. To select all the systems in the network, select the Select “All Systems” itself check box, as shown in Figure 4-5. Click Apply. The Verify Target Systems window is displayed, as shown in Figure 4-6.
7. Select the appropriate check box to verify the target systems and click Next, as shown in Figure 4-6. The Enter credentials window is displayed, as shown in Figure 4-7. Figure 8 Enter credentials 8. Enter your credentials in the given fields, as shown in Figure 4-7. Click Next. The Configure or Repair settings window is displayed, as shown in Figure 4-8. Figure 9 Configure or Repair settings 9. On the Configure or Repair settings window, click Run Now.
Figure 10 Task Results 10. To obtain a printable report of the indication subscription details, click View Printable Report at the bottom of the window. The report is displayed, as shown in Figure 4-10. Figure 11 Printable Report of the indication Subscription NOTE: For more information, see the HP Systems Insight Manager 6.3 Installation and Configuration Guide for HP-UX at: http://www.hp.
1. Select All Events in the left pane of the HP SIM window. The list of events is displayed, as shown in Figure 4-11. Figure 12 Events list 2. To view the details of an event, select the event. The details are displayed at the bottom of the same window, as shown in Figure 4-12.
3. To obtain the printable version of the event details, click View Printable Details at the bottom of the window. The printable report is displayed in a new window, as shown in Figure 4-13.
Table 6 EMS, WBEM and Evweb events severity values (continued) EMS severity WBEM severity Evweb severity 4 Serious 6 Critical 7 Critical 5 Critical 7 Fatal/Non-recoverable 7 Critical NOTE: • Perceived severities in Syslog is same as WBEM severities. • The WBEM severities are standard. Their number can be seen as the severity value for the actual events recorded in /var/opt/sfm/log/event.log. The Evweb severity numbering matches the HP SMH system status.
Table 8 Property Representation EMS Hardware Monitors EMS wrapper provider / Native indication provider Event Time EventTime Severity PerceivedSeverity Event EventID System SystemName Summary Summary Description of Error Description Probable Cause/ Recommended Action ProbableCauseDescription and RecommendedAction (these two are separate fields) System Serial Number SystemSerialNumber InquiryVendorID HWManufacturer Physical Device Path HWLogicalLocation InquiryProductID DeviceModel Ph
NOTE: The Severity levels in Table 4-5 indicate EMS severity. Table 11 (page 43) displays the default event destinations for SysFaultMgmt. Table 11 Default monitoring requests for each monitor Default notification method Severity levels SysFaultMgmt Textlog All textlog: /var/opt/sfm/log/event.log Syslog MAJOR Available CRITICAL FATAL/NON-RECOVERABLE E-MAIL None Not Available Evweb DB All Available (evweb eventviewer -L) NOTE: The Severity levels in Table 4-6 indicate WBEM severity.
The sfmconfig -a -r command is used to change the state of a subsystem. When this command is not working with processor, the user should check for errors. When a CPU is deactivated on a system due to an action taken against an error symptom, the user tries to use sfmconfig command to make the CPU state back to OK The change does not happen unless the processor which is faulty is replaced or it is acquitted from the Onboard Administrator on HP Superdome 2.
5 Administering indications and instances using HP SMH This chapter describes the SFM administration tasks that you can perform using HP SMH on a local system.
NOTE: Starting September 2009 release, in HP SMH GUI, you can refer to “The equivalent command line” option, to view command line information about processors. For more information, view cprop manpage. See "man cprop" 1. Select Show All under System on the HP SMH home page. The system page is displayed. Figure 16 System Management Homepage 2. Select Processors under System on the HP SMH home page. Information about the processors is displayed. 3. To return to the HP SMH home page, click on Home.
Viewing information about System Summary To obtain information about system summary, such as the model, role, UUID, UUID (Logical), Serial number, Serial number (Logical) and many more, complete the following steps: 1. Select System Summary under System on the HP SMH home page. System summary information is displayed. 2. To return to the HP SMH home page, click on Home.
Viewing information about Management Processor To obtain information about the Management Processor (MP), such as its IP address, status, and URL, complete the following steps: 1. Select Management Processor under System on the HP SMH home page. Information about the management processor is displayed. 2. To return to the HP SMH home page, click on Home.
1. Select Blade under System on the HP SMH home page. Information about the Blade is displayed. 2. To return to the HP SMH home page, click on Home. Viewing information about Cell Blade To obtain information about the Cell Blade, such as the Status, Hardware Path and OA partition Information of the enclosures, complete the following steps: 1. Select Cell Blade under System on the HP SMH home page. Information about the Cell Blade is displayed. 2. To return to the HP SMH home page, click on Home.
For more information, see HP System Management Homepage Online Help. In HP SMH, go to the Help menu. Administering indications using Evweb This section provides an overview of Evweb and describes how to use Evweb for administrative tasks, such as creating and managing subscriptions for indications.
Launching Evweb for administration You can launch Evweb either through the CLI or through the HP SMH GUI. To launch Evweb for administering event subscriptions using the CLI, enter the following command at the HP-UX prompt: # evweb subscribe To use HP SMH GUI to launch Evweb for administering event subscription, complete the following steps: 1. Log in to HP SMH. To log in to HP SMH, enter http://:2301 in the address bar of the Web browser. The HP SMH login screen is displayed. 2. 3.
1. 2. Repeat steps 1-5 from “Launching Evweb for administration” (page 51). Select Create subscription in the action pane on the top right corner of the Event subscription administration page. The Create subscription page is displayed. 3. Provide appropriate information in the fields present in the Create subscription page. NOTE: 4. 5. It is mandatory to specify a unique name for creating an event subscription.
IMPORTANT: The subscription criteria is not copied when you copy an HP Advised event subscription. Therefore, ensure that you specify the subscription criteria in the Copy and create subscription page. NOTE: 5. It is mandatory to specify a unique name in the Subscription name. Select Create on the Copy and create subscription page. Evweb creates the event subscription and displays a confirmation message. 6. Click OK on the confirmation message window. NOTE: There is no CLI equivalent for this action.
6. Select Modify in the Modify subscription page. Evweb modifies the event subscription and displays a confirmation message. 7. Click OK on the confirmation message window. For more information on modifying an event subscription using the HP SMH GUI, select Help on the action pane of the Modify event subscription page.
Example 1 # evweb subscribe -L Subscription Name HP Known Is Deprecated Event Archive Email Syslog ====================== ======== ============== ============== ======== ======== HP_defaultSyslog FALSE FALSE FALSE FALSE TRUE test FALSE FALSE TRUE FALSE FALSE FALSE TRUE FALSE FALSE HP_General Filter@1_V1 TRUE # evweb subscribe -Mn test -r The execution of 'evweb subscribe' command was successful.
IMPORTANT: The subscription criteria are not copied when you copy an HP Advised event subscription. Therefore, ensure that you specify the subscription criteria in the Copy and modify subscription page. For more information on copying and modifying an event subscription using the HP SMH GUI, select Help on the action pane of the Copy and Modify event Subscription page. Deleting Evweb event subscriptions You must periodically delete event subscriptions that are not required.
For more information on deleting event subscriptions using the CLI, see evweb_subscribe(1). Configuring E-mail Consumer The E-mail Consumer is a component of Evweb that receives indications from the WBEM Services and redirects them to an SMTP server. Normally, the local system itself is the e-mail server. In such cases, you need not configure the E-mail Consumer. If the e-mail server is not on the local system, you must configure the E-mail Consumer.
The Event subscription administration page is displayed. The Event subscription administration page displays a summary of the the event subscriptions, in a tabular format. In this document, this table is referred to as the event subscription summary table. For more information on viewing a summary of an evweb event subscription using the HP SMH GUI, select Help on the action pane of the event subscription page.
Viewing details of an event subscription using the GUI To view details of an event subscription, complete the following steps: 1. Repeat steps 1-5 from “Launching Evweb for administration” (page 51). The Event subscription administration page displays the event subscription Table. 2. Select the event subscription from the event subscription Table. The details of the event subscription is displayed at the end of the event subscription Table.
1. Repeat steps 1-5 from “Launching Evweb for administration” (page 51). The Event subscription administration page displays the event subscription table. 2. Select View external subscription in the action pane on the top right corner of the page. The View external subscriptions page is displayed. For more information on viewing external event subscription using the HP SMH GUI, select Help on the action pane of the View external event subscription page.
This section addresses the following topics: • “Launching Evweb for viewing WBEM indications” (page 61) • “Searching for the subscribed WBEM events” (page 62) • “Viewing summary information about WBEM events” (page 63) • “Viewing detailed information about WBEM events” (page 63) • “Deleting WBEM Events from the Event Archive” (page 64) NOTE: Evweb enables both administrators and non-administrators to search and view WBEM events. However, only administrators can delete WBEM events.
Searching for the subscribed WBEM events Evweb enables you to search the Event archive for subscribed WBEM events. The Evweb GUI provides a link, EMT Search, using which you can obtain error, cause, and recommended solutions for errors that may be generated on an HP-UX 11i v3 system. For more information about how to use EMT to search for WBEM events, see “Querying the Common Error Repository” (page 72).
• -t[eq|le|ge|bw] ()[,] • -v • -x For information on searching the WBEM events using the CLI, see evweb_eventviewer(1). Viewing summary information about WBEM events You can view summary information about events stored in the Event Archive database. Viewing summary information using GUI To view summary information about WBEM events, repeat steps 1-5 from “Launching Evweb for viewing WBEM indications” (page 61).
The screen displays detailed information about the WBEM events. For information on viewing detailed information of WBEM events using the CLI, see evweb_eventviewer(1). Deleting WBEM Events from the Event Archive You can delete a single event or multiple events at a time. Deleting an Event using GUI To delete an event, complete the following steps: 1. Repeat steps 1-5 from “Launching Evweb for viewing WBEM indications” (page 61). 2.
This section discusses the following topics: • “Overview” (page 65) • “Searching Low Level Logs using Simple Search” (page 65) • “Searching Low Level Logs using Advanced Search” (page 66) • “Viewing List of Low Level Logs” (page 67) • “Viewing Details of Low Level Logs” (page 68) Overview The low level log is required to view information about hardware details and system errors. The Log Viewer enables you to view low level log information from the log database on a local HP-UX system.
3. Click Sign In on the login screen. The HP SMH home page is displayed. 4. Select Logs on the main menu. The Logs page is displayed. 5. Select Log Viewer in the Evweb box. The Log Viewer page is displayed. 6. 7. Provide appropriate information in the fields present in the Log Viewer page. Click Search on the Log Viewer page. Based on the search criteria, the log records are displayed in a tabular format.
• -s [LogId|LogIndex|DeviceId|DeviceType|TimeOfOccurence] • -o -c NOTE: The -s, -o,and the -c switches can be used with the -L option only. For information on searching for low level log information using the CLI, see evweb_logviewer(1). Viewing List of Low Level Logs You can view a list of low level logs summary using the Log Viewer.
41 40 39 38 37 0 0 0 0 0 0 0 0 0 0 Memory Memory Memory Memory Memory Wed Thu Thu Thu Sun Dec Dec Dec Dec Dec 30 2009 00:36:1 10 2009 20:38:2 10 2009 20:31:3 10 2009 20:25:0 6 2009 23:37:08 For information on viewing a list of low level logs using the CLI, see evweb_logviewer(1). Viewing Details of Low Level Logs You can view a details of low level logs using the Log Viewer.
Tracing Evweb This section provides an overview of tracing and information about the various trace levels in Evweb. This section also describes administrative tasks, such as enabling, modifying, and disabling tracing.
Table 17 Trace Levels Trace Level Description 1-Critical The system logs only those situations in Evweb that cause major failures. Example: The database server is not functioning properly or is down. 2-Error The system logs those situations that generate an error. Example: There is more than one subscription name. Evweb accepts only one subscription name. Critical situations are also logged at the Error trace level. 3-Warning The system logs situations that result in warning messages.
For more information on enabling tracing using the HP SMH GUI, select Help on the action pane of either the Event Viewer or the Event Subscription Administration page. Enabling Tracing using the Evweb CLI To enable tracing using the Evweb CLI, you must export the environment variable, EVWEB_TRACE_LEVEL. To export the environment variable, enter the following command at the HP-UX prompt: # export EVWEB_TRACE_LEVEL= Tracing is now enabled. The trace value is the trace level that you have set.
3. Click Sign In on the login screen. The HP SMH home page is displayed. 4. Do one of the following: • Select Tools -> Subscription Administration. or • Select Logs -> Event Viewer. NOTE: 5. The Disable Tracing option is not displayed if tracing is not enabled. Select Disable Tracing available on the top right corner of the page. The tracing is disabled and a confirmation message is displayed. 6. Click OK on the confirmation message window.
The EMT supports the following user groups: • Administrator • Non-administrator In the CLI, any user with superuser privileges is an administrator. However, in the HP System Management Homepage (HP SMH) GUI, the user groups in EMT are mapped internally to the user groups defined in the HP SMH. The Administrator user group in the HP SMH maps to administrators in EMT. The Operator and the User user groups in the HP SMH map to non-administrators in EMT.
Querying CER for Events using the CLI To query CER for events using the CLI, enter the following command at the HP-UX prompt: #emtui { -q [-w]} | { -i } Where: -q is an option that enables you to specify a query string to query the CER for information about errors, cause, and recommended actions. -w is an option that enables you to specify the match type. Following are the match types: -i • any (default) - Searches for at least one word specified in the query string.
The List Events page, which contains the Error Summary Table, is displayed. For information about viewing summary information of events in CER using the HP SMH GUI, select Help on the action pane of the List Events page. Viewing Summary Information using the CLI To view summary information about events in CER using the CLI, enter the following command at the HP-UX prompt: # emtui -b Where: -b is an option used to view information about events in brief. A list of events in CER is displayed.
Adding a Custom Solution If you are a system administrator, you can add your own solution for an error generated on a HP-UX system. The custom solution is permanently stored in CER and is available to all EMT users. Adding a Custom Solution using the GUI To add a custom solution using the HP SMH GUI, complete the following steps: 1. Repeat steps 1-5 from “Launching EMT” (page 73). 2. Search for the event from CER using either the Simple Search or the Advanced Search feature. 3.
3. If there is no cause associated with the event, skip to Step 4. If a cause is associated with the event, select the cause for the event from the Detailed Error Information (Administrative View) pane. You can select multiple causes for an event. 4. Click Modify Selected Solution on the right corner of the Detailed Error Information (Administrative View) pane. The Modify a Custom Solution page is displayed. 5.
# emtui -d -u Where: -d is an option used to delete a custom solution present in the CER. -u is an option used to specify the number associated with the custom solution that you want to delete. You can also use the following switches with the -m option: • -c • -i For information on deleting custom solution in the CER using the CLI, see emtui(1).
1. Repeat steps 1-5 from “Launching EMT” (page 73). NOTE: The Enable Tracing option is not displayed if tracing is already enabled. Instead, the Disable Tracing and the Modify Tracing option are displayed. 2. Select Enable Tracing on the top right corner of the page. The Enable Tracing page is displayed. 3. 4. Set the trace level by selecting the level from the trace level list. Select Enable Tracing. The tracing level is set and a confirmation message is displayed. 5.
3. Click OK on the confirmation message window. For more information about disabling tracing using the HP SMH GUI, select Help on the action pane of the Disable Tracing page. Disabling Tracing using the EMT CLI To disable tracing using the EMT CLI, you must reset the trace value. To reset the trace value, enter the following command at the HP-UX prompt: # unset EMT_TRACE_LEVEL Tracing is now disabled. NOTE: 80 Tracing is automatically disabled at the end of an EMT session.
6 Troubleshooting SFM This chapter describes how to troubleshoot SFM providers and EVWEB. This chapter addresses the following topics: • “Troubleshooting instance providers” (page 81) • “Troubleshooting indication providers” (page 86) • “Troubleshooting EVWEB” (page 91) For information on Upgrade Installation of the Postgres 8.4.8, see the Installation scenarios of Postgres 8.4.8 at www.hp.
Table 19 Troubleshooting instance providers (continued) Problem Cause Solution 6. After the provider module is registered, create a link between the SFM providers and the CIMOM by entering the following command at the HP-UX prompt: On Itanium-based systems, enter: # ln -s /opt/sfm/lib/libsfmproviders.1\ /opt/wbem/providers/lib/libsfmproviders.so On PA-RISC-based systems, enter: # ln -s /opt/sfm/lib/libsfmproviders.1\ /opt/wbem/providers/lib/libsfmproviders.sl 7.
Table 19 Troubleshooting instance providers (continued) Problem Cause Solution 1. Enter the following command at the HP-UX prompt to disable SFMProviderModule: # cimprovider –d –m SFMProviderModule 2. Enter the following command at the HP-UX prompt to enable SFMProviderModule: # cimprovider –e –m SFMProviderModule Alternatively, you can enter the following command at the HP-UX prompt to start SFMProviderModule: # sh /opt/sfm/bin/restart_sfm.sh The logs to /var/opt/sfm/log/state.
Table 21 Troubleshooting instance providers (continued) Problem: Requests for instances do not return any value. Causes Solution # cimserver Cause 2 The provider is not registered properly. To register the provider, complete the following steps: 1. Enter the following command at the HP-UX prompt: # cimprovider -ls | grep SFMProviderModule 2. If the following output is displayed, all the providers are registered properly: SFMProviderModule OK 3.
Table 21 Troubleshooting instance providers (continued) Problem: Requests for instances do not return any value. Causes Solution MODULE OperatingSystemModule ComputerSystemModule ProcessModule IPProviderModule SFMProviderModule STATUS OK OK OK OK Degraded If the status of SFMProviderModule is Degraded as displayed in the given output, SFMProviderModule is not running. To enable SFMProviderModule, complete the following steps: 1.
Table 23 Troubleshooting instance providers (continued) Problem: Indications fulfilling the conditions defined in the HP-Known HP-Defined filters, are not logged in the Event Archive. Cause Solution To execute the file, enter the following command at the HP-UX prompt: # wbemexec /EnumerateInstances.
Table 25 Troubleshooting indication providers (continued) Problem: Indications corresponding to events generated by the Event Monitoring Service (EMS) monitors, are not logged in the Events List. Causes Solution If the status displayed is not OK, the provider module is not registered properly. To register the provider module, enter the following command at the HP-UX prompt: # cimmof -nroot/PG_InterOp /opt/sfm/schemas/mof/SFMProvidersR.
Table 25 Troubleshooting indication providers (continued) Problem: Indications corresponding to events generated by the Event Monitoring Service (EMS) monitors, are not logged in the Events List.
Table 25 Troubleshooting indication providers (continued) Problem: Indications corresponding to events generated by the Event Monitoring Service (EMS) monitors, are not logged in the Events List. Causes Solution Cause 4 Create the following enumerateInstances_sub.xml file and save it in any location: Subscriptions do not exist. PAGE 90Table 25 Troubleshooting indication providers (continued) Problem: Indications corresponding to events generated by the Event Monitoring Service (EMS) monitors, are not logged in the Events List. Causes Solution CIM_ComputerSystem hpdst348 Cause 5 The indication providers are not loaded properly.
Table 25 Troubleshooting indication providers (continued) Problem: Indications corresponding to events generated by the Event Monitoring Service (EMS) monitors, are not logged in the Events List.
Table 26 Troubleshooting EVWEB (continued) Problem Cause following errors are displayed: Solution in the Event Archive. The second message Could not fetch the indicates that the EVWEB is unable to details of the establish a connection events. with the Event Archive. The connection to The Event Archive the database could Database service is not not be established. running properly. If the output of this command is sfmdb, the Event Archive Database service is running properly.
Table 26 Troubleshooting EVWEB (continued) Problem Cause WBEM Indications are not SFMProviderModule is mailed to your email not running. address.
Table 26 Troubleshooting EVWEB (continued) Problem Cause Solution Both EMS and SFM log the same symptom in the syslog. The syslog functionality is available from SFM Version C.06.00.07.01, September 2009 release, to provide a summary of event information of critical and serious events. The default subscription to syslog HP_defaultSyslog is configured.
A EMT Message Definition Following is a sample EMT Message file: $ Descriptor Header begins $ <> DescriptorID=0000023100000010800000AA006D2EA3 $ <> ProductName = myProduct $ <> ProductID = ID $ <> ProductEmailAlias=myproduct@abc.com $ <> OrgName = myorg $ <> OrgType = ISV $ <> Subsystem={(Type=EMS, Name= dm_chassis),(Type=WBEM, Name=FileSystemProvider)} $ <> ProductCategory=Kernel $ <> MsgCat={ID=1,(Path=./ lvmcommonmessages.cat, Locale= ja_JP.
Table 27 EMT Message File Description (continued) Tag Description Usage ProductCategory Specify one or more of the following product ProductCategory= categories that best describes your product: Kernel,IO,Network • Hardware • Network • IO • Kernel • Commands • Others MsgCat Specify a list of message catalogs. The MsgCat tag has the following attribute: ID – A unique number used to identify an error message. MsgCat={(ID=1, Path=../../../bin/cat/en_US.iso88591/module1.cat,LocaleName= en_US.
Table 27 EMT Message File Description (continued) Tag Description Usage only one cause and one or more corrective action for a given error message, the Action tags are associated with the Cause tag. In such a situation, the Cause_Action tag is not mandatory. For any error, a cause can be specified without specifying the corrective action. However, a corrective action cannot be specified without specifying a cause. WBEMDetail Specify WBEM specific details of a message.
B Interpretation of HP SMH instances This appendix describes the fields and enables you to interpret the instances in the HP SMH property pages.
Processor instances This section describes the processor instances. Figure 20 Sample Processors property page Table 28 (page 99) describes the fields and enables you to interpret the values displayed in Figure 20 (page 99). Table 28 Description of the Processors Fields and Values Fields and Values Description Status Indicates the status of the processors. An OK status indicates that all the processors are functioning properly. Click Events to see the details of the errors.
Memory instances This section describes the memory instances. Figure 21 Sample Memory property page Table 29 (page 100) and Table 30 (page 101) describes the fields and enables you to interpret the values displayed in Figure 21 (page 100). Table 29 Description of the Memory Slots Fields and Values Fields and Values Description Status Indicates the status of the memory module. An OK status indicates that all the modules are configured properly.
Table 29 Description of the Memory Slots Fields and Values (continued) Fields and Values Description Part Number Indicates the part number of the memory. HashID Identifies an instance of the device. Table 30 Description of the Empty Slots Fields and Values Fields and Values Description Location Indicates the location of the memory. Attributes such as Cabinet Number, Cell Slot, and DIMM Slot help narrow down the location of the memory module.
Table 31 Description of the Memory Slots Fields and Values (continued) Fields and Values Description Logical memory information Physical memory information Device Bay Information NOTE: Indicates the URL to launch the blade information page on the OA. Memory information displayed is as viewed from a hard partition (nPar).
System Summary instances This section describes the system summary instances. Figure 23 Sample System Summary property page Table 32 (page 103), Table 33 (page 104) and Table 34 (page 104) describes the fields and enables you to interpret the values displayed in Figure 23 (page 103). Table 32 Description of the General Information Fields and Values Fields and Values Description Model Describes the system model.
Table 32 Description of the General Information Fields and Values (continued) Fields and Values Description UUID UUID (Logical) Universally Unique ID (UUID) indicates the asset number of the system. Indicates the UUID of the logical server. A logical server is a software configuration that can be applied to a server blade or a virtual machine. Also, you can move a logical server from one server blade or a virtual machine to another.
Cooling Device instances This section describes the cooling device instances. Figure 24 Sample Cooling device property page Table 35 (page 105) describes the fields and enables you to interpret the values displayed in Figure 24 (page 105). Table 35 Description of the Cooling Device Fields and Values Fields and Values Description Status Indicates the status of the fans. An OK status indicates that all the modules are configured properly.
Power supply instances This section describes the power supply instances. Figure 25 Sample Power property page Table 36 (page 106) describes the fields and enables you to interpret the values displayed in Figure 25 (page 106). Table 36 Description of the Power Supply Fields and Values Fields and Values Description Status Indicates the status of the power supply. An OK status indicates that the power supplies are configured properly.
Temperature instances This section describes the temperature instances. Figure 26 Sample Temperature property page Table 37 (page 107) describes the fields and enables you to interpret the values displayed in Figure 26 (page 107). Table 37 Description of the Temperature Fields and Values Fields and Values Description Status Indicates whether the sensor temperature in the system is normal or not. However, the status of the sensor temperature does not reflect the status of the cooling devices.
Voltage instances This section describes the voltage instances. Figure 27 Sample Voltage property page Table 38 (page 108) describes the fields and enables you to interpret the values displayed in Figure 27 (page 108). Table 38 Description of the Voltage Fields and Values Fields and Values Description Status Indicates whether the sensor voltage in the system is normal or not. An OK status indicates that the sensor voltage in the system is normal. HashID Identifies an instance of the device.
FRU Information instances This section describes the FRU Information instances. Figure 28 Sample FRU Information property page Table 39 (page 109) describes the fields and enables you to interpret the values displayed in Figure 28 (page 109). Table 39 Description of the MP Fields and Values Fields and Values Description Name Indicates the FRU Name of the Physical Element. Serial Number Indicates the serial number of the FRU. HashID Identifies an instance of the device.
Management Processor instances This section describes the Management Processor (MP) instances. Figure 29 Sample MP property page Table 40 (page 110) describes the fields and enables you to interpret the values displayed in Figure 29 (page 110). Table 40 Description of the MP Fields and Values 110 Fields and Values Description Status Indicates whether the Management Processor (MP) is functioning properly or not. An OK status indicates that the MP is functioning properly.
Firmware Information instances This section describes the Firmware Information instances. Figure 30 Sample Firmware Information property page Table 41 (page 111) describes the fields and enables you to interpret the values displayed in Figure 30 (page 111). Table 41 Description of the Firmware Information Fields and Values Fields and Values Description Name Indicates the name of the entity, such as the system firmware, MP, or the system backplane cell, whose firmware information is displayed.
Enclosure Information instances This section describes the Enclosure instances. Figure 31 Sample Enclosure property page Table 42 (page 112) describes the fields and enables you to interpret the values displayed in Figure 31 (page 112). Table 42 Description of the Enclosure Information Fields and Values 112 Fields and Values Description Status Indicates the status of the enclosure. An OK status indicates that the components of the enclosure are functioning properly.
Complex-wide Info instances This section describes the Complex-wide Info instances. Figure 32 Sample Complex-wide Info property page Table 43 (page 114), Table 44 (page 114) and Table 45 (page 114) describes the fields and enables you to interpret the values displayed in Figure 32 (page 113).
Table 43 Description of the Complex Information Fields and Values Fields and Values Description Complex Name Describes user defined name for the complex. Model Defines Model identification string. Serial Number Indicates the serial number of the complex as assigned by the original manufacturer. Revision Displays string for the revision number of the profile, consisting of the major and minor revision numbers concatenated with a period as a separator.
Cell Board instances This section describes the Cell Board instances. Figure 33 Sample Cell Board property page Table 46 (page 115) describes the fields and enables you to interpret the values displayed in Figure 33 (page 115). Table 46 Description of the Cabinet Fields and Values Fields and Values Description Firmware Version Displays string for the firmware revision number, consisting of the major number separated from the minor number by a period. Status Indicates the status of the component.
Table 46 Description of the Cabinet Fields and Values (continued) Fields and Values Description Total Processor Slots Indicates the number of processor module slots on the cell. Total Empty Processor Slots Indicates the number of all empty processor slots. Processors Per Module Indicates the number of processors per processor module on the cell. Total Installed Processor Modules Indicates the number of all installed processor modules in the cell.
Partition Information instances This section describes the Partition Information instances. Figure 34 Sample Partition Information property page Table 47 (page 117) describes the fields and enables you to interpret the values displayed in Figure 34 (page 117). Table 47 Description of the Partition Fields and Values Fields and Values Description Partition Name Describes user defined name with the numeric label for the Partition. nPartition ID Indicates the ID of the nPartition in the complex.
Table 47 Description of the Partition Fields and Values (continued) 118 Fields and Values Description Total Deconfigured Processor Modules Indicates the number of all deconfigured processor modules in the partition. Total Installed Memory Displays the total amount of memory installed in the partition, in megabytes. Total Installed Cells Indicates the number of all cells installed in the partition. Total Active Cells Indicates the number of all active cells in the partition.
Blade instances This section describes the Blade instances. Figure 35 Sample Blade property page Table 48 (page 119) describes the fields and enables you to interpret the values displayed in Figure 35 (page 119). Table 48 Description of the Blade Fields and Values Fields and Values Description Status Indicates the status of the blade. Hardware Path Indicates the hardware path of the blade. Serial Number Indicates the serial number of the blade.
Cell Blade instances This section describes the Cell Blade instances. Figure 36 Sample Cell Blade property page Table 49 (page 120) describes the fields and enables you to interpret the values displayed in Figure 36 (page 120). Table 49 Description of the Cell Blade Fields and Values Fields and Values Description Status Indicates the status of the blade. Hardware Path Indicates the hardware path of the blade.
Launch the Onboard Administrator To access the Onboard Administrator (OA) from the property pages, complete the following steps: 1. Click on the Onboard Administrator link from the property page. Figure 37 Onboard Administrator 2. The OA login page opens in a new browser window. Figure 38 OA login page 3. Enter the Onboard Administrator User name and Password.
C Syslog property order This appendix describes the order for the properties (IndicationIdentifier, EventID, PerceivedSeverity, ProviderName and Summary) in the event message which is written in syslog by the HP_defaultSyslog subscription. NOTE: The term legacy refers to HP Integrity Servers with Intel(R) Itanium(R) processors older than 9300. The term HP Integrity Servers refers to Intel(R) Itanium(R) 9300 processors.
D SFM configuration files This appendix describes the items of different configuration files in SFM. The user can configure only the following three files: • “DBConfig.xml” (page 123) • “FMLoggerConfig.xml” (page 123) • “evweb.conf” (page 124) DBConfig.xml The DBConfig.xml is a configuration file used to set SFM DB parameters of the Event storage (evweb) and the Common Log storage (LOGDB) databases, and their corresponding archive database (evweb_history and LOGARCHDB). SFM allows the DBConfig.
evweb.conf The evweb.conf is a configuration file used to set parameters used by the email consumer. SFM allows the evweb.conf being modified at anytime and take effect immediately by executing the following command: NOTE: If the email server is not set to local machine, it is required to add hostname of generating events server into /etc/mail/sendmail.cw file on the email server and restart sendmail of the email server. Else, the events mail will not be delivered to subscribed-ID defined in subscription.
Glossary A-B Admin-defined event subscription Subscriptions created by the administrator using the CLI. These subscriptions cannot be deleted. Admin-defined filters Filters that can be created, deleted, and modified to set the criteria for indications that must be logged. C Central Management Server (CMS) The server monitoring the client systems in the network using SFM. CIM client An entity in WBEM architecture which sends CIM Operation requests and receives CIM Operation responses.
External subscriptions These are subscriptions created by tools other than EVWEB. H HP System Management Homepage (HP SMH) HP's management application installed on the local system that uses WBEM instrumentation on operating systems such as HP-UX, Linux, and Windows. HP Systems Insight Manager (HP SIM) HP's management application installed on the CMS that uses WBEM instrumentation on operating systems such as HP-UX, Linux, and Windows.
SysFaultMgmt The name of the bundle that includes the SFM software. T-V Tracing Tracing is an error-logging and reporting facility provided by EVWEB and EMT. W-Z WBEM (Web-Based Enterprise Management) A collection of standards that aid large-scale systems management. WBEM allows management applications to monitor systems in a network.
Index Central Management Server see CMS CER, 72 CIMOM, 16 cimserver, 81, 84 -s option, 81, 83 cimserver -s, 81, 84 CMS, 16 command-line interface, 19 Common Information Model Object Manager see CIMOM configuration monitor mode, 31 SFM, 20 cooling devices on a system, 47 creation subscription, 33 cron, 15 custom solution adding, 76 deleting, 77 modifying, 76 view, 74 Enforce dependency, 19 error metadata, 72 Event Archive, 30, 92 HP-Known HP-Defined filter, 86 troubleshooting, 91, 92, 93 event list, 38 Eve
Filter Metadata, 81 Memory, 9, 81 IPMI Event Viewer slview, 9 J jobid, 24 L Log Viewer, 65 Archive Log Database, 65 Current Log Database, 65 Logfile, 23, 24 logs /var/opt/sfm/log/sfm.log file, 69, 78 /var/sam/log/samlog.
7 Support and other resources About this document This document describes how to install, administer, and troubleshoot the System Fault Management (SFM) software and its components. Document updates may be issued between editions to correct errors or to document product changes. To ensure that you receive the updated or new editions, subscribe to the appropriate product support service. Contact your local HP sales representative for more information. This document can also be found online at: http://www.hp.
Chapter 5 Administering Indications and instances using HP SMH Describes how to use the HP System Management Homepage (HP SMH) GUI to administer indications and view instances on the local system. Chapter 6 Troubleshooting SFM Describes how to troubleshoot SFM providers and EVWEB. Appendix A Appendix A Describes the EMT message file. Appendix B Appendix B Interpretation of HP SMH instances. Appendix C Appendix C Describes the Syslog property order.
New and changed information in this edition • The Table 3 (page 14) lists the instance and indication providers support on different platforms. • A new appendix, “ Syslog property order” (page 122) describes the order for the three properties (EventID, PerceivedSeverity and ProviderName) in the event message which is written in syslog by the HP_defaultSyslog subscription. Related information Additional information about SFM is available at: http://www.hp.
8 Documentation feedback HP is committed to providing documentation that meets your needs. To help us improve the documentation, send any errors, suggestions, or comments to Documentation Feedback (docsfeedback@hp.com). Include the document title and part number, version number, or the URL when submitting your feedback.
