Distributed Systems Administration Utilities User’s Guide HP Part Number: T2786-90337 Published: March 2009 Edition: 2.
Copyright Copyright ©2007, 2009 Hewlett-Packard Company Legal Notices Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor’s standard commercial license. The information contained herein is subject to change without notice.
Table of Contents About this Document..........................................................................................................9 Intended Audience.................................................................................................................................9 Typographic Conventions......................................................................................................................9 Related Information.........................................................
3.3.1 Using the Log Consolidation Wizard......................................................................................50 3.3.1.1 Configuring a Log Consolidation Standalone Server with clog_wizard........................50 3.3.1.2 Configuring a Serviceguard Cluster as a Log Consolidation Server with clog_wizard................................................................................................................................52 3.3.1.3 Cluster Configuration Notes for clog..................
A.3 Uninstalling the DSAU rpm from your System.............................................................................91 B HP-Supported Open Source pdsh Options................................................................93 Index.................................................................................................................................
List of Figures 2-1 3-1 3-2 4-1 6 cfengine Overview.........................................................................................................................19 syslog-ng Log-Forwarding Configuration....................................................................................48 syslog-ng Log Consolidator Configuration...................................................................................49 pdsh Architecture ..................................................................
List of Tables 1 2 1-1 1-2 1-3 1-4 1-5 1-6 1-7 1-8 2-1 3-1 3-2 3-3 4-1 4-2 4-3 A-1 Conventions.....................................................................................................................................9 DSAU User's Guide Publishing History.........................................................................................9 Configuration Synchronization Command...................................................................................12 Consolidated Logging Commands...
About this Document Distributed Systems Administration Utilities provide tools to simplify the management of groups of systems and of Serviceguard clusters. This document provides information on each component tool in separate chapters. Intended Audience This document is written for system administrators to assist them in learning about Distributed Systems Administration Utilities and how to use them. Typographic Conventions Table 1 Conventions Book Title Title of a book or other document.
HP Encourages Your Comments HP encourages your comments concerning this document. We are truly committed to providing documentation that meets your needs. Please submit comments to: http://docs.hp.com/assistance/feedback.html Please include the following information: • Document title (Distributed Systems Administration Utilities User’s Guide ) • Manufacturing part number (T2786-90337) • Any comment, error found, or suggestion for improvement you have concerning this document.
1 Introduction The Distributed Systems Administration Utilities provide several tools for simplifying the management of groups of systems and of Serviceguard clusters. There are three utilities: • Configuration Synchronization: - with this utility, based on the open source tool cfengine or “configuration engine,” the administrator can centrally define management actions to be applied to a set of managed systems. cfengine is a client/server based tool.
the distribution of ssh keys. The companion utility Parallel Distributed Copy (pdcp) enables file and directory copies to be performed in parallel to a set of remote systems. The dshbak filter allows the output from multiple systems to be formatted and consolidated for better on-screen presentation. The cexec, ccp, ckill, cps, and cuptime tools are wrappers around the pdsh and pdcp commands optimized for use in a Serviceguard cluster. They default to executing commands clusterwide.
Table 1-3 Command Fanout Commands (continued) Command What it Does When to Use it cuptime Reports uptime(1) information for multiple systems. In a Serviceguard cluster, issues command clusterwide by default. To check uptime, users, and load averages. cwall Displays a wall(1) broadcast message on multiple hosts. In a Serviceguard cluster, issues command clusterwide by default. To broadcast a message to all logged-in users across a group of systems.
Table 1-7 Open Source syslog-ng Command Command What it Does syslog-ng Tool that performs consolidated logging. 1.
The values of the environment variables such as $SGCONF are used in shell command examples. 1.5 Red Hat and SLES File Path Difference The location of the syslog-ng configuration file in Red Hat is different from the location in SUSE Linux Enterprise Server (SLES). In Red Hat the file is located at /etc/syslog-ng.conf and in SLES it is located at /etc/syslog-ng/syslog-ng.conf. 1.
2 Configuration Synchronization Managing the configuration and configuration drift of a set of distributed systems is a constant challenge for system administrators. There are a variety of tools available to help manage various aspects of multi-system configuration management. For example, for account management, standard solutions include the Network Information System (NIS) and Lightweight Directory Access Protocol (LDAP).
appropriate for each group of managed clients. For example, every five minutes, once an hour, or once a day. The administrator can also invoke cfagent directly for on-demand synchronization runs. 2.1.1 cfengine Daemons and Commands cfengine employs several daemons and commands to perform configuration synchronization operations. The following list describes the primary cfengine components. • cfagent -- the cfagent command is cfengine’s workhorse.
Figure 2-1 cfengine Overview cfrun 2 cfservd 3 cfagent 1 cron cfexecd 5 + /var/opt/dsau/cfengine/inputs -update.conf -cfagent.conf -cfservd.conf -cfrun.hosts 2. 3. 4. 5. cfron cfagent cfexecd 4 Master Policy Files: +/dir/cfengine_master/master_files/ - +/dir/cfengine_master/inputs/ -update.conf -cfagent.conf -cfservd.conf -cfrun.hosts + /var/opt/dsau/cfengine/inputs -update.conf -cfagent.conf -cfservd.conf -cfrun.hosts Client Master Server 1.
synchronization service to groups of remote client systems. Those clients can be standalone systems or Serviceguard clusters. The cluster providing the cfengine service can be a client of itself and also take advantage of cfengine’s configuration synchronization features. A possible but somewhat unusual configuration is to have a fixed member of a Serviceguard cluster act as the master server but no package is configured so cfservd will not be highly available.
NOTE: If you have used the wizard previously to configure a cfengine master server and rerun it to reconfigure the master server, stop the currently running configuration first. For example, use the following command to stop the running cfservd: /etc/init.d/cfservd stop If you have cfagent in cron or are using cfexed, disable them so they do not run while the wizard reconfigures the system.
Enter choice: 1 The cfengine master server is being configured on: local_hostname The wizard then asks if you would like to additionally configure managed clients immediately after configuring the server. For this example, answer no. Managed clients will be added later. You can optionally specify additional remote clients to manage at this time. If you are running in an HA environment, you do not need to specify the cluster members.
Note that when configuring a master server but not adding any managed clients during the server configuration, the members entry (list of managed clients), is empty, as shown in the above example. A file containing the answers for this run of the Configuration Synchronization Wizard is stored here... /var/opt/dsau/cfengine/tmpdir/csync_wizard_input.txt This configuration can be reestablished by issuing the following command: /opt/dsau/sbin/csync_wizard \ -f /var/opt/dsau/cfengine/tmpdir/csync_wizard_input.
Configuration Synchronization Wizard Menu ========================================= (1) Set up a cfengine master server (2) Add a client (3) Remove a client (4) Manage keys for cfengine clients (5) Display current configuration (9) Exit Enter choice: 1 After you choose 1 and press Enter, the wizard displays the following text: This system is a member of a Serviceguard cluster. The cfengine configuration will be defined as a package for high availability unless you answer no to the question below.
Once the storage infrastructure is configured and the IP address obtained, press return to access the default answer of ‘yes’ and proceed with creating the package. The wizard now prompts for the package information: Configuring the csync Serviceguard package for a highly available cfengine master. The cfengine master server is being configured as a HA Serviceguard Package on this cluster.
Verifying that the master has an entry in the /etc/hosts file on each client... Starting cfengine on the master server and any managed clients. This may take a few minutes.... When the configuration is done, the wizard displays the following summary screens which direct the administrator to the main policy file, /mount_point/cfengine_master/inputs/cf.main, and the recorded answer file for this run of the wizard. The policy file is located on the newly configured filesystem associated with the package.
update.conf and cfagent.conf define the master configuration synchronization server to be the registered DNS name for the relocatable IP address of the package. When managed clients run cfagent (see cfagent(8)), cfagent connects to cfservd on the package’s adoptive node. Thus the cluster members themselves are all managed clients. The member hosting the package additionally acts as the master server for the policy files. When booting the cluster, each member will start a client cfservd.
NOTE: When adding members to a cluster, consider the following: • When adding a member to a cluster that is configured as a highly available master server, the csync package must be running when the member is added. The add member processing task copies the configuration data from the package’s mounted filesystem to the new member’s /var/opt/dsau/cfengine directories. If the package is not running, the filesystem will not be accessible and the new member will not be properly configured.
A remote Serviceguard cluster can be configured as a managed client. However, each member must be configured individually. Repeat the configuration tasks described below for each member of the cluster. Start by logging in as root on the master server and configure ssh access to the remote system: # csshsetup hostname_of_managed_client csshsetup first tests ssh access to the remote system. If it is not configured, ssh prompts for the managed client’s password.
NOTE: You can use csshsetup to configure a trust relationship between the master server and the managed clients. This will allow you to use command fanout commands such as cexec and ccp (see cexec(1) and ccp(1)). Using these commands can simplify the configuration steps described below when files need to be distributed to managed clients. 2.3.2.1 Manually Configuring a Standalone Synchronization Server Perform the following one-time steps to configure a standalone system as a cfengine master server: 1.
the client’s domain could be determined based on the client’s IP address or subnet, as follows: classes: # host in these ip address ranges xyz_domain = ( IPRange(10.0.0.1-15) ) abc_domain = ( IPRange(192.0.0.1-254) ) control: xyz_domain:: domain = ( “xyz.example.com” ) abc_domain:: domain = ( “abc.example.com”) Use the cfagent -p (or --parse-only) flag to verify the syntax of update.conf. 4. 5. Distribute the master update.conf to each managed client.
9. The file /var/opt/dsau/cfengine_master/inputs/cfservd.conf controls which managed clients have access to the files served by cfservd on the master. Make the following edits to cfservd.conf: • Replace the “<%CFSERVD_DOMAIN_LISTS%>“token with a comma-separated list of wildcarded DNS domains or hostnames for the systems that are allowed to access this server. For example: domain_list = ( “*.abc.xyz.com,*.cde.xyz.com“ ) This statement allows all hosts in the abc.xyz.com and cde.xyz.
Serviceguard package and the mechanism used to distribute cfengine’s security keys. Follow all the steps described below. • Initial Serviceguard Package Preparation 1. 2. Start by obtaining an IP address for the package. This address is typically registered in DNS to simplify management of remote clients. If you are using cfengine for intra-cluster use only, it is sufficient to make sure the address is added to each member’s /etc/ hosts file.
It is critical to keep this file very simple and avoid errors. Errors in this file will require manually copying a new version to each managed client. The file contains tokens in the form <%token name%> which are replaced by the csync_wizard with the administrator’s answers to questions. Replace the tokens as follows: — Replace the <%POLICYHOST_NAME%> token with the fully qualified domain name of the Serviceguard csync package. For example: policyhost = ( “csync.abc.xyz.
• Edit the cfservd.conf File The file /var/opt/dsau/cfengine_master/inputs/cfservd.conf controls which managed clients have access to the files served by cfservd on the master. Make the following edits to cfservd.conf: — Replace the “<%CFSERVD_DOMAIN_LIST%>” token with a comma-separated list of wildcarded DNS domains or hostnames for the systems that are allowed to access this server. For example: domain_list = ( “*.abc.xyz.com,*.cde.xyz.com” ) This statement allows all hosts in the abc.xyz.com and cde.
This will create keys named localhost.priv and localhost.pub in the directory /var/opt/dsau/cfengine/ppkeys. 2. The public key, localhost.pub is then copied to root-package IP address.pub. For example, # cp localhost.pub root-192.10.25.12.pub where 192.10.25.12 is the relocatable IP address of the csync package. 3. This member’s localhost.pub is then used to create the member-specific keys for each member: # cp localhost.pub root-.pub # cp localhost.pub root-.
# cp /dsau/share/serviceguard/templates/csync.script.template csync # chmod +x csync 3. Edit the csync.conf package ASCII configuration file to replace the placeholder tokens with the appropriate values. The tokens are in the form <%token name%>. a. Find the line "SUBNET <%SG_PKG_SUBNET%>" b. 4. Replace the token <%SG_PKG_SUBNET%> with the subnet for the csync package. (Use netstat -i to help identify the subnet.) Find the lines that contain the <%SG_PKG_SGCONF%> token.
5. Distribute the package control script and package ASCII configuration files clusterwide: # ccp csync csync.conf $SGCONF/csync/ 6. Apply the package and start it: # cmapplyconf -P csync.conf # cmmodpkg -e csync • Test the csync Package Configuration Test the configuration by performing the following steps: 1.
# scp localhost.pub master_server:\ /var/opt/cfengine/ppkeys/root-client_IP_address.pub It is important to use a utility such as secure copy (see scp(1)) when transferring the key in order to protect its integrity. 3. Finally, copy the master server’s key to this managed client: # scp master_server:/var/opt/cfengine_master/ppkeys/localhost.pub root-master_IP_address.pub 4. Next, copy the master server update.
synchronization. For details on using cfexecd in daemon-mode, refer to the cfengine tutorial located in /opt/dsau/doc/cfengine/. 2.4 Security Notes cfengine has many security features that range from parameters that control denial-of-service attacks to access control lists that prevent managed clients from accessing reference file directories on the server. For details on cfengine security features, refer to the reference manual located in /opt/dsau/doc/cfengine/.
2.4.3 Encryption In general, file transfer traffic between the master server and a managed client is not encrypted. For many system management related configuration files this is acceptable. For certain files, an encrypted file transfer is desirable. The copy action in cfagent.conf has an "encrypt = true" option to encrypt the specified file. For additional encryption options, refer to the cfengine reference manual located in /opt/dsau/doc/cfengine. 2.4.
• • • • Most cfagent.conf actions such as "copy," "editfiles," and "processes," support a syslog = true option to cause the specific action to be logged to syslog. Similarly, most actions support an "inform = true" option to cause cfagent to report any changes. cfagent.conf’s control section supports global "inform = (true)" and "syslog = true" options. cfagent (see cfagent(8)) supports the --inform switch on the command line.
cfrun(0): .......... [ Hailing host1 ] .......... cfrun(0): .......... [ Hailing host2 ] .......... cfrun:host2: Couldn’t open a socket cfrun:host2: socket: Connection refused Check that the cfservd daemon on host2 is configured and running. Use the command # /etc/init.d/cfservd start to start cfservd if it is not running. 6. “Can’t stat” messages When running using either cfrun or cfagent, you might get “can’t stat” errors.
3 Consolidated Logging Distributed Systems Administration Utilities offers consolidated logging features, including the standard logging features offered by syslogd, and syslog Next Generation (syslog-ng) features in standalone and cluster log consolidation environments. The next sections of this document describe their use. 3.1 Introduction to syslog syslogd (see ) (see syslogd(8)) is a ubiquitous component of UNIX systems that performs system logging activities.
Table 3-2 syslog Facilities Messages (continued) Message Description LOG_NEWS USENET news subsystem. LOG_SYSLOG Messages generated internally by syslogd. LOG_USER (default) Generic user-level messages. LOG_UUCP UUCP subsystem. 3.1.2 Message Filtering Using /etc/syslog.conf, messages can be filtered based on their priority level and facility. Messages can be directed to: • • • • • Specific log files The console A specified user. The message is sent to the user's terminal if the user is logged in.
• • Improved filtering functionality. In addition to syslog's facility/priority level filtering, syslog-ng can perform regular expression filtering against the program name, hostname, text of the message itself, the sender's IP address, and so on. TCP transport - In addition to syslogd’s UDP transport, syslog-ng supports a TCP transport which offers better delivery guarantees. NOTE: syslog-ng's support for a TCP transport does not imply that it safeguards against all message loss.
Figure 3-1 syslog-ng Log-Forwarding Configuration TCP/IP or UDP 4 syslog-ng 3 syslog-ng fifo Log reader 2 1 syslogd cmcld +/var/log/ messages maillog + /usr/local/cmcluster/conf// -clog.log -csync.log -xclock.log NOTE: Actual path for cmcluster may be different 1. 2. 3. 4. The gray area represents standard syslogd operation. Applications such as Serviceguard’s cmcld daemon call syslog (see syslog(3C)) to send messages to syslogd.
Figure 3-2 syslog-ng Log Consolidator Configuration TCP/IP or UDP 1 A B syslog-ng syslog-ng fifo C Consolidated Logs: + /clog/syslog -syslog.log -mail.log -syslog-ng.log +/clog/packages -clog.log -csync.log -xclock.log 3 Log reader syslogd 2 cmcld +/var/log/ messages mail log + /usr/local/cmcluster/conf// -clog.log -csync.log -xclock.log NOTE: Actual path for cmcluster may be different 1. 2. 3. The syslog-ng server reads the incoming log data from the UDP or TCP connected clients.
IMPORTANT: The following notes are particularly for SLES 10 users. On Red Hat syslogd and syslog-ng coexist and both have different start up scripts whereas SLES has only syslog-ng. On SLES syslog-ng is managed by two start up scripts - one provided by default in SLES and the other provided by DSAU. The proper sequencing is that the SLES startup script runs first, and then the default startup script runs. If the system startup script runs first, then the DSAU startup script displays a warning message.
For a standalone system, the wizard first displays introductory paragraphs explaining log consolidation and then asks: Do you want to configure log consolidation? (y/n) [y]: Answer yes (y) or press Enter. The next question is: You can configure this system hostname as either a: - Consolidation server - Client that forwards logs to a remote consolidation server Do you want to configure hostname as a Consolidation Server? (y/n) [y]: Answer yes (y).
source s_syslog_tcp { tcp(port(tcp_port) keep-alive(yes) max-connections(N)); }; where N is the expected number of clients. Next, the wizard prompts for which local logs should be consolidated: Log files that reside on this system can be consolidated. Would you like to consolidate this system's syslogs? (y/n) [y]: Answering yes places this log consolidation system’s own local syslog data in the consolidated log along with the client system's syslog data.
The wizard will set up and create a Serviceguard package for the consolidated logging service. Make sure that this cluster’s MAX_CONFIGURED_PACKAGES value can accommodate the additional package. For more information on this setting, please refer to the Managing Serviceguard manual which is part of the Serviceguard documentation set.
LVM” in the chapter “Building an HA Cluster Configuration,” in the Managing Serviceguard manual. NOTE: The wizard only supports creating packages based on LVM volume groups. When using CFS or VxVM, manual configuration is required. See the section “Manually Configuring Log Consolidation” (page 59) for details. The wizard prompts for the following, all of which should have already been configured: 1. 2. 3. 4. 5. 6. 7. LVM volume group name (for example, /dev/vgclog).
to local port port_number using the TCP protocol. For high availability the Serviceguard package "clog" will be configured with the following attributes: Volume Group: /dev/vgclog Logical Volume: /dev/vgclog/lvol1 Filesystem: /clog Mount Options: -o rw Filesystem Type: ext3 IP Address: 192.10.25.12 Subnet: 192.10.25.
DSAU maintains two configuration files that control whether the instance of syslog-ng on a particular cluster member operates as a consolidation server or a log forwarding client: /etc/syslog-ng.conf.server and /etc/syslog-ng.conf.client on Red Hat and /etc/syslog-ng/syslog-ng.conf.server and /etc/syslog-ng/syslog-ng.conf.client on SLES. The symbolic link /etc/ syslog-ng.conf or /etc/syslog-ng/syslog-ng.conf points to one of the configuration files.
— stanzas added in this section direct syslog-ng to filter package log messages into the appropriate consolidated package logs. The clog_tail log monitor adds or deletes the package log file from its list of files to monitor. 3.3.1.5 Minimizing Message Loss During Failover When there is a failure on the adoptive node, it takes a finite amount of time for the clog package to fail over to another cluster member.
After entering the hostname or IP address of the log consolidation server, the wizard asks if you want to use the TCP transport when forwarding log messages: You can choose to forward logs to the consolidator using either the UDP protocol or the TCP protocol (recommended). Do you want to use the TCP protocol? (y/n) [y]: Standard syslogd forwards messages using the UDP protocol. UDP is a high-performance, broadcast-oriented protocol with no flow control or message delivery verification.
When forwarding a cluster’s package logs, manual configuration is required on the consolidation server in order to add the syslog-ng filtering lines to cause these package logs to be consolidated into their own unique files. See “Manually Configuring a Serviceguard Cluster as a Log Forwarding Client” (page 70) for details.
Manual configuration is required for the following cases: • • When a cluster is a log forwarding client and forwarding package logs, manual configuration is required on the consolidation server (standalone or cluster) to filter the package logs appropriately.
destination d_syslog { file(“<%FS%>/syslog/syslog.log”); }; becomes: destination d_syslog { file(“/clog/syslog/syslog.log”); }; Make sure that this directory exists or the appropriate filesystem is mounted. Since consolidated logs can grow quite large, HP recommends that this filesystem use the largefiles option and that there is sufficient room for growth. • When using TCP, record the port number you choose above in the /etc/services file.
default suppresses duplicate messages. If you issue multiple logger test messages, make sure each is unique. 3.3.2.2 Manually Configuring a Serviceguard Cluster as a Log Consolidation Server Configuring a Serviceguard cluster as a log consolidation server is similar to the steps for a single system. All cluster members must be up and accessible before proceeding. Create the configuration files described below on every cluster member.
1. To configure syslog-ng, start with the same syslog-ng.conf templates used by the clog_wizard. On one cluster member, copy /opt/dsau/share/clog/templates/syslog-ng.conf.server.template to /etc/syslog-ng.conf.server on Red Hat or /etc/syslog-ng/ syslog-ng.conf.server on SLES. Then copy an /opt/dsau/share/clog/templates/syslog-ng.conf.client.template to /etc/syslog-ng.conf.client on Red Hat or /etc/syslog-ng/ syslog-ng.conf.client on SLES.
2. Manually replace the tokens in /etc/syslog-ng.conf.client on Red Hat or /etc/ syslog-ng/syslog-ng.conf.client on SLES as follows: a. Delete the <%UDP_LOOPBACK_SOURCE%> and <%UDP_LOOPBACK_LOG%> tokens. b. Replace all the <%TYPE%> tokens with either tcp or udp depending on the desired log transport. c. Find the line: “destination d_syslog_<%TYPE%>{ <%TYPE%>(“<%IP%>”port(<%PORT>%>)); };.” Replace <%IP%> with the IP address of the clog package.
3.3.2.2.1 Creating the clog Package To create the consolidated logging or clog package, start by copying the package templates: # mkdir $SGCONF/clog # cd $SGCONF/clog # cp /opt/dsau/share/serviceguard/templates/clog.conf.template clog.conf # cp /opt/dsau/share/serviceguard/templates/clog.script.template clog # chmod +x clog Both the clog.conf package configuration file and the clog package control script need to be edited to replace marker tokens with the correct values. For clog.
7. Find the line “FS_FSCK_OPT[0]=“<%SG_PKG_FS_FSCK_OPT%>”” and replace the token with any filesystem specific fsck options. The token can be deleted and this option left blank. For example: FS_FSCK_OPT[0]= 8. Find the line “IP[0]=“<%SG_PKG_IP%>”” and replace the token with the IP address of the clog package. For example: IP[0]= 192.119.152.3 9. Find the line “SUBNET[0]=“<%SG_PKG_SUBNET%>”” and replace the token with the subnet for the packages IP address. Use netstat -i to help determine the subnet.
1. Run /sbin/syslog-ng with the -s or --syntax-only option to verify the syntax of the/etc/syslog-ng.conf.server and/etc/syslog-ng.conf.client files on Red Hat or /etc/syslog-ng/syslog-ng.conf.server and /etc/syslog-ng/ syslog-ng.conf.client on SLES. For the package’s adoptive node, a symbolic link will be created named /etc/syslog-ng.conf on Red Hat or /etc/syslog-ng/ syslog-ng.conf on SLES and this symbolic link will point to the .server file.
5. Validate that log forwarding is working properly. If consolidating the cluster’s local syslogs, use “logger ” and make sure this message is in the consolidated syslog.log. If you are not consolidating local logs, use the logger command from a log forwarding client. Note that logger messages are first sent to the local syslogd, which forwards them to syslog-ng. By default, syslogd suppresses duplicate messages. If you issue multiple logger test messages, make sure each is unique.
3.3.2.3.1 Manually Configuring a Standalone Log Forwarding Client 1. To configure syslog-ng, start with the same syslog-ng.conf templates used by the clog_wizard. Copy /opt/dsau/share/clog/templates/syslog-ng.conf.client.template to /etc/syslog-ng.conf.client on Red Hat or /etc/syslog-ng/ syslog-ng.conf.client on SLES. This file has tokens named <%token-name%> that are replaced by the wizard based on the administrator’s answers to the wizard’s questions. Manually replace the tokens in /etc/syslog-ng.conf.
2. The syslog-ng startup procedure, /etc/init.d/syslog-ng, relies on several configuration variables. Edit as follows:/etc/sysconfig/syslog-ng a. Change the CLOG_CONFIGURED line to: CLOG_CONFIGURED=1 b. Add the following lines: CLOG_CONSOLIDATOR=0 CLOG_CONS_IP= c.
1. To configure syslog-ng, start with the same syslog-ng.conf templates used by the clog_wizard. On one cluster member, copy the /opt/dsau/share/clog/templates/ syslog-ng.conf.client.template to /etc/syslog-ng.conf.client on Red Hat or /etc/syslog-ng/syslog-ng.conf.client on SLES. This file contains tokens named <%token-name%> which are replaced by the wizard based on the administrator’s answers to the wizard’s questions. Manually replace the tokens in /etc/syslog-ng.conf.
2. The syslog-ng startup procedure, /etc/init.d/syslog-ng, relies on several configuration variables. Edit /etc/sysconfig/syslog-ng as follows: a. Change the CLOG_CONFIGURED line to: CLOG_CONFIGURED=1 b. Add the following lines: CLOG_CONSOLIDATOR=0 CLOG_CONS_IP= c.
6. Test the configuration by performing the following steps: a. Run /sbin/syslog-ng with the -s or --syntax-only option to verify the syntax of the /etc/syslog-ng.conf file on Red Hat or /etc/syslog-ng/ syslog-ng.conf file on SLES. This should be a symbolic link to /etc/ syslog-ng.conf.client on Red Hat or /etc/syslog-ng/ syslog-ng.conf.client on SLES as described above. b. Start syslog-ng on all cluster members using # cexec “/etc/init.d/syslog-ng start” c.
CLOG_TEXT_LOG[1]=/var/adm/logs/mylog.log CLOG_TEXT_FORMAT[1]="custom" If the text file is already formatted using the syslog-compatible format shown above, then add the corresponding CLOG_TEXT_FORMAT[n]entry with a value of “syslog”. For example, CLOG_TEXT_LOG[2]=/var/adm/app/logs/app.log CLOG_TEXT_FORMAT[2]="syslog" If no CLOG_TEXT_FORMAT[]entry is made for a corresponding CLOG_TEXT_LOG[]entry, the default is “custom”.
1. For each text log that will be forwarded from a client, add the following destination, filter and log lines to the file syslog-ng.conf.server, after the section called HP_AUTOMATED_LOG_FILE_CONSOLIDATION: For the destination line: destination d_node1_text1{ file(“fs/textdir/node1_text1.log”); }; For the filter line: filter f_node1_text1{ program(node1_text1.
Restart syslog-ng on all cluster nodes. 3. For each text log that is deleted from a client that is forwarding its text logs, delete the corresponding destination, filter and log lines from the /etc/syslog-ng.conf.server file on Red Hat or /etc/syslog-ng/syslog-ng.conf.server file SLES of the log consolidator. syslog-ng on the log consolidator must be sighup’d so that it rereads this configuration file. On a Serviceguard log consolidator, the updated /etc/syslog-ng.conf.
3.4 Disabling Log Consolidation The clog_wizard enables log consolidation configurations but does not have an unconfigure or deconfigure option. Thus you must disable a system from participating in log consolidation manually, using the following instructions for each system type to: • Disable a standalone log consolidation system • Disable a Serviceguard cluster log consolidation system • Disable a standalone log forwarding client • Disable a Serviceguard log forwarding client 3.4.
2. Edit the /etc/sysconfig/syslog-ng file and change the CLOG_CONFIGURED line to the following: CLOG_CONFIGURED=0 Remove all other CLOG lines except for the following: CLOG_LAYOUTS_DIR=/var/opt/dsau/layouts CLOG_ADDITIONAL_LOG_DIRS[0]=/var/log 3. If ssh port forwarding had been configured, remove the following line from /etc/ services: clog_ssh /tcp # Consolidated logging with ssh port forwarding 3.4.
traffic within a Serviceguard cluster (member to member). Standard TCP or UDP is used for intra-cluster log traffic. ssh port forwarding is transparent to syslog-ng. The /etc/syslog-ng.conf.client file on Red Hat or /etc/syslog-ng/syslog-ng.conf.client file on SLES is configured so that syslog-ng forwards log traffic to a local port managed by ssh. The local ssh connects to the remote sshd on the log consolidation server and sshd opens the standard syslog-ng TCP port.
Then from each log consolidation client, perform a standard ssh key exchange with the relocatable IP address of the clog package. One way to do this is using the csshsetup tool (see csshsetup(1)), as follows: # csshsetup csshsetup will prompt for the password of the cluster in order to do the initial key exchange. 3.5.3 clog Network Port Usage syslog and syslog-ng require specific network ports to be available for correct operation.
3.6.2 Using the System Log Viewer The System Log Viewer will display the syslog-related logs for the system. By default, this includes the local logs for the system from /var/log. If this system is also a log consolidator, the consolidated logs will also be listed. NOTE: In a Serviceguard cluster configured as a log consolidation server, the consolidated logs are placed on the filesystem associated with the “clog” package. See “Cluster Configuration Notes for clog” (page 55) for additional details.
4 Command Fanout Command fanout utilities allow the system administrator to replicate shell commands across multiple systems. Traditionally, administrators have created wrappers around tools such as remote shell (see remsh(1)) and secure shell (see ssh(1)) to provide command fanout functions. 4.1 Parallel Distributed Shell The Distributed Systems Administration Utilities (DSAU) include the open source tool Parallel Distributed Shell (pdsh).
• Parallel copy command The pdcp command provides a parallelized copy command to copy a local source file to multiple targets. • Setting ssh arguments Users can set the PDSH_SSH_ARGS environment variable with any ssh arguments that ssh should use while connecting to remote systems. For example, to obtain verbose output, use the following command; # export PDSH_SSH_ARGS="-v" Figure 4-1: “pdsh Architecture ”, shows the components of pdsh and its architecture.
ckill ckill allows the administrator to signal a process by name since the pid of a specific process will vary across a set of systems or the members of a cluster. cuptime cuptime displays the uptime statistics for a set of systems or a cluster. cwall cwall displays a wall(1) broadcast message on multiple hosts. All the wrappers support the CFANOUT_HOSTS environment variable when not executing in a Serviceguard cluster.
If the “r” services are not disabled, use of the pdsh -R rsh option by unprivileged users is still disabled by default because of the inherent security risk. By default, only users with root privileges can use the pdsh -R rsh option. This is because the remote shell rcmd library call requires the use of a privileged port. Even though privileged users can use -R rsh, the ssh transport is still preferred.
Table 4-3 Target Node Error Messages Message Cause To Correct :sh::not The command does not exist on the Use full paths to specify commands. target node. The remote shell found invoked by pdsh has only a minimal path, and a user's login scripts are not executed on remote nodes. 4.
A Installing DSAU on Linux This appendix describes how to install DSAU on Linux to run on supported HP Integrity and Proliant servers. These instructions are written for experienced Linux system administrators. Refer to your Linux documentation for Linux-specific information. This section describes the installation and setup process for DSAU. There are two parts to this process: 1. Satisfying prerequisites 2. Performing the installation Uninstall instructions are also provided. A.
Configuring the syslog-ng startup scripts. Configuring System Log Viewer for Systems Insight Manager Configuring System Log Viewer for System Management Homepage Note: DSAU provides the System Log Viewer tool for use from System Management Homepage (HPSMH). If PHSMH is currently running, it must be restarted before the System Log Viewer will appear in its menus. Use “etc/init.d/hpsmhd restart”. The installation on the HP Distributed Systems Administration Utilities (DSAU) has finished.
Use "/etc/init.d/hpsmhd restart". The installation of the HP Distributed Systems Administration Utilities (DSAU) has finished. To complete the DSAU/Linux installation: - add "/opt/dsau/sbin:/opt/dsau/bin" to your path. After the command is executed, the system prompt reappears. A.2.2.2 Installing DSAU on HP x86 Servers (32 and 64–bit systems) To install DSAU on 32-bit HP x86 servers, issue the following command: # rpm -i hpdsau-2.4-1.sles10.i586.
B HP-Supported Open Source pdsh Options This release of DSAU includes open source pdsh code that was compiled with the following options: • readline support. • The secure shell (ssh) remote command module (pdsh —R ssh). • The remote shell (rsh) remote command module (pdsh —R rsh). Because of security concerns, this option is disabled by default at installation time. Refer to Chapter 4 (page 83) for details. • The machines module (pdsh —a option).
Index A adoptive node, 28, 55 alert checksum, 41 authentication errors, 42 B backup file, 26 Bastille, 80 Berkeley commands, 85 blanks in configuration files, 42 C can’t stat messages, 43 ccp, 36 central management server, 19 cf.main, 22, 39 cfagent, 18, 19, 39 cfagent.
K key exchange, 40 integrity, 39 public/private, 27, 35 security, 29, 38 known host fingerprint, 79 L lockdown tool Bastille, 80 log consolidation, 55 configuration, 49 disabling, 77 overview, 46 log forwarding, 55 client, 57 disabling, 77 log traffic intra-cluster, 78 logs package, 72 loss message, 47 LVM storage configuration, 24 volume groups, 24 M managed client, 28, 38 marker tokens, 65 master policy files, 38 server, 28, 31 update.
LVM, 24 subdirectories timestamped, 26 synchronization client configuring, 28 syslog introduction, 45 message format, 45 syslog-ng, 47 syslogd, 45, 47 System Management Homepage, 80 Systems Insight Manager, 19 T TCP port, 40 port errors, 43 protocol, 58 transport, 78 TCP port, 51 timestamped subdirectories, 26 tokens marker, 65 transport TCP or UDP, 78 troubleshooting cfengine, 42 trust relationships, 85 U UDP protocol, 58 unconfigure, 41 update.
