HPSS Installation Guide High Performance Storage System Release 6.2 July 2008 (Revision 2.0) HPSS Installation Guide Release 6.2 (Revision 2.
© Copyright (C) 1992, 2008 International Business Machines Corporation, The Regents of the University of California, Los Alamos National Security, LLC, Lawrence Livermore National Security, LLC, Sandia Corporation, and UT-Battelle. All rights reserved. Portions of this work were produced by Lawrence Livermore National Security, LLC, Lawrence Livermore National Laboratory (LLNL) under Contract No. DE-AC52-07NA27344 with the U.S.
Table of Contents Chapter 1. Release 6.2...........................................................................................................15 1.1.New Features.................................................................................................................15 1.1.1.DCE Replacement .............................................................................................................................15 1.1.2.Linux Support ............................................................
2.3.3.HPSS Storage Subsystems.................................................................................................................44 2.3.4.HPSS Infrastructure ..........................................................................................................................44 2.3.5.HPSS User Interfaces ........................................................................................................................46 2.3.6.HPSS Management Interfaces..................................
3.4.2.3.STK L40, STK SL500, STK SL8500..........................................................................................63 3.4.2.4.STK............................................................................................................................................63 3.4.2.5.ADIC AML ...............................................................................................................................63 3.4.3.Manually Mounted Tape.....................................................
3.7.6.6.SCSI PVR..................................................................................................................................88 3.7.7.Mover ................................................................................................................................................89 3.7.7.1.AIX Asynchronous I/O..............................................................................................................89 3.7.7.2.Tape Devices...........................................
3.10.4.File Families...................................................................................................................................112 3.11.HPSS Performance Considerations...........................................................................112 3.11.1.DB2................................................................................................................................................112 3.11.2.Bypassing Potential Bottlenecks ............................................
5.1.4.Installation Target Directory Preparation........................................................................................136 5.2.Install Prerequisite Software........................................................................................137 5.2.1.Install Java........................................................................................................................................137 5.2.2.Install MIT Kerberos (If Using Kerberos Authentication)............................
Chapter 6. Upgrading to HPSS Release 6.2 .....................................................................179 6.1.Special Instructions for Upgrading to HPSS 6.2.2......................................................179 6.2.Planning for the HPSS 6.2 Upgrade............................................................................180 6.2.1.Metadata changes in HPSS 6.2........................................................................................................180 6.2.2.
6.3.16.Enable DB2 Backup......................................................................................................................215 6.3.17.Perform the DCE Export: hpss_dce_export..................................................................................215 6.3.18.Perform the Unix, LDAP or Kerberos Import...............................................................................215 6.3.19.Prepare the 6.2 System..........................................................................
D.6. Transfer Agent Stanza..............................................................................................287 D.7. Stanzas Reserved for Future Use..............................................................................291 Appendix E. hpss_env_defs.h.............................................................................................293 Appendix F. /var/hpss files.................................................................................................
List of Figures Figure 1. File Migration and Stage Operations..................................................................37 Figure 2. Class of Service / Hierarchy / Storage Class.......................................................38 Figure 3. HPSS Components................................................................................................40 Figure 4. HPSS Generic Configuration...............................................................................52 Figure 5.
List of Tables Table 1. HPSS Client Interface and Mover Platforms.......................................................49 Table 2. Supported Platform/Driver/Tape Drive Combinations......................................64 Table 3. Cartridge/Drive Affinity Table.............................................................................66 Table 4. Paging Space Info...................................................................................................79 Table 5. Gatekeeping Call Parameters.......
Preface About this book The HPSS Installation Guide is for use both at system installation time as well as throughout the lifetime of the system. It will guide system administrators through the planning and installation of a new HPSS system. It also guides system administrators through the conversion process to upgrade existing HPSS systems to Release 6.2.
Chapter 1. Release 6.2 This chapter summarizes HPSS changes for Release 6.2 into four categories: new features, retired features, deferred features, and changed features. Changes since release 4.5 and 5.1 are described. 1.1. New Features This section describes the new HPSS features added to Release 6.2. 1.1.1. DCE Replacement Previous HPSS releases used DCE to provide authenticated client/server remote procedure calls.
• Storage Class (but not subsystem-specific storage class options) • Global configuration • Accounting policy • Location policy • All server configuration. Newly supported options include Core Server, Gatekeeper, Location Server, Log Daemon, Migration/Purge Server, PVL, all PVRs, and SSM. The Mover, Log Client, and Startup Daemon were previously supported and still are. • Modifying devices and drives.
1.1.12. Drive Pools HPSS provides HPSS end clients the ability to direct tape read I/O requests to a predefined group of tape drives referred to as a Drive Pool. This ability helps HPSS administrators manage tape drive scheduling and thus availability. For more information, please refer to Section 7.3: Drive Pools of the HPSS Management Guide. 1.1.13.
1.1.18. Mover Enhancement Multiple Movers for one or more HPSS instances can now be configured to run on the same machine. The -c flag is added to the Mover entry in the inetd configuration file to specify an alternate "/var/hpss" path to be used by the Mover. In addition, the -s flag is added to enable syslog logging for the Mover. This feature is intended for multiple HPSS test systems to share the same set of Mover machines. 1.2.
Core Server's memory image • Modified the DMG table. In support of the new HPSS RPC library, the TCP port was eliminated and the Program and Version numbers were added to DMG specific configuration. • Modified the DMGFILESET table. The TCP port was eliminated and the TCP hostname and RPC endpoint information was added. • Modified the GATEKEEPER table. The length of site policy pathname was changed from 127 to 1023 characters. • Modified the LSPOLICY table.
filesystems do not support mirrored namespaces, fields which were specific to managing mirrored filesets are also no longer available. This includes the following configuration options: • · Global Mount Point · Local Mount Point · Fileset Owner · Fileset Permissions Communication between the HDM and the DMAP Gateway now uses the HPSS RPC library. Fields which were used to specify TCP communication information have been removed. 1.4.3.2.
1.4.4. SSM Changes Significant changes were made in SSM between Releases 4.5 and 5.1 and again between 5.1 and 6.2. For the reader's convenience, all changes between 4.5 and 6.2 are summarized in Section 3.3.4.1: Changes Affecting Sites Upgrading Directly from 4.5. Changes between 5.1 and 6.2 are summarized in Section 3.3.4.2: Changes Affecting Sites Upgrading from 5.1.
Changes since 4.5 Server GUI ADM n n The scripts are dependent upon a login.conf file and, if using Kerberos authentication, a krb5.conf file. Both files are created by hpssuser. The scripts will look in the current working directory for these files unless they are specified on the command line (-C and k options). n n It is no longer necessary to add a security provider entry to the java system java.security file. n n The pathname to the java executable can be specified by using the -j option.
Changes since 4.5 Server GUI ADM n n n Access to port 88 is needed for Kerberos authentication. Access to port 111 is needed if the RPC portmapper is needed to find the System Manager. n n n The user_authorization.dat and hpssadm.config files provided in HPSS 4.5 are no longer available. n n n The hpssgui and hpssadm programs will now optionally record every error and informational message in an ASCII log file.
Changes since 4.5 Server GUI ADM The ability to see which users are logged into SSM, referred to in HPSS 4.5 as a list of SSM "consoles", is available in 6.2 as part of the System Manager Statistics window from both the hpssgui and hpssadm. n n The menu bar has been reorganized extensively. n The "Set Keyboard", "Show Sammi Environment", "View Sammi Errlog", and "About Sammi" items from the 4.5 Session menu and the "SSM consoles" item from the 4.5 Monitor menu are no longer available.
Changes since 4.5 Server GUI Column ordering is now controlled by dragging columns to the desired location. The modified order is preserved automatically in the user preferences across hpssgui restarts. n When messages have been written to the status bar, the most recent messages can be viewed in the status bar's tooltip. Rolling the mouse over the status bar without clicking gives a tooltip that says, "Click mouse in status bar to view messages" if there are status messages to view.
Changes since 4.5 Server The HPSS 4.5 hpssadm commands • pvr_cartridge • pvl_volume • ss_pv • ss_map • ss_vv GUI ADM n have been replaced by a single command "volume". The volume command has a required option "-type" for which these types may be specified: • PVL Volume Information • PVR Cartridge Information • CS Disk Volume • CS Tape Volume The new volume command behaves similarly to the config command, where the types are the titles of the window screens for the structures.
Changes since 4.5 Server GUI The hpssadm config command now supports the additional structures: • Class of Service Config • Storage Class (but not subsystem-specific storage class options) • Global Config • Accounting Policy • Location Policy • All server configurations; servers which were not supported before are the core server, gatekeeper, location server, log daemon, migration/purge server, pvl, all pvrs, and ssm. The mover, log client and startup daemon are still supported as previously.
Changes since 5.1 Server GUI ADM n n The scripts are dependent upon an ssm configuration file (ssm.conf), created by mkhpss, which contains some site-specific configuration values. The SSM client script will attempt to read the ssm configuration file in the current working directory unless the pathname to this file is specified by the -m option. n n The SSM client scripts now use an internal polling mechanism for getting window updates (as opposed to being notified of the update by the server).
Changes since 5.1 Server GUI ADM The SSM client script options for connecting to the System Manager across a Virtual Private Network connection (VPN) have changed. See the -p and -h options on the hpssgui and hpssadm man pages. n n n n n The SSM client script option for using ports exempted by the network administrator as firewall exceptions has changed; see the -n option on the hpssadm/hpssgui man pages.
Changes since 5.1 Server The menu bar has been reorganized slightly. GUI ADM n The "Data Server Statistics" menu item has been replaced by the "System Manager Statistics" menu item, available from the Monitor->SSM Information menu path. "Column View" was added to the menu bar for SSM windows that display an SSM table. See Section 3.6: Common Window Elements of the HPSS Management Guide. In HPSS 5.1, this ability was part of the Preference window associated with the SSM table window.
Changes since 5.1 Server GUI The mechanism for selecting columns to be displayed in each list window has moved from the list's Preferences window to the "Column View" menu of the list window itself. See the description of the "Column View" menu item in Section 3.6: Common Window Elements of the HPSS Management Guide for details. n Column ordering is now controlled solely by dragging columns to the desired location.
Changes since 5.1 Server GUI Since SSM now represents the three Core Server volume structures as a single structure, the types specified to the hpssadm volume command have changed. The 5.1 types: • Disk Storage Map Information • Disk Physical Volume Information • Disk Virtual Volume Information ADM n have been replaced in 6.2 by the single type: • CS Disk Volume Similarly, the 5.
Changes since 5.1 Server A new subcommand “update” has been added to the hpssadm device command for updating the mover device and pvl drive objects. HPSS Installation Guide Release 6.2 (Revision 2.
HPSS Installation Guide Release 6.2 (Revision 2.
Chapter 2. HPSS Basics 2.1. Introduction The High Performance Storage System (HPSS) provides hierarchical storage management and services for very large storage environments. HPSS may be of interest to organizations having present and future scalability requirements that are very demanding in terms of total storage capacity, file sizes, data rates, number of objects stored, and numbers of users.
2.2.4. Based on Standard Components HPSS runs on UNIX and is written in ANSI C and Java. It uses remote procedure calls, a selectable security service (Kerberos or UNIX), UNIX or LDAP for user configuration information, and DB2 as the basis for its portable, distributed, transaction-based architecture. These components are offered on many vendors’ platforms. The full HPSS system has been implemented on IBM AIX and LINUX platforms, and some components of HPSS have been ported to other platforms.
2.3. HPSS Components The components of HPSS include files, filesets, junctions, virtual volumes, physical volumes, storage segments, metadata, servers, infrastructure, user interfaces, a management interface, and policies. Media and file metadata are represented by data structures that describe the attributes and characteristics of storage system components such as files, filesets, junctions, storage segments, and volumes.
Figure 2. Class of Service / Hierarchy / Storage Class 2.3.1. HPSS Files, Filesets, Volumes, Storage Segments and Related Metadata The various metadata constructs used to describe the HPSS namespace and HPSS storage are described below: • Files (Bitfiles). Files in HPSS, called bitfiles in deference to IEEE Mass Storage Reference Model terminology, are logical strings of bytes, even though a particular bitfile may have a structure imposed by its owner.
recorded on these tapes. HPSS supports grouping files on tape volumes only. Families can only be specified by associating the family with a fileset. All files created in the fileset belong to the family. When one of these files is migrated from disk to tape, it is recorded on a tape with other files in the same family. If no tape virtual volume is associated with the family, a blank tape is reassigned from the default family. The family affiliation is preserved when tapes are repacked. • Physical Volumes.
• Class of Service (COS). Each bitfile has an attribute called Class Of Service. The COS defines a set of parameters associated with operational and performance characteristics of a bitfile. The COS results in the bitfile being stored in a storage hierarchy suitable for its anticipated and actual size and usage characteristics. Figure 2: Class of Service/Hierarchy/Storage Class shows the relationship between COS, storage hierarchies, and storage classes. 2.3.2.
consisting of directories, files, and links. These name space objects may exist within filesets that are connected via junctions. Second, the Core Server provides the abstraction of logical bitfiles to its clients. A bitfile is identified by a Core Server generated name called a bitfile ID. Clients may reference portions of a bitfile by specifying the bitfile ID and a starting address and length. The Core Server supports random access to files and sparsely written files.
The purpose of tape file migration is to make an additional copy (or multiple additional copies) of a file, in a tape storage class, to a lower level in the hierarchy. It is also possible to move files downwards instead of copying them. In this case there is no duplicate copy maintained. There is no separate purge component to tape file migration. Empty volumes must be reclaimed using the reclaim utility. Tape Volume Migration: The purpose of tape volume migration is to free tape volumes for reuse.
• Physical Volume Repository (PVR). PVRs manage HPSS cartridges. Though an HPSS system may contain multiple PVRs, each cartridge is managed by only one. PVRs provide APIs for clients to request cartridge mounts and dismounts and query the status of cartridges. For convenience, PVRs are often configured in one-to-one correspondence to tape libraries. For information on the types of tape libraries supported by HPSS PVRs, see Section 3.4.2: Robotically Mounted Tape on page 62.
2.3.3. HPSS Storage Subsystems The goal of storage subsystems (or just “subsystems”) is to increase the scalability of HPSS by allowing multiple Core Servers to be used within a single HPSS system. Every HPSS system is partitioned into one or more subsystems. Each subsystem contains a single Core Server. If migration and purge are needed, then the subsystem should contain a single Migration/Purge Server. Each Core Server and each Migration/Purge Server must exist within a storage subsystem.
HPSS Core Server performs most of the HPSS metadata changes using the transaction management tools provided by DB2. For the most part, these metadata transactions are managed entirely within the Core Server. Other servers such as MPS and PVL modify their metadata transactionally, and those transactions are entirely contained within those servers. A very small number of rarely performed operations require distributed transaction management, and these are handled by DB2 as well.
HPSS. A delog function is provided to extract and format log records from a central or archived log file. Delog options support filtering by time interval, record type, server, and user. • Accounting. The HPSS accounting system provides the means to collect usage information in order to allow a particular site to charge its users for the use of HPSS resources. It is the responsibility of the individual site to sort and use this information for subsequent billing based on site-specific charging policies.
GUI. The hpssgui simplifies the management of HPSS by organizing a broad range of technical data into a series of easy-to-read graphic displays. The hpssgui allows monitoring and control of virtually all HPSS processes and resources from windows that can easily be added, deleted, moved, or overlapped as desired. HPSS also provides a command line SSM interface, hpssadm.
HPSS provides facilities for recording information about authentication and object (file/directory) creation, deletion, access, and authorization events. The security audit policy for each server determines the records that each individual server will generate. All servers can generate authentication records. • Accounting Policy. The accounting policy provides runtime information to the accounting report utility and to the Account Validation service of the Gatekeeper.
The following matrix illustrates which platforms support HPSS interfaces. Table 1. HPSS Client Interface and Mover Platforms Platform PFTP Client Client API HPSS HPSS VFS FTP Clients Mover Client IBM AIX X X X Sun Solaris (Big Endian ONLY) Digital UNIX X X X Hewlett-Packard HPUX Silicon Graphics IRIX (32-bit) X Not Tested X Not Tested X Compaq Tru64 Linux (Intel) Any platform running standard FTP clients. GUIbased Clients may not function correctly for some commands.
HPSS Installation Guide Release 6.2 (Revision 2.
Chapter 3. HPSS Planning 3.1. Overview This chapter provides HPSS planning guidelines and considerations to help the administrator effectively plan, and make key decisions about, an HPSS system. The planning process for HPSS must be done carefully to ensure that the resulting system satisfies the site’s requirements and operates in an efficient manner. We recommend that the administrator read this entire chapter before planning the system.
Figure 4. HPSS Generic Configuration 3.1.2. HPSS Configuration Planning Before beginning the planning process, there is an important issue to consider. HPSS was designed to optimize the transfer of large files at the expense of some small file transfer performance. If at all possible, try to reduce the number of small files that are introduced into your HPSS system. For example, if you plan to use HPSS to backup all of the PCs in your organization, it would be best to HPSS Installation Guide Release 6.
aggregate the individual files into large individual files before moving them into the HPSS name space. The following planning steps must be carefully considered for the HPSS infrastructure configuration and the HPSS configuration phases: 1. Identify the site’s storage requirements and policies, such as the initial storage system size, anticipated growth, usage trends, average file size, expected throughput, backup policy, and availability. For more information, see Section 3.
more information about configuring families. • Plan for filesets and junctions, if any. Refer to Chapter 10: Filesets and Junctions in the HPSS Management Guide for more information. • Plan for HPSS storage classes. Refer to Section 3.10.1: Storage Class on page 102 for more information on the storage class configuration. • Plan for HPSS storage hierarchies. Refer to Section 3.10.2: Storage Hierarchy on page 109 for more information on the storage hierarchy configuration.
3.1.4. HPSS Operational Planning The following planning steps must be carefully considered for the HPSS operational phase: 1. Define the site guidelines for the HPSS users and SSM users. • Each HPSS user who uses the storage services provided by HPSS should be assigned an Accounting ID and one or more appropriate Classes of Service (COS) to store files. • Each SSM user (administrator or operator) should be assigned an appropriate SSM security level.
guide that the HPSS support/deployment team uses to monitor and check the progress of an installation. 3.2. Requirements and Intended Uses for HPSS This section provides some guidance for the administrator to identify the site’s requirements and expectations of HPSS. Issues such as the amount of storage needed, access speed and data transfer speed, typical usage, security, expected growth, data backup, and conversion from an old system must be factored into the planning of a new HPSS system. 3.2.1.
3.2.5. Duplicate File Policy The policy on duplicating user data files impacts the amount of data stored and the amount of data moved. If all user files are duplicated, the system will require twice as much tape storage. If users perform their own duplication of files, the system may consume a smaller amount of storage space. Users can be given control over duplication of their files by allowing them a choice between hierarchies which provide duplication and hierarchies which do not. 3.2.6.
authentication and authorization (subject to certain caveats – See FTP documentation for details). The pftp_client binary must be built using the distributed source code. However, it is the site's responsibility to obtain the necessary Kerberos components. ACLs entries in the AUTHZACL table and/or ACLs on HPSS directories and files may need to be added for appropriate foreign_user and/or foreign_group entries. 3.2.8.
If UNIX authentication will be used, this product is not required. 3.3.1.3. LDAP and IBM Kerberos HPSS can be configured to use an LDAP directory to store its authorization information such as users' names, UIDs, GIDs, and home directories. The supported LDAP server product for this release is IBM Tivoli Directory Server. It can be downloaded from the http://www-306.ibm.com/software/tivoli/resource-center/security/code-directory-server.jsp webpage.
• IBM ATL: atldd.driver 6.5.2.0 and Atape.driver 10.2.8.0 (if planning to control the IBM tape library and drives from this node) 3.3.2.1.2. Linux Requirements Each Linux server node must have the following installed: • Linux machine (eServer zSeries) with a minimum of 2 GB RAM • Red Hat Enterprise Linux AS release 4 (Nahant Update 3, kernel 2.6.9-34.ELsmp) • DB2 UDB V8.2 Enterprise Server Edition (ESE) for Linux (32-bit) with ixPak 4. Note that FixPak 12 for DB2 8.1 is the same as DB2 UDB Version 8.
• 1 GB RAM 3.3.2.2.3. Solaris Requirements Each Solaris Mover node must have the following prerequisites: • Solaris UltraSPARC based processor • Solaris 8+ (32-bit or 64-bit) • C compiler: Forte Developer 7 C 5.4 2002/03/09 (if planning to recompile Mover code) 3.3.2.2.4. IRIX Requirements Each IRIX Mover node must have the following prerequisites: • SGI machine • IRIX 6.5 (with latest/recommended patch set) • C compiler (if planning to recompile Mover code) 3.3.2.3.
• xfsdump-2.2.25-0.2 or higher • xfsprogs-2.6.25-0.2 or higher 3.4. Hardware Considerations This section describes the hardware infrastructure needed to operate HPSS and includes considerations about infrastructure installation and operation that may impact HPSS. 3.4.1. Network Considerations Because of its distributed nature and high-performance requirements, an HPSS system is highly dependent on the networks providing connectivity among the HPSS servers and clients. For control communications (i.e.
• STK SL500 and SL8500 • STK Tape Libraries that support ACSLS • ADIC i500 • ADIC AML (supported by special bid only) 3.4.2.1. IBM 3494 The 3494 PVR supports Ethernet and RS-232 (TTY) attached robots. If appropriately configured, multiple robots can be accessible from a single node. 3.4.2.2.
9840C drives should not be used in conjunction with either 9840A or 9840B drives. Table 2.
(LTO) Gen4 drive can read 3580 (LTO) Gen4, 3580 (LTO) Gen3, and 3580 (LTO) Gen2 cartridges, but can only write 3580 (LTO) Gen4 and 3580 (LTO) Gen3 cartridges), HPSS will mount a 3580 (LTO) Gen2 cartridge into a 3580 (LTO) Gen4 drive only if 3580 (LTO) Gen2 drives are not defined in HPSS and 3580 (LTO) Gen3 drives are either busy or not defined. In this case, it is up to the HPSS Administrator to make sure these 3580 (LTO) Gen2 cartridges are read only. HPSS Installation Guide Release 6.2 (Revision 2.
Table 3.
3580 (LTO) Gen 4 3580 (LTO) Gen 4 3592 J1A Short Tape 3592 J1A 3592 J1A Standard Tape 3592 EO5 3592 EO6 3592 EO5 JJ Short Tape 3592 EO5 3592 EO5 JA Standard Tape 3592 EO6 3592 EO5 JB XL Tape 3592 EO6 JJ Short Tape 3592 EO6 3592 EO6 JA Standard Tape 3592 EO6 JB XL Tape STK 9840A STK 9840A STK 9840B STK 9840C STK 9840D STK 9840B STK 9840B STK 9840C STK 9840D STK 9840C STK 9840C STK 9840D STK 9840D STK 9840D STK 9940A STK 9940A STK 9940B STK 9940B STK 9940B STK T10000A STK T10000A STK
Fibre Channel. For these devices, operating system disk partitions of the desired size must be created (e.g., AIX logical volume or Linux/Solaris/IRIX disk partition), and the raw device name must be used when creating the Mover Device configuration (see Section 7.1: Configure a New Device & Drive of the HPSS Management Guide for details on configuring storage devices). 3.4.6.
Starting with HPSS 6.2 there are many enhancements to the storage system to take advantage of Storage Area Networks. Though separated in Figure 4, in reality there is usually only one SAN at an installation and all the resources are attached to it. Besides the HPSS Movers being connected to SAN, the end-user clients are often SAN attached as well. The result is that the data paths take greater advantage of the SAN architecture and fewer store-and-forward operations are needed through Movers (i.e.
Figure 5. Basic HPSS Metadata & Filesystem Allocation On the left hand side of the diagram, the raw physical volumes are shown attached to the disk array controller. The configuration of the disks by the controller and its software should be divided into three separate LUNs: 1) HPSS Filesystems and DB2 Backups, 2) DB2 Logs, 3) and the DB2 Tables. One disk may be kept as as a "hot spare" in the event that one of the other disks fails.
mirrored log components will need to be determined by HPSS and the customer based on transaction performance requirements. Potentially, disk resources primarily allocated for HPSS disk cache can be used or the site may want to dedicate a second disk array for this purpose to prevent any possible interference. 3.5.3. HPSS Filesystems The following sections describe the various filesystems used by HPSS. 3.5.3.1. /opt/hpss The HPSS software is installed in the /opt/hpss directory.
to create a site policy configuration file, usually named /var/hpss/gk/gksitepolicy. The size of this file depends on the site-implemented gatekeeping policy. If the Gatekeeper Service is not used, there is a minimal amount of disk space used in this directory. • If an Accounting report is requested, a report file and a checkpoint file are created in the directory specified in the Accounting Policy, usually /var/hpss/acct. The sizes of these files depend upon the number of files stored in HPSS.
3.5.3.7. /db2/backups/subsys1 & subsysX Similar to /db2/backups/cfg, the /db2/backups/subsys1 filesystem temporarily stores backup images of the subsystem archived logs and database as they are generated. The backup files are then transferred to long-term media, such as tape, using a backup file manager such as TSM. Details are described in the Section 3.5.4.4: DB2 Disk Space on page 77. Separate filesystems are allocated for each subsystem defined in the system.
Section 5.3.1.2: Install HPSS Documentation and DB2 Software on page 141. The tables and indexes are separated into separate logical volumes/partitions to ease future expansion of the database and to maximize performance of database operations. For Linux, access to a /dev/hdxy partition is through the Linux buffered I/O system. While this is an appropriate access method for a filesystem that supports journaled logging, for DB2 and Mover accesses, non-buffered IO is required. Linux, up to release RHEL 4.
disk devices. The “6.2 Sizing Spreadsheet” input tab is shown below. Based on the input, the resulting output is show below: HPSS Installation Guide Release 6.2 (Revision 2.
Definitions for the DB2 tables are as follows: Bitfile Disk Allocation Maps. (BFDISKALLOCREC) For each bitfile stored on disk, one or more rows will be created in the Disk Allocation Maps table. The number of rows is determined by the storage segment size in the Storage Class in which the files are stored and the average file size stored in that Storage Class. Bitfile Disk Segments. (BFDISKSEG) For a bitfile stored on disk, one or more rows will be created in the bitfile disk segment table.
bytes. Disk Storage Segments. (STORAGESEGDISK) Expect the size of the disk storage segment metadata table to be quite volatile. As files are added to HPSS, disk storage segments will be created, and as files are migrated to tape and purged from disk, they will be deleted.
created will require approximately 10MB of disk space in a filesystem that should be used only for database home directories and other DB2 home directories (such as the DB2 Administration Server and Fenced User). This filesystem should be protected by RAID since DB2 may require information in the database home directory to properly recover a database. DB2 backups Full backups of DB2 databases may be larger than the databases themselves.
The amount of memory for nodes running HPSS Movers, and no DB2 servers, is dependent on the number and types of devices configured on the Mover node, the expected usages of those devices, and the configuration of the Movers. In general, Movers supporting disk devices will require more memory than Movers supporting tape devices because disk devices are likely to have more outstanding requests. At least 1GB of memory should be configured on the Mover nodes.
Class of Service and Accounting Code information. In addition, the chgrp, chmod, and chown commands are supported as quote site options. The FTP daemon is built on top of the Client API and must be run on a node that supports Kerberos clients. Note that FTP clients can run on computers that do not have Kerberos installed. The size of the buffer, used for reading and writing HPSS files, can be specified in the FTP daemon configuration.
3.7.1. Core Server The Core Server is responsible for managing the HPSS name space (files, directories, links, etc.), bitfiles, and storage (physical volumes, virtual volumes, etc.) for a single subsystem. Each of these areas of responsibility are outlined in greater detail below. Core Server at large The Core Server uses POSIX threads to service concurrent requests.
Figure 6. The Relationship of Various Server Data Structures Disk Storage Allocation Each Core Server manages disk storage units for HPSS. It maps each disk storage unit onto an HPSS disk Physical Volume (PV) and records configuration data for the PV. Groups of one or more PVs (disk stripe groups) are managed by the server as disk Virtual Volumes (VVs). The server also maintains a storage map for each VV that describes which portions of the VV are in use and which are free.
The Core Server is designed to scale up its ability to manage tapes as the number of tapes increases. As long as sufficient memory and CPU capacity exist, threads can be added to the server to increase its throughput. Additional subsystems can also be added to a system, increasing concurrency even further. Note that the number of tape drives the server manages has much more to do with the throughput of the server than the number of tape volumes the server manages.
metadata updates and can impose extra overhead on DB2. Also, unpredictable purge behavior may be observed if the purge record ordering is changed with existing purge records in the system until these existing records are cleared. A purge run ends when either the supply of purge records is exhausted or the purge target is reached. There are two different tape migration methods, tape volume migration and tape file migration.
Sites can choose to configure zero (0) or more Gatekeepers per HPSS system. Gatekeepers are associated with storage subsystems. Each storage subsystem can have zero or one Gatekeeper associated with it and each Gatekeeper can support one or more storage subsystems. Gatekeepers are associated with storage subsystems using the Storage Subsystem Configuration screen (see Section 4.2: Storage Subsystems of the HPSS Management Guide).
requests from a particular host or user. The Site Interfaces will be located in a shared library that is linked into the Gatekeeper. It is important that the Site Interfaces return a status in a timely fashion. Create, open, and stage requests from MPS are timing sensitive, thus the Site Interfaces won't be permitted to delay or deny these requests, however the Site Interfaces may choose to be involved in keeping statistics on these requests by monitoring requests from Authorized Callers.
PVL. It is possible for multiple HPSS PVRs to manage a single robot. This is done if it is necessary to organize the tape drives in the robot into partitions. Each tape drive in the robot is assigned to exactly one PVR. Additionally, each cartridge is assigned to only one PVR. The PVRs can be configured identically and can communicate with the robot through the same interface. The following sections describe the considerations for the various types of PVRs supported by HPSS. 3.7.6.1.
3.7.6.3. 3494 PVR The 3494 PVR can manage an IBM 3494 tape robot attached via Ethernet or SCSI. The PVR will create a process to receive asynchronous notifications from the robot. At least one PVR should be created for every robot managed by HPSS. If multiple 3494 robots are managed, care must be taken to ensure that the PVRs are configured to communicate with the correct /dev/lmcp devices. The PVRs can run on the same CPU or different CPUs as long as the proper /dev/lmcp devices are available.
3.7.7. Mover The Mover configuration is largely dictated by the hardware configuration of the HPSS system. Each Mover can handle both disk and tape devices and must run on the node to which the storage devices are attached. The Mover is also capable of supporting multiple data transfer mechanisms for sending data to or receiving data from HPSS clients (e.g., TCP/IP and shared memory). 3.7.7.1. AIX Asynchronous I/O Asynchronous I/O must be enabled manually on AIX platforms.
Driver has a built-in configuration table for all HPSS supported tape drives. This configuration provides variable block size for most HPSS supported drives. In order to override the built-in configuration, device information can be supplied in the /dev/kernel/st.conf as global properties that apply to each node. Consult the tape device driver documentation for instructions on installation and configuration. 3.7.7.2.3.
HBAs with a 7000 series chip set, uncomment the following lines in the ".config" file and follow the procedure for rebuilding your Linux kernel. CONFIG_SCSI_AIC7XXX=y CONFIG_AIC7XXX_CMDS_PER_DEVICE=253 CONFIG_AIC7XXX_RESET_DELAY_MS=15000 3.7.7.4. Performance The configuration of the Movers and attached devices can have a large impact on the performance of HPSS because of constraints imposed by a number of factors e.g., device channel bandwidth, network bandwidth, processor power.
3.7.9. Startup Daemon The Startup Daemon is responsible for starting, monitoring, and stopping the HPSS servers. The Daemon responds only to requests from the SSM System Manager. It shares responsibility with each HPSS server for ensuring that only one copy of the server runs at a given time. It helps the SSM determine whether servers are still running, and it allows the SSM to send signals to servers.
values may need to be overridden in env.conf.
Variable Default Value HPSS_SM_SRV_MAX_CONNECTIONS 50 Functionality Number of HPSS server connections to maintain at once. If this number of connections is exceeded, then old connections will be closed to maintain this number of connections * The SM attempts to throttle the connection attempts to other servers. It will attempt to reconnect to each server every HPSS_SM_SRV_CONNECT_INTERVAL_MIN seconds until the number of failures for that server has reached HPSS_SM_SRV_CONNECT_FAIL_COUNT.
Disk migration is controlled by several parameters. By default, these parameters are the same across all subsystems. However, subsystem-specific policies may be created which override all of these values. For a list of these parameters, refer to Section 6.4.2.2: Disk Migration Policy Configuration in the HPSS Management Guide. 3.9.1.2. Migration Policy for Tape There are two tape migration algorithms: tape file migration and tape volume migration.
storage class by periodically extracting this information from the HPSS Core Server. Based upon parameters in the purge policy, a purge run will be started when appropriate. The administrator can also force the start of a purge run via SSM. The disk purge is controlled by several parameters: • The Do not purge files accessed within minutes parameter determines the minimum amount of time a site wants to keep a file on disk.
out of the hpssGECOS field, it does not exist in UNIX. It only exists in LDAP. The metadata for each file and directory in an HPSS system contains an Account field, which determines how the storage will be charged. Each user has at least one default account index, which is put into the Account field of all new files and directories . When using UNIX-style accounting, the account index is the user's UID. When the user's UID is combined with the user's Realm Id, a unique Account is created.
the principal's LDAP hpssGECOS attribute, then Site-style accounting will be used. Otherwise UNIX-style accounting will be used. To keep the accounting information consistent, it is important to set up all users in the HPSS Authorization services with the same style of accounting (i.e. they should all have the AA= string in their hpssGECOS attribute or none should have this string.) The hpss_ldap_admin tool can be used to set attributes for a user including the hpssGECOS field.
3.9.4.5. Security Audit HPSS provides the ability to record information about authentication, file creation, deletion, access, and authorization events. The security audit policy in each HPSS server determines what audit records a server will generate. In general, all servers can create authentication events, but only the Core Server will generate file events. The security audit records are sent to the log file and are recorded as security type log messages. 3.9.5.
Gatekeeper configured in its storage subsystem. If one is found, the Core Server asks the Gatekeeper for its monitor types by calling the gk_GetMonitorTypes function which calls the locally implemented gk_site_GetMonitorTypes function which determines which types of requests to monitor. This query by the Core Server occurs each time the Core Server connects to the Gatekeeper, which occurs whenever the Core Server or Gatekeeper is restarted.
DOWN or when the user application is aborted. NOTES: 1. All open requests to the Core Server will call the Gatekeeping Service open API (gk_Open). This includes opens that end up invoking a stage. 2. Any stage call that is invoked on behalf of open will NOT call the Gatekeeping Service stage API (gk_Stage). (e.g. The ftp site stage command will use the Gatekeeping Service open API, gk_Open, rather than the Gatekeeping Service stage API, gk_Stage.) 3.
(VVs). Each VV contains one or more PVs. The VV characteristics described by a Storage Class include the VV Block Size and VV Stripe Width. If PVs are grouped one at a time, so that their Stripe Width is one, they are still defined as VVs. A number of additional parameters are defined in Storage Classes. These include migration and purge policies, minimum and maximum storage segment sizes, and warning thresholds.
SSM provides the capability to define storage classes and to add and delete virtual volumes to and from the defined storage classes. A storage class is identified by a storage class ID and its associated attributes. For detailed descriptions of each attribute associated with a storage class, see Section 6.1.1: Configuring a Storage Class of the HPSS Management Guide. The sections that follow give guidelines and explanations for creating and managing storage classes. 3.10.1.1.
utility programs copy data from one tape VV to another, so the number of available tape drives of the appropriate type must be at least twice the tape Stripe Width, for these programs to function. Migration of files between tape storage classes in a hierarchy, that are of the same media type, requires at least twice as many available tape drives as the Stripe Width of the Storage Class.
streaming to maximize performance. For recommended values for various media types, see Section 3.10.1.12: Some Recommended Parameter Values for Supported Storage Media on page 106. 3.10.1.6. Minimum Storage Segment Size Selection (disk only) The Core Server maps disk files onto a series of disk storage segments. The size of the storage segments is controlled by the Min Storage Segment Size parameter, the Max Storage Segment Size parameter, and the Average Number of Segments parameter.
3.10.1.8. Maximum VVs to Write (tape only) This parameter restricts the number of tape VVs, per Storage Class, that can be concurrently written by the Core Server. Its purpose is to minimize the number of tape mounts and to limit the number of tape VVs being written, to prevent files from being scattered over a number of tapes. The number of tape drives used to write files in the Storage Class will be limited to approximately the value of this field times the stripe width defined for the Storage Class.
3.10.1.12.1. Disk Media Parameters Table 7 contains attributes settings for the supported disk storage media types. Table 6. Suggested Block Sizes for Disk Disk Type Media Block Size Minimum Access Size Minimum Virtual Volume Block Size SCSI Attached 4 KB 0 1 MB SSA Attached 4 KB 0 1 MB Fibre Channel Attached 4 KB 0 1 MB In Table 7: • Media Block Size is the block size to use in the Storage Class definition.
Table 7.
StorageTek T10000A 256 KB 2560 500 GB StorageTek T10000B 256 KB 2560 1000 GB In the above table: • Media Block Size is the block size to use in the Storage Class definition. This is the size of the data blocks written to tape. Note that for tape devices, the Mover configuration does not contain the Media Block Size. This value may have a significant impact on data transfer performance, as for most tape devices each input/output request must be for the media block size.
3.10.3. Class of Service Each HPSS file belongs to a single Class of Service (COS) which is selected when the file is created. It is selected via Class of Service Hints information passed to the Core Server when the bitfile is created. If using the Client API, the application program has full access to this hints information. FTP users can use the “quote” command to set the COS. A pput request in PFTP automatically selects a COS based on file size unless the user explicitly selects the COS.
guaranteed that the file is completely and successfully staged before it is read. If the stage operation fails, the open will return with an error. Guideline 3: Select the Stage on Open Async option if you wish to stage the entire file to the top level in the hierarchy and do not want the Client API open to block.
Latency field from the Storage Class at the top level in the hierarchy. 3.10.3.6. Selecting Transfer Rate This field can be used via the COS Hints mechanism to affect COS selection. Guideline 1: This field should generally be set to the value of the Transfer Rate field in the Storage Class that is at the top level in the hierarchy. This should always be the case if the data is being staged on open.
• • · Maximum users and applications connected to DB2 · Nature of usage: read or update Database logging · Hardware or software mirroring · Disk speed and reliability: select the fastest, most reliable disk · Location of physical disks and database data: it is recommended that they be separate Database recovery · • • Enabling dropped table recovery will decrease database performance Amount of available memory and size of buffer pools · Buffer pools should not be larger than the tables using
directly to tape, the buffer size used by the FTP server and the virtual volume block size defined for the Storage Class being written to will have a significant impact. If the buffer size used by the FTP server is not a multiple of the virtual volume block size, each buffer written will result in a distinct storage segment on the tape. This will cause additional metadata to be stored in the system and extra synchronization processing of the tape.
3.11.5. Client API The Client API provides the capability to perform data transfer of any size (the size being parameters supplied by the client to the read and write interfaces). The size of the data transfers can have a significant impact on the performance of HPSS. In general, larger transfers will generate less overhead than a series of smaller transfers for the same total amount of data.
stage requests. The site policy could limit the maximum number of non-Authorized Caller requests allowed at once by either delaying or denying particular requests. To delay the request, the site policy may return a special retry status along with the number of seconds to wait before the Client API retries the request. Delaying requests should limit the number of create, open, and/or stage requests performed at a particular point in time, thus decreasing the load on the system.
3.12. HPSS Metadata Backup Considerations This section contains guidelines for proper maintenance of the HPSS metadata stored in DB2. The policies described should be fully understood and implemented to protect the HPSS metadata. Failure to follow these policies can lead to unrecoverable data loss. The remainder of this section is a set of rules associated with backing up HPSS metadata.
HPSS Installation Guide Release 6.2 (Revision 2.
Chapter 4. System Preparation This section covers the steps that must be taken to appropriately prepare your system for installation and configuration of HPSS and its infrastructure. • General setup (Section 4.1) • Setup filesystems (Section 4.2) • Setup tape libraries (Section 4.3) • Verify tape drives (Section 4.4) • Setup disk drives (Section 4.5) • Setup network parameters (Section 4.
• Configure the Perl prerequisite software on HPSS nodes. • Configure the SSH prerequisite software on the core HPSS server node (at a minimum) and configure SSH to accept connections from IBM Houston. Include the Houston subnet IP addresses 192.94.47 and 12.39.169 in the local firewall routing rules, as necessary. • Obtain the HPSS deployment tool package from your customer support representative and install it on each HPSS node in /opt/hpss/tools/deploy.
/db2/mirror-backups/subsys1 4.2.2. HPSS Filesystem Configure /var/hpss as a separate file system on each HPSS server node. This filesystem will store HPSS configuration files, log files, MPS reports, and other HPSS related files. It is recommended that this filesystem be at least 1GB in size. Configure /var/hpss/adm/core as a separate file system on each HPSS server node. If an HPSS process fails and creates a core file, it will be placed in this location.
(SMC) library device on the node that will run the HPSS LTO PVR, and verify that it is operational.
To start the lmcp daemon: root% /etc/methods/startatl To test whether lmcp daemon is configured and working correctly: % mtlib -l -qL where lmcpDevice is usually /dev/lmcp0.
For AML tape libraries: • If using an AML PVR, configure the Insert/Eject ports using the configuration files /var/hpss/ etc/AML_EjectPort.conf and /var/hpss/etc/AML_InsertPort.conf. Refer to Section 5.2.8.3: AML PVR Specific Configuration of the HPSS Management Guide for more information. 4.4. Verify Tape Drives Verify that the correct number and type of tape devices are available on each Tape Mover node. 4.4.1. AIX The tape devices section of the lsnode report displays all available tape drives.
To measure read performance on drive rmt1 using the previously-written uncompressed and compressed files: % iocheck -r -t 20 -b 1mb /dev/rmt1.1 % iocheck -r -t 20 -b 1mb /dev/rmt1.1 To unload a tape: % tctl -f rewoffl Repeat the above steps for each tape drive. 4.4.2. Solaris On each Tape Mover node, verify that each tape drive has the variable-length block size option enabled.
4.4.3. IRIX On each Tape Mover node, verify that each tape drive has the variable-length block size option enabled. To determine if the variable block size option is enabled, the following should complete successfully: % dd if=/dev/null of=/dev/rmt/tps2d6nr bs=80 count=1 % dd if=/dev/null of=/dev/rmt/tps2d6nr bs=1024 count=1 If the variable-length block size option is not enabled, consult your driver documentation for procedures to enable it.
4.5.1. AIX • Verify that the correct number and type of disk devices are available on each DB2 and Disk Mover node. The disk devices section of the lsnode report displays all available disk devices. To determine the type and number of available disk devices: % lsdev -C -S a -c disk • If using SSA disks spread the SSA disks equally across the two loops on each SSA adapter. The SSA configuration section of the lsnode report provides details on the SSA configuration.
Use the iocheck.ksh script from the deployment tools package to show the performance of one or more individual disk devices and the peak aggregate performance of concurrent I/O across multiple disks (e.g., to show the peak performance of adapters). WARNING: The contents of this logical volume will be overwritten by iocheck so be sure to use the correct logical volume name. To measure the individual and aggregate throughput of hdisks 4, 5, 6, and 7: % iocheck.
4.6. Setup Network Parameters • Install and configure all network interfaces and corresponding network connections. Refer to IBM's internal network technologies home page for resources on configuring and tuning networks and TCP/IP. The network interfaces section of the lsnode report from each node shows the network interfaces that are configured.
will be used. For Linux, a similar change should be made to /etc/nsswitch.conf: hosts: nis dns files • For each AIX ethernet network interface, verify that the en0 and et0 interfaces are not both configured at the same time (we recommend only using en0 unless the other nodes in the network are all using the 802.3 et* interface). Configure the local name service with the unique hostname for each network interface on all nodes and verify that each hostname is resolvable from other nodes.
... ccs0 ccs0 ... 64k 64k 64k 32k On On --- --- You are looking for the best values possible for each network connection. These values will be used by HPSS to optimize its data transfers. This example is, by no means, a complete picture of what controls network performance. In fact, it is assumed that you have already optimized the networks. The reason for gathering these values is to optimize HPSS performance on an already tuned network, not to fix underlying network problems.
udp_recvspace Controls the default size of the receive buffer for UPD/IP sockets. A value that is too small can cause server RPC sockets to be overrun. tcp_recvspace, tcp_sendspace Controls the default size for the receive and send buffers for TCP/IP sockets. Internally, HPSS servers and clients attempt to set these buffers sizes explicitly, but other utilities may not. rfc1323 Controls whether large TCP window sizes are used. Usually set to ON for higher throughput networks (e.g.
4.6.2. SP/x Switch Device Buffer Driver Buffer Pools IBM SP/x systems provide the capability to tune the buffer pool allocation in the switch device driver. Two variables can be changed: rpoolsize, which is the size of the buffer pool for incoming data, and spoolsize which is the buffer pool size for outgoing data. If these values are too small, then buffer overruns may occur. The current values of these variables can be interrogated with the lsattr command (e.g.
HPSS Installation Guide Release 6.2 (Revision 2.
Chapter 5. HPSS Installation and Infrastructure Configuration This chapter provides instructions and supporting information for installing the HPSS prerequisite software, the HPSS and DB2 software from the HPSS distribution media, and performing the HPSS infrastructure configuration. To install and setup an HPSS system, we recommend that the administrator be familiar with UNIX commands and configuration, be familiar with a UNIX text editor, and have some experience with shell scripts.
Table 9. Installation Package Sizes and Disk Requirements HPSS Package Name AIX HPSSServer-6.2.0.0.lpp < 65MB 450 MB HPSSClientMover-6.2.0.0.lp p < 10MB 70 MB HPSSSource-6.2.0.0.lpp < 40MB 130 MB HPSS Source Code HPSSServer-6.2.0.0.rpm < 20 MB 175 MB All HPSS Components HPSSClientMover-6.2.0.0.rp m < 5 MB 70 MB HPSSSource-6.2.0.0.rpm < 10 MB 130 MB IRIX HPSSClientMover-6.2.0.0.ta r.Z < 5 MB 75 MB Client and Mover Components SUN HPSSClientMover-6.2.0.0.
• The disk, where the installation target directory resides, has enough space to hold all the HPSS packages to be installed on this node. • WARNING: Do not use NFS mounted directories for installing nor allocating space for HPSS related components. Installing on NFS is problematic and the errors can be difficult to diagnose. 5.2. Install Prerequisite Software This section provides an overview of how to install the prerequisite software to prepare for the upcoming HPSS configuration.
If UNIX authorization will be used, this product is not required. LDAP authorization is supported on AIX only. If LDAP will be used for HPSS authorization, download the IBM Tivoli Directory Server from http://www.ibm.com. Search for "IBM Tivoli Directory Server" on the site to find the download page. The download page presents many options for downloading the software. Download version 6.0 of the package that is a tar file containing the LDAP software and its prerequisites.
XFS uses the Linux udev daemon to dynamically configure the DMAPI device node. Use the chkconfig command to make sure that boot.udev is "on". Note that if a change is required, then the machine needs to be rebooted in order for it to take effect. 5.3.
5. From the "Root Subsystem Machine" submenu, click on the 'Pre-Install Config' icon in the left panel. mkhpss will display the following screen: 6. Verify that the default values are correct for the given installation and modify if necessary. Click the 'Configure' button to perform the pre-installation setup. This will run a set of HPSS Installation Guide Release 6.2 (Revision 2.
scripts to verify/create the 'hpss' account and group, setup the /var/hpss (default location) directory with the required subdirectories and initialize the HPSS environment file, env.conf in /var/hpss/etc. 7. If the pre-installation configuration succeeds, you will see the following message in the command output window: ## run command exited with status 0 Click the 'Done' button on the Command Output window to close the window. 5.3.1.2.
DB2 software will be installed in the /usr/opt/db2_08_01 (AIX) or /opt/IBM/db2/v8.1 (Linux) directory. 2. For 6.2.0, the HPSS software and documentation should already be installed according to the installation step in section 5.3.1.1. Installing from CD will be an option starting in 6.2.1. 3. For 6.2.0, the hpssuser utility should be used to package the HTML files for delivery to each hpssgui machine or to a common shared file system on a remote node. Installing from CD will be an option starting in 6.2.1.
% su % cd /usr/opt/db2_08_01/adm % ./db2licm -a The generic DB2 license file (*/db2/license/db2ese.lic) can be found on the DB2 Installation CD or image. It can also be obtained by contacting your HPSS Support Representative. To update the license with the appropriate number of of processors, issue the following command: % .
3. Using the scrollbar, move the right-panel display until the "Authorization Service" information is seen. It should look like the following: HPSS Installation Guide Release 6.2 (Revision 2.
4. Select the "Configure Authorization Service" checkbox. Set the "Authorization Service" to "Unix and config files". 5. Review and modify (if necessary) the active fields: • Local Site Name. The value is usually set to the full machine name of the local host which can be determined using the 'hostname' and 'domainname' commands. • Local Realm Name. The value is usually set to the "Local Site Name" all capitalized. • Local Realm ID. The field is set to a unique ID number for each site.
configuration step. Other HPSS utilities are available to administer these HPSS configuration files. See section 2.2.2: Security Mechanisms in the HPSS Management Guide for more information. To use the HPSS configuration files, select the "Enable local Unix Passwd Files" checkbox. The default names for the files should be used as displayed. 7. Select the "Configure Server accounts" checkbox to create UNIX accounts for HPSS servers. 8.
• KDC Directory. The pathname of the KDC directory. This directory should be set to /var/hpss/krb5kdc. • Master Password. The Kerberos administration password. Be sure to remember this password to be able to administer the Kerberos environment later. • Verify Password. Re-enter the Kerberos administration password. • Kerberos Admin Principal. The userid to administer the Kerberos environment. • Authentication Type. There are two supported options: Keytab File or Password.
5. Select the "Configure Authorization Service" checkbox. Set the "Authorization Service" to "Unix and config files". 6. Review and modify (if necessary) the following authorization fields: • Local Site Name. The value is usually set to the full host name of the local host. This can be determined by using the 'hostname' and 'domainname' commands. • Local Realm Name. The value is usually set to the "Local Site Name" all capitalized. • Local Realm ID. The field is set to a unique ID number for each site.
8. Select the "Configure Server accounts" checkbox to create UNIX accounts for HPSS servers. 9. Click on the "Configure Security Services" button at the bottom of the screen to perform the specified security configuration. 10. If the configuration was a success, you should see the following message in the command output window: ## run command exited with status 0 Click the 'Done' button on the Command Output window to close the window. 5.3.1.4.3.
4. Select the "Configure Authorization Service" checkbox. Set the "Authorization Service" to "LDAP". 5. Review and modify (if necessary) the following authentication fields: • Kerberos Install Path. The pathname where Kerberos is installed. The default directory is /usr/kerberos. • KDC Directory. The pathname of the KDC directory. This directory should be set to /var/hpss/krb5kdc. • Master Password. The Kerberos administration password.
• Authentication Type. There are two supported options: Keytab File or Password. The Keytab File option allows HPSS servers or utilities to read a keytab file to authenticate. The Password option requires a password to be supplied each time an HPSS server or utility is invoked. • Password. The password used to authenticate the caller when the HPSS server or utility is invoked. This field is not enterable when the Authentication Type field is set to Keytab File. • Keytab File.
Unix Passwd Files" checkbox as shown in the example above. The default names for the files should be used as displayed. 8. Select the "Configure Server accounts" checkbox to create accounts for HPSS servers. 9. Click on the "Configure Security Services" button at the bottom of the screen to perform the specified security configuration. 10.
3. Review and modify (if necessary) the following fields: • Instance Owner. The name of the DB2 instance owner, normally 'hpssdb'. • Schema Name. The name of the DB2 schema containing the HPSS metadata table, normally 'hpss'. • Create DB2 Instance. Select this checkbox to create the DB2 instance. • Instance Owner Group. The UNIX group to which the Instance Owner is assigned, normally 'hpssdb'. • Instance Owner Home Directory.
• Create Subsys Database. Select this checkbox to create the "subsys1" database. • Subsystem Database Alias. The "real" database is configured as 'hsubsys1', while the database alias is the name used by HPSS servers and utilities to reference the database. The default value of 'subsys1' should normally be used. • Log File Directory. The DB2 log files will be placed in the /var/hpss/hpssdb filesystem by default.
B. On the “Tablespace Config” window, set the Tablespace Type from the drop down menu. For the USERSPACE1 tablespace, select “SMS” System Managed Space. For the remaining tablespace entries, select Tablespace Type to "DMS" Database Managed Space. C. For USERSPACE1, which uses System Managed space, from the "Tablespace Config" window, click the 'Add' button and select the directory you would like to allocate as the containers for this tablespace.
HPSS Installation Guide Release 6.2 (Revision 2.
F. Select the "Add" button to add "raw devices" to Container Path list. The "raw devices" must already exist. For AIX, the logical volumes must have already been created and the "raw" form be specified (i.e. LV "dbs1.tables", specify / dev/rdbs1.tables") For Linux, not only must the partition exist, but the "raw" mapping must also be in place prior to selecting the container path. See Section 3.5.3: HPSS Filesystems on page 71 for a listing of these mappings.
a variable named DB2_USE_LOCAL_RESYNC set to the value of 'true' when starting DB2 in order for the aforementioned fileset operations to complete successfully: csh & tcsh: setenv DB2_USE_LOCAL_RESYNC true sh & bash: export DB2_USE_LOCAL_RESYNC=true 5.3.1.6. Configure Other Services This menu configures various services such as Parallel FTP, Java security for SSM, and SSM start scripts. To configure Other Services, perform the following steps: 1.
4. After exiting mkhpss, verify permissions on the generated files. In particular, note the permissions on the keytab files. The hpss.keytab is used by HPSS servers to establish credentials. The mm.keytab is used by hpss utility programs. The kadm5.keytab is used to establish credentials as the Kerberos admin. Be certain that the permissions on these files allow access only to the appropriate users. /var/hpss/etc/hpss.keytab /var/hpss/etc/mm.keytab /var/hpss/krb5kdc/kadm5.keytab 5.3.1.7.
5.3.2. Install and Configure HPSS – Secondary Subsystem Machine For the secondary subsystem machine, the following configuration steps must be performed: • Pre-installation Configuration • Install HPSS documentation and DB2 • Set up DB2 permanent license • Install configuration bundle • Configure Security Services • Configure DB2 • Configure Other Services The following sections describe the procedure to perform the above steps. 5.3.2.1.
3. Verify that the default values are as desired. Modify them, if necessary. Click the 'Configure' button to perform the pre-installation setup. 4. If the pre-installation configuration was a success, you will see the following message in the command output window: ## run command exited with status 0 Click the 'Done' button on the Command Output window to close the window. 5.3.2.2.
This panel allows you to install the DB2 software, HPSS software (option starting in 6.2.1), and HPSS documentation. Perform the following steps to install the software: 1. To install DB2, check the 'Install DB2' checkbox. Click on the 'Source Path' button and select the directory that contains the DB2 filesets you wish to install. DB2 software will be installed in the /usr/opt/db2_08_01 (AIX) or /opt/IBM/db2/v8.1 (Linux) directory. 2. For 6.2.
% su % cd /usr/opt/db2_08_01/adm % ./db2licm -a The generic DB2 license file (*/db2/license/db2ese.lic) can be found on the DB2 Installation CD or image. It can also be obtained by contacting your HPSS Support Representative. To update the license with the appropriate number of of processors, issue the following command: % .
5.3.2.5. Configure HPSS Security Services This section describes the procedure to configure a Kerberos or UNIX Security Client on the secondary subsystem machine. The security client must match the security mechanism on the root machine. To configure the security client for the secondary subsystem machine, perform the following steps: 1. From the Secondary Subsystem Machine submenu in the left panel, click on the "Configuration" icon and then the "Security Services" icon.
• Configure server accounts. This checkbox is flagged when the server accounts should be created on the local machine. This is usually not required when the Kerberos server has already been configured with the HPSS server accounts. 3. To configure the UNIX client, un-check the "Configure Kerberos Client" checkbox and select the "Configure server accounts" checkbox. 4. Click on the "Configure Security Services" button to configure the security client on the Secondary Subsystem machine. 5.
2. Using the scrollbar, move the right-hand panel until you see the following screen: 3. Review and modify (if necessary) the following fields: • Instance Owner. The name of the instance owner HPSS will use. Should use the default value of 'hpssdb' in most cases. • Schema Name. The name of the HPSS schema used by the database to reference the HPSS DB2 tables. Should use the default value of 'HPSS' in most cases. • Create DB2 Instance. Select this checkbox to create the DB2 instance on this machine.
• Authentication Type. Currently there is only one authentication type allowed. • User Name. The UNIX userid to be used for DB2 authentication. This userid must have a valid UNIX account before running this configuration option. • Password and Verify Password. Enter and verify the UNIX password for the userid entered in the User Name field above. • Create Subsys Database. Select this checkbox to create the "subsysX" database (where X will be 2, 3, ...). • Subsystem Database Alias.
"Configuration" icon and then the "Other Services" icon. The following window will be shown: 2. Click on the "Configure Other Services" button to complete the configuration of the Secondary Subsystem Machine. If the configuration was a success, you should see the following message in the command output window: ## run command exited with status 0 Click the 'Done' button on the Command Output window to close the window. 5.3.3.
Source Tree to obtain the source code from the root machine and install on the Mover/Client machine. 5.3.3.2. Install Configuration Bundle The configuration bundle is created on the root subsystem and contains configuration files used by the root, Mover/Client and subsystem machines. See page 159, section 5.3.1.7 Create Configuration Bundle for instructions on creating the Configuration Bundle on the root machine. To install the configuration bundle, perform the following steps: 1.
/opt/hpss/config/ /opt/hpss/stk/ /opt/hpss/src/ Included only if the hpss-src package is installed. /var/hpss/ 2.
bundle the HTML files for delivery to the hpssgui host machine. The recommended installation location for the HTML files on each hpssgui host is /var/hpss/doc for AIX and Linux platforms and c:\hpss\doc for Windows platforms. 5.5.2. Manual Page Setup Perform the following steps to set up HPSS manual pages: 1. Create a symbolic link for the HPSS manual catalog. Assuming that the HPSS tree is installed at /opt/hpss, use the following commands: % su - root % cd /opt/hpss % ln -s man cat7 2.
wishing to learn more about DB2 tuning are referred to the HPSS DB2 Tuning Guide, available from your support representative, the DB2 Administrative Guide: Performance, available on-line from the IBM DB2 website, the IBM DB2 Tuning Redbook, and the many IBM and after-market books available on this subject. Be sure to contact your HPSS support representative for advice before making changes to DB2 configuration parameters.
Makefile.macros: BUILD_PLATFORM = AIX BUILD_TOP_ROOT = /opt/hpss KRB5_AUTH_SUPPORT=on UNIX_AUTH_SUPPORT=on LDAP_AUTH_SUPPORT=off GSI_AUTH_SUPPORT=off CONVERSION_FROM_45 = off CONVERSION_FROM_51 = off BUILD_UNSUPPORTED = off MVR1_PROGRAM_NAME = $(LOCAL_BIN)/hpss_mvr_tcp MVR1_OPTIONS = "" # MVR2_PROGRAM_NAME = $(LOCAL_BIN)/hpss_mvr_ssd # MVR2_OPTIONS = "SSD" # MVR3_PROGRAM_NAME = $(LOCAL_BIN)/hpss_mvr_omi # MVR3_OPTIONS = "OMI" # MVR4_PROGRAM_NAME = $(LOCAL_BIN)/hpss_mvr_dd2 # MVR4_OPTIONS = "DD2" Makefile.
1. Log on as hpss. 2. Change directory to /opt/hpss/bin. 3. Run the following command: % hpss_db2_bindall.ksh 5.8.2. Construct and Build the HPSS Mover/Client Source Tree This section describes the procedures to extract the Mover/Client code from the HPSS base source tree and to build the Mover/Client binaries. 5.8.2.1. Construct the HPSS Mover/Client Source Tree The HPSS Mover/Client source tree can be extracted from the HPSS base source tree.
5.8.3. Construct and Build the HPSS HDM Source Tree XFS is not supported in HPSS 6.2. XFS references have been left in the HPSS documentation to support the option of re-enabling XFS in future releases. This section describes the procedures to extract the HDM source code from the HPSS base source tree and to build the HDM binaries. 5.8.3.1. Construct the HPSS HDM Source Tree The HPSS HDM source tree can be extracted from the HPSS base source tree.
Unix authentication with LDAP authorization is not supported at this time, so it only makes sense to do this if you are using Unix authorization.
serverinterfaces where server_id = (select server_id from server where desc_name = 'SSM System Manager') and descriptive_name = 'Administrative Client Interface' AUTHN_MECH_SET_NUM_MECHS AUTHN_MECH_SET_MECHS1 ------------------------ --------------------1 0 1 record(s) selected. db2 => update server set (num_auth_mechs, authn_mechs1_mechanism, authn_mechs1_auth_type_key) = (2, 2, 1) where desc_name = 'SSM System Manager' DB20000I The SQL command completed successfully.
HPSS Installation Guide Release 6.2 (Revision 2.
Chapter 6. Upgrading to HPSS Release 6.2 This chapter is only intended for sites upgrading HPSS from either version 4.5 or 5.1 to version 6.2. Sites wishing to upgrade from prior HPSS releases must first upgrade to HPSS 4.5 or HPSS 5.1. Sites that are going to install and configure HPSS 6.2 from scratch will not need to perform the upgrade. The upgrade procedures in this chapter are intended only for HPSS systems running on AIX.
6.2. Planning for the HPSS 6.2 Upgrade This section provides information necessary to plan the upgrade of an HPSS 4.5 or HPSS 5.1 system to HPSS 6.2. It is important that all planning information be reviewed carefully before performing the upgrade. 6.2.1. Metadata changes in HPSS 6.2 From 4.5 to 6.2: • Encina's Structured File System (SFS) product is replaced by DB2 Universal Database (UDB) as the metadata database for HPSS. • Migration policies are now either of type disk or tape, but not both.
• Modification of NFS metadata. In support of new authentication mechanisms, elimination of credential object Id. Change privileged caller principal length from 15 to 255 characters. NFS is no longer supported. The NFS table contains converted metadata, but will be renamed to PRE62_NFS to prevent use. • Modification of server metadata. In support of new HPSS RPC library, elimination of authorization service and authentication service information.
information. New index definition for table. The metadata conversion is performed by the hpss_51_62_server conversion program. • New SERVERINTERFACES table. This table is populated by the hpss_51_62_server conversion with default server interface information. • Modification of SITE table. In support of new authorization mechanism, elimination of descriptive and LS group names and addition of site and realm names. The pre-6.
hpss_ldap_import to convert DCE authorization information into LDAP. • Kerberos authentication and Unix authorization. In this case, the site determines on its own how to convert DCE authentication information into Kerberos. The site will then use hpss_unix_import to convert DCE authorization information into Unix. Depending on environment variables, the hpss_unix_import program may import authentication information (i.e. Create a password for the Unix user) into Unix.
• The hpss_dce_export and hpss_unix_import utilities are provided to convert DCE authorization information. LDAP: • Configuring LDAP is more complex than Unix. • Managing LDAP is fairly simple and does not require regular maintenance. • The hpss_dce_export and hpss_ldap_import utilities areprovided to convert DCE authorization information. 6.2.4. New HPSS 6.
three minutes, and the subsystem conversion took approximately 40 minutes. 6.2.6.1. Running Time for the Long Running Metadata Conversion Utilities (for 4.5 upgrades only) A system with more than 3 million HPSS files will see a significant performance gain from choosing to run the long running metadata conversion programs in parallel. With 3 million files, the serial conversion should take approximately 11 hours to finish, while the parallel conversions would take only 2 hours.
running the metadata conversion. This directory will be used to store one text-based restart file, various text-based db2 output files, and other text files. Prior to performing the metadata conversion, become familiar with the conversion program output for the HPSS 4.5 conversion in Section 6.5: HPSS 4.5 Conversion Utilities Output on page 232 or for the HPSS 5.1 conversion in Section 6.3.15: Upgrade from HPSS 5.1 to HPSS 6.2 on page 213.
% DB2 UPDATE DB CFG FOR USING To reset to default settings, execute: % DB2 RESET DB CFG FOR Review and update the following: • Associated with the database log are the log file size (LOGFILSIZ), the number of primary logs (LOGPRIMARY), and number of secondary log files (LOGSECOND). A good size for LOGFILSIZ prior to performing a metadata conversion is 50,000 (these are 4K pages, so each log file will be about 200 MB).
attributes pertinent to authorization are retained in HPSS 6.2. The program also optionally supports Unix authentication in HPSS 6.
load into the new table. • db_convert_nsobject - This program converts the name server object metadata from the HPSS 4.5 nsobjects SFS file to the HPSS 6.2 DB2 NSOBJECT and NSACL tables by reading records from SFS and performing DB2 loads into the new tables. • db_convert_nstext - This program converts the name server text metadata from the HPSS 4.5 nstext SFS file to the HPSS 6.2 DB2 NSTEXT table by reading records from SFS and performing a DB2 load into the new table.
hpss_md_convert_51. This program will rename the HPSS 5.1 tables by prepending the original table name with PRE62_ so that the original metadata in HPSS 5.1 is preserved. The hpss_md_convert_51 utility calls the following programs to perform the conversion: • hpss_51_62_dmg – Reads the 5.1 metadata from the PRE62_DMG table, modifies the metadata as needed, and inserts into the DMG table for use in HPSS 6.2 • hpss_51_62_dmgfileset - Reads the 5.
• Acquire Kerberos or LDAP software, as needed. See Section 6.2.3: New Authentication and Authorization Mechanisms on page 182 for a list of valid authentication and authorization combinations. See Section 5.2: Install Prerequisite Software on page 137 for more information on obtaining MIT or IBM Kerberos and LDAP. • Acquire DB2 UDB • Acquire software to upgrade AIX, if necessary • Acquire Java software • Acquire HPSS Release 6.2 distribution images 6.3.3.
instance (e.g. . /home/hpssldap/sqllib/db2profile) 7. Setup DB2 instance for LDAP: % /opt/IBM/ldap/V6.0/sbin/idsicrt –n -e -I -t -l e.g. % /opt/IBM/ldap/V6.0/sbin/idsicrt -n -e [asUf4As].f -I idsldap -t idsldap -l /home/idsldap 8. Set the LDAP admin distinguished name (DN) and password by: % /opt/IBM/ldap/V6.0/sbin/idsdnpw -I -u -p e.g. % /opt/IBM/ldap/V6.
% /opt/IBM/ldap/V6.0/bin/idsldapmodify –c –i /opt/hpss/config/ HPSS_ldap_schema.ldif –D “cn=hpssldap” –w -v Note: where passwd is the password for the DN. In the example above, it would be the hpssldap user’s password since hpssldap is the DN used above and configured with LDAP on this system. 14. Setup LDAP stash file in /var/hpss/etc/ldap_stash: % cat /var/hpss/etc/ldap_stash ldap { host=hpss.acme.
6.3.5. Upgrade AIX This step cannot be performed while the HPSS 4.5 or 5.1 system is running. This section applies to site that are upgrading to HPSS 6.2 from both HPSS 4.5 and HPSS 5.1. See Section 3.3.2: Prerequisite Summary By HPSS Node Type on page 59 for the the appropriate version of AIX required for HPSS 6.2. This step should be performed on all systems used with HPSS 6.2. 6.3.6. Install or Upgrade Java This step can be performed while the HPSS 4.5 or 5.1 system is running. See Section 3.3.
6.3.8.1. Install HPSS 6.2 Distribution Image This step can be performed while the HPSS 4.5 or 5.1 system is running. Prior to installing the HPSS 6.2 code, sites should not disturb the current HPSS release code. It is possible for a site to revert to the previous release of HPSS at any point prior to the last step of starting the HPSS 6.2 servers. In order to facilitate reverting to the previous release of HPSS, sites are recommended to install the HPSS 6.
code. Check $HPSS_ROOT/bin/convert62 to ensure the conversion programs compiled properly. The conversion utilities must include several Encina SFS files to compile and link using /usr/lpp/encina as the default pathname. If this problem is encounted during building the source code, create a link from /usr/lpp/encina to where the root Encina SFS directory resides. 6.3.8.3.
Next, use mkhpss and select the Configuration option and then Other Services option to configure the IEEE address for HPSS. This will create the /var/hpss/etc/ieee_802_addr file. This is only used by HPSS and will not affect other services on this system. Without performing this option the HPSS RPC mechanisms will fail. Note that an error may occur about not finding the SSM server configuration, and this is expected and should be ignored because the SSM server metadata has not been converted yet.
Environment variables are no longer kept in the $HPSS_ROOT/config subdirectory. Many are now kept in /var/hpss/etc/env.conf. The mkhpss utility will assist in creating the env.conf file and HPSS specific environment variables can be added to it. Before running the conversion utilities, ensure the following environment variables are properly defined. These environment variables can be defined in the new env.conf file or any other way a site defines environment variables.
• MKHPSS_ROOT – set to the path of HPSS 6.2 code (e.g. "/opt/hpss") • HPSS_DB_INSTANCE_OWNER – the Unix User ID of the DB2 HPSS instance (e.g. "hpssdb") • HPSS_LDAP_BIND_TYPE – should correspond with the type of bindings allowed for LDAP (e.g. "SIMPLE") • HPSS_LDAP_URL – the valid URL for the HPSS LDAP server (e.g. "ldap://hpss.acme.com/cn=hpss.acme.com") • HPSS_LDAP_BIND_ARG – the name of the file created previously to hold the user name and password of the LDAP owner (e.g.
HPSS_PRIMARY_AUTHN_MECH=krb5 HPSS_PRIMARY_AUTHENTICATOR=/krb5/hpss.keytabs Valid settings for unix authentication are: HPSS_PRIMARY_AUTHN_MECH=unix HPSS_PRIMARY_AUTHENTICATOR=/etc/passwd The new authorization is performed based on the contents of /var/hpss/etc/site.conf. This file will tell HPSS to use unix authorization, or to use LDAP and how to contact the LDAP server based on the URL entered.
Using the scrollbar, move the right-panel scrollbar until the "Authorization Service" information is seen. It should look like the following: HPSS Installation Guide Release 6.2 (Revision 2.
6.3.11. Pre-Conversion System Check • It is strongly recommended that a test conversion be performed prior to running the conversion on a production system. See Section 184: Testing the Metadata Conversion on page 184 • Ensure that the DB2 database has been configured prior to the conversion • Conversion output should be captured and any problems corrected before continuing. See Section 6.2.
6.3.12. Take a full backup of SFS or DB2 This step should not be performed while the HPSS 4.5 or 5.1 system is running. The SFS or DB2 backup should be performed on the root and each non-root subsytem. Refer to the HPSS 4.5 or 5.1 Management Guide or contact your HPSS customer support representative for instructions on how to complete a SFS or DB2 backup. 6.3.13. Upgrade from HPSS 4.5 to HPSS 6.2 Only follow procedures in this section to convert an HPSS 4.5 system to HPSS 6.2.
The first program used in the 4.5 conversion is the information collection utility, db_convert_collect_info. This utility will gather configuration related information based on input provided about the 4.5 HPSS system. The program will save the information to several text files located in the /var/hpss/convert/6.2 directory. This utility must be run once for each subsystem and must be run prior to invoking any other conversion utilities.
subsystem Id of the subystem being converted as the command line arguments. For example, % su – root % db_subsys_convert /.:/encina/sfs/hpss/globalconfig 1 This program provides an option to change all partially written HPSS 4.5 tapes to be read-only. When invoked, the option will force the storage map tape conversion to look at the current state (administrative and map states associated with storage map tape and tape virtual volume metadata) of each tape volume in HPSS 4.
-db /var/hpss/convert/6.2/Convert.DB.Names \ -ss /var/hpss/convert/6.2/Convert.SS.Server.Ids \ -sub 1 % db_convert_bitfile –g /.:/encina/sfs/hpss/globalconfig \ -db /var/hpss/convert/6.2/Convert.DB.Names \ -sub 1 % db_convert_nsobject –g /.:/encina/sfs/hpss/globalconfig \ -db /var/hpss/convert/6.2/Convert.DB.Names \ -ns /var/hpss/convert/6.2/Convert.NS.Server.Ids \ -sub 1 % db_convert_nstext –g /.:/encina/sfs/hpss/globalconfig \ -db /var/hpss/convert/6.2/Convert.DB.
root 1456 2034 root 1567 1456 0 16:04:08 pts/13 0 16:04:08 pts/13 0:00 db_convert_bitfile ... 0:00 ksh db2 load ... Next, issue a 'kill -15 1381', sending a SIGTERM (kill -l, will list all signals and their appropriate number on your system) to the only the conversion C program, and not to any of the other bitfile conversion processes. Note that all C conversion programs begin with lr_db_convert.
-db /var/hpss/convert/6.2/Convert.DB.Names \ -sub 1 -restart When given the ‘-restart’ flag, the program will first attempt to determine that no other conversions or DB2 loads are currently running on the specific table. If a load or conversion is still running (due to failure to cleanup from an abnormal termination), the conversion should output a message and quit. If no conversion or load is running, the restart will perform a db2 load insert to append metadata to the table.
error occurs while reading metadata. A restart from a user termination (CTRL-C) works well 90% of the time. Sometimes a user termination can cause DB2 to output many system log errors (usually means the db2as, administrative server died). If the program is unable to perform a restart (i.e. an attempt to perform the restart and the program hangs or exits with an error again), there are steps to take to discover the problem.
environment variables to HPSS 4.5 values. Examples follow: • HPSS_PRINCIPAL_SSM=hpss_ssm • HPSS_PRINCIPAL_FTPD=hpss_ftp • HPSS_PRINCIPAL_MPS=hpss_mps • HPSS_PRINCIPAL_NDCG=hpss_ndcg • HPSS_PRINCIPAL_NFSD=hpss_nfs • HPSS_PRINCIPAL_DMG=hpss_dmg This allows the db_convert_dce_cds program to add proper DCE principals from HPSS 4.5 to a newly created core server ACL. Upon completion of the db_convert_dce_cds program, ensure the values are returned to valid HPSS 6.2 values.
machine to enable the HPSS 6.2 system to work properly. On several test machines, we had to open world access to the devices used by the HPSS Core Server and PVL in order for them to initialize properly. 6.3.14. Verify HPSS 4.5 Conversion Results • Prior to running the verification utilities, we recommend that running the DB2 utility RUNSTATS on each table in every database. This will enable the verification utilities to run quicker than if RUNSTATS is not used. For example: “db2 runstats on table hpss.
ensure that the site has SFS and DB2 configured as expected. This program is not able to check the Name Server Objects (NSOBJECT) or ACLs (NSACL) table. The object count between two systems does not match because objects associated with deleted bitfiles are not converted. To invoke this utility, see the usage below: db_convert_size_check -g -db [-m] (Metadata Consistency Check - sfs vs. db2) [-t ] (check specified table) [-v] (verbose) 6.3.14.3.check every bitfile descriptor in the 6.2 system. The -f and -l options check only the first and last number of bitfile descriptors provided with these options. To invoke this utility, see the usage below: db_convert_address_check -g -db -ss -s [-a] (check all files) [-f ] (check first files) [-l ] (check last files) 6.3.14.5. Terminate Scripting Session Terminate the scripting session.
Option 1 – Rename old tables [ [ [ [ [ [ renaming renaming renaming renaming renaming renaming LSPOLICY table to PRE62_LSPOLICY ] SERVER table to PRE62_SERVER ] DMG table to PRE62_DMG ] GATEKEEPER table to PRE62_GATEKEEPER ] MOVERDEVICE table to PRE62_MOVERDEVICE ] SITE table to PRE62_SITE ] Option 2 – Create new tables Note: This option uses the hpss_managetables program which will assist in creating the newly formatted HPSS 6.2 tables.
6.3.16. Enable DB2 Backup Set up the DB2 backup process for HPSS metadata. Ensure that an initial backup of all HPSS databases are available at this point. Perform this step on each root and non-root subsystem. 6.3.17. Perform the DCE Export: hpss_dce_export This step applies to upgrades for both HPSS 4.5 and 5.1. The program that exports principal and group information from DCE is called hpss_dce_export.
5.
The conversion program that imports DCE authorization information into LDAP is called hpss_ldap_import. Usage: hpss_ldap_import -realmname % hpss_ldap_import /var/hpss/convert/6.2 -realmname "cn=hpss.acme.com" Where the realmname option should use the name of the realm desired in LDAP. The program requires a path to the directory where expected input files reside (the same path used when running hpss_dce_export).
6.3.19.1. Tune DB2 for normal operations Refer to Section 5.7: Tune DB2 on page 171. Perform this step on each root and non-root subsystem. 6.3.19.2. Modify Accounting, if applicable If the HPSS system consists of multiple subsystems and accounting by subsystems is desired, accounting metadata should be modified to support accounting by subsystem. The necessary procedure is quite complex. Contact your HPSS customer support representative for information.
that each caller of the interface is given. Run the hpss_init_server_acls program as follows: % /opt/hpss/bin/convert62/hpss_init_server_acls Error in stat of Keytab File, /var/hpss/etc/mm.keytab, 2 Note: The error is expected and does not indicate that the hpss_init_server_acls program did not complete successfully. In the above example, the error is displayed because the system did not have a mm lib or DB2 username and password setup for this system.
hpssgui.pl hpssgui.vbs hpss.jar login.conf The -ssmclientpkg option creates a tar file and places it in the specified directory (e.g. /tmp/ssmclientpkg.tar) which will contain the following files: • hpss.jar – Java executables for running the SSM GUI • hpssgui.pl – Perl version of the SSM GUI start up script • hpssgui.vbs – Visual Basic version of the SSM GUI start up script • krb5.conf – Kerberos environment variable file • login.
6.3.20.1. Invoke the SSM System Manager, Startup Daemon and prerequisite software Perform this step on the root subsystem only. Use the rc.hpss script to invoke the Startup Daemon, SSM System Manager and the prerequisite software. % /opt/hpss/bin/rc.hpss or $ /etc/rc.hpss If problems are encountered, try starting the software independantly using the rc.hpss script. The default operation is "start". Usage: rc.
The default name of the SSM configuration file is ssm.conf. Its default location is the current directory or, if not there, in $HPSS_PATH_SSM (/var/hpss/ssm by default). The name and location may be overridden by the -m option to the hpssgui.pl script or by setting the value in the user's environment. The value of HPSS_SSM_SM_PORT_NUM can be obtained by connecting to the DB2 configuration database (e.g. CFG) and issuing the following SQL statement: % db2 select rpc_prog_num,rpc_prog_vers from hpss.
MPS 10 10 5 20 PVL 50 10 1 2 PVR 30 20 5 30 Mover 20 20 1 5 Startup Daemon 10 10 0 2 SSM System Manager 100 100 5 20 4. Review and update the Core Server(s) other configuration parameters, if necessary 5. For 4.5 upgrades only: Review all migration policies to ensure that they were converted correctly In pre-5.1 HPSS systems, migration policies could apply to both a disk and tape storage class. In HPSS 5.
fields will display the current value. Change the accounting bits such that if the initial value is 1 then the new value will be 2 and conversely if the initial value is 2 then the new value will be 1. At this point, the HPSS system has no accounting metadata for existing files. Any new files that are added will generate new acounting metadata but existing files need to have their accounting metadata created using an additional conversion routine (acct_convert).
• Using the GUI, mark the MPS configuration executable and start it • Verify that Migration and Purge are working correctly • Repack and reclaim tape volumes 6.3.22. Clean Up After a 4.5 to 6.2 Upgrade After the 6.2 system has been operational for an extended period, clean up the following: • If it is still running, shutdown the SFS server • Unconfigure SFS server(s) using mkhpss from the /opt/hpss_45 code • Uninstall Encina code • Uninstall Sammi code • Delete HPSS 4.
in tape mount errors for read operations Once it is determined that it’s safe to revert back to Release 6.2, perform the steps as follows. Note that this procedure assumes that the upgrade stopped after the 6.2 servers are up but before changes were made that may affect the 4.5 user data or invalidate the 4.5 metadata. Perform the following steps for each root and non-root subsystem: 1. Shutdown all HPSS servers, if running. 2. Shutdown DB2. This step is optional since it does not affect the 4.5 system. 3.
the "PRE62_". Next, beginning with step 4, follow the procedures in Section 6.3.24.1: Revert the HPSS 6.2 System to Version 4.5 on page 225. The site will also have to revert the operating system software and any other upgraded software back to the HPSS 4.5 prerequisite versions so that the HPSS 4.5 code is not running on unsupported software. 6.4.
6.4.1.2. db_config_convert, db_subsys_convert, and db_lr_convert Errors and Warnings 1SQL3304N The table does not exist. => The program is attempting to insert data into a table that has not been created under the database name and schema it is running under. Determine the database name entered for the subsystem during the db_convert_collect_info program (i.e. Suppose “dwcntrl” was entered for the database name for subsystem 1; however, the table was created under a different database name).
Resolution: Check that the table exists in the schema and that the db_convert_collect_info utility has been run recently. The empty table scripts pull their information about which schema to use from the text file /var/hpss/convert/6.2/Convert.DB.Names, created by running db_convert_collect_info. Enter the appropriate schema name for the global database (which will be used for all databases) when rerunning db_convert_collect_info.
Resolution: Either the wrong filename or the wrong subsystem Id for that filename was entered. Rerun the conversion program with the correct subsystem Id and filename. SQL0968C The file system is full. SQLSTATE=57011 => This error message usually follows 10 or so other SQL messages, and means that the load operation for the long running conversion program failed because, most likely, the system temporary tablespace (which defaults to TEMPSPACE1) has run out of room.
Error: write_bitfile_to_ASC failed to write to pipe, fd is -1, errno = 9 => This error results when running a long running conversion program that cannot write to /var/hpss/convert/6.2 and create a file called asc_fifo_bitfile (the named pipe for the C program to communicate with the DB2 load). Resolution: Ensure that the user running the conversion has write permissions on /var/hpss/convert/6.2. Rerun the conversion program. Could not find database 'subsys1' using schema 'hpss'.
trying to insert the converted metadata. If this resulted from previously running the metadata conversion, and the metadata should be re-converted, then identify the table name (e.g. “HPSS.SERVERINTERFACES”) and empty the table by executing a % db2 delete from
statement where is the exact table name in the statement (e.g. HPSS.SERVER).through several syslogd error messages output to the monitor or a failed attempt at restarting the failed conversion program. Should DB2 be placed in an inconsistent state, the tablespace will likely have to be dropped and recreated along with the table being converted. Then the database will need to be recycled (using db2stop and db2start).
The output displays “Re-running” which indicates that the utility is performing a restart. When performing a restart and there are already rows in the DB2 table, rows committed should still equal rows submitted to the load. The conversion program will determine the number of records already loaded into the table, find the next consecutive SFS record to load, and continue counting until all SFS records are converted properly and inserted into the DB2 table(s).
$ db_config_convert /.:/encina/sfs/hpss/globalconfig Logged into DCE as hpss_ssm. Emptying all configuration tables in global database... All config tables emptied in global database, under schema hpss. Running db_convert_global utility... Converting SFS file /.:/encina/sfs/hpss/globalconfig Converted 1 record successfully from /.:/encina/sfs/hpss/globalconfig db_convert_global complete, inserted 1 record into DB2 global table Running db_convert_storsubsys utility... Converting SFS file /.
Running db_convert_gatekeeper utility... Converting SFS file /.:/encina/sfs/hpss/gkconfig Converted 1 records successfully from /.:/encina/sfs/hpss/gkconfig db_convert_gatekeeper complete, inserted 1 records into DB2 gatekeeper table Running db_convert_logclient utility... Converting SFS file /.:/encina/sfs/hpss/logclient Converted 1 records successfully from /.:/encina/sfs/hpss/logclient db_convert_logclient complete, inserted 1 records into DB2 logclient table Running db_convert_logdaemon utility...
/.:/encina/sfs/hpss/cartridge_stk Converting SFS file /.:/encina/sfs/hpss/cartridge_stk_rait Converted 4 records successfully from /.:/encina/sfs/hpss/cartridge_stk_rait db_convert_cartridge complete, inserted 32 records into DB2 cartridge table Running db_convert_cos utility... Converting SFS file /.:/encina/sfs/hpss/cos Converted 21 records successfully from /.:/encina/sfs/hpss/cos db_convert_cos complete, inserted 21 records into DB2 cos table Running db_convert_dmgfileset utility...
Running db_convert_purgepol utility... Converting SFS file /.:/encina/sfs/hpss/purgepolicy Converted 1 records successfully from /.:/encina/sfs/hpss/purgepolicy db_convert_purgepol complete, inserted 1 records into DB2 purgepol table Running db_convert_pvldrive utility... Converting SFS file /.:/encina/sfs/hpss/pvldrive Converted 17 records successfully from /.:/encina/sfs/hpss/pvldrive db_convert_pvldrive complete, inserted 17 records into DB2 pvldrive table Running db_convert_pvlpv utility...
$ db_subsys_convert /.:/encina/sfs/hpss/globalconfig 1 Logged into DCE as hpss_ssm. Emptying all subsystem tables in subsys1 database... All subsystem tables emptied in subsys1 database, under schema hpss. Running db_convert_acctlog utility... Converting SFS file /.:/encina/sfs/hpss/acctlog.1 Converted 0 records successfully from /.:/encina/sfs/hpss/acctlog.1 db_convert_acctlog complete, inserted 0 records into DB2 acctlog table for subsystem 1 Running db_convert_acctsnap utility... Converting SFS file /.
Converted 600 records successfully from /.:/encina/sfs/hpss/bfmigrrec.1 db_convert_bfmigrec complete, inserted 600 records into DB2 bfmigrec table for subsystem 1 Running db_convert_bfpurgerec utility... Converting SFS file /.:/encina/sfs/hpss/bfpurgerec.1 Converted 0 records successfully from /.:/encina/sfs/hpss/bfpurgerec.1 db_convert_bfpurgerec complete, inserted 0 records into DB2 bfpurgerec table for subsystem 1 Running db_convert_bfssegchkpt utility... Converting SFS file /.
table for subsystem 1 Running db_convert_storagemapdisk utility... Converting SFS file /.:/encina/sfs/hpss/storagemapdisk.1 Converted 3 records successfully from /.:/encina/sfs/hpss/storagemapdisk.1 db_convert_storagemapdisk complete, inserted 3 records into DB2 storagemapdisk table for subsystem 1 db_convert_storagemapdisk complete, inserted 1 records into DB2 diskspace table for subsystem 1 Running db_convert_storagemaptape utility... Converting SFS file /.:/encina/sfs/hpss/storagemaptape.
Converting SFS file /.:/encina/sfs/hpss/bftapesegment.1 Converted 1159 records successfully from /.:/encina/sfs/hpss/bftapesegment.1 lr_db_convert_bftapeseg complete, submitted 1159 records to DB2 bftapeseg load for subsystem 1, 1596 operations per sec, 0.725914 total time Number Number Number Number Number Number of of of of of of rows rows rows rows rows rows read skipped loaded rejected deleted committed = = = = = = 1159 0 1159 0 0 1159 Running lr_db_convert_bitfile utility...
Converting SFS file /.:/encina/sfs/hpss/nstext.1 Converted 408 records successfully from /.:/encina/sfs/hpss/nstext.1 lr_db_convert_nstext complete, submitted 408 records to DB2 nstext load for subsystem 1, 1233 operations per sec, 1.001456 total time Number Number Number Number Number Number of of of of of of rows rows rows rows rows rows read skipped loaded rejected deleted committed = = = = = = 408 0 408 0 0 408 Running lr_db_convert_storagesegdisk utility... Converting SFS file /.
HPSS Installation Guide Release 6.2 (Revision 2.
Appendix A. Glossary of Terms and Acronyms ACI Automatic Media Library Client Interface ACL Access Control List ACSLS Automated Cartridge System Library Software (Science Technology Corporation) ADIC Advanced Digital Information Corporation accounting The process of tracking system usage per user, possibly for the purposes of charging for that usage. Also, a log record message type used to log information to be used by the HPSS Accounting process. This message type is not currently used.
Bitfile Service Portion of the HPSS Core Server that provides a logical abstraction of bitfiles to its clients. BMUX Block Multiplexer Channel bytes between tape marks The number of data bytes that are written to a tape virtual volume before a tape mark is required on the physical media. CAP Cartridge Access Port cartridge A physical media container, such as a tape reel or cassette, capable of being mounted on and dismounted from a drive.
device A physical piece of hardware, usually associated with a drive, that is capable of reading or writing data. directory An HPSS object that can contain files, symbolic links, hard links, and other directories. dismount An operation in which a cartridge is either physically or logically removed from a device, rendering it unreadable and unwritable. In the case of tape cartridges, a dismount operation is a physical operation. In the case of a fixed disk unit, a dismount is a logical operation.
Gatekeeping Service A registered interface in the Gatekeeper that provides a site the mechanism to create local policy on how to throttle or deny create, open and stage requests and which of these request types to monitor. Gatekeeping Site Interface The APIs of the gatekeeping site policy code. Gatekeeping Site Policy The gatekeeping shared library code written by the site to monitor and throttle create, open, and/or stage requests.
I/O Input/Output IOD/IOR Input/Output Descriptor / Input/Output Reply. Structures used to send control information about data movement requests in HPSS and about the success or failure of the requests. IP Internet Protocol IRIX SGI’s implementation of UNIX junction A mount point for an HPSS fileset.
LS Location Server LTO Linear Tape-Open. A half-inch open tape technology developed by IBM, HP and Seagate. MAC Mandatory Access Control managed object A programming data structure that represents an HPSS system resource. The resource can be monitored and controlled by operations on the managed object. Managed objects in HPSS are used to represent servers, drives, storage media, jobs, and other resources.
notification A notice from one server to another about a noteworthy occurrence. HPSS notifications include notices sent from other servers to SSM of changes in managed object attributes, changes in tape mount information, and log messages that are alarm, event, and status log record message types.
RAM Random Access Memory reclaim The act of making empty tape virtual volumes available for reuse. Reclaimed tape virtual volumes are assigned a new Virtual Volume ID, but retain the rest of their previous characteristics. registration The process by which SSM requests notification of changes to specified attributes of a managed object.
SS HPSS Storage Service SSA Serial Storage Architecture SSM Storage System Management SSM session The environment in which an SSM user interacts with the SSM System Manager to monitor and control HPSS. This environment may be the graphical user interface provided by the hpssgui program, or the command line user interface provided by the hpssadm program. stage To copy file data from a level in the file’s hierarchy onto the top level in the hierarchy.
stripe length The number of bytes that must be written to span all the physical storage media (physical volumes) that are grouped together to form the logical storage media (virtual volume). The stripe length equals the virtual volume block size multiplied by the number of physical volumes in the stripe group (i.e., stripe width). stripe width The number of physical volumes grouped together to represent a virtual volume. System Manager The Storage System Management (SSM) server.
HPSS Installation Guide Release 6.2 (Revision 2.
Appendix B. References 4. 3580 Ultrium Tape Drive Setup, Operator and Service Guide GA32-0415-00 5. 3584 UltraScalable Tape Library Planning and Operator Guide GA32-0408-01 6. 3584 UltraScalable Tape Library SCSI Reference WB1108-00 7. AIX Performance Tuning Guide 8. Data Storage Management (XDSM) API, ISBN 1-85912-190-X 9. HACMP for AIX, Version 4.4: Concepts and Facilities 10. HACMP for AIX, Version 4.4: Planning Guide 11. HACMP for AIX, Version 4.4: Installation Guide 12. HACMP for AIX, Version 4.
Storage System (HPSS),” from the 1995 IEEE MSS Symposium, courtesy of the IEEE Computer Society Press. 33. T.W. Tyler and D.S. Fisher, “Using Distributed OLTP Technology in a High-Performance Storage System,” from the 1995 IEEE MSS Symposium, courtesy of the IEEE Computer Society Press. 34. J.K. Deutsch and M.R. Gary, “Physical Volume Library Deadlock Avoidance in a Striped Media Environment,” from the 1995 IEEE MSS Symposium, courtesy of the IEEE Computer Society Press. 35. R. Grossman, X. Qin, W. Xu, H.
Appendix C. Developer Acknowledgments HPSS is a product of a government-industry collaboration. The project approach is based on the premise that no single company, government laboratory, or research organization has the ability to confront all of the system-level issues that must be resolved for significant advancement in highperformance storage system technology.
Appendix D. HPSS.conf Configuration File The HPSS.conf configuration file contains tuning options to be used by HPSS clients and servers. For additional information, please see the HPSS.conf manual page. General HPSS.conf Rules/Suggestions: • Keywords MUST be specified precisely as shown (no extra spaces) • Lines are comprised of comments, blank lines, simple specifiers, specifiers and values - “abc = def”, or compound specifiers and terminators - “def = { …}.
Stanza (CMPD) PFTP Client = { E.g. PFTP Client = { Optional Reserved Stanza specifier. Must be terminated with a matching “}” SubStanza SYSLOG Facility = Values: DAEMON, LOCAL0 ... LOCAL7 E.g. SYSLOG Facility = LOCAL2 Optional SubStanza specifying the Syslog Facility for the MultiNode Daemon SubStanza Debug Value = Values: 0 – 3 E.g. Debug Value = 1 Optional SubStanza specifying the Level of Debugging for the Parallel FTP Client. Larger number provides more information.
SubStanza Auto Parallel Size = Value: Size - May be specified as a decimal number or “xMB” style notation. E.g. Auto Parallel Size = 4MB Optional SubStanza specifying the minimum file size to start using the “auto-parallel” features of the PFTP Client. SubStanza PortRange = Value: ncadg_ip_tcp[StartPort-EndPort] E.g. PortRange = ncadg_ip_tcp[10100-12100] Optional SubStanza specifying the TCP port range to use between the PFTP Client and the Mover(s).
SubStanza No SAN3P E.g. No SAN3P Optional SubStanza specifying not to use SAN3P even if it is available. Default is to use SAN3P if available. SubStanza No Transfer Agent Support E.g. No Transfer Agent Support Optional Substanza specifying to NOT use the Transfer Agent. Default is to use the Transfer Agent. SubStanza No 64-bit Support E.g. No 64-bit Support Optional Substanza specifying to NOT use the 64-bit protocol. Default is to use the 64-bit protocol. SubStanza Special Features Enabled E.g.
The pftp_client automatically performs conversion of get and put commands to their parallel equivalents, pget and pput. Some sites have reported degraded performance as a result of this substitution occurring with small file transfers. To accommodate this problem, the Auto Parallel Size = value substanza may be specified in the HPSS.conf file where the “automatic” parallel features will NOT be invoked if the size of the file is less than the value provided.
; Transfer Buffer Size = 1MB ; Socket Buffer Size = 16MB # PFTP sets an Artificial (Compiled in) Maximum of 250GB MAX Ptran Size = 10GB # Disable SAN3P ; No SAN3P # Disable Transfer Agent ; No Transfer Agent Supported # Disable 64-bit Protocol - Default is on ; No 64-bit Support # Special Features ; Special Features Enabled } D.2. PFTP Client Interfaces Stanza Many systems have multiple interfaces, some of which may not have access to all destination hosts.
SubSection or Name: Valid Interface Name Dotted IP Address: 132.175.1.1 E.g. eth0 132.175.1.1 Optional parameter containing the name or Dot Notation IP Address specification for the interface on the local host (PFTP Client) to use to connect to the Mover(s) associated with the specified PFTP Daemon. The PFTP Client Interfaces = { … } stanza contains several configuration options for the pftp_client executables.
PFTP Client Interfaces Stanza Example: ; PFTP Client Interfaces = { # PFTP Client Host Name(s) ; water.clearlake.ibm.com water = { # Next Specification is the PFTP Daemon Host # water has 3 specific interfaces that can talk # to the HPSS Movers associated with the PFTP # Daemon Host "water", as well as various # interfaces of the form 192.2.nnn.nnn ; water.clearlake.ibm.com water = { # Interfaces on the host specified as the Client Machine ; 192.94.47.227 ; 192.175.14.35 ; 192.222.197.
Configuration Type Description Stanza (CMPD) Multinode Table = { E.g. Multinode Table = { Optional Reserved Stanza specifier. Must be terminated with a matching “}” SubStanza Sleep for Debugger = values Value: Time in seconds. E.g. Sleep for Debugger = 15 Optional parameter to specify a delay in the Multinode Daemons to allow diagnosis. This should ONLY be specified for diagnostics and will unncessarily cause degradation if misused! Leave commented out. SubStanza (CMPD) E.g.
superdaemon (/etc/inetd.conf and /etc/services) to initiate the multinoded. The sections may be either a simple section or a valued section. A simple substanza is a single name/Dot Notation IP Address to be used for both “Control” connection and “Data” connection. The valued substanza is used to specify the name/Dot Notation IP Address for the “Control” connection (specifier) and the name/Dot Notation IP Address for the “Data” connection (value.
Destination IP address “masked” by the NetMask value. The calling application (PFTP Client, Client API, or Mover) will apply the value of the NetMask specification in the configuration file entry to the specified destination address. A “Default” destination may be specified for all sources/destinations not explicitly specified in the HPSS.conf file. Table 15. Network Options Stanza Fields Configuration Type Description Stanza (CMPD) Network Options = { E.g.
Section (CMPD) = { E.g. 100.101.102.0 = { Optional SubStanza specifying a dotted decimal address of the destination interface. Only one address is allowed; however, networks and sub-networks may be chosen by appropriate specification of the NetMask. May contain: “Default = {“ (Reserved Specification) for inclusion of entries not explicitly specified. Must be terminated with a matching “}” SubSection NetMask = Value: Viable netmask as IP Address E.g. NetMask = 255.255.255.
SubSection TcpNoDelay = Values: 0, 1 E.g. TcpNoDelay = 1 Optional parameter Indicates whether the TCP Delay option should be disabled (0) or enabled (any other value) SendSpace & RecvSpace Controls the size of the receive and send buffers for TCP/IP sockets. Internally, HPSS servers and clients attempt to set these buffers sizes explicitly, but other utilities may not. Typically, the RecvSpace and the SendSpace are equal; however, this is not mandated.
• Multiple Sections may be included in any SubStanza. A “Default” Destination Interface Name Section may be specified. NOTE: Do not include the quotes when specifying Default. • The NetMask must be specified in Decimal Dot IP Address Notation • All SubSections must be specified in every Section. Network Options Stanza Example: NOTE: Tuning is a “fine art” and may vary dramatically within any network configuration and may change with only very minor network configuration modifications.
} } D.5. Default = { NetMask = 255.255.255.0 RFC1323 = 1 SendSpace = 256KB RecvSpace = 128KB WriteSize = 512KB TCPNoDelay = 0 } PFTP Daemon Stanza A large number of options are available for configuring the PFTP daemon and tuning its performance. These options were previously specified in the ftpaccess file or via command line switches. These options have now been consolidated into the PFTP daemon stanza in the HPSS.conf file. The options are described below: Table 16.
SubStanza SYSLOG Facility = Value: DAEMON, LOCAL0 ... LOCAL7 E.g. SYSLOG Facility = LOCAL0 Replaces -s option. Optional SubStanza specifying the syslog facility for the HPSS PFTPD. The default syslog facility is LOG_DAEMON (reference:/usr/include/sys/syslog.h). Alternatives are LOCAL0 LOCAL7. Incorrect specification will default back to LOG_DAEMON. To make use of the alternates, modify /etc/syslog.conf to use the alternate facility. Note, the file specified in the /etc/syslog.
SubStanza FTP Access File = Value: filename E.g. FTP Access File = myftpaccess Replaces -F option. Optional SubStanza setting the {FTP_FtpAccessFile}. Default: ftpaccess. Located in the directory {FTPBaseDir}/etc. SubStanza Disable Slash Home Directory E.g. Disable Slash Home Directory Replaces -Z option. Optional SubStanza disabling use of / as the user’s home directory. Normally, this should be active for Security reasons. Substanza Disable Access if no Home Directory E.g.
SubStanza FTP Principal Keytab File = Value: Pathname/Filename E.g. FTP Principal Keytab File = /var/hpss/etc/hpss.keytabs Optional SubStanza specifying the keytab containing the FTP principal. SubStanza Allow Passive Connections E.g. Allow Passive Connections Replaces -A option. Optional SubStanza enabling passive connections. Note: Not supported in the HPSS parallel FTP daemon for parallel operations. Client requests to use passive mode will be rejected.
SubStanza PFTP IO Buffer Size = E.g. PFTP IO Buffer Size = 4MB Replaces -b option. Optional SubStanza setting the preferred IO Buffer Size for the PFTP Server SubStanza Debug Value = E.g. Debug Value = 3 Replaces -d option(s). Optional SubStanza specifying the level of debugging desired (1-3). Used internally to determine the quantity and detail of syslog messages from the PFTP Daemon. SubStanza Non-Parallel HostName = E.g. Non-Parallel HostName = aixrahe.sandia.
SubStanza Client API Verbose Value = E.g. Client API Verbose Value = 1 Replaces -v option(s). Optional SubStanza specifying the level of HPSS Client API Logging to use (1-3). The Client API will perform logging specified by the HPSS_DEBUG environment variable in a file specified by the HPSS_DEBUGPATH environment variable (default name is /var/hpss/ftp/adm/hpss_ftpd.log.) The default value is 1. (HPSS only.) SubStanza Disallow User Setting of COS E.g.
SubStanza Maximum Time Out = Value: Time in seconds E.g. Maximum Time Out = 86400 Replaces -T option and hpss_option MTO time in ftpaccess. Optional SubStanza specifying the maximum timeout in seconds. SubStanza Use UDP ONLY E.g. UDP ONLY Replaces -U option. Optional SubStanza specifying use of UDP RPCs Only. Sets the environment variable: RPC_SUPPORTED_PROTSEQS=ncadg_ip_udp (HPSS only.) SubStanza Use Extended Registry Attributes E.g. Use Extended Registry Attributes Replaces -X option.
SubStanza PortRange = E.g. PortRange = ncadg_ip_udp[10100-12100]:ncacn_ip_tcp[10100-12100] Optional SubStanza specifying the port range to be used for the non-HPSS parallel FTP daemon which is necessary for parallel transfers. NOTE: This is ignored for passive listings, etc. SubStanza Socket Buffer Size = Vlaues: Viable Socket Sizes E.g. Socket Buffer Size = 1MB Optional SubStanza specifying the socket buffer size. (non-HPSS PFTP daemon only.
SubSection COS = E.g. COS = 2 Optional SubSection specifying the Class of Service to be used based on file size. 0 means allow the Core Server to determine the optimal COS. (Has no meaning for the Non-HPSS PFTD daemon.) SubStanza Service Name = E.g. sunrahe Service Name = sunrahe.sandia.gov Optional SubStanza specifying the service name to be used by the PFTP daemon node when acquiring credentials.
SubStanza PFTP Group File = Value: Pathname/Filename E.g. PFTP Group File = /var/hpss/etc/pftp_groups Optional Substanza used to specify the file containing the group information for PFTP clients. Default is /var/hpss/etc/group. SubStanza Primary Authentication Mechanism = Values: KRB5, IBM_HPSS_PKEY, GSI, UNIX E.g. Primary Authentication Mechanism = KRB5 Optional Substanza used to specify the default Authentication mechanism. NOTE: IBM_HPSS_PKEY is NOT implemented.
SubSection Mapfile Specifier = Values: Pathname/filename E.g. Mapfile Specifier = /var/hpss/etc/MapfileName Optional Substanza used to specify a file containing username mappings. A different file can exist for each Authentication Type. This file provides the ability to Authenticate as one user and be authorized as another user (entity account). These files MUST be protected for security reasons. These files should be owned by root and readable/writeable ONLY by root.
sizes in the range [1MB, 2MB). PFTP Daemon Stanza Example: PFTP Daemon = { # Allow the Daemon to take Core Dumps ; Allow Core Files # Directory to put core files in (Default = .) ; Core File Directory = /var/hpss/adm/core # Specify the SYSLOG facility to use for all syslog messages # except Authentication Messages. # Values: DAEMON, LOCAL<0-7> # Replaces -sstring option.
) ; Default Time Out = 1500 # Specify (in octal) the Default umask ; Default Umask = 077 # Specification of the Level of HPSS Client API logging to use ( 0 - 3 ; Client API Verbose Value = 0 # Do NOT allow the user to specify Classes of Service (HPSS Only) ; Disallow User Setting of COS # Name the Kerberos Keytab file for "Kerberized" PFTP Daemon (HPSS Only) ; Keytab File Name = /etc/v5srvtab # Name of the Realm associated with the PFTP Daemon (HPSS Only) ; HPSS RealmName = realm.
; 100MB = { ; BlockSize = 4MB ; StripeWidth= 0 ; COS = 0 ; } ; 1GB = { ; BlockSize = 8MB ; StripeWidth= 0 ; COS = 0 ; } ; } # Use the System Password file routines (TRUE or FALSE) # The Default for PFTP is FALSE (Case Sensitive!) Use System Password Files = FALSE # Path and Name for the PFTP Password File PFTP Password File = /var/hpss/etc/passwd # # # # # # # ; Path and Name for the PFTP Shadow Password File NOTE: PFTP does not currently use the value.
} # # # # # # # # # used } ; GSI = { ; Mapfile Specifier = LDAP:/var/hpss/etc/KRB2UnixMapfile ; Use Site Auth Method = CryptoCard ; Default Authorization Mechanism = LDAP ; } ; IDENT = { ; Mapfile Specifier = FILE:/var/hpss/etc/IDENT2UnixMapfile ; Use Site Auth Method = SecurId ; Default Authorization Mechanism = DBAS ; } USER_PASS = { ; Use Site Auth Method = KRB5KDC ; Mapfile Specifier = FILE:/var/hpss/etc/Unix2UnixMapfile Default Authorization Mechanism = UNIX } Keytab Hostname Mapping Section Syntax:
SubStanza Nodeset File = Value: Pathname/filename E.g. Nodeset File = /var/hpss/etc/PMTA_NodeSets Optional Substanzaused to specify the location of the Transfer Agent file containing Nodesets. SubStanza Node Affinity File = Value: Pathname/filename E.g. Node Affinity File = /var/hpss/etc/PMTA_NodeAffinity Optional Substanzaused to specify the location of the Transfer Agent file containing Node Affinities. SubStanza Shared FS File = Value: Pathname/filename E.g.
SubStanza Debug Logfile = Value: Pathname/filename E.g. Debug Logfile = /var/hpss/log/PMTA_Debugfile Optional Substanzaused to specify the location of the Transfer Agent Debugging File. Substanza SYSLOG Facility = Values: DAEMON, LOCAL0 ... LOCAL7, NONE E.g. SYSLOG Facility = LOCAL0 Optional SubStanza specifying the syslog facility for the HPSS PFTPD. The default syslog facility is LOG_DAEMON (specified by DAEMON reference: /usr/include/sys/syslog.h).
# Parallel Multinode Transfer Agent (PMTA) Section Transfer Agent = { # The (optional) NodeSet File contains named sets of # Nodes that can be referred to via the "SET:setname" # notation. Nodeset File = /var/hpss/etc/ta/nodeset.conf # The (optional) Node Affinity file is used to specify # groups of nodes are able to communicate in a # network whose topology does not support full interconnection Node Affinity File = /var/hpss/etc/ta/node_affinity.
D.7. Stanzas Reserved for Future Use The following stanza names (specifiers) are reserved for future implementation in HPSS and should not be used by application developers. • Transfer Agent (Partially Implemented) • Pipe File • Local File Path • PSI HPSS Installation Guide Release 6.2 (Revision 2.
HPSS Installation Guide Release 6.2 (Revision 2.
Appendix E. hpss_env_defs.h The HPSS environment variables are defined in /opt/hpss/include/hpss_env_defs.h. These environment variables can be overridden in /var/hpss/etc/env.conf or in the local environment.
{ "HPSS_GRP_NAME_CLIENT", "hpss_client", NULL}, /* ************************************************************************** * * HPSS Server Principal names * * HPSS_PRINCIPAL - Principal name for SEC Server * HPSS_PRINCIPAL_CORE - Principal name for CORE Server * HPSS_PRINCIPAL_DMG - Principal name for DMAP Gateway * HPSS_PRINCIPAL_FTPD - Principal name for FTP Daemon * HPSS_PRINCIPAL_GK - Principal name for Gatekeeper Server * HPSS_PRINCIPAL_HDM - Principal name for HDM * HPSS_PRINCIPAL_HPSSD - Principal
* HPSS_PRINCIPAL_MPS_UID - Principal UID for Migration/Purge Server * HPSS_PRINCIPAL_MVR_UID - Principal UID for Mover * HPSS_PRINCIPAL_NFSD_UID - Principal UID for NFS Daemon * HPSS_PRINCIPAL_NS_UID - Principal UID for Name Server * HPSS_PRINCIPAL_PFSD_UID - Principal UID for PFS Daemon * HPSS_PRINCIPAL_PVL_UID - Principal UID for PVL * HPSS_PRINCIPAL_PVR_UID - Principal UID for PVR * HPSS_PRINCIPAL_SS_UID - Principal UID for Storage Server * HPSS_PRINCIPAL_SSM_UID - Principal UID for SSM * * NOTE: Princip
* HPSS_EXEC_PVR_SCSI - executable name for PVR SCSI * HPSS_EXEC_SSMSM - executable name for SSM Storage Manager * ************************************************************************** * */ { "HPSS_EXEC_ACCT", "${HPSS_PATH_BIN}/hpss_acct", NULL}, { "HPSS_EXEC_CORE", "${HPSS_PATH_BIN}/hpss_core", NULL}, { "HPSS_EXEC_DMG", "${HPSS_PATH_BIN}/hpss_dmg", NULL}, { "HPSS_EXEC_FTPD", "${HPSS_PATH_BIN}/hpss_pftpd", NULL}, { "HPSS_EXEC_GK", "${HPSS_PATH_BIN}/hpss_gk", NULL}, { "HPSS_EXEC_HPSSD", "${HPSS_PATH_B
* HPSS_EXEC_DELOG - Pathname for the delog utility * HPSS_EXEC_RECLAIM - Pathname for the reclaim utility * HPSS_EXEC_REPACK - Pathname for the repack utility * ************************************************************************** * */ { "HPSS_EXEC_ACL_EDIT", NULL}, { "HPSS_EXEC_CDSCP", NULL}, { "HPSS_EXEC_DELOG", NULL}, { "HPSS_EXEC_RECLAIM", NULL}, { "HPSS_EXEC_REPACK", NULL}, "${HPSS_PATH_SLASH_BIN}/acl_edit", "${HPSS_PATH_SLASH_BIN}/cdscp", "${HPSS_PATH_BIN}/hpss_delog", "${HPSS_PATH_BIN}/recla
* HPSS_UNIX_NFS_CHECKPOINT - checkpoint * HPSS_NFS_DISABLE_JUNCTIONS - disable traversal of junctions * HPSS_MNT_DISABLE_JUNCTIONS - disable mounting junctions ************************************************************************** */ { "HPSS_PATH_NFS", "${HPSS_PATH_VAR}/nfs", NULL}, { "HPSS_UNIX_NFS_EXPORTS", "${HPSS_PATH_NFS}/exports", NULL}, { "HPSS_UNIX_NFS_CREDMAP", "${HPSS_PATH_NFS}/credmap.nfs", NULL}, { "HPSS_UNIX_NFS_CACHEFILE", "${HPSS_PATH_NFS}/cachefile.
{ { { { { { { "HPSS_SUBSYS_DB_NAME", "HPSS_MM_SCHEMA_NAME", "HPSS_MM_LOW_WATER_MARK", "HPSS_MM_HIGH_WATER_MARK", "HPSS_MM_CACHE", "HPSS_SERVER_DB_GROUP", "HPSS_SERVER_DB_KEYTAB", NULL}, "subsys", NULL}, "HPSS", NULL}, "1", NULL}, "5", NULL}, "", NULL}, "HPSSSRVR", NULL}, "${HPSS_PATH_ETC}/mm.
*/ { "HPSS_HOST_TMP", { "HPSS_DESC_CORE", { "HPSS_DESC_DMG", { "HPSS_DESC_FTPD", { "HPSS_DESC_GK", { "HPSS_DESC_HPSSD", {HPSS_HOST_TMP})", NULL}, { "HPSS_DESC_HPSSD_SHORT", { "HPSS_DESC_LOGC", NULL}, { "HPSS_DESC_LOGC_SHORT", { "HPSS_DESC_LOGD", { "HPSS_DESC_LS", { "HPSS_DESC_MM", { "HPSS_DESC_MOUNTD", { "HPSS_DESC_MPS", NULL}, { "HPSS_DESC_MVR", NULL}, { "HPSS_DESC_MVR_SHORT", { "HPSS_DESC_NFSD", { "HPSS_DESC_PVL", { "HPSS_DESC_PVR_AMPEX", { "HPSS_DESC_PVR_OPER", { "HPSS_DESC_PVR_STK", { "HPSS_DESC_PVR_ST
updates * HPSSSSM_UI_MO_RATE - Time the GUI will wait between MO update tries * HPSSSSM_UI_LIST_RATE - Time the GUI will wait between List update tries * HPSSSSM_UI_ALARM_RATE - Time the GUI will wait between Alarm update tries * HPSSGUI_SEC_MECH - security mechanism used for SM communication * HPSSADM_SEC_MECH - security mechanism used for SM communication * HPSSGUI_USER_CFG_PATH - Directroy which holds GUI config files * HPSSADM_USER_CFG_PATH - Directory which holds ADM config files * * HPSS_SSMUSER_JAVA_
* to maintain at once.
* HPSS_API_MAX_CONN - Defines the number of connections that * are supported by the Client API within * a single client process * HPSS_API_DEBUG - Used to enable debug messages * HPSS_API_RETRIES - Used to control the number of retries * when operations fail with a "retryable" * return code * HPSS_API_BUSY_DELAY - Used to control the number of seconds * to delay between retry attempts.
{ { { { { { { { { { "HPSS_API_TOTAL_DELAY", "0", NULL}, "HPSS_API_LIMITED_RETRIES", "1", NULL}, "HPSS_API_DMAP_WRITE_UPDATES","20", NULL}, "HPSS_API_REUSE_CONNECTIONS", "0", NULL}, "HPSS_API_USE_PORT_RANGE", "0", NULL}, "HPSS_API_RETRY_STAGE_INP", "1", NULL}, "HPSS_API_DISABLE_CROSS_REALM","0", NULL}, "HPSS_API_DISABLE_JUNCTIONS", "0", NULL}, "HPSS_API_SITE_NAME", "${HPSS_SITE_NAME}", NULL}, "HPSS_API_AUTHN_MECH", "${HPSS_CLIENT_AUTHN_MECH}", NULL}, { "HPSS_API_RPC_PROT_LEVEL", "${HPSS_RPC_PROT_LEVEL}", NU
* [ stdout | syslog ] * HPSS_CONFIG_CAT_NAME - HPSS config message catalog full pathname * HPSS_INFRA_LOG_TYPES - Default types of infrastructure messages * to be logged * HPSS_INFRA_LOG_CONF - The infrastructure logging configuration file * HPSS_MASTER_CAT_NAME - HPSS master message catalog full pathname * ************************************************************************** * */ { "HPSSLOG_SHMKEY", "2801", NULL}, { "HPSSLOG_SHMID", NULL, NULL}, { "HPSSLOG_HOSTNAME", "${HPSS_HOST}", NULL}, { "HPSSLOGG
* HPSS_AUTHZ_TYPES - Supported authorization types * HPSS_SITE_LOCATION - Site Location * KRB5_INSTALL_PATH - Kerberos installation path * no default - platform dependent * KRB5_KDC_DIR - Kerberos directory containing local config * files for KDC * KRB5_KDC_HOST - Host for Kerberos KDC (just used by mkhpss) * HPSS_LDAP_URL * If set and non-empty, specifies the URL of the LDAP server that * the hpss ldap admin tool should connect to by default.
NULL}, { "HPSS_PRIMARY_AUTHENTICATOR", "${HPSS_KRB5_KEYTAB_FILE}", NULL}, { "HPSS_CLIENT_AUTHN_MECH", "${HPSS_PRIMARY_AUTHN_MECH}", NULL}, { "IEEE_802_FILE", "${HPSS_PATH_ETC}/ieee_802_addr", NULL}, { "HPSS_AUTH_SERVICE_CONF", "${HPSS_PATH_ETC}/auth.conf", NULL}, { "HPSS_AUTHZ_SERVICE_CONF", "${HPSS_PATH_ETC}/authz.conf", NULL}, { "HPSS_SEC_EP_CONF", "${HPSS_PATH_ETC}/ep.conf", NULL}, { "HPSS_SEC_SITE_CONF", "${HPSS_PATH_ETC}/site.conf", NULL}, { "KRB5_CONF", "${HPSS_PATH_SLASH_ETC}/krb5.
* HPSS_PATH_CORE - Path where subsystem core files are placed * HPSS_PATH_TMP - Path where temporary files are placed * HPSS_PATH_ETC - Path where runtime config files are placed * HPSS_ENV_CONF - The path to the environment override file * HPSS_PTHREAD_STACK - Stack size for HPSS pthreads ************************************************************************** * */ { "HPSS_PATH_ADM", "${HPSS_PATH_VAR}/adm", NULL}, { "HPSS_PATH_CORE", "${HPSS_PATH_ADM}/core",NULL}, { "HPSS_PATH_TMP", "${HPSS_PATH_VAR}/tmp
Appendix F. /var/hpss files The /var/hpss directory tree is the default location of a number of HPSS configuration files, log files, and other files needed by the servers. The directories and configuration files can be created with the mkhpss utility or hand created. Be very careful when using mkhpss utility as selecting the wrong option can damage the already partially configured HPSS 6.2 system. The log files and other files are created by the servers. The directories in /var/hpss include: acct.
authorization mechanism and authorization URL to utilize. • rc.db2. Created by mkhpss. Script for starting DB2. • rc.krb. Created by mkhpss. Script for starting the Kerberos servers. • passwd. Created by mkhpss. A local HPSS-only password file for use with Unix authentication and authorization. Optionally, the system password file can be used instead. • group. Created by mkhpss. A local HPSS-only group file for use with Unix authentication and authorization.
of the HPSS servers it brought up in the node. HPSS may also write diagnostic log files and disk allocation maps in this directory, when configured to do so. The lock files are very small, but the log files and maps can be very large. HPSS Installation Guide Release 6.2 (Revision 2.