VERITAS File System 4.1 (HP OnlineJFS/JFS) Administrator’s Guide HP-UX 11i v2 Fifth Edition Manufacturing Part Number: 5991-1833 September 2005 Printed in the United States © Copyright 2005-2006 Hewlett-Packard Development Company L.P.
Legal Notices Confidential computer software. Valid license from HP required for possession, use or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation and Technical Data for Commercial Items are licensed to the U.S. Government under vendor’s standard commercial license. The information contained herein is subject to change without notice.
Contents 1. The VERITAS File System Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . VxFS Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Disk Layouts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . File System Performance Enhancements . . . . . . . . . . . . . . . . . . . . . . . . .
Contents Quality of Storage Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 2. VxFS Performance: Creating, Mounting, and Tuning File Systems Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Choosing mkfs Command Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Block Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Contents Other Controls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 Commands Related to Extent Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 Failure to Preserve Extent Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 4. Application Interface Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Contents Space Management Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . File System Restore From Storage Checkpoints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Example of Restoring a File From a Storage Checkpoint. . . . . . . . . . . . . . . . . . . . . Example of Restoring a File System From a Storage Checkpoint . . . . . . . . . . . . . . Storage Checkpoint Quotas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Contents Reverse Path Name Lookup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 9. Multi-Volume File Systems Features Implemented Using MVS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Volume Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Creating MVS File Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Contents Enabling and Disabling Quick I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cached Quick I/O For Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Enabling Cached Quick I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tuning Cached Quick I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Quick I/O Statistics . . . . . . . . . . . . .
Preface The VERITAS File System Administrator’s Guide provides information on the most important aspects of VERITAS File System (VxFS) administration.
Typographic Conventions Table 1 describes the typographic conventions used in this document. Table 1 Typographic Conventions Typeface monospace Usage Computer output, files, directories, software elements such as command options, function names, and parameters Examples Read tunables from the /etc/vx/tunefstab file. See the ls (1) manpage for more information. italic New terms, book titles, emphasis, variables replaced with a name or value See the VERITAS 4.1 Installation Guide for details.
• VERITAS Volume Manager 4.1 Hardware Notes • VERITAS Volume Manager 4.1 Release Notes • VERITAS Volume Manager 4.1 Troubleshooting Guide • VERITAS Volume Manager 4.1 Migration Guide • VERITAS Volume Manager 4.1 Administrator’s Guide • VERITAS Enterprise Administrator (VEA 500 Series) Getting Started • VERITAS Storage Foundation 4.
1 The VERITAS File System Introduction VxFS is an extent based, intent logging file system. VxFS is designed for use in UNIX environments that require high performance and availability and deal with large volumes of data. This chapter provides an overview of major VxFS features that are described in detail in later chapters.
The VERITAS File System VxFS Features VxFS Features Basic features include: • Extent based allocations • Extent attributes • Fast file system recovery • Access control lists (ACLs) • Online administration • Online backup • Enhanced application interface • Enhanced mount options • Improved synchronous write performance • Support for file systems up to 32 terabytes in size • Support for files up to 2 terabytes in size • Enhanced I/O performance • Quotas • Cluster file systems • I
The VERITAS File System Disk Layouts Disk Layouts The disk layout is the way file system information is stored on disk. On VxFS, five disk layout versions, numbered 1 through 5, were created to support various new features and specific UNIX environments. Currently, only Version 4 or Version 5 disk layouts can be created, but file systems with Version 2 and Version 3 disk layouts can be mounted. See “Disk Layout” on page 251 for a description of the disk layouts.
The VERITAS File System File System Performance Enhancements The rest of this chapter, as well as “VxFS Performance: Creating, Mounting, and Tuning File Systems” on page 37 and “Application Interface” on page 75 provide details on many of these features.
The VERITAS File System VERITAS Enterprise Administrator Graphical User Interface VERITAS Enterprise Administrator Graphical User Interface The VERITAS Enterprise AdministratorTM (VEA) is a Java-based GUI that consists of a server and a client. The server runs on a UNIX system that is running the VERITAS Volume Manager and VxFS. The client runs on any platform that supports the Java Runtime Environment. You can use VEA to perform a subset of VxFS administrative functions on a local or remote system.
The VERITAS File System Extent Based Allocation Extent Based Allocation Disk space is allocated in 1024-byte sectors to form logical blocks. VxFS supports logical block sizes of 1024, 2048, 4096, and 8192 bytes. The default block size is IK. An extent is defined as one or more adjacent blocks of data within the file system. An extent is presented as an address-length pair, which identifies the starting block address and the length of the extent (in file system or logical blocks).
The VERITAS File System Extent Based Allocation Typed Extents VxFS has an inode block map organization for indirect extents known as typed extents. Each entry in the block map has a typed descriptor record containing a type, offset, starting block, and number of blocks. Indirect and data extents use this format to identify logical file offsets and physical disk locations of any given extent.
The VERITAS File System Extent Attributes Extent Attributes VxFS allocates disk space to files in groups of one or more extents. VxFS also allows applications to control some aspects of the extent allocation. Extent attributes are the extent allocation policies associated with a file. The setext and getext commands allow the administrator to set or view extent attributes associated with a file, as well as to preallocate space for a file.
The VERITAS File System Fast File System Recovery Intent Log Resizing The VxFS intent log is allocated when the file system is first created. The size of the intent log is based on the size of the file system-the larger the file system, the larger the intent log. The maximum default intent log size of disk layout Versions 4, 5 and 6 is 16 megabytes. With the Version 6 disk layout, you can dynamically increase or decrease the intent log size using the log option of the fsadm command.
The VERITAS File System Online System Administration Online System Administration A VxFS file system can be defragmented and resized while it remains online and accessible to users. The following sections provide an overview of these features. Defragmentation Free resources are initially aligned and allocated to files in the most efficient order possible to provide optimal performance.
The VERITAS File System Application Interface Application Interface VxFS conforms to the System V Interface Definition (SVID) requirements and supports user access through the Network File System (NFS). Applications that require performance features not available with other file systems can take advantage of VxFS enhancements that are introduced in this section and covered in detail in “Application Interface” on page 75.
The VERITAS File System Extended mount Options Extended mount Options The VxFS file system supports extended mount options to specify: • Enhanced data integrity modes • Enhanced performance modes • Temporary file system modes • Improved synchronous writes • Large file sizes See “VxFS Performance: Creating, Mounting, and Tuning File Systems” on page 37 and the mount_vxfs (1M) manual page for details on the VxFS mount options.
The VERITAS File System Extended mount Options Using the log Option for Data Integrity File systems are typically asynchronous in that structural changes to the file system are not immediately written to disk, which provides better performance. However, recent changes made to a system can be lost if a system failure occurs. Specifically, attribute changes to files and recently created files may disappear.
The VERITAS File System Extended mount Options The mount convosync=dsync option improves the performance of applications that require synchronous data writes but not synchronous inode time updates. CAUTION The use of the convosync=dsyncoption violates POSIX semantics. Support for Large Files VxFS can support files up to two terabytes in size. See “largefiles | nolargefiles” on page 47 for information on how to create, mount, and manage file systems containing large files.
The VERITAS File System Enhanced I/O Performance Enhanced I/O Performance VxFS provides enhanced I/O performance by applying an aggressive I/O clustering policy, integrating with VxVM, and allowing application specific parameters to be set on a per-file system basis. Enhanced I/O Clustering I/O clustering is a technique of grouping multiple I/O operations together for improved performance.
The VERITAS File System Access Control Lists Access Control Lists An Access Control List (ACL) stores a series of entries that identify specific users or groups and their access privileges for a directory or file. A file may have its own ACL or may share an ACL with other files. ACLs have the advantage of specifying detailed access permissions for multiple users and groups. Refer to the getacl (1) and setacl (1) manual pages for information on viewing and setting ACLs.
The VERITAS File System Storage Checkpoints Storage Checkpoints To increase availability, recoverability, and performance, the VERITAS File System offers on-disk and online backup and restore capabilities that facilitate frequent and efficient backup strategies. Backup and restore applications can leverage the VERITAS Storage Checkpoint, a disk- and I/O-efficient copying technology for creating periodic frozen images of a file system.
The VERITAS File System Quotas Quotas VxFS supports quotas, which allocate per-user quotas and limit the use of two principal resources: files and data blocks. You can assign quotas for each of these resources. Each quota consists of two limits for each resource: • The hard limit represents an absolute limit on data blocks or files. A user can never exceed the hard limit under any circumstances. • The soft limit is lower than the hard limit and can be exceeded for a limited amount of time.
The VERITAS File System Cluster File Systems Cluster File Systems Clustered file systems are an extension of VxFS that support concurrent direct media access from multiple systems. CFS employs a master/slave protocol. All cluster file systems can read file data directly from a shared disk. In addition, all systems can write "in-place" file data. Operations that require changes to file system metadata, such as allocation, creation, and deletion, can only be performed by the single primary file system node.
The VERITAS File System Cross-Platform Data Sharing Cross-Platform Data Sharing Cross-platform data sharing allows data to be serially shared among heterogeneous systems where each system has direct access to the physical devices that hold the data.This feature can be used only in conjunction with VERITAS Volume Manager. See the VERITAS Storage Foundation Cross-Platform Data Sharing Administrator’s Guide for more information.
The VERITAS File System File Change Log File Change Log The VxFS File Change Log (FCL) tracks changes to files and directories in a file system.The File Change Log can be used by applications such as backup products, webcrawlers, search and indexing engines, and replication software that typically scan an entire file system searching for modifications since a previous scan. FCL functionality is a separately licensable feature. See “File Change Log” on page 33" for more information.
The VERITAS File System Multi-Volume Support Multi-Volume Support The multi-volume support (MVS) feature allows several volumes to be represented by a single logical object. All I/O to and from an underlying logical volume is directed by way of volume sets. This feature can be used only in conjunction with VERITAS Volume Manager. MVS functionality is a separately licensable feature. See Chapter 9, “Multi-Volume File Systems,” on page 139 for more information.
The VERITAS File System Quality of Storage Service Quality of Storage Service The Quality of Storage Service (QoSS) option is built on the multi-volume support technology introduced in this release. Using QoSS, you can map more than one volume to a single file system. You can then configure policies that automatically relocate files from one volume to another, or relocate files by running file relocation commands.
The VERITAS File System Quality of Storage Service 36 Chapter 1
2 VxFS Performance: Creating, Mounting, and Tuning File Systems Introduction For any file system, the ability to provide peak performance is important. Adjusting the available VERITAS File System (VxFS) options provides a way to increase system performance. This chapter describes the commands and practices you can use to optimize VxFS. For information on optimizing an application for use with VxFS, see “Application Interface” on page 75.
VxFS Performance: Creating, Mounting, and Tuning File Systems Introduction — Number of Links to a File — VxFS Inode Free Time Lag — VxVM Maximum I/O Size • Monitoring Free Space — Monitoring Fragmentation • I/O Tuning — Tuning VxFS I/O Parameters — Tunable VxFS I/O Parameters 38 Chapter 2
VxFS Performance: Creating, Mounting, and Tuning File Systems Choosing mkfs Command Options Choosing mkfs Command Options There are several characteristics that you can select when you create a file system. The most important options pertaining to system performance are the block size and intent log size. Block Size The unit of allocation in VxFS is a block.
VxFS Performance: Creating, Mounting, and Tuning File Systems Choosing mount Command Options Choosing mount Command Options In addition to the standard mount mode(delaylog mode), VxFS provides blkclear, log, tmplog, and nodatainlog modes of operation. Caching behavior can be altered with the mincache option, and the behavior of O_SYNC and D_SYNC (see the fcntl (2) manual page) writes can be altered with the convosync option. The delaylog and tmplog modes can significantly improve performance.
VxFS Performance: Creating, Mounting, and Tuning File Systems Choosing mount Command Options log is written. If a system failure occurs, recent changes may be lost. This mode approximates traditional UNIX guarantees for correctness in case of system failures. Fast file system recovery works with this mode. The rename(2) system call flushes the source file to disk to guarantee the persistence of the file data before renaming it.
VxFS Performance: Creating, Mounting, and Tuning File Systems Choosing mount Command Options logiosize The logiosize=size option is provided to enhance the performance of storage devices that employ a read-modify-write feature. If you specify logiosize when you mount a file system, VxFS writes the intent log in at least size bytes to obtain the maximum performance from such devices. The values for size can be 1024, 2048, or 4096.
VxFS Performance: Creating, Mounting, and Tuning File Systems Choosing mount Command Options nodatainlog Use the nodatainlog mode on systems with disks that do not support bad block revectoring. Usually, a VxFS file system uses the intent log for synchronous writes. The inode update and the data are both logged in the transaction, so a synchronous write only requires one disk write instead of two.
VxFS Performance: Creating, Mounting, and Tuning File Systems Choosing mount Command Options switched off, only files that are currently open can lose data. A mincache=closesync mode file system should be approximately 15 percent slower than a standard mode VxFS file system, depending on the workload.
VxFS Performance: Creating, Mounting, and Tuning File Systems Choosing mount Command Options convosync NOTE Use of the convosync=dsync option violates POSIX guarantees for synchronous I/O. The convosync (convert osync) mode has five suboptions: • convosync=closesync • convosync=delay.
VxFS Performance: Creating, Mounting, and Tuning File Systems Choosing mount Command Options CAUTION 46 Before using convosync=dsync, convosync=unbuffered, or convosync=direct, make sure that all applications that use the file system do not require synchronous inode time updates for O_SYNC writes.
VxFS Performance: Creating, Mounting, and Tuning File Systems Choosing mount Command Options ioerror Sets the policy for handling I/O errors on a mounted file system. I/O errors can occur while reading or writing file data, or while reading or writing metadata. The file system can respond to these I/O errors either by halting or by gradually degrading. The ioerror option provides four policies that determine how the file system responds to the various errors.
VxFS Performance: Creating, Mounting, and Tuning File Systems Choosing mount Command Options NOTE Applications and utilities such as backup may experience problems if they are not aware of large files. In such a case, create your file system without large file capability.
VxFS Performance: Creating, Mounting, and Tuning File Systems Choosing mount Command Options You cannot change a file system to nolargefiles if it holds large files. See the mount_vxfs (1M), fsadm_vxfs (1M), and (1M) manual pages.
VxFS Performance: Creating, Mounting, and Tuning File Systems Choosing mount Command Options Combining mount Command Options Although mount options can be combined arbitrarily, some combinations do not make sense. The following examples provide some common and reasonable mount option combinations. Example 1 - Desktop File System # mount -F vxfs -o log,mincache=closesync /dev/dsk/c1t3d0 /mnt This guarantees that when a file is closed, its data is synchronized to disk and cannot be lost.
VxFS Performance: Creating, Mounting, and Tuning File Systems Kernel Tunables Kernel Tunables This section describes the kernel tunable parameters in VxFS. Internal Inode Table Size VxFS caches inodes in an inode table. There is a dynamic tunable in VxFS called vx_ninode that determines the number of entries in the inode table. A VxFS file system obtains the value of vx_ninode from the system configuration file used for making the HP-UX kernel (/stand/system for example).
VxFS Performance: Creating, Mounting, and Tuning File Systems Kernel Tunables Changing the value of a tunable does not resize the internal hash tables and structures of the caches. These sizes are determined at system boot up based on either the system memory size, which is the default, or the value of the tunable if explicitly set, whichever is larger.
VxFS Performance: Creating, Mounting, and Tuning File Systems Kernel Tunables The initial value of vx_bc_bufhwm is zero. When the operating system reboots, VxFS sets the value of vx_bc_bufhwm based on the amount of system memory. You can explicitly reset the value of vx_bc_bufhwm by changing the value of vxfs_bc_bufhwm using the sam or kctune commands (see the sam(1M) and kctune(1M) manual pages). You can also set the value by adding an entry to the system configuration file.
VxFS Performance: Creating, Mounting, and Tuning File Systems Kernel Tunables The vxfs_ifree_timelag tunable specifies the minimum amount of time an unused inode spends on a freelist before its memory space is freed.vxfs_ifree_timelag is a dynamic tunable. Any changes to vxfs_ifree_timelag take affect immediately. The default value of vxfs_ifree_timelag is 0. By setting vxfs_ifree_timelag to 0, the inode free timelag is autotuned to 1800 seconds.
VxFS Performance: Creating, Mounting, and Tuning File Systems Monitoring Free Space VxVM Maximum I/O Size When using VxFS with the VERITAS Volume Manager (VxVM), VxVM by default breaks up I/O requests larger than 256K. When using striping, to optimize performance, the file system issues I/O requests that are up to a full stripe in size. If the stripe size is larger than 256K, those requests are broken up.
VxFS Performance: Creating, Mounting, and Tuning File Systems Monitoring Free Space • Less than 5 percent of free space in extents of less than 64 blocks in length • More than 5 percent of the total file system size available as free extents in lengths of 64 or more blocks A badly fragmented file system will have one or more of the following characteristics: • Greater than 5 percent of free space in extents of less than 8 blocks in length • More than 50 percent of free space in extents of less than 6
VxFS Performance: Creating, Mounting, and Tuning File Systems I/O Tuning I/O Tuning NOTE The tunables and the techniques described in this section work on a per file system basis. Use them judiciously based on the underlying device properties and characteristics of the applications that use the file system. Performance of a file system can be enhanced by a suitable choice of I/O sizes and proper alignment of the I/O requests based on the requirements of the underlying special device.
VxFS Performance: Creating, Mounting, and Tuning File Systems I/O Tuning /dev/vx/dsk/userdg/metasave read_pref_io=128k,write_pref_io=128k,read_nstream=4,write_nstream=4 /dev/vx/dsk/userdg/solbuild read_pref_io=64k,write_pref_io=64k,read_nstream=4,write_nstream=4 /dev/vx/dsk/userdg/solrelease read_pref_io=64k,write_pref_io=64k,read_nstream=4,write_nstream=4 /dev/vx/dsk/userdg/solpatch read_pref_io=128k,write_pref_io=128k,read_nstream=4,write_nstream=4 58 Chapter 2
VxFS Performance: Creating, Mounting, and Tuning File Systems I/O Tuning Tunable VxFS I/O Parameters read_pref_io The preferred read request size. The file system uses this in conjunction with the read_nstream value to determine how much data to read ahead. The default value is 64K. write_pref_io The preferred write request size. The file system uses this in conjunction with the write_nstream value to determine how to do flush behind on writes. The default value is 64K.
VxFS Performance: Creating, Mounting, and Tuning File Systems I/O Tuning default_indir_ size On VxFS, files can have up to ten direct extents of variable size stored in the inode. Once these extents are used up, the file must use indirect extents which are a fixed size that is set when the file first uses indirect extents. These indirect extents are 8K by default.
VxFS Performance: Creating, Mounting, and Tuning File Systems I/O Tuning fcl_maxalloc Specifies the maximum amount of space that can be allocated to the VxFS file change log (FCL). The FCL file is a sparse file that grows as changes occur in the file system. When the space allocated to the FCL file reaches the fcl_maxalloc value, the oldest FCL records are purged from the FCL and the extents nearest to the beginning of the FCL file are freed. This process is referred to as "punching a hole.
VxFS Performance: Creating, Mounting, and Tuning File Systems I/O Tuning hsm_write_ prealloc For a file managed by a hierarchical storage management (HSM) application, hsm_write_prealloc preallocates disk blocks before data is migrated back into the file system. An HSM application usually migrates the data back through a series of writes to the file, each of which allocates a few blocks.
VxFS Performance: Creating, Mounting, and Tuning File Systems I/O Tuning inode_aging_count Specifies the maximum number of inodes to place on an inode aging list. Inode aging is used in conjunction with file system Storage Checkpoints to allow quick restoration of large, recently deleted files. The aging list is maintained in first-in-first-out (fifo) order up to maximum number of inodes specified by inode_aging_count.
VxFS Performance: Creating, Mounting, and Tuning File Systems I/O Tuning max_seqio_extent_ size Increases or decreases the maximum size of an extent. When the file system is following its default allocation policy for sequential writes to a file, it allocates an initial extent which is large enough for the first write to the file.
VxFS Performance: Creating, Mounting, and Tuning File Systems I/O Tuning read_ahead The default for all VxFS read operations is to perform sequential read ahead. You can specify the read_ahead cache advisory to implement the VxFS enhanced read ahead functionality. This allows read aheads to detect more elaborate patterns (such as increasing or decreasing read offsets or multithreaded file accesses) in addition to simple sequential reads.
VxFS Performance: Creating, Mounting, and Tuning File Systems I/O Tuning write_throttle The write_throttle parameter is useful in special situations where a computer system has a combination of a large amount of memory and slow storage devices. In this configuration, sync operations (such as fsync()) may take long enough to complete that a system appears to hang.
VxFS Performance: Creating, Mounting, and Tuning File Systems I/O Tuning number of columns in the stripe. For striped arrays, use the same values for write_pref_io and write_nstream, but for RAID-5 arrays, set write_pref_io to the full stripe size and write_nstream to 1. For an application to do efficient disk I/O, it should issue read requests that are equal to the product of read_nstream multiplied by read_pref_io.
VxFS Performance: Creating, Mounting, and Tuning File Systems I/O Tuning 68 Chapter 2
3 Extent Attributes Introduction The VERITAS File System (VxFS) allocates disk space to files in groups of one or more adjacent blocks called extents. VxFS defines an application interface that allows programs to control various aspects of the extent allocation for a given file (see “Extent Information” on page 80). The extent allocation policies associated with a file are referred to as extent attributes. The VxFS getext(1M) and setext(1M) commands let you view or manipulate file extent attributes.
Extent Attributes Attribute Specifics Attribute Specifics The two basic extent attributes associated with a file are its reservation and its fixed extent size. You can preallocate space to the file by manipulating a file’s reservation, or override the default allocation policy of the file system by setting a fixed extent size. Other policies determine the way these attributes are expressed during the allocation process.
Extent Attributes Attribute Specifics Reservation: Preallocating Space to a File VxFS makes it possible to preallocate space to a file at the time of the request rather than when data is written into the file. This space cannot be allocated to other files in the file system. VxFS prevents any unexpected out-of-space condition on the file system by ensuring that a file’s required space will be associated with the file before it is required. Persistent reservation is not released when a file is truncated.
Extent Attributes Attribute Specifics Other Controls The auxiliary controls on extent attributes determine: • Whether allocations are aligned • Whether allocations are contiguous • Whether the file can be written beyond its reservation • Whether an unused reservation is released when the file is closed • Whether the reservation is a persistent attribute of the file • When the space reserved for a file will actually become part of the file Alignment Specific alignment restrictions coordinate a f
Extent Attributes Commands Related to Extent Attributes Reservation Persistence A reservation request can ensure that the reservation does not become a persistent attribute of the file. The unused reservation is discarded when the file is closed. Including Reservation in the File A reservation request can make sure the size of the file is adjusted to include the reservation. Normally, the space of the reservation is not included in the file until an extending write operation requires it.
Extent Attributes Commands Related to Extent Attributes Failure to Preserve Extent Attributes Whenever a file is copied, moved, or archived using commands that preserve extent attributes, there is nevertheless the possibility of losing the attributes. Such a failure might occur for three reasons: • The file system receiving a copied, moved, or restored file from an archive is not a VxFS type.
4 Application Interface Introduction The VERITAS File System (VxFS) provides enhancements that can be used by applications that require certain performance features. This chapter describes cache advisories and provides information about fixed extent sizes and reservation of space for a file. If you are writing applications, you can optimize them for use with the VxFS. To optimize VxFS for use with applications, see Chapter 2, “VxFS Performance: Creating, Mounting, and Tuning File Systems,” on page 37.
Application Interface Cache Advisories Cache Advisories VxFS allows an application to set cache advisories for use when accessing files. These advisories are in memory only and they do not persist across reboots. Some advisories are currently maintained on a per-file, not a per-file-descriptor, basis. This means that only one set of advisories can be in effect for all accesses to the file. If two conflicting applications set different advisories, both use the last advisories that were set.
Application Interface Cache Advisories Unbuffered I/O If the VX_UNBUFFERED advisory is set, I/O behavior is the same as direct I/O with the VX_DIRECT advisory set, so the alignment constraints that apply to direct I/O also apply to unbuffered I/O. For unbuffered I/O, however, if the file is being extended, or storage is being allocated to the file, inode changes are not updated synchronously before the write returns to the user. The VX_UNBUFFERED advisory is maintained on a per-file-descriptor basis.
Application Interface Cache Advisories Discovered Direct I/O Discovered Direct I/O is a file system tunable you can set using the vxtunefs command. When the file system gets an I/O request larger than the discovered_direct_iosz, it tries to use direct I/O on the request. For large I/O sizes, Discovered Direct I/O can perform much better than buffered I/O.
Application Interface Cache Advisories The VX_RANDOM advisory indicates that the file is being accessed randomly. For reads, this disables read-ahead. For writes, this disables the flush-behind. The data is flushed at a rate based on memory contention. If VX_NOREUSE is set when doing sequential I/O, buffers are also freed when they are flushed to disk.The VX_NOREUSE advisory is used as a modifier.
Application Interface Extent Information Extent Information The VX_SETEXT ioctl command allows an application to reserve space for a file, and set fixed extent sizes and file allocation flags. Applications can obtain status information on VxFS ioctls by using the VX_GETEXT ioctl. The getext command also provides access to this information. See the getext (1), setext (1), and vxfsio (7) manual pages for more information.
Application Interface Extent Information If the VX_TRIM flag is set, when the last close occurs on the inode, the reservation is trimmed to match the file size and the VX_TRIM flag is cleared. Any unused space is freed. This can be useful if an application needs enough space for a file, but it is not known how large the file will become. Enough space can be reserved to hold the largest expected file, and when the file has been written and closed, any extra space will be released.
Application Interface Extent Information Space reservation is used to make sure applications do not fail because the file system is out of space. An application can preallocate space for all the files it needs before starting to do any work. By allocating space in advance, the file is optimally allocated for performance, and file accesses are not slowed down by the need to allocate storage. This allocation of resources can be important in applications that require a guaranteed response time.
Application Interface Extent Information Fixed Extent Sizes VxFS uses the I/O size of write requests, and a default policy, when allocating space to a file. For some applications, this may not work out well. These applications can set a fixed extent size, so that all new extents allocated to the file are of the fixed extent size. By using a fixed extent size, an application can reduce allocations and guarantee good extent sizes for a file.
Application Interface Freeze and Thaw Freeze and Thaw The VX_FREEZE ioctl command is used to freeze a file system. Freezing a file system temporarily blocks all I/O operations to a file system and then performs a sync on the file system. When the VX_FREEZE ioctl is issued, all access to the file system is blocked at the system call level. Current operations are completed and the file system is synchronized to disk. Freezing provides a stable, consistent file system.
Application Interface Named Data Streams Named Data Streams Named data streams associate multiple data streams with a file. Access to the named data stream can be done through a file descriptor using the named data stream library functions. Applications can open the named data stream to obtain a file descriptor and perform read(), write(), and mmap() operations using the file descriptor. These system calls would work as though they are operating on a regular file.
Application Interface Named Data Streams Programmatic Interface Named Data Streams Programmatic Interface VxFS named data stream functionality is available only through the following application programming interface (API) functions: vxfs_nattr_link Links to a named data stream. vxfs_nattr_open Open a named data stream. vxfs_nattr_rename Renames a named data stream. vxfs_nattr_unlink Removes a named data stream.
Application Interface Named Data Streams Programmatic Interface O_RDWR|O_CREAT, 0777); /* create attribute attribute1 for file foo */ write(afd, buf, 1024); /* writes to attribute file */ read(afd, buf, 1024); /* reads from attribute file */ dfd = vxfs_nattr_open(fd, ".", O_RDONLY);/* opens attribute directory for file foo */ getdents(dfd, buf, 1024); /* reads directory entries for attribute directory */ Namespace for Named Data Streams Names starting with $vxfs: are reserved for VxFS.
Application Interface Named Data Streams Programmatic Interface 88 Chapter 4
5 Storage Checkpoints Storage Checkpoints are a feature of the VERITAS File System (VxFS ) provide point-in-time images of file system contents. These frozen images of VxFS file systems can be used in a variety of applications such as full and incremental online backups, fast error recovery, and product development testing. Storage Checkpoint replicas of real time databases can also be used for decision support and an assortment of database analyses.
Storage Checkpoints What is Storage Checkpoint ? What is Storage Checkpoint ? VERITAS File System provides a unique Storage Checkpoint facility that quickly creates a persistent image of a file system at an exact point in time. Storage Checkpoints significantly reduce I/O overhead by identifying and maintaining only the file system blocks that have changed since the last Storage Checkpoint or backup through a copy-on-write technique (See “How a Storage Checkpoint Works” on page 92).
Storage Checkpoints What is Storage Checkpoint ? Storage Checkpoints can be taken in environments with a large number of files (for example, file servers with millions of files) with little adverse impact on performance. Because the file system does not remain frozen during Storage Checkpoint creation, applications can access the file system even while the Storage Checkpoint is taken.
Storage Checkpoints How a Storage Checkpoint Works How a Storage Checkpoint Works The Storage Checkpoint facility freezes the mounted file system (known as the primary fileset), initializes the Storage Checkpoint, and thaws the file system. Specifically, the file system is first brought to a stable state where all of its data is written to disk, and the freezing process momentarily blocks all I/O operations to the file system.
Storage Checkpoints How a Storage Checkpoint Works Figure 5-1 Primary Fileset and its Storage Checkpoint In the figure below, each block of the file system is represented by a square. Similar to the previous figure, this figure shows a Storage Checkpoint containing pointers to the primary fileset at the time the Storage Checkpoint is taken. Figure 5-2 Initializing a Storage Checkpoint The Storage Checkpoint presents the exact image of the file system by finding the data from the primary fileset.
Storage Checkpoints How a Storage Checkpoint Works primary fileset do not result in additional updates to the Storage Checkpoint because the old data needs to be saved only once. As blocks in the primary fileset continue to change, the Storage Checkpoint accumulates the original data blocks. In the following figure, the third block originally containing C is updated. Before the block is updated with new data, the original data is copied to the Storage Checkpoint.
Storage Checkpoints Types of Storage Checkpoints Types of Storage Checkpoints You can create the following types of Storage Checkpoints: • Data Storage Checkpoints • Nodata Storage Checkpoints • Removable Storage Checkpoints • Non-mountable Storage Checkpoints Data Storage Checkpoints A data Storage Checkpoint is a complete image of the file system at the time the Storage Checkpoint is created. This type of Storage Checkpoint contains the file system metadata and file data blocks.
Storage Checkpoints Types of Storage Checkpoints Non-mountable Storage Checkpoints A non-mountable Storage Checkpoint cannot be mounted. You can use this type of Storage Checkpoint as a security feature which prevents other applications from accessing the Storage Checkpoint and modifying it.
Storage Checkpoints Storage Checkpoint Administration Storage Checkpoint Administration Storage Checkpoint administrative operations require the utility (see the fsckptadm(1M) manual page). You can use the fsckptadm utility to create and remove Storage Checkpoints, change attributes, and ascertain statistical data. Every Storage Checkpoint has an associated name, which allows you to manage Storage Checkpoints; this name is limited to 127 characters and cannot contain a colon (:).
Storage Checkpoints Storage Checkpoint Administration Use the fsvoladm command to determine if the volume set has enough free space (seethe fsvoladm(1M) manual page): # fsvoladm list /mnt0 devid size used avail name 0 20971520 8497658 12473862 mnt1 1 20971520 6328993 14642527 mnt2 2 20971520 4458462 16513058 mnt3 Creating a Storage Checkpoint You can create a Storage Checkpoint using the fsckptadm utility.
Storage Checkpoints Storage Checkpoint Administration ctime= Thu 3 Mar 2005 7:00:17 PM PST mtime= Thu 3 Mar 2005 7:00:17 PM PST flags= nodata, largefiles Removing a Storage Checkpoint You can delete a Storage Checkpoint by specifying the remove keyword of the fsckptadm command. Specifically, you can use either the synchronous or asynchronous method of removing a Storage Checkpoint; the asynchronous method is the default method.
Storage Checkpoints Storage Checkpoint Administration • If a Storage Checkpoint is originally mounted as a read-only Storage Checkpoint, you can remount it as a writable Storage Checkpoint using the -o remount option. • To mount a Storage Checkpoint of a file system, first mount the file system itself. • To unmount a file system, first unmount all of its Storage Checkpoints. WARNING If you create a Storage Checkpoint for backup purposes, do not mount it as a writable Storage Checkpoint.
Storage Checkpoints Storage Checkpoint Administration /dev/vx/dsk/fsvol / - vol1:may_23 /fsvol_ may_2 3 vxfs 0 yes ckpt= may_23 To mount a Storage Checkpoint of a cluster file system, you must also use the -o cluster option: # mount -F vxfs -o cluster,ckpt=may_23 \ /dev/vx/dsk/fsvol/vol1:may_23 /fsvol_may_23 You can only mount a Storage Checkpoint clusterwide if the file system that the Storage Checkpoint belongs to is also mounted clusterwide.
Storage Checkpoints Storage Checkpoint Administration to the pool of free blocks in the file system. The Storage Checkpoint deallocates all of its file data blocks in the background and eventually returns them to the pool of free blocks in the file system. If all of the older Storage Checkpoints in a file system are nodata Storage Chekpoints, use the synchronous method to convert a data Storage Checkpoint to a nodata Storage Checkpoint.
Storage Checkpoints Storage Checkpoint Administration hello, world # cat /mnt0@5_30pm/file hello, world 4. Change the content of the original file: # echo "goodbye" > /mnt0/file 5. Examine the content of the original file and the Storage Checkpoint file. The original file contains the latest data while the Storage Checkpoint file still contains the data at the time of the Storage Checkpoint creation: # cat /mnt0/file goodbye # cat /mnt0@5_30pm/file hello, world 6.
Storage Checkpoints Storage Checkpoint Administration Conversion with Multiple Storage Checkpoints The following example highlights the conversion of data Storage Checkpoints to nodata Storage Checkpoints, particularly when dealing with older Storage Checkpoints on the same file system: To convert Storage Checkpoints : 1.
Storage Checkpoints Storage Checkpoint Administration flags = largefiles oldest: ctime = Mon 26 Jul 11:56:41 2004 mtime = Mon 26 Jul 11:56:41 2004 flags = largefiles 3. Try to convert synchronously the "latest" Storage Checkpoint to a nodata Storage Checkpoint.
Storage Checkpoints Storage Checkpoint Administration flags = largefiles 5. You can combine the three previous steps and create the "latest" Storage Checkpoint as a nodata Storage Checkpoint. The creation process will detect the presence of the older data Storage Checkpoints and create the "latest" Storage Checkpoint as a delayed nodata Storage Checkpoint.
Storage Checkpoints Storage Checkpoint Administration flags = largefiles older: ctime = Mon 26 Jul 11:56:46 2004 mtime = Mon 26 Jul 11:56:46 2004 flags = largefiles oldest: ctime = Mon 26 Jul 11:56:41 2004 mtime = Mon 26 Jul 11:56:41 2004 flags = largefiles 6.
Storage Checkpoints Storage Checkpoint Administration 7. Remove the "older" and "old" Storage Checkpoints.
Storage Checkpoints Space Management Considerations Space Management Considerations Several operations, such as removing or overwriting a file, can fail when a file system containing Storage Checkpoints runs out of space. Usually these operations do not fail because of insufficient space on the file system, but these operations on a file system containing Storage Checkpoints can cause a data block copy that, in turn, may require extent allocation.
Storage Checkpoints File System Restore From Storage Checkpoints File System Restore From Storage Checkpoints Mountable data Storage Checkpoints on a consistent and undamaged file system can be used by backup and restore applications to restore either individual files or an entire file system. Restoration from Storage Checkpoints can also help recover incorrectly modified files, but typically cannot recover from hardware damage or other file system integrity problems.
Storage Checkpoints File System Restore From Storage Checkpoints 3. Delete the file MyFile.txt from your home directory. $ cd /home/users/me $ rm MyFile.txt 4. Go to the /home/checkpoints/mar_4/users/me directory, which contains the image of your home directory. $ cd /home/checkpoints/mar_4/users/me $ ls -l -rw-r--r-- 1 me staff 14910 Mar 4 17:09 MyFile.txt 5. Copy the file MyFile.txt to your home directory. $ cp MyFile.
Storage Checkpoints File System Restore From Storage Checkpoints /dev/vx/dsk/dg1/vol2: UNNAMED: ctime = Thu 08 May 2004 06:28:26 PM PST mtime = Thu 08 May 2004 06:28:26 PM PST flags = largefiles, file system root CKPT6: ctime = Thu 08 May 2004 06:28:35 PM PST mtime = Thu 08 May 2004 06:28:35 PM PST flags = largefiles CKPT5: ctime = Thu 08 May 2004 06:28:34 PM PST mtime = Thu 08 May 2004 06:28:34 PM PST flags = largefiles, nomount CKPT4: ctime = Thu 08 May 2004 06:28:33 PM PST mtime = Th
Storage Checkpoints File System Restore From Storage Checkpoints Select Storage Checkpoint for restore operation or (EOF) to exit or to list Storage Checkpoints: CKPT3 CKPT3: ctime = Thu 08 May 2004 06:28:31 PM PST mtime = Thu 08 May 2004 06:28:36 PM PST flags = largefiles UX:vxfs fsckpt_restore: WARNING: V-3-24640: Any file system changes or Storage Checkpoints made after Thu 08 May 2004 06:28:31 PM PST will be lost. 3.
Storage Checkpoints File System Restore From Storage Checkpoints ctime = Thu 08 May 2004 06:28:31 PM PST mtime = Thu 08 May 2004 06:28:36 PM PST flags = largefiles, file system root CKPT2: ctime = Thu 08 May 2004 06:28:30 PM PST mtime = Thu 08 May 2004 06:28:30 PM PST flags = largefiles CKPT1: ctime = Thu 08 May 2004 06:28:29 PM PST mtime = Thu 08 May 2004 06:28:29 PM PST flags = nodata, largefiles Select Storage Checkpoint for restore operation or (EOF) to exit or to list Storage Check
Storage Checkpoints Storage Checkpoint Quotas Storage Checkpoint Quotas VxFS provides options to the fsckptadm command interface to administer Storage Checkpoint quotas. Storage Checkpoint quotas set limits on the number of blocks used by a primary fileset and all of its related Storage Checkpoints. hard limit An absolute limit that cannot be exceeded. If a hard limit is exceeded, all further allocations on any of the Storage Checkpoints fail, but existing Storage Checkpoints are preserved.
Storage Checkpoints Storage Checkpoint Quotas 116 Chapter 5
6 Online Backup Using File System Snapshots This chapter describes the online backup facility provided with the VERITAS File System (VxFS). The snapshot feature of VxFS can be used to create a snapshot image of a mounted file system, which becomes a duplicate read-only copy of the mounted file system. This chapter also provides a description of how to create a snapshot file system and some examples of backing up all or part of a file system using the snapshot mechanism.
Online Backup Using File System Snapshots Snapshot File Systems Snapshot File Systems A snapshot file system is an exact image of a VxFS file system, referred to as the snapped file system, that provides a mechanism for making backups. The snapshot is a consistent view of the file system “snapped” at the point in time the snapshot is made.
Online Backup Using File System Snapshots Using a Snapshot File System for Backup Using a Snapshot File System for Backup After a snapshot file system is created, the snapshot performs a consistent backup of data in the snapped file system. Backup programs (such as cpio) that back up a standard file system tree can be used without modification on a snapshot file system because the snapshot presents the same data as the snapped file system.
Online Backup Using File System Snapshots Creating a Snapshot File System Creating a Snapshot File System You create a snapshot file system by using the -o snapof= option of the mount command. The -o snapsize= option may also be required if the device you are mounting does not identify the device size in its disk label, or if you want a size smaller than the entire device.
Online Backup Using File System Snapshots Making a Backup Making a Backup Here are some typical examples of making a backup of a 300,000 block file system named /home using a snapshot file system on /dev/vx/dsk/fsvol/vol1 with a snapshot mount point of /backup/home: • To back up files changed within the last week using cpio: # mount -F vxfs -o snapof=/home,snapsize=100000 \ /dev/vx/dsk/fsvol/vol1 /backup/home # cd /backup # find home -ctime -7 -depth -print | cpio -oc > /dev/rmt/0m # umount /backup/home
Online Backup Using File System Snapshots Performance of Snapshot File Systems Performance of Snapshot File Systems Snapshot file systems maximize the performance of the snapshot at the expense of writes to the snapped file system. Reads from a snapshot file system typically perform at nearly the throughput rates of reads from a standard VxFS file system. The performance of reads from the snapped file system are generally not affected.
Online Backup Using File System Snapshots Differences Between Snapshots and Storage Checkpoints Differences Between Snapshots and Storage Checkpoints While snapshots and Storage Checkpoints both create a point-in-time image of a file system and only the changed data blocks are updated, there are significant differences between the two technologies: • Snapshots require a separate device for storage. Storage Checkpoints reside on the same device as the original file system. • Snapshots are read-only.
Online Backup Using File System Snapshots Snapshot File System Internals Snapshot File System Internals The following sections describe the internal structure of a snapshot file system and how it copies changed data blocks from the original snapped file system.
Online Backup Using File System Snapshots Snapshot File System Internals The blockmap contains one entry for each block on the snapped file system. Initially, all entries are zero. When a block is copied from the snapped file system to the snapshot, the appropriate entry in the blockmap is changed to contain the block number on the snapshot file system that holds the data from the snapped file system.
Online Backup Using File System Snapshots Snapshot File System Internals should only require two to six percent of the blocks of the snapped file system. During periods of high activity, the snapshot might require 15 percent of the blocks of the snapped file system. These percentages tend to be lower for larger file systems and higher for smaller ones. CAUTION 126 If a snapshot file system runs out of space for changed data blocks, it is disabled and all further access to it fails.
7 Quota The VERITAS File System (VxFS) supports user quotas. The quota system limits the use of two principal resources of a file system: files and data blocks. For each of these resources, you can assign quotas to individual users to limit their usage. The following topics are covered in this chapter: • Quota Limits • Quota Files on VxFS • Quota Commands • Using Quotas For more information on VxFS quotas, see .
Quota Quota Limits Quota Limits You can set limits for individual users to file and data block usage on a file system. You can set two kinds of limits for each of the two resources: • The hard limit is an absolute limit that cannot be exceeded under any circumstances. • The soft limit, which must be lower than the hard limit, can be exceeded, but only for a limited time. The time limit can be configured on a per-file system basis only. The VxFS default limit is seven days.
Quota Quota Commands Quota Commands Most of the quotas commands in VxFS are similar to BSD quotas commands. In general, quota administration for VxFS is performed using commands similar to HFS quota commands. The VxFS mount command supports a special mount option (–o quota), that can be used to turn on quotas at mount time. NOTE For additional information on the quota commands, see the corresponding manual pages.
Quota Using Quotas edquota You can set up user quotas using the edquota command. You must have superuser privileges to edit quotas: # edquota username edquota creates a temporary file for the given user; this file contains on–disk quotas for each mounted file system that has a quotas file. It is not necessary that quotas be turned on for edquota to work. However, the quota limits are applicable only after quotas are turned on for a given file system.
8 File Change Log CAUTION File Change Log is currently not officially supported, and VERITAS strongly cautions against using it in a production environment. Although FCL is not 100% complete, it is functional and can be used to begin developing new applications. File Change Log will be fully operational in the next VERITAS File System maintenance release. The VxFS File Change Log (FCL) tracks changes to files and directories in a file system.
File Change Log The File Change Log File The File Change Log File FCL stores changes in a sparse file in the file system namespace. The FCL log file is always located in mount_point/lost+found/changelog. The FCL file behaves like a regular file, but some operations are prohibited.The standard system calls open(2), lseek(2), read(2) and close(2) can access the data in the FCL. The system calls write(2), mmap(2) and rename(2) are not allowed.
File Change Log File Change Log Administrative Interface File Change Log Administrative Interface The FCL can be set up and tuned through the VxFS administrative commands fcladm and vxtunefs. The FCL tunable parameters are: fcl_keeptime Specifies the duration in seconds that FCL records stay in the FCL file before they can be purged. The first records to be purged are the oldest ones, which are located at the beginning of the file.
File Change Log File Change Log Administrative Interface Print the on-disk FCL super-block in text format to obtain information about the FCL by using offset 0. Because the FCL on-disk super-block occupies the first block of the FCL file, the first and last valid offsets into the FCL file can be determined by reading the FCL super-block and checking the fc_foff field.
File Change Log File Change Log Programmatic Interface File Change Log Programmatic Interface The standard system calls open(2), lseek(2), read(2) and close(2) can be used on the FCL file at mount_point/lost+found/changelog. Only one programmatic interface is exposed through libvxfsutil, the vxfsu_fcl_sync API (see the vxfsu_fcl_sync(3) manual page). The prototype is available at /opt/VRTSfssdk/4.0/include/vxfsutil.h.
File Change Log File Change Log Programmatic Interface if ((fd = open(fclname, O_RDONLY)) < 0) { return ENOENT; } if ((off = lseek(fd, 0, SEEK_SET)) != 0) { close(fd); return EIO; } size = read(fd, &fclsb, sizeof (struct fcl_sb)); if (size < 0) { close(fd); return EIO; } if (fclsb.fc_state == VX_FCLS_OFF) { close(fd); return 0; } if (err = vxfsu_fcl_sync(fclname, &lastoff)) { close(fd); return err; } if ((off = lseek(fd, fclsb.fc_foff)) != fclsb.
File Change Log File Change Log Programmatic Interface close(fd); return 0; } Chapter 8 137
File Change Log Reverse Path Name Lookup Reverse Path Name Lookup The reverse path name lookup feature obtains the full path name of a file or directory from the inode number of that file or directory. The inode number is provided as an argument to the vxlsino administrative command, or the vxfs_inotopath application programming interface library function.
9 Multi-Volume File Systems VxFS provides support for multi-volume file systems when used in conjunction with the VERITAS Volume Manager.Using the multi-volume support (MVS) feature, you can create a single file system over multiple volumes and set the properties for each volume. For example, it is possible to place metadata on mirrored storage while placing file data on better performing volume types such as RAID5.
Multi-Volume File Systems Features Implemented Using MVS Features Implemented Using MVS You can implement the following features using multi-volume support: • Controlling where files are stored can be selected at multiple levels so that specific files or file hierarchies can be assigned to different volumes. This functionality is available in the VERITAS File System Quality of Storage Service (QoSS) feature (see Chapter 10, “Quality of Storage Service,” on page 151).
Multi-Volume File Systems Volume Sets Volume Sets The VERITAS Volume Manager exports a feature called Volume Sets to VxFS. Unlike a Volume Manager volume, which is used for accessing raw I/O or contain a file system, a volume set is a container for multiple different volumes. Each volume in a volume set has its own internal structure. The Volume Manager vxvset is used to create and manage volume sets.
Multi-Volume File Systems Volume Sets # vxvset addvol myvset vol4 # ls -l /dev/vx/rdsk/rootdg/vol4 /dev/vx/rdsk/rootdg/vol4: No such file or directory Volume sets cannot be empty. So, a volume set is removed when the last entry in that set is removed.
Multi-Volume File Systems Creating MVS File Systems Creating MVS File Systems After a volume set is created, creating a VxFS file system is the same as creating a file system on a raw device or volume.
Multi-Volume File Systems Creating MVS File Systems 3 51200 16 51184 vol4 4 51200 16 51184 vol5 You must empty a volume before you remove it from the file system. All the file system commands except volume addition and deletion work alike on volumes within a volume set.
Multi-Volume File Systems Allocation Policies Allocation Policies To make full use of the MVS features, VxFS supports allocation policies that allow assigning files or groups of files to specified volumes within the volume set. A policy specifies a list of volumes and the order of allocations. You can assign a policy to a file, to a file system, or to a Storage Checkpoint created from a file system.
Multi-Volume File Systems Allocation Policies 0 51200 1250 49950 vol1 1 51200 16 51184 vol2 The assignment of policies on a file system-wide basis ensures that any metadata allocated is stored on the device with the policy metadatapolicy (vol2) and all the user data is stored on vol1 with the associated datapolicy policy.
Multi-Volume File Systems Volume Encapsulation Volume Encapsulation Multi-volume support enables the ability to encapsulate an existing raw volume and make the volume contents appear as a file in the file system. There are two steps required to achieve this: • Add the volume to an existing volume set. • Add the volume to the file system using fsvoladm. As an example: Assume that the following volume set and new volume exist.
Multi-Volume File Systems Volume Encapsulation # ls -l /mnt1/dbfile -rw------- 1 root other 104857600 May 22 11:30 /mnt1/dbfile # head -2 /mnt1/dbfile root:x:0:1:Super-User:/:/sbin/sh daemon:x:1:1::/: Now, you can view the passwd file that was written to the raw volume . NOTE 148 If the encapsulated file is changed in any way, for example, extended, truncated, or moved with an allocation policy or resized volume, or the volume is encapsulated with a bias, the file cannot be de-encapsulated.
Multi-Volume File Systems Converting from QuickLog to MVS Converting from QuickLog to MVS The 4.0 release of the VERITAS File System is the last to support Quicklog. The Version 6 disk layout does not support QuickLog. The functionality provided by the VERITAS Multi-Volume Support feature replaces most of the functionality provided by QuickLog. The following procedure describes how to migrate from QuickLog to MVS.
Multi-Volume File Systems Converting from QuickLog to MVS 150 Chapter 9
10 Quality of Storage Service The VERITAS File System allows you to create a file system that spans multiple volumes ,known as a multi-volume file system. The component volumes compose a volume set. See “Quality of Storage Service” on page 151. On top of this basic capability is a set of services known as Quality of Storage Service (QoSS).
Quality of Storage Service How File Relocation Works How File Relocation Works File relocation is the process of searching a file system to select files listed in a relocation policy for relocation, then relocating the selected files. The volume on which files are originally located is referred to as a source component volume, and the volume to which files are relocated is referred to as a destination component volume.
Quality of Storage Service Configuring Relocation Policies Configuring Relocation Policies Relocation policies define the files to relocate and the physical devices on which the files are located. Relocation policies are based on the following properties of a file: • Age: The time since the file was last accessed or last modified. • File size • Path: The location of a file in a directory structure.
Quality of Storage Service Configuring Relocation Policies Option Description -n count Sets a limit for the maximum number of files for the fsmove utility to relocate. -p policy Evaluates files based on the specified policy only. If not specified, fssweep evaluates files based on all configured policies. -r Reports a statistical summary of files that can be relocated to standard output without actually relocating any files. The report is the same as the fsmove statistical summary.
Quality of Storage Service Configuring Relocation Policies source_volume The source component volume where the file originally resided. destination_volume The destination component volume to which to relocate the file. number_of_files_relocated The cumulative number of all files relocated. size_of_all_files_relocated The cumulative size of all files relocated. Scheduling Example You can use crontab for schedule file selection and relocation at specified intervals.
Quality of Storage Service Customizing QoSS Customizing QoSS The following information is not essential to QoSS daily operation and management. This section describes the relationship between the fssweep/fsmove utilities and the allocation policies and how files and volumes are selected for relocation. Mapping Relocation Policies to Allocation Policies The fssweep and fsmove utilities use relocation policies to relocate existing files.
Quality of Storage Service Customizing QoSS You can use the fsapadm utility to create allocation policies. Avoid using names that would be created by fsmove. If a file was moved using fsapadm into an allocation policy whose name is something other than the allocation policies used by fssweep and fsmove, the fssweep utility disregards the file when it searches the file system. You can use the fsapadm utility to create a special allocation policy named fsmove_SITE.
Quality of Storage Service Customizing QoSS 158 Chapter 10
11 Quick I/O for Databases VERITAS Quick I/O for Databases (referred to as Quick I/O) allows applications access preallocated VxFS files as raw character devices. This provides the administrative benefits of running databases on file systems without the performance degradation usually associated with databases created on file systems.
Quick I/O for Databases Quick I/O is part of the VRTSvxfs package, but is available for use only with other VERITAS products. See the VERITAS File System Release Notes for current product information.
Quick I/O for Databases Quick I/O Functionality and Performance Quick I/O Functionality and Performance Many database administrators (DBAs) create databases on file systems because it makes common administrative tasks (such as moving, copying, and backup) much simpler. However, putting databases on file systems significantly reduces database performance. By using VERITAS Quick I/O, you can retain the advantages of having databases on file systems without performance degradation.
Quick I/O for Databases Quick I/O Functionality and Performance Avoiding Kernel Write Locks When database I/O is performed via the write() system call, each system call acquires and releases a write lock inside the kernel. This lock prevents simultaneous write operations on the same file. Because database systems usually implement their own locks for managing concurrent access to files, write locks unnecessarily serialize I/O operations.
Quick I/O for Databases Using VxFS Files as Raw Character Devices Using VxFS Files as Raw Character Devices When VxFS with Quick I/O is installed, there are two ways of accessing a file: • The VxFS interface treats the file as a regular VxFS file • The Quick I/O interface treats the same file as if it were a raw character device, having performance similar to a raw device This allows a database server to use the Quick I/O interface while a backup server uses the VxFS interface.
Quick I/O for Databases Quick I/O Naming Convention Quick I/O Naming Convention To treat a file as a raw character device, Quick I/O requires a file name extension to create an alias for a regular VxFS file. Quick I/O recognizes the alias when you add the following suffix to a file name: ::cdev:vxfs: Whenever an application opens an existing VxFS file with the suffix ::cdev:vxfs (the cdev portion is an acronym for character device), Quick I/O treats the file as if it were a raw device.
Quick I/O for Databases Use Restrictions Use Restrictions • — The name xxx::cdev:vxfs: is recognized as a special name by VxFS only when: VxFS with Quick I/O has a valid license — the regular file xxx is physically present on the VxFS file system — there is no regular file named xxx::cdev:vxfs: on the system • If the file xxx is being used for memory mapped I/O, it cannot be accessed as a Quick I/O file.
Quick I/O for Databases Creating a Quick I/O File Using qiomkfile Creating a Quick I/O File Using qiomkfile The best way to make regular files accessible to the Quick I/O interface and preallocate space for them is to use the qiomkfile command. Unlike the VxFS setext command, which requires superuser privileges, any user who has read/write permissions can run qiomkfile to create the files. The qiomkfile command has five options: -a Creates a symbolic link with an absolute path name for a specified file.
Quick I/O for Databases Creating a Quick I/O File Using qiomkfile or: $ ls -lL crw-r----- 1oracle dba 43,0 Oct 22 15:04 dbfile -rw-r--r-- 1oracle dba 10485760 Oct 22 15:04 .dbfile If you specify the -a option to qiomkfile, an absolute path name (See “Using Absolute or Relative Path Names” on page 168) is used so /database/dbfile points to /database/.dbfile::cdev:vxfs:. To check the results, enter: $ ls -al -rw-r--r-- 1 oracle dba 104857600 Oct 22 15:05 .
Quick I/O for Databases Accessing Regular VxFS Files Through Symbolic Links Accessing Regular VxFS Files Through Symbolic Links Another way to use Quick I/O is to create a symbolic link for each file in your database and use the symbolic link to access the regular files as Quick I/O files. The following commands create a 100 MB Quick I/O file named dbfile on the VxFS file system /database. The dd command preallocates the file space: $ cd /database $ dd if=/dev/zero of=/database/.
Quick I/O for Databases Accessing Regular VxFS Files Through Symbolic Links # touch .dbfile # setext -r 102400 -f noreserve -f chgsize .dbfile # ln -s .dbfile::cdev:vxfs: dbfile # chown oracle dbfile # chgrp dba dbfile See the setext(1M) manual page for more information.
Quick I/O for Databases Using Quick I/O with Oracle Databases Using Quick I/O with Oracle Databases The following example shows how a file can be used by an Oracle database to create a tablespace.
Quick I/O for Databases Using Quick I/O with Sybase Databases Using Quick I/O with Sybase Databases Quick I/O works similarly on Sybase database devices. To create a new database device, preallocate space on the file system by using the qiomkfile command, then use the Sybase buildmaster command for a master device, or the Transact SQL disk init command for a database device.
Quick I/O for Databases Enabling and Disabling Quick I/O Enabling and Disabling Quick I/O If the Quick I/O feature is licensed and installed, Quick I/O is enabled by default when a file system is mounted. Alternatively, the VxFS mount -o qio command enables Quick I/O. The mount -o noqio command disables Quick I/O. If Quick I/O is not installed or licensed, a file system mounts by default without Quick I/O and no error message is displayed.
Quick I/O for Databases Cached Quick I/O For Databases Cached Quick I/O For Databases 32-bit applications (such as 32-bit databases) can use a maximum of only 4 GB of memory because of the 32-bit address limitation. The Cached Quick I/O feature improves database performance on machines with sufficient memory by also using the file system cache to store data.
Quick I/O for Databases Cached Quick I/O For Databases NOTE This command enables caching for all the Quick I/O files on this file system. You can make this setting persistent across mounts by adding a file system entry in the file /etc/vx/tunefstab. For example: /dev/vx/dsk/datadg/database01 qio_cache_enable=1 /dev/vx/dsk/datadg/database02 qio_cache_enable=1 For information on how to add tuning parameters, see the tunefstab(4) manual page.
Quick I/O for Databases Cached Quick I/O For Databases names.dbf,OFF To check the setting of the qio_cache_enable flag for a file system, enter: $ vxtunefs -p /database01 qio_cache_enable = 1 For more information on the format of the /etc/vx/qioadmin file and the command syntax, see the qioadmin(1) manual page. NOTE Check the setting of the flag qio_cache_enable using the vxtunefs command, and the individual cache advisories for each file, to verify caching.
Quick I/O for Databases Quick I/O Statistics Quick I/O Statistics Quick I/O provides the qiostat utility to collect database I/O statistics generated over a period of time. qiostat reports statistics such as the number of read and write operations, the number of blocks read or written, and the average time spent on read and write operations during an interval. See the qiostat(1) manpage for more information.
Quick I/O for Databases Quick I/O Summary Quick I/O Summary To increase database performance on a VxFS file system using Quick I/O: 1. Make sure that the VERITAS Editions product is installed: # swinstall | grep VRTSdbed 2. Make sure that the VERITAS Quick I/O package is licensed: # vxlicrep | VXFDD 3. Create a regular VxFS file and preallocate it to required size, or use the qiomkfile command. The size of this preallocation depends on the size requirement of the database server. 4.
Quick I/O for Databases Quick I/O Summary 178 Chapter 11
A VERITAS File System Quick Reference Veritas File System Quick Reference This appendix lists the VERITAS File System (VxFS) commands and manual pages.
VERITAS File System Quick Reference Veritas File System Quick Reference Table A-1 (Continued) Command Description cfsmntadm3 Adds, deletes, modifies, and sets policy on cluster mounted file systems. cfsmount,cfsumount 3 Mounts or unmounts a cluster file system. df Reports the number of free disk blocks and inodes for a VxFS file system. fcladm5 VxFS File Change Log administration utility. ff Lists file names and inode information for a VxFS file system.
VERITAS File System Quick Reference Veritas File System Quick Reference Table A-1 (Continued) Command Description fssweep4,5 Sweep a multiple-volume VxFS file system for files to relocate. fstyp Returns the type of file system on a specified disk partition. fsvoladm5 VxFS device administration utility. getext Gets extent attributes for a VxFS file system. glmconfig3 Group Lock Manager (GLM) configuration utility. mkfs Constructs a VxFS file system. mount Mounts a VxFS file system.
VERITAS File System Quick Reference Veritas File System Quick Reference Table A-1 (Continued) Command Description vxtunefs Tunes a VxFS file system. vxupgrade Upgrades the disk layout of a mounted VxFS file system.
VERITAS File System Quick Reference Veritas File System Quick Reference Table A-2 (Continued) Section1 Description cfscluster3 CFS cluster configuration command. cfsdgadm3 Adds or deletes shared disk groups to/from a cluster configuration. cfsmntadm3 Adds, deletes, modifies, and sets policy on cluster mounted file systems. cfsmount, cfsumount3 Mounts or unmounts a cluster file system. df_vxfs Reports the number of free disk blocks and inodes for a VxFS file system.
VERITAS File System Quick Reference Veritas File System Quick Reference Table A-2 (Continued) Section1 Description fstyp_vxfs Returns the type of file system on a specified disk partition. glmconfig3 Group Lock Manager (GLM) configuration utility. mkfs_vxfs Constructs a VxFS file system. mount_vxfs Mounts a VxFS file system. ncheck_vxfs Generates path names from inode numbers for a VxFS file system. setext Sets extent attributes on a file in a VxFS file system.
VERITAS File System Quick Reference Veritas File System Quick Reference Table A-2 (Continued) Section1 Description vxfs_ap_enforce_file5 Ensures that all blocks in a specified file match the file allocation policy. vxfs_ap_enumerate5 Returns information about all allocation policies. vxfs_ap_query5 Returns information about a specific allocation policy. vxfs_ap_query_ckpt5 Returns information about allocation policies for each Storage Checkpoint.
VERITAS File System Quick Reference Veritas File System Quick Reference Table A-2 (Continued) Section1 Description vxfs_vol_encapsulate _bias5 Encapsulates a volume within a multi-volume file system. vxfs_vol_enumerate5 Returns information about the volumes within a multi-volume file system. vxfs_vol_remove5 Removes a volume from a multi-volume file system. vxfs_vol_resize5 Resizes a specific volume within a multi-volume file system.
VERITAS File System Quick Reference Veritas File System Quick Reference Before running mkfs, you must create the target device. Refer to your operating system documentation for more information. If you are using a logical device (such as a VxVM volume), see the VxVM documentation for instructions on device initialization. How to Create a File System To create a file system, use the mkfs command: mkfs [-F vxfs] [generic_options] [-o specific_options] special [size] vxfs The file system type.
VERITAS File System Quick Reference Veritas File System Quick Reference Converting a UFS File System to VxFS The vxfsconvert command can be used to convert a UFS file system to a VxFS file system. How to Convert a File System To convert a UFS file system, use the vxfsconvert command: vxfsconvert [-l logsize] [-s size] [-efnNvyY] special -e Estimates the amount of space required to complete the conversion. -f Displays the list of supported file system types.
VERITAS File System Quick Reference Veritas File System Quick Reference mount [-F vxfs] [generic_options] [-r] [-o specific_options] \special mount_point vxfs The file system type. generic_options Options common to most other file system types. specific_options Options specific to VxFS. -o ckpt=ckpt_name Mounts a VERITAS Storage Checkpoint. -o cluster Mounts a file system in shared mode. Available only with the VxFS cluster file system feature. special Block special device.
VERITAS File System Quick Reference Veritas File System Quick Reference do not have Quick I/O, mount ignores the qio option. Alternatively, you can increase database performance using the mount option convosync=direct, which utilizes direct I/O. See “Quick I/O Functionality and Performance” on page 161 for more information. • News file systems If you are using cnews, use delaylog (or tmplog),mincache=closesync because cnews does an fsync() on each news file before marking it received.
VERITAS File System Quick Reference Veritas File System Quick Reference Each entry must be on a single line. See the fstab(4) manual page for more information about the /etc/fstab file format. Here is a typical fstab file with the new file system on the last line: # System /etc/fstab file. Static # information about the file systems # See fstab(4) and sam(1M) for further # details on configuring devices.
VERITAS File System Quick Reference Veritas File System Quick Reference # umount -a This unmounts all file systems except /(root), /usr, /var, /opt, and /tmp. Displaying Information on Mounted File Systems You can use the mount command to display a list of currently mounted file systems. How to Display File System Information To view the status of mounted file systems, use the syntax: mount -v This shows the file system type and mount options for all mounted file systems.
VERITAS File System Quick Reference Veritas File System Quick Reference Jun 6 4:17:20 2004 /dev/vg00/lvol4 on /home type vxfs ioerror=mwdisable,delaylog on Thu \ Jun 6 4:17:20 2004 /dev/vgdb/lvol9 on /bench type vxfs ioerror=mwdisable,delaylog on Thu \ Jun 6 4:17:11 2004 Identifying File System Types Use the fstyp command to determine the file system type for a specified file system. This is useful when a file system was created elsewhere and you want to know its type.
VERITAS File System Quick Reference Veritas File System Quick Reference f_files: 256548 f_ffree: 256516 f_favail: 256516 f_fsid: 520114176 f_basetype: vxfs f_namemax: 254 f_magic: a501fcf5 f_featurebits: 0 f_flag: 0 f_fsindex: 7 f_size: 4194304 Resizing a File System You can extend or shrink mounted VxFS file systems using the fsadm command. Use the extendfs command to extend the size of an unmounted file system. A file system using the Version 4 disk layout can be up to two terabytes in size.
VERITAS File System Quick Reference Veritas File System Quick Reference newsize The size (in sectors) to which the file system will increase. mount_point The file system’s mount point. -r rawdev Specifies the path name of the raw device if there is no entry in /etc/fstab and fsadm cannot determine the raw device. NOTE Example A-7 The device must have enough space to contain the larger file system.
VERITAS File System Quick Reference Veritas File System Quick Reference NOTE After this operation, there is unused space at the end of the device. You can then resize the device, but be careful not to make the device smaller than the new size of the file system. How to Reorganize a File System You can reorganize (or compact) a fragmented file system using fsadm, even while the file system is mounted. This may help shrink a file system that could not previously be decreased.
VERITAS File System Quick Reference Veritas File System Quick Reference extendfs [-F vxfs] [-q] [-v] [-s size] special vxfs The file system type -q Displays the size of special without resizing it -v Specifies verbose mode -s size Specifies the number of blocks to add to the file system (maximum if not specified) special Either a logical volume or a disk partition NOTE The device must have enough space to hold the new larger file system.
VERITAS File System Quick Reference Veritas File System Quick Reference • vxdump(1M) • vxrestore(1M) How to Create and Mount a Snapshot File System The first step in backing up a VxFS file system is to create and mount a snapshot file system. To create and mount a snapshot of a VxFS file system, use the syntax: mount [-F vxfs] -o snapof=source,[snapsize=size] \destination snap_mount_point source The special device name or mount point of the file system to copy.
VERITAS File System Quick Reference Veritas File System Quick Reference How to Restore a File System After backing up the file system, you can restore it using the vxrestore command. First, create and mount an empty file system. To restore a VxFS snapshot file system, use the syntax: vxrestore [-v] [-x] [filename] -v Specifies verbose mode. -x Extracts the named files from the tape. filename The file or directory to restore.
VERITAS File System Quick Reference Veritas File System Quick Reference If the root directory does not contain a quotas file, the mount command succeeds, but quotas are not turned on.
VERITAS File System Quick Reference Veritas File System Quick Reference How to View Quotas The superuser or individual user can view disk quotas and usage on VxFS file systems using the quota command. To view quotas for a specific user, use the syntax: quota -v username This command displays the user’s quotas and disk usage on all mounted VxFS file systems where the quotas file exists. You will see all established quotas regardless of whether or not the quotas are actually turned on.
VERITAS File System Quick Reference Veritas File System Quick Reference 202 Appendix A
B Kernel Messages Kernel Messages This appendix contains a listing of diagnostic or error messages generated by the VERITAS File System (VxFS) kernel. Each message has a description and a suggestion on how to handle or correct the underlying problem.
Kernel Messages Kernel Messages Disabling Transactions If the file system detects an error while writing the intent log, it disables transactions. After transactions are disabled, the files in the file system can still be read or written, but no block or inode frees or allocations, structural changes, directory entry changes, or other changes to metadata are allowed. Disabling a File System If an error occurs that compromises the integrity of the file system, VxFS disables itself.
Kernel Messages Kernel Messages Each message is also written to an internal kernel buffer that you can view in the file /var/adm/syslog/syslog.log. In some cases, additional data is written to the kernel buffer. For example, if an inode is marked bad, the contents of the bad inode are written. When an error message is displayed on the console, you can use the unique message ID to find the message in /var/adm/syslog/syslog.log and obtain the additional information.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 002 Message and Definition WARNING: msgcnt x: mesg 002: V-2-02: vx_snap_strategy - mount_point file system write attempt to read-only file system WARNING: msgcnt x: mesg 002: V-2-02: vx_snap_copyblk - mount_point file system write attempt to read-only file system • Description The kernel tried to write to a read-only file system. This is an unlikely problem, but if it occurs, the file system is disabled.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 003, 004, 005 Message and Definition WARNING: msgcnt x: mesg 003: V-2-03: vx_mapbad - mount_point file system free extent bitmap in au aun marked bad. WARNING: msgcnt x: mesg 004: V-2-04: vx_mapbad - mount_point file system free inode bitmap in au aun marked bad. WARNING: msgcnt x: mesg 005: V-2-05: vx_mapbad - mount_point file system inode extended operation bitmap in au aun marked bad.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 006, 007 Message and Definition WARNING: msgcnt x: mesg 006: V-2-06: vx_sumupd - mount_point file system summary update in au aun failed WARNING: msgcnt x: mesg 007: V-2-07: vx_sumupd - mount_point file system summary update in inode au iaun failed • Description An I/O error occurred while writing the allocation unit or inode allocation unit bitmap summary to disk. This sets the VX_FULLFSCK flag on the file system.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 008, 009 Message and Definition WARNING: msgcnt x: mesg 008: V-2-08: vx_direrr - mount_point file system inode inumber block blkno error errno WARNING: msgcnt x: mesg 009: V-2-09: vx_direrr - mount_point file system inode inumber immediate directory error errno • Description A directory operation failed in an unexpected manner. The mount point, inode, and block number identify the failing directory.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 011 Message and Definition NOTE: msgcnt x: mesg 011: V-2-11: vx_noinode - mount_point file system out of inodes • Description The file system is out of inodes. • Action Monitor the free inodes in the file system. If the file system is getting full, create more inodes either by removing files or by expanding the file system.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 013 Message and Definition WARNING: msgcnt x: mesg 013: V-2-13: vx_iposition - mount_point file system inode inumber invalid inode list extent • Description For a Version 2 and above disk layout, the inode list is dynamically allocated. When the kernel tries to read an inode, it must look up the location of the inode in the inode list file. If the kernel finds a bad extent, the inode can’t be accessed.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 015 Message and Definition WARNING: msgcnt x: mesg 015: V-2-15: vx_ibadinactive - mount_point file system can’t mark inode inumber bad msgcnt x: mesg 015: V-2-15: vx_ilisterr - mount_point file system can’t mark inode inumber bad • Description An attempt to mark an inode bad on disk, and the super-block update to set the VX_FULLFSCK flag, failed.
Kernel Messages Kernel Messages Table B-1 Message Number 017 (Continued) Message and Definition WARNING: msgcnt x: mesg 017: V-2-17: vx_attr_getblk - mount_point file system inode inumber marked bad in core WARNING: msgcnt x: mesg 017: V-2-17: vx_attr_iget - mount_point file system inode inumber marked bad in core WARNING: msgcnt x: mesg 017: V-2-17: vx_attr_iget - mount_point file system inode inumber marked bad in core WARNING: msgcnt x: mesg 017: V-2-17: vx_attr_indadd - mount_point file system inode in
Kernel Messages Kernel Messages Table B-1 Message Number 017 (Continued) Message and Definition WARNING: msgcnt x: mesg 017: V-2-17: vx_enter_ext4 - mount_point file system inode inumber marked bad in core WARNING: msgcnt x: mesg 017: V-2-17: vx_exttrunc - mount_point file systeminode inumber marked bad in core WARNING: msgcnt x: mesg 017: V-2-17: vx_get_alloc - mount_point file system inode inumber marked bad in core WARNING: msgcnt x: mesg 017: V-2-17: vx_ilisterr - mount_point file system inode inumber
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 017 Message and Definition WARNING: msgcnt x: mesg 017: V-2-17: vx_stablestore - mount_point file system inode inumber marked bad in core WARNING: msgcnt x: mesg 017: V-2-17: vx_tranitimes - mount_point file system inode inumber marked bad in core WARNING: msgcnt x: mesg 017: V-2-17: vx_trunc - mount_point file system inode inumber marked bad in core WARNING: msgcnt x: mesg 017: V-2-17: vx_write_alloc2 - mount_point file system inode in
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 017 Message and Definition • Action Check the console log for I/O errors. If the problem is a disk failure, replace the disk. If the problem is not related to an I/O failure, find out how the disk became corrupted. If no user or process is writing to the device, report the problem to your customer support organization. In either case, unmount the file system.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 021 Message and Definition WARNING: msgcnt x: mesg 021: V-2-21: vx_fs_init - mount_point file system validation failure • Description When a VxFS file system is mounted, the structure is read from disk. If the file system is marked clean, the structure is correct and the first block of the intent log is cleared. If there is any I/O problem or the structure is inconsistent, the kernel sets the VX_FULLFSCK flag and the mount fails.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 022 Message and Definition WARNING: msgcnt x: mesg 022: V-2-22: vx_mountroot - root file system remount failed • Description The remount of the root file system failed. The system will not be usable if the root file system can’t be remounted for read/write access. When a VERITAS root file system is first mounted, it is mounted for read-only access. After fsck is run, the file system is remounted for read/write access.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 023 Message and Definition WARNING: msgcnt x: mesg 023: V-2-23: vx_unmountroot - root file system is busy and can’t be unmounted cleanly • Description There were active files in the file system and they caused the unmount to fail. When the system is halted, the root file system is unmounted. This happens occasionally when a process is hung and it can’t be killed before unmounting the root.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 025 Message and Definition WARNING: msgcnt x: mesg 025: V-2-25: vx_wsuper - mount_point file system super-block update failed • Description An I/O error occurred while writing the super-block during a resize operation. The file system is disabled. • Action Unmount the file system and use fsck to run a full structural check. Check the console log for I/O errors.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 027 Message and Definition WARNING: msgcnt x: mesg 027: V-2-27: vx_snap_bpcopy - mount_point snapshot file system write error • Description A write to the snapshot file system failed. As the primary file system is updated, copies of the original data are read from the primary file system and written to the snapshot file system. If one of these writes fails, the snapshot file system is disabled.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 029,030 Message and Definition WARNING: msgcnt x: mesg 029: V-2-29: vx_snap_getbp - mount_point snapshot file system block map write error msgcnt x: mesg 030: V-2-30: vx_snap_getbp - mount_point snapshot file system block map read error • Description During a snapshot backup, each snapshot file system maintains a block map on disk.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 032 Message and Definition WARNING: msgcnt x: mesg 032: V-2-32: vx_disable - mount_point snapshot file system disabled • Description Snapshot file system disabled, preceded by a message that specifies the reason. • Action Unmount the snapshot file system, correct the problem specified by the message, and rerun any backups that failed due to the error.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 034 Message and Definition WARNING: msgcnt x: mesg 034: V-2-34: vx_resetlog - mount_point file system can’t reset log • Description The kernel encountered an error while resetting the log ID on the file system. This happens only if the super-block update or log write encountered a device failure. The file system is disabled to preserve its integrity. • Action Unmount the file system and use fsck to run a full structural check.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 037 Message and Definition WARNING: msgcnt x: mesg 037: V-2-37: vx_metaioerr - file system metadata error • Description A read or a write error occurred while accessing file system metadata. The full fsck flag on the file system was set. The message specifies whether the disk I/O that failed was a read or a write. File system metadata includes inodes, directory blocks, and the file system log.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 038 Message and Definition WARNING: msgcnt x: mesg 038: V-2-38: vx_dataioerr - file system file data error • Description A read or a write error occurred while accessing file data. The message specifies whether the disk I/O that failed was a read or a write. File data includes data currently in files and free blocks.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 039 Message and Definition WARNING: msgcnt x: mesg 039: V-2-39: vx_writesuper - file system super-block write error • Description An attempt to write the file system super block failed due to a disk I/O error. If the file system was being mounted at the time, the mount will fail.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 041 Message and Definition WARNING: msgcnt x: mesg 041: V-2-41: vx_dqget - mount_point file system user quota file can’t read quota for id id • Description A read of the user quotas file failed for the uid. The quotas file keeps track of the total number of blocks and inodes used by each user, and contains soft and hard limits for each user ID. The VX_FULLFSCK flag is set in the super-block.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 044 Message and Definition WARNING: msgcnt x: mesg 044: V-2-44: vx_bsdquotaupdate mount_point file system user id disk quota exceeded • Description The soft limit on blocks is exceeded. Users can exceed the soft limit for a limited amount of time before allocations begin to fail. After the soft quota time limit has expired, subsequent attempts to allocate blocks for files fail. • Action Remove some files to free up space.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 047 Message and Definition WARNING: msgcnt x: mesg 047: V-2-47: vx_bsdiquotaupdate - warning: mount_point file system user id inode quota exceeded • Description The soft limit on inodes was exceeded. The soft limit can be exceeded for a certain amount of time before attempts to create new files begin to fail. Once the time limit has expired, further attempts to create files owned by the user will fail.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 056 Message and Definition WARNING: msgcnt x: mesg 056: V-2-56: vx_mapbad - mount_point file system extent allocation unit state bitmap number number marked bad • Description If there is an I/O failure while writing a bitmap, the map is marked bad. The kernel considers the maps to be invalid, so does not do any more resource allocation from maps.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 057 Message and Definition WARNING: msgcnt x: mesg 057: V-2-57: vx_esum_bad - mount_point file system extent allocation unit summary number number marked bad • Description An I/O error occurred reading or writing an extent allocation unit summary. The VX_FULLFSCK flag is set. If the VX_FULLFSCK flag can’t be set, the file system is disabled. • Action Check the console log for I/O errors.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 059 Message and Definition WARNING: msgcnt x: mesg 059: V-2-59: vx_snap_getbitbp mount_point snapshot file system bitmap write error • Description An I/O error occurred while writing to the snapshot file system bitmap. There is no problem with the snapped file system, but the snapshot file system is disabled. • Action Check the console log for I/O errors. If the problem is a disk failure, replace the disk.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 061 Message and Definition WARNING: msgcnt x: mesg 061: V-2-61: vx_resize - mount_point file system remount failed • Description During a file system resize, the remount to the new size failed. The VX_FULLFSCK flag is set and the file system is disabled. • Action Unmount the file system and use fsck to run a full structural check. After the check, the file system shows the new size.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 064 Message and Definition WARNING: msgcnt x: mesg 064: V-2-64: vx_ivalidate - mount_point file system inode number version number exceeds fileset’s • Description During inode validation, a discrepancy was found between the inode version number and the fileset version number. The inode may be marked bad, or the fileset version number may be changed, depending on the ratio of the mismatched version numbers. VX_FULLFSCK flag is set.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 067 Message and Definition WARNING: msgcnt x: mesg 067: V-2-67: mount of device_path requires HSM agent • Description The file system mount failed because the file system was marked as being under the management of an HSM agent, and no HSM agent was found during the mount. • Action Restart the HSM agent and try to mount the file system again.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 070 Message and Definition WARNING: msgcnt x: mesg 070: V-2-70: checkpoint checkpoint_name removed from file system mount_point • Description The file system ran out of space while updating a Storage Checkpoint. The Storage Checkpoint was removed to allow the operation to complete. • Action Increase the size of the file system.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 072 Message and Definition WARNING: msgcnt x: vxfs: mesg 072: vx_iaualloc - mount_point file system mount_point fileset (index number) fileset header checksum bad • Description An incorrect fileset header checksum was found while adding a new inode allocation unit. The VX_FULLFSCK flag is set and the file system is disabled to preserve integrity.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 075 Message and Definition WARNING: msgcnt x: mesg 075: V-2-75: replay fsck failed for mount_point file system • Description The log replay failed during a failover or while migrating the CFS primary-ship to one of the secondary cluster nodes. The file system was disabled. • Action Unmount the file system from the cluster. Use fsck to run a full structural check and mount the file system again.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 078 Message and Definition WARNING: msgcnt x: mesg 078: V-2-78: vx_ilealloc - mount_point file system mount_point fileset (index number) ilist corrupt • Description The inode list for the fileset was corrupted and the corruption was detected while allocating new inodes. The failed system call returns an ENOSPC error. Any subsequent inode allocations will fail unless a sufficient number of files are removed.
Kernel Messages Kernel Messages Table B-1 Message Number 079 (Continued) Message and Definition WARNING: msgcnt x: mesg 017: V-2-79: vx_attr_getblk - mount_point file system inode inumber marked bad on disk WARNING: msgcnt x: mesg 017: V-2-79: vx_attr_iget - mount_point file system inode inumber marked bad on disk WARNING: msgcnt x: mesg 017: V-2-79: vx_attr_indadd - mount_point file system inode inumber marked bad on disk WARNING: msgcnt x: mesg 017: V-2-79: vx_attr_indtrunc mount_point file system inode
Kernel Messages Kernel Messages Table B-1 Message Number 079 (Continued) Message and Definition WARNING: msgcnt x: mesg 017: V-2-79: vx_do_getpage - mount_point file system inode inumber marked bad on disk WARNING: msgcnt x: mesg 017: V-2-79: vx_enter_ext4 - mount_point file system inode inumber marked bad on disk WARNING: msgcnt x: mesg 017: V-2-79: vx_exttrunc - mount_point file system inode inumber marked bad on disk WARNING: msgcnt x: mesg 017: V-2-79: vx_get_alloc - mount_point file system inode inumb
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 079 Message and Definition WARNING: msgcnt x: mesg 017: V-2-79: vx_stablestore - mount_point file system inode inumber marked bad on disk WARNING: msgcnt x: mesg 017: V-2-79: vx_tranitimes - mount_point file system inode inumber marked bad on disk WARNING: msgcnt x: mesg 017: V-2-79: vx_trunc - mount_point file system inode inumber marked bad on disk WARNING: msgcnt x: mesg 017: V-2-79: vx_write_alloc2 - mount_point file system inode in
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 079 Message and Definition • Action Check the console log for I/O errors. If the problem is a disk failure, replace the disk. If the problem is not related to an I/O failure, find out how the disk became corrupted. If no user or process is writing to the device, report the problem to your customer support organization. In either case, unmount the file system and use fsck to run a full structural check.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 081 Message and Definition WARNING: msgcnt x: mesg 081: V-2-81: possible network partition detected • Description This message displays when CFS detects a possible network partition and disables the file system locally, that is, on the node where the message appears. • Action There are one or more private network links for communication between the nodes in a cluster.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 083 Message and Definition WARNING: msgcnt x: mesg 083: V-2-83: mount_point file system log is not compatible with the specified intent log I/O size • Description Either the specified mount logiosize size is not compatible with the file system layout, or the file system is corrupted.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 086 Message and Definition WARNING: msgcnt x: mesg 086: V-2-86: Checkpoint quota - warning: file_system file system fileset quota soft limit exceeded • Description The system administrator sets the quotas for checkpoints in the form of a soft limit and hard limit. This message displays when the soft limit is exceeded. • Action Delete checkpoints or increase the soft limit. This is not a mandatory action, but is recommended.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 089 Message and Definition WARNING: msgcnt x: mesg 089: V-2-89: quotaon on file_system invalid; disk usage for group/user id uid exceeds sectors sectors • Description The supported quota limit is up to 2147483647 sectors. When quotas are turned on by the quotaon command, this message displays when a user exceeds the supported quota limit. • Action Ask the user to delete files to lower the quota below the limit.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 092 Message and Definition WARNING: msgcnt x: mesg 092: V-2-92: vx_mkfcltran - failure to map offset offset in File Change Log file • Description The vxfs kernel was unable to map actual storage to the next offset in the File Change Log file. This is mostly likely caused by a problem with allocating to the FCL file. Because no new FCL records can be written to the FCL file, the FCL has been deactivated. • Action Re-activate the FCL.
Kernel Messages Kernel Messages Table B-1 (Continued) Message Number 095 Message and Definition WARNING: msgcnt x: mesg 095: V-2-95: Setting vxfs_ifree_timelag to time since the specified value for vxfs_ifree_timelag is less than the recommended minimum value of time. • Description The value for vxfs_ifree_timelag specified by the system administrator is less than the recommended minimum value, time, and so the value of vxfs_ifree_timelag has been automatically changed to time.
C Disk Layout Disk Layout The disk layout is the way file system information is stored on disk. On VxFS, six different disk layout versions were created to take advantage of evolving technological developments. The disk layout versions used on VxFS were: Table C-1 Version 1 The Version 1 disk layout is the original VxFS disk layout provided with pre-2.0 versions of VxFS.
Disk Layout Disk Layout Some of the disk layout versions were not supported on all UNIX operating systems. Version 2 and 3 file systems can still be mounted, but this will be disallowed in future releases. Currently, the Version 4, Version 5, and Version 6 disk layouts can be created and mounted. Version 6 is the default disk layout version. The vxupgrade command is provided to upgrade an existing VxFS file system to the Version 4, Version 5, or Version 6 disk layout while the file system remains online.
Disk Layout Disk Layout The VxFS Version 4 disk layout divides the entire file system space into fixed size allocation units. The first allocation unit starts at block zero and all allocation units are a fixed length of 32K blocks. (An exception may be the last AU, which occupies whatever space remains at the end of the file system).
Disk Layout Disk Layout extent allocation unit summary file Contains the AU summary for each allocation unit, which contains the number of free extents of each size. The summary for an extent is created only when an allocation unit is expanded for use. free extent map file Contains the free extent maps for each of the allocation units. quotas files There is a quotas file which is used to track the resources allocated to each user and a quotas.grp file to track the resources allocated to each group.
Disk Layout Disk Layout Figure C-1 Appendix C VxFS Version 4 Disk Layout 255
Disk Layout Disk Layout The VxFS Version 5 Disk Layout VxFS disk layout Version 5 is similar to Version 4. Structural files in Version 5 are the same in Version 4. However, the Version 5 disk layout supports file systems up to 32 terabytes. For a file system to take advantage of VxFS 32-terabyte support, it must be created on a VERITAS Volume Manager volume, and only on a 64-bit kernel operating system. The maximum file system size on a 32-bit kernel is still one terabyte.
Disk Layout Disk Layout Table C-3 Block Size (Continued) Maximum File System Size 4096 bytes 17,179,868,156 sectors (~ 16 TB) 8192 bytes 34,359,736,312 sectors (~ 32 TB) Note: Sector size in bytes specified by the DEV_BSIZE system parameter. If you specify the file system size when creating a file system, the block size defaults to the appropriate value as shown above. See the mkfs(1M) manual page for more information.
Disk Layout Disk Layout 258 Appendix C
Glossary A B Acess Control Lists The information that identifies specific users or groups and their access privileges for a particular file or directory. Block-Level Incremental Backup (BLI Backup) A VERITAS backup capability that does not store and retrieve entire files. Instead, only the data blocks that have changed since the previous backup are backed up. agent A process that manages predefined VERITAS Cluster Server (VCS) resource types.
Glossary data block D E data block A block that contains the actual data belonging to files and directories. encapsulation A process that converts existing partitions on a specified disk to volumes. If any partitions contain file systems, /etc/fstab entries are modified so that the file systems are mounted on volumes instead. Encapsulation is not applicable on some systems.
Glossary local mounted file system fragmentation The on-going process on an active file system in which the file system is spread further and further along the disk, leaving unused gaps or fragments between areas that are in use. This leads to degraded performance because the file system has fewer options when assigning a file to an extent. G GB Gigabyte (230 bytes or 1024 megabytes).
Glossary metadata M metadata Structural data describing the attributes of files on a disk. object location table replica A copy of the OLT in case of data corruption. The OLT replica is written to a fixed location on the underlying media (or disk). MB Megabyte (220 bytes or 1024 kilobytes). P mirror A duplicate copy of a volume and the data therein (in the form of an ordered collection of subdisks). Each mirror is one copy of the volume with which the mirror is associated.
Glossary Unbuffered I/O quotas file The quotas commands read and write the external quotas file to get or change usage limits. When quotas are turned on, the quota limits are copied from the external quotas file to the internal quotas file. See also quotas, internal quotas file, and external quotas file.. R reservation An extent attribute used to preallocate space for a file. root disk group A special private disk group that always exists on the system. The root disk group is named rootdg.
Glossary volume file is extended; for direct I/O, the inode is written to disk synchronously, for unbuffered I/O, the inode update is delayed. See also buffered I/O and direct I/O. volume A virtual disk which represents an addressable range of disk blocks used by applications such as file systems or databases. volume set A container for multiple different volumes. Each volume can have its own geometry. vxfs The VERITAS File System type. Used as a parameter in some commands. VxFS The VERITAS File System.
