MemFS 2.0 - A Memory-based File System on HP-UX 11iv3

ManualsBrandsHP ManualsSoftwareHP-UX 11i v3 Foundation OE

Introduction ....................................................................................................................................... 2

Implementation Goals ........................................................................................................................ 2

Design .............................................................................................................................................. 3

Data structures ............................................................................................................................... 4

Directory entries ......................................................................................................................... 4

Inodes ....................................................................................................................................... 4

File cache .................................................................................................................................. 4

MemFS Daemon (memfsd) ........................................................................................................... 4

Operations .................................................................................................................................... 4

Mount ....................................................................................................................................... 4

Directory entries operation .......................................................................................................... 4

read/write operation .................................................................................................................. 4

Usage .............................................................................................................................................. 5

Performance ...................................................................................................................................... 6

Postmark Benchmark ...................................................................................................................... 6

Connectathon Tests ........................................................................................................................ 8

SDET Benchmark .......................................................................................................................... 10

Configuration Guidelines .................................................................................................................. 11

Recommended Use .......................................................................................................................... 12

Limitations and Future Work .............................................................................................................. 12

Summary ........................................................................................................................................ 12

MemFS 2.0 – A Memory-based File System

on HP-UX 11iv3

Summary of content (13 pages)

PAGE 1
MemFS 2.0 – A Memory-based File System on HP-UX 11iv3 Introduction ....................................................................................................................................... 2 Implementation Goals ........................................................................................................................ 2 Design..............................................................................................................................................
PAGE 2
Introduction A Memory-based File System (MemFS) is a file system that resides in primary memory. It does not normally write data out to stable storage. Thus, MemFS is able to provide extremely high throughput. The purpose of such a file system is to provide fast access for temporary or short lived files that do not need to be kept for an indeterminate time. Keeping data in memory comes at a cost. It consumes system physical memory. Even with today‟s large-memory systems, physical memory comes at a premium.
PAGE 3
     All the inodes are pre-allocated during the creation of file system, which wastes kernel memory. The maximum file system size and file size of MemFS is restricted to 256GB due to the design of inodes in HFS and cannot grow up to the size of available system memory. It does not scale well for large directories, as the directory entries are stored and searched sequentially by HFS, which in turn degrades the performance of operations like create, search on large directories.
PAGE 4
Data structures Directory entries A directory entry, which converts the file name to an inode, is created for each file/directory. Directory entries are maintained in the hash table using the unique tuple. As directory entries of a directory are evenly distributed across the hash table, the lookup of files in the directory is faster and it scales well for large directories.
PAGE 5
Usage MemFS file systems are created by invoking the mount command with „„memfs‟‟ specified as the file system type. Device argument is not used by MemFS as it uses swap device as the back store. It allocates the required metadata for the file system. The metadata related to MemFS is created from kernel memory. The metadata of MemFS is not swappable. It shares the kernel memory with other kernel resources. The memory usage by MemFS can be limited using memfs_metamax(5) tunable.
PAGE 6
Performance A MemFS is expected to perform much better than normal disk file system for smaller file and file system sizes. As the memory occupied by MemFS increases to a level where swapping starts happening, the performance may begin to taper off. If the swapping becomes excessive, performance will decrease. MemFS performance varies depending on usage and machine configurations. The benchmark tests chosen to characterize the performance of MemFS are: Postmark, Connectathon basic tests and SDET.
PAGE 7
Figure 2. Postmark benchmark results for 10,000 simultaneous files and 100 subdirectories The file systems under test were: MemFS on HP-UX 11iv3 of size=2GB. memfs_metamax set to 30%. tmpfs on Red Hat Linux 2.4.217 tmpfs on Red Hat Linux 2.6 MemFS on HP-UX 11iv2 of size=2GB. dbc_max_pct was set to 50% (4GB) RAMdisk on HP-UX 11iv3: VxFS was the base file system installed on it using the delaylog mount option. VxFS 4.1 was used.
PAGE 8
Figure 3. Postmark benchmark results for 10,000 files and no sub-directories Connectathon Tests The Connectathon tests are a collection of micro-benchmarks used to benchmark NFS. The basic tests are used to verify basic operations of “NFS” file system implementations. Figure 3 shows the performance of various file systems on various file operations.
PAGE 9
symlink statfs create and read 400 symlinks on 10 files stat the mount point 1500 times The system under test consisted of HP rx5670 comprising of 2 x 900MHz Itanium CPUs, 8GB RAM. The Connectathon basic test results indicate that MemFS performs slightly better than RAMdisk/VxFS, and far better than disk based file systems as it takes less time compared to them. It is in close proximity with Linux tmpfs. Figure 4.
PAGE 10
SDET Benchmark The Software Development Environment Throughput (SDET) benchmark is a highly scalable, multi process benchmark which exercises most of the kernel (except networking). The benchmark was designed to apply a representative model workload to a system at increasing levels of concurrency in order to generate a graph of throughput vs. offered load. SDET is however, a system benchmark. The system under test consisted of HP rx7620 comprising 8 * 1.6GHz Itanium CPUs and 32GB RAM.
PAGE 11
Figure 5.
PAGE 12
As mentioned earlier, the amount of free space on the file system depends on the free swap space at that instant. memfs_swapmax_pct(5) tunable can be used to increase the swap space available for MemFS. 0-80% of the swap space can be specified for MemFS usage. This tunable will just specify the maximum percentage of swap space MemFS can use but this will not reserve the space for MemFS and assures the space for MemFS.
PAGE 13
© Copyright 2009 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein. Linux is a U.S. registered trademark of Linus Torvalds.