MemFS 2.0 - A Memory-based File System on HP-UX 11iv3

2
Introduction
A Memory-based File System (MemFS) is a file system that resides in primary memory. It does not
normally write data out to stable storage. Thus, MemFS is able to provide extremely high
throughput. The purpose of such a file system is to provide fast access for temporary or short lived
files that do not need to be kept for an indeterminate time.
Keeping data in memory comes at a cost. It consumes system physical memory. Even with today‟s
large-memory systems, physical memory comes at a premium. A system or application that runs
out of available memory at a critical time can cause irreparable loss to the user. As a result, less
frequently used data or pages are paged out to a swap device. Under memory pressure, the
Virtual Memory (VM) sub-system can de-allocate MemFS pages and reassign them where needed.
The MemFS uses system swap device transparently to back the data from the reallocated pages.
When needed, this data is brought back into memory.
Implementation Goals
Disk based file system caches file system data in memory and a disk is used as persistent storage.
Based on different technologies, disk based file systems require metadata to be updated to the
disk either synchronously or asynchronously. Now consider a work load which continuously
creates or deletes intermediate file system objects. This kind of work load will generate a stream
of metadata update operations to disk. Though journaling can assist in reducing the updates, it
still needs metadata to be updated to disk. Due to slow disk transfer rate, performance of such
operations reduces.
RAM disk can be alternative for physical disks to overcome slow I/O rates. RAM disk allocates an
exclusive chunk of physical memory to host file system data to operate as a device. The I/O to
RAM disk is different from I/O to hard disk in a sense that RAM disk driver uses memory to
memory copy but hard disk drivers use direct memory access(DMA)for the data transfer. The RAM
disk has its own limitations:
 RAM disk requires exclusive allocation of physical memory. Memory is still considered as
a sacred resource and file systems already enjoy exclusive reservation of memory in the
form of buffer cache. So this exclusive allocation of memory for RAM disk is considered
an overhead and this overhead increases by the number of instances of RAM disk based
file systems.
 RAM disk uses two memory to memory copies, one from RAM disk to cache and another
from cache to user space.
 RAM disk implementation keeps multiple copies of data.
 RAM disk requires a free chunk of contiguous physical memory of the requested size to
successfully allocate the RAM disk.
On HP-UX 11iv2, MemFS was developed by reusing the Hierarchical disk based file system (HFS)
to treat memory as stream of blocks of specified file system size. As the memory sub-system does
not allow the file data pages to use the system swap device, each MemFS instance was
associated with a user process which pre-allocates virtual memory. Under system memory/buffer
cache pressure, the MemFS file data was swapped into the user process‟s address space via
MemFS pseudo swap driver, which in turn gets swapped on to system swap device by memory
management sub-system. As swapping in/out of data between kernel and user space is costly,
buffer cache was enhanced to keep data and metadata of MemFS files for longer time than other
file systems. Although this file system was substantially faster than any disk based and RAM-disk
based file systems, it has several drawbacks, largely due to the limitations of disk based file
system layout (HFS) and HP-UX memory subsystem architecture. The drawbacks are as follows:
 Number of files that can be created on a MemFS instance is restricted by file system size
rather than the available memory, as HFS dedicates some fixed number of blocks to
inodes.