Best Practices When Deploying Linux on the HP ProLiant DL980 (updated December 2012)

Where:

 numa_hit is the number of successful allocations for a node.

 numa_miss is the number of failed allocations for a node because of low memory (allocations are successful on

another node).

 numa_foreign is the number of allocations that were intended for another node but ended up on the local node.

 local_node is incremented when a process running on the local node has allocated memory on the same node.

 other_node is incremented when a process running on another node allocated memory on that node.

 interleave_hit is the number of interleave policy allocations that were intended for a specific node and

succeeded there.

Controlling tasks and memory object placement

Best performance is obtained when tasks and their associated memory objects are close together, preferably in the

same node. To control the placement of memory objects and task threads, use either the numactl command or the

cpuset command.

Using the numactl command

For example, the following command binds a shell, its children processes and associated memory objects to node 1:

[root@tux ~]# numactl --membind=1 --cpunodebind=1 bash

[root@tux ~]# numactl --show

policy: bind

preferred node: 1

physcpubind: 10 11 12 13 14 15 16 17 18 19

cpubind: 1

nodebind: 1

membind: 1

The numactl command uses the NUMA policy library (libnuma) interface for policy retrieval and control. For more

information, see the numactl(8) and numa(3) manpages. The original A NUMA API for Linux white paper is at:

halobates.de/numaapi3.pdf

Using the cpuset command

The interface to cpusets is the cpuset pseudo-filesystem, which was first introduced on Red Hat Enterprise Linux 5 and

SUSE Linux Enterprise Server 10 and remains available today. Once the pseudo-file system is mounted, you can use the

cpuset command to partition system resources by creating subdirectories under the mount point. These

subdirectories, or cpusets, contain pseudo-files and are used for specifying resources such as the logical CPUs allowed

and the memory nodes.

In the following example, the first command verifies that the kernel is aware of the cpuset subsystem. The next

commands mount the subsystem and list the pseudo-files automatically created in a subdirectory called mycpuset.

The echo commands place logical processors 2 and 3 and memory node 1 into this cpuset, and attach the current shell

to it.

[root@tux ~]# grep cpuset /proc/filesystems

nodev cpuset

[rorot@tux ~]# mkdir /dev/cpusets

[root@tux ~]# mount –t cpuset none /dev/cpusets

[root@tux ~]# mkdir /dev/cpusets/mycpuset ; cd /dev/cpusets/mycpuset

[root@tux ~]# ls

cpu_exclusive memory_pressure mems

cpus memory_pressure_enabled notify_on_release

mem_exclusive memory_spread_page sched_relax_domain_level

memory_migrate memory_spread_slab tasks

[root@tux ~]# echo 2-3 > cpus