Users Guide

Broadcom DRVLin-UG128-100
111
Emulex Drivers for Linux User Guide
Appendix C: Native NVMe Multipathing
Two multipathing options can be used with NVMe storage: DM and native NVMe multipathing.
DM is a general block-device multipathing framework that can work with devices of any protocol. DM supports the addition
of hardware-specific handlers to help manage the storage device and its path management. DM has been used extensively
with SCSI storage. DM can be used in its generic form with NVMe storage. However, DM currently does not support
advanced NVMe features, such as ANA support. Refer to the operating system release documentation for the level of DM
and NVMe storage integration supported.
Native NVMe multipathing is a kernel-level feature that can be enabled to add an NVMe-specific multipath module to the
kernel, which automatically detects subsystems and manages paths to the subsystems. The NVMe-specific module
supports the NVMe ANA feature. Inclusion and enablement of the Native NVMe Multipath module is
operating-system-specific. Refer to the operating system release documentation for the level of native NVMe multipathing
supported.
Native NVMe multipathing is included, but is not enabled by default, in SLES 12 SP4 and SLES 15 and later operating
systems. To enable native NVMe multipathing on SLES 12 SP4 and SLES 15 and later initiators, add
nvme-core.multipath=Y in the kernel command line, either in the grub boot loader or during bootup.
To verify whether native NVMe multipathing is enabled, run the following command:
# cat /sys/module/nvme_core/parameters/multipath
If native NVMe multipathing is enabled, the output displayed is Y. If native NVMe multipathing is not enabled, the output
displayed is N.
When enabled, native NVMe multipathing changes the manner in which NVMe devices are presented and related in
/sys/class/nvme, especially in cases where multiple paths to target namespaces exist. These changes significantly
affect the views generated by the kernel and system utilities, such as nvme-cli. This appendix describes these new views.
When native NVMe multipathing is not enabled, if an NVMe device is connected to a subsystem, a controller element is
created. If the subsystem is connected using multiple paths, multiple controller elements are created. Each controller
element is considered a unique and separate storage entity, even though the subsystem and namespaces that the controller
could access might be the same.
When native NVMe multipathing is enabled, controller elements are still created for each connection to a subsystem.
However, when a controller is created, the subsystem is compared to the list of subsystems that have already been found
in the system, and one of the following actions occurs:
 If the subsystem is not found, a new subsystem element is created and added to the system list, with the controller
linked to it as a path.
 If the subsystem is found, the controller is linked to the subsystem as an additional path.
Namespaces are elements of the NVMe subsystem and not of the NVMe controller. A controller acts as a communication
port to a subsystem. If a subsystem contains two controllers (communication ports), and namespace 1 is changed by
controller A, controller B sees this change as an action on namespace 1 because the namespace is a singular component
of the subsystem.