Concept Guide

Dell EMC BeeGFS Storage Solution Reference Architecture
12 Dell EMC Ready Solutions for HPC BeeGFS High Performance Storage | ID 460
3.6 Advantages of using NVMe devices in the R740xd servers
Figure 7 shows the storage layout in the Dell EMC PowerEdge Enterprise Servers.
Storage Layout in the Dell EMC Enterprise Servers
Use of the NVMe devices in the solution provides the following distinct advantages:
1. While the legacy storage protocols such as SCSI, SAS and SATA communicate with the host using
an integrated on-chip or an external Host Bus Adapter (HBA), the NVMe based devices connect
through the PCIe interface as shown in Figure 1. A single Gen 3 PCIe lane allows transfers up to 985
MB/s. The P4600 SSDs used in the solution are x4 PCIe Gen 3 devices and so a x4 PCIe connection
supports close to 4 GB/s.
2. The I/O paths of traditional SATA/SAS SSDs and NVMe SSDs are different. In case of a traditional
I/O path, an I/O request arriving at the block layer is first inserted into a request queue (Elevator).
The elevator then reorders and combines multiple requests to sequential requests. The most
commonly used elevator scheduler for flash media is the noop scheduler, which implements a First in
First out (FIFO) policy without any reordering. An NVMe request bypasses the conventional block
layer queue. Instead it uses a paired submission and completion queue mechanism and offers
latency savings by removing the scheduler and queuing bottlenecks from the block layer.
3. NVMe standard supports up to 64 K I/O queues and up to 64K commands per queue. The large pool
of NVMe queues provides higher levels of I/O parallelism.
4. After a CPU writes a command to the tail of the submission queue, the CPU finishes the I/O request
and can continue servicing other requests. Thus, the CPU time spent to issue an I/O request is
significantly less in the case of NVMe drives unlike in case of SATA/SAS drives, where the CPU is
responsible to pass the request through several functions before the command is finally issued to the
device.