White Papers

ManualsBrandsDell ManualsConverged InfrastructureHigh Performance Computing Solution Resources

Unbalanced Memory Performance

Joseph Stanfield Sept. 26 2012

It is well understood that a server with a “balanced” memory configuration yields the best performance for your

servers (See Memory Selection Guidelines for HPC and 11G PowerEdge Servers). Balanced implies that all

memory channels of the server are populated equally and with identical memory modules (DIMMS). But there

are certain situations where an unbalanced configuration might be needed. Cost limitations, capacity

requirements, and application needs area all possible factors. This blog will provide a brief overview of how to

gain the best performance from an unbalanced memory configuration.

To better understand the demerits of unbalanced configurations and to determine which unbalanced

configuration is the best, several tests were conducted in our lab. We have seen many requests for servers

configured with 48GB and 96GB of memory. Satisfying these capacity requirements on the latest generation of

servers that have four memory channels per socket is only possible with unbalanced configurations. Using the

available 2GB, 4GB, 8GB and 16GB DIMMs, we tested the configurations described below.

For the purpose of this study, a Dell PowerEdge M620 was used with the following configuration:

Figure 1: PowerEdge M620 configuration and memory used for testing.

Two capacity tests (48GB and 96GB) were performed with eight different memory organization options using

the STREAM memory bandwidth benchmark. Due to the similarities in memory channel population and

benchmarking results, this blog will focus on the 96GB options. For a comparison of the capacities tested, see

the figure 6 at the end of the blog. All results report the total measured system memory bandwidth.

The first test utilized fully populated memory banks across all four channels (see figure 2). Each CPU in this case

supports up to 3 DIMMs per channel but, a maximum capacity configuration reduces the speed at which the

memory operates significantly, impacting the overall performance as evident by the result.

Dual CPU

Intel Xeon E5-2680 @ 2.70GHz

BIOS

1.1.2

CPLD

1.0.2

iDRAC Version

1.06.06

Node Interleaving

Disabled

Memory Mode

Optimized

Memory Used For

Testing

2GB 1Rx8 @1600 MT/s

4GB 2Rx8 @1600 MT/s

8GB 2Rx4 @1600 MT/s

16GB 2Rx4 @1600 MT/s

Summary of content (5 pages)

PAGE 1
Unbalanced Memory Performance Joseph Stanfield Sept. 26 2012 It is well understood that a server with a “balanced” memory configuration yields the best performance for your servers (See Memory Selection Guidelines for HPC and 11G PowerEdge Servers). Balanced implies that all memory channels of the server are populated equally and with identical memory modules (DIMMS). But there are certain situations where an unbalanced configuration might be needed.
PAGE 2
Option CPU1 CPU2 Triad 12x4GB 12x4GB 56GB/s Figure 2 For the remaining seven unbalanced tests, three of the options were unbalanced across processors and four were unbalanced across memory channels. Balanced Configuration Reference Before we began the unbalanced testing, we needed a reference point and some actual STREAM results from a balanced configuration. Six valid configurations were benchmarked which gave us an average benchmark outcome to baseline against when testing the unbalanced options.
PAGE 3
The table below shows the exact configurations used for each option and the corresponding STREAM Triad results. Option CPU1 CPU2 Triad 8x8GB 8x4GB 75GB/s 8x8GB 4x8GB 77GB/s 4x16GB 4x8GB 76GB/s Unbalanced Across Channels The unbalanced channel configurations that were tested had partially populated memory channels resulting in bandwidth bottlenecks. Figure 5 shows an example of unequally populated memory channels.
PAGE 4
3x16GB 3x16GB 62GB/s As mentioned earlier, 48GB capacity configurations were also tested with the eight organization options populated similarly to the 96GB tests. The DIMM channels were populated similarly between both sets of tests with smaller capacity DIMMs used for the 48GB tests. The results followed similar trends, with the exception of Option 5 on the 48GB test. The 48GB benchmark mixed 2GB and 4GB DIMM capacities with mixed ranks resulting in a lower performance.
PAGE 5
http://content.dell.com/us/en/enterprise/d/business~solutions~whitepapers~en/Documents~11gmemory-selection-guidelines.pdf.