White Papers

ManualsBrandsDell ManualsConverged InfrastructureHigh Performance Computing Solution Resources

Measuring Performance of Intel

Broadwell Processors with High

Performance Computing

Benchmarks

Authors: Ashish Kumar Singh, Mayura Deshmukh and Neha Kashyap

The increasing demand for more compute power pushes servers to be upgraded with higher and more powerful

hardware. With the release of the new Intel® Xeon® processor E5-2600 v4 family of processors (architecture

codenamed “Broadwell”), Dell has refreshed the 13

generation servers to benefit from the increased number of cores

and higher memory speeds thus benefiting a wide variety of HPC applications.

This blog is part one of “Broadwell performance for HPC” blog series and discusses the performance characterization

of Intel Broadwell processors with High Performance LINPACK (HPL) and STREAM benchmarks. The next three blogs

in the series will discuss the BIOS tuning options and the impact of Broadwell processors on Weather Research

Forecast (WRF), NAMD, ANSYS® Fluent®, CD-adapco® STAR-CCM+®, OpenFOAM, LSTC LS-DYNA® HPC

applications as compared to the previous generation processor models.

In this study, performance was measured across five different Broadwell processor models listed in Table2 along with

2400 MT/s DDR4 memory. This study focuses on HPL and STREAM performance for different BIOS profiles across all

five Broadwell processor models and compares the results to previous generations of Intel Xeon processors. The

platform we used is a PowerEdge R730, which is a 2U dual socket rack server with two processors. Each socket has

four memory channels and can support up to 3 DIMMs per channel (DPC). For our study, we used 2 DPC for a total of

16 DDR4 DIMMs in the server.

Broadwell (BDW) is a tick in Intel’s tick-tock principle as the next step in semiconductor fabrication. It is a 14nm

processor with the same microarchitecture as the Haswell-based (HSW, Xeon E5-2600 v3 series) processors with the

same TDP range. Broadwell E5-2600 v4 series processors support up to 22 cores per socket with up to 55MB of LLC,

which is 22% more cores and LLC than Haswell. Broadwell supports DDR4 memory with max memory speed of up to

2400 MT/s, 12.5% higher than the 2133 MT/s that is supported with Haswell.

Summary of content (8 pages)

PAGE 1
Measuring Performance of Intel Broadwell Processors with High Performance Computing Benchmarks Authors: Ashish Kumar Singh, Mayura Deshmukh and Neha Kashyap The increasing demand for more compute power pushes servers to be upgraded with higher and more powerful hardware.
PAGE 2
Broadwell introduces a new snoop mode option in the BIOS memory setting, Directory with Opportunistic Snoop Broadcast (DIR+OSB), which is the default snoop mode for Broadwell. In this mode, the memory snoop is spawned by the Home Agent and a directory is maintained in the DRAM ECC bits. DIR+OSB mode allows for low local memory latency, high local memory bandwidth and I/O directory cache to reduce directory update overheads for I/O accesses.
PAGE 3
Intel Broadwell supports Intel® Advanced Vector Extensions 2 (Intel AVX2) vector technology, which allows a processor core to execute 16 FLOPs per cycle. HPL is a benchmark that solves a dense linear system. The HPL problem size (N) was chosen to be 177408 along with a block size (NB) of 192. The theoretical peak value of HPL was calculated using the AVX base frequency, which is lower than rated base frequency of the processor model.
PAGE 4
BIOS options System profile – Performance and Performance Per Watt (DAPC) Logical Processor – Disabled Power Supply Redundant Policy – Not Redundant Power Supply Hot Spare Policy – Disabled I/O Non-Posted Prefetch - Disabled Snoop modes – OSB, ES, HS and COD Node Interleaving - Disabled BIOS Firmware iDRAC Firmware HPL MPI STREAM 2.0.0 2.30.30.02 From Intel Parallel Studio 2016 update1 Intel MPI - 5.1.2 5.
PAGE 5
the efficiency of that processor. The “X” value on top of each bar shows the performance gain over 12 core Broadwell processor. The HPL performance improvement with top bin Broadwell processor is not correspondingly increasing as number of cores. For example, adding 83% more cores in top bin 22 core than 12 core Broadwell processor, allows HPL a 57% performance improvement. The line pattern on the graph shows the HPL performance per core.
PAGE 6
Figure 3: Comparing BIOS profiles with HPL Figure 3 plots HPL performance with two BIOS system profile options for all four snoop modes across all five Broadwell processors. As Directory + Opportunistic Snoop Broadcast (DIR+OSB) snoop mode performs well for all workloads and DAPC system profile balances performance and energy efficiency, these options are set as default in the BIOS and so has been chosen as the baseline.
PAGE 7
profiles are almost same. The memory bandwidth with Early snoop (ES) mode for both system profiles is lower by 8 to 20 % and the difference is more apparent for 22 core processor up to 25%. The Early Snoop (ES) mode have less Requester Transaction IDs (RTIDs) distributed across all the cores, while other snoop modes gets higher RTIDs, that is higher number of credits for local and remote traffic at the home agent.
PAGE 8
Figure 6: Generation over generation comparison with STREAM Figure 6 plots performance comparison of STREAM (TRIAD) for multiple generations of Intel processors. From the graph, it can be seen that the memory bandwidth on the system has increased over generations. The theoretical maximum memory frequency increased by 12.5% in Broadwell over Haswell (2133 MT/s to 2400 MT/s) and this translates into 10 to 12% better measured memory bandwidth as well.