White Papers

ManualsBrandsDell ManualsConverged InfrastructureHigh Performance Computing Solution Resources

Need for Speed: Comparing FDR and EDR InfiniBand (Part 2)

By Olumide Olusanya and Munira Hussain

This is the second part of this blog series. In the first part, we shared OSU Micro-Benchmarks (latency

and bandwidth) and HPL performance between FDR and EDR Infiniband. In this part, we will further

compare performance using additional real-world applications such as ANSYS Fluent, WRF, and NAS

Parallel Benchmarks. For my cluster configuration, please refer to part 1.

Results

ANSYS Fluent

Fluent is a Computational Fluid Dynamics (CFD) application used for engineering design and analysis. It

can be used to simulate the flow of fluids, with heat transfer, turbulence and other phenomena,

involved in various transportation, industrial and manufacturing processes.

For this test we ran Eddy_417k which is one of the problem sets from ANSYS Fluent Benchmark suits. It

is a reaction flow case based on the eddy dissipation model. In addition, it has around 417,000

hexahedral cells and is a small dataset with a high communication overhead.

Figure 1 - ANSYS Fluent 16.0 (Eddy_417k)

From Figure 1 above, EDR shows a wide performance advantage over FDR as the number of cores

increase to 80. We continue to see an even wider difference as the cluster scales. While FDR’s

performance seems to gradually taper off after 80 cores, EDR’s performance continues to scale as the

number of cores increase and performs 85% better than FDR on 320 cores (16 nodes).

3050

4975

8882

10916

11799

3044

4987

9250

14961

21784

5000

10000

15000

20000

25000

20 40 80 160 320

Solver Rating

Number of cores

ANSYS Fluent 16.0

(Eddy_417k)

FDR EDR

HIGHER IS BETTER

Summary of content (5 pages)

PAGE 1
Need for Speed: Comparing FDR and EDR InfiniBand (Part 2) By Olumide Olusanya and Munira Hussain This is the second part of this blog series. In the first part, we shared OSU Micro-Benchmarks (latency and bandwidth) and HPL performance between FDR and EDR Infiniband. In this part, we will further compare performance using additional real-world applications such as ANSYS Fluent, WRF, and NAS Parallel Benchmarks. For my cluster configuration, please refer to part 1.
PAGE 2
WRF (Weather Research and Forecasting) WRF is a modelling system for weather prediction. It is widely used in atmospheric and operational forecasting research. It contains two dynamic cores, a data assimilation system, and a software architecture that allows for parallel computation and system extensibility. For this test, we are going to study the performance of a medium size case, Conus 12km. Conus 12km is a resolution case over the Continental US domain.
PAGE 3
data movement. For my test, we ran four of these benchmarks: CG, MG, FT, and IS. In the figures below, the performance difference is in an oval right above the corresponding run. CG 7.5% jobs/day 2000 1669 1500 973 995 1000 500 1790 266 266 496 485 0 32 64 128 256 Number of cores FDR EDR MG 1.
PAGE 4
FT 7.5% jobs/day 2000 1518 1632 1500 1000 500 780 837 417 393 0 64 128 256 Number of cores FDR EDR jobs/day IS 30000 25000 20000 15000 10000 5000 0 16% 12% 4628 5031 32 7826 8819 26182 22539 14203 12725 64 128 256 Number of cores FDR EDR Figure 5 - FT Figure 6 - IS CG is a benchmark which computes an approximation of the smallest eigenvalue of a large, sparse, symmetric positive-definite matrix using a conjugate gradient method.
PAGE 5
Conclusion In both blogs, we have shown several micro-benchmark and real-world application results to compare FDR with EDR Infiniband. From these results, EDR has shown a higher performance and better scaling than FDR on our 16-node Dell PowerEdge C6320 cluster. Also, some applications have shown a wider performance margin between these interconnects than other applications.