White Papers

ManualsBrandsDell ManualsConverged InfrastructureHigh Performance Computing Solution Resources

Ready Solutions Engineering Test Results

the property of their respective owners. Published in the USA. Dell EMC believes the information in this document is accurate as of its publication date. The information is

subject to change without notice.

Cryo-EM – Relion benchmarks on Skylake

Optimized Relion Tests on Dell EMC PowerEdge C6420s with Dell EMC Isilon

Overview

The purpose of this blog is to validate the optimized Relion (for REgularised LIkelihood OptimizatioN) on Dell EMC PowerEdge C6420s

with Skylake CPUs. Relion was developed from the Scheres lab at MRC Laboratory of Molecular Biology. It uses an empirical Bayesian

approach to refining multiple 3D images or 2D class averages for the data generated from CryoElectron Microscopy (Cryo-EM). The

impressive performance gain from Intel

’s efforts in the collaboration of Relion development team reduced the performance gap

between CPUs and GPUs. The CPU/GPU performance comparison results are not shown here; however, the performance gap

becomes single digit fold between Skylake CPU systems and Broadwell CPU/Tesla P100 GPU systems.

Essentially, Cryo-EM is a type of transmission electron microscopy (TEM)

for imaging frozen-hydrated specimens at cryogenic

temperatures

. Specimens remain in their native state without the need for dyes or fixatives allowing the study of fine cellular structures,

viruses and protein complexes at molecular resolution. A rapid vitrification at cryogenic temperature is the key step to avoid water

molecule crystallization and forming amorphous solid that does almost no damage to the sample structure. Regular electron

microscopy requires samples to be prepared in complex ways, and the sample preparations make hard to retaining the original

molecular structures. Cryo-EM is not perfect like X-ray crystallography; however, it gains the popularity in the research community fast

due to the simple sample preparation steps and flexibility to the sample size, complexity, and non-rigid structure. As the resolution

revolution in Cryo-EM progresses due to the 40+ years of dedicated work from the structural biology community, we now have the

ability to yield accurate, detailed, 3D models of intricate biological structures at the sub-cellular and molecular scales.

The tests were performed on 8 nodes of Dell PowerEdge C6420s which is a part of Dell EMC Ready Bundle for HPC Life Sciences. A

summary of the configuration is listed in Table 1.

Table 1 Test Cluster Configurations

Dell EMC PowerEdge C6420

CPU

2x Xeon® Gold 6148F 20c 2.4GHz (Skylake)

RAM

12x 32GB @2666 MHz

RHEL 7.3

Interconnect

Intel® Omni-Path

BIOS System Profile

Performance Optimized

Logical Processor

Enabled

Virtualization Technology

Disabled

Intel Compiler and Libraries

2017.4.196

GLIBC

2.17-196.el7

Summary of content (5 pages)

PAGE 1
Ready Solutions Engineering Test Results Cryo-EM – Relion benchmarks on Skylake Optimized Relion Tests on Dell EMC PowerEdge C6420s with Dell EMC Isilon Overview The purpose of this blog is to validate the optimized Relion (for REgularised LIkelihood OptimizatioN) on Dell EMC PowerEdge C6420s with Skylake CPUs. Relion was developed from the Scheres lab at MRC Laboratory of Molecular Biology.
PAGE 2
Ready Specs The test clusters and F800/H600 storage systems were connected via 4 x 100GbE links between two Dell Networking Z9100-ON switches. Each of the compute nodes was connected to the test cluster side Dell Networking Z9100-ON switch via single 10GbE. Four storage nodes in the Dell EMC Isilon F800/H600 were connected to the other switch via 8x 40GbE links. The configuration of the storage is listed in Table 2. The detailed network topology is illustrated in Figure 1.
PAGE 3
Ready Specs 3 Table 2 Storage Configurations Dell EMC Isilon F800 Dell EMC Isilon H600 4 4 Intel® Xeon™ CPU E5-2697A v4 @2.60 GHz Intel® Xeon™ CPU E5-2680 v4 @2.40GHz 256GB 256GB Total usable space: 166.8 TB, 41.7 TB per node Total usable space: 126.8 TB, 31.7 TB per node N/A 2.9 TB per node Network Front end network: 40GbE Back end network: 40GbE Front end network: 40GbE Back end network: IB QDR OS Isilon OneFS v8.1.0 DEV.0 Isilon OneFS v8.1.0.
PAGE 4
Ready Specs 4 Figure 2 Optimized Relion Benchmark Comparisons with Intel®’s results and Dell EMC PowerEdge C6420. The storage used for the tests were Dell EMC Isilon, F800 and H600. Intel’s results shown here were using Lustre as a storage system; However, we were not able to run our tests with Dell EMC Lustre Storage due to the limited availability. Performance Comparisons of Relion without the optimization Figure 3 shows Relion performances without the code optimization.
PAGE 5
Ready Specs 5 Figure 3 Relion performance comparisons with CPU and GPU Conclusion Dell EMC PowerEdge C6420 shows that it is an ideal compute platform for the Optimized Relion. It scales well over various number of compute nodes with Plasmodium ribosome data. In the future study, we plan to use a larger protein data and more compute nodes to accomplish more comprehensive scaling tests. Resources Internal web page External web page Contacts Americas Kihoon Yoon Sr. Principal Systems Dev Eng Kihoon.