Keeping Researchers Connected
CERN IT simplifies HPC cluster connectivity with 10GbE iWARP from Intel
The European Organization for Nuclear Research, CERN, is the world's largest particle physics laboratory, located near Geneva, Switzerland.
Its IT department is tasked with providing the information technology the laboratory needs to carry out its complex and innovative research
both efficiently and effectively. This requires that the department not only develop leading research facilities, but also that it engineer and
maintain a world-class infrastructure on which to run them.
CASE STUDY
Intel®Ethernet 10 Gigabit Server Adapter
Performance: Data-Intensive Computing
Keeping Researchers Connected
CHALLENGES
• Drive performance. Boost research application results by enhancing cluster connectivity
• Simplify management. Reduce the complexity involved in managing network connections
• Save money. Create a more cost-effective connectivity model
SOLUTIONS
• Simplified network fabric. Replace the existing InfiniBand* fabric with 10GB Ethernet
using Intel® Ethernet 10 Gigabit Server Adapters
• High-performance compute. Improve application performance with optimized compute
in server clusters based on Intel® Xeon® processor 5500 series
IMPACT
• Optimal performance. Latency and application performance meet high-performance
application requirements
• Efficient user experience. More flexible cluster connectivity makes resource allocation
quicker and easier to manage, keeping the IT team and users happy
• Reduced total cost of ownership. The platform is simpler to maintain, enabling CERN IT
to reduce TCO and focus on other areas
Complex applications
One of CERN IT’s most important responsibilities is ensuring that the various engineering
teams based at the facility have the tools they need to carry out calculations and simulations
most effectively. It operates a wide variety of applications covering computational fluid
dynamics (CFD), electronic data automation (EDA), structural and thermal analysis, and many
other workloads.
A typical application is the CFD* software used for simulating heat distribution inside CERN’s
particle accelerators and Large Hadron Collider* experiments. This is an essential process to
ensure that some of the facility’s most challenging and high-profile experiments are carried
out successfully.
These applications are often very complex, processing large volumes of data. They require
a strong server platform to generate the best results — including excellent processing
performance and optimized network connectivity. In terms of networking, latency and
bandwidth are particularly crucial factors.
Many of the key applications, including STAR-CCM+* (CFD), GdfidL* (electromagnetic field
calculations), ORBIT* (beam dynamics simulation), and Pelegant* (accelerator simulation), are
run on a specialized server cluster, which previously used InfiniBand* adapters to provide
connectivity.
The team managing the servers, headed by Helge Meinhard, group leader of the Platform
and Engineering Services Group, wanted to replace the six-year-old cluster with newer,
more powerful processor technology based on 10GB Ethernet Internet Wide-Area RDMA
Protocol (iWARP) connectivity. “We wanted to simplify the connectivity we used, but also
to improve the energy-efficiency and cost-effectiveness of our computing cluster,” explains
Meinhard. Another important criterion was that the transition itself was as smooth and
seamless as possible.
“With reduced maintenance
complexity, we don’t need to
invest as many resources in
day-to-day upkeep of the cluster.
Instead, we can focus on making
sure it’s optimized to deliver the
absolute best results for our
research teams.”
Helge Meinhard,
Group leader, Platform and Engineering
Services Group, IT Department, CERN