Cray EX

Cray EX
Name	Cray EX
Manufacturer	Cray Inc.
Predecessor	Cray XC40
Successor	Cray Shasta
Operating system	Cray Linux Environment
Power	~1.5 MW (typical cabinet)

Cray EX. The Cray EX is a massively scalable, liquid-cooled supercomputer architecture developed by Cray Inc. (now part of Hewlett Packard Enterprise). It represents a significant evolution from prior systems like the Cray XC40, designed to support exascale computing workloads for government, academic, and industrial research. The architecture integrates advanced Slingshot networking and dense compute blades to deliver extreme performance for complex computational science and artificial intelligence applications.

Overview

The system is engineered for leadership-class high-performance computing facilities, such as those funded by the United States Department of Energy and the European High-Performance Computing Joint Undertaking. It builds upon Cray's deep heritage in vector processing and scalable systems, incorporating modern technologies to address challenges in climate modeling, nuclear fusion simulation, and genomics. Key design goals include exceptional energy efficiency through direct liquid cooling and robust integration with large-scale data storage systems like the Cray ClusterStor.

Architecture

The architecture is based on a modular, cabinet-scale building block utilizing the Shasta platform. Each cabinet houses multiple compute blades, which integrate next-generation x86 processors from AMD, specifically the AMD EPYC CPU and AMD Instinct GPU accelerators. The heart of the system's communication is the high-performance, Ethernet-compatible Slingshot interconnect, which provides low latency and high bandwidth for data-intensive workloads. This network topology supports advanced adaptive routing and congestion control, critical for massive MPI jobs. The entire system employs direct-to-chip warm water cooling, a technology refined from the Cray XC50, which dramatically reduces Power usage effectiveness and operational costs.

System Specifications

A standard cabinet can incorporate hundreds of compute nodes, each node typically configured with multiple AMD EPYC cores and several AMD Instinct GPU accelerators. Peak performance for a full-scale system can reach multiple petaflops, with memory bandwidth exceeding several terabytes per second. The Slingshot network offers port speeds of 200 gigabits per second and can scale to connect tens of thousands of endpoints. Power demands are substantial, with a single cabinet often requiring approximately 1.5 megawatts, though the liquid cooling system efficiently manages the resulting waste heat.

Software and Programming Environment

The primary operating environment is the Cray Linux Environment, a robust, scalable software stack derived from SUSE Linux Enterprise Server. Programming is supported through a comprehensive suite of tools including the Cray Compiling Environment, which features optimized Fortran, C, and C++ compilers. For parallel execution, the system fully supports OpenMP and MPI libraries, alongside frameworks for heterogeneous computing like ROCm for AMD accelerators. Management and monitoring are handled by the Cray System Management software, while workload scheduling is typically done via Slurm Workload Manager. The environment also integrates libraries such as the Cray Scientific Libraries and supports emerging paradigms like Kubernetes for containerized applications.

Deployment and Applications

Major deployments include systems at Lawrence Berkeley National Laboratory and the Finnish IT Center for Science (CSC). These supercomputers are tasked with grand challenge simulations in fields like astrophysics, exemplified by projects modeling the Big Bang, and materials science for discovering new pharmaceuticals. At institutions like the National Center for Atmospheric Research, the systems run advanced weather forecasting and climate change models. Further applications span computational fluid dynamics for aerospace engineering, cryptanalysis for national security agencies, and training large-scale neural networks for AI research.

Historical Context and Development

The development was driven by the global race toward exascale computing, notably influenced by programs like the United States Department of Energy's Exascale Computing Initiative and the European Union's EuroHPC strategy. It succeeded the Cray XC series and was contemporary with other exascale contenders like IBM's Power-based systems and Fujitsu's Fugaku. The acquisition of Cray Inc. by Hewlett Packard Enterprise in 2019 provided additional resources for its final development and market rollout. The architecture's design principles directly informed its successor, the Cray Shasta platform, which would form the basis for the first United States exascale systems, Frontier and Aurora.

Category:Supercomputers Category:Cray