DOEGrids

DOEGrids
Name	DOEGrids
Formation	2000s
Type	Consortium
Headquarters	United States
Region served	National laboratories, universities, research centers
Leader title	Director
Parent organization	United States Department of Energy

DOEGrids

DOEGrids is a distributed computational and data infrastructure initiative that connects Argonne National Laboratory, Lawrence Berkeley National Laboratory, Oak Ridge National Laboratory, Los Alamos National Laboratory, and other national laboratories to support large-scale scientific computing and data-intensive research. It provides middleware, high-throughput computing, data management, and identity services that enable collaboration among researchers at University of California, Berkeley, Massachusetts Institute of Technology, Stanford University, Fermi National Accelerator Laboratory, and international partners. DOEGrids integrates with supercomputing facilities such as Summit (supercomputer), Frontier (supercomputer), NERSC, and with community projects like Open Science Grid, Globus and HTCondor.

Overview

DOEGrids serves as an operational and organizational framework linking computational resources across national laboratories, research universities, and science facilities including Brookhaven National Laboratory, Pacific Northwest National Laboratory, Sandia National Laboratories, Princeton Plasma Physics Laboratory, and SLAC National Accelerator Laboratory. It emphasizes interoperability with middleware stacks from Globus Toolkit, GridFTP, gLite, and tools used by collaborations such as CMS (particle detector), ATLAS experiment, LIGO Scientific Collaboration, and the Human Genome Project. The initiative supports workloads ranging from high-performance computing jobs on systems like Titan (supercomputer) to data staging for observatories such as the Large Synoptic Survey Telescope and the James Webb Space Telescope instrument teams. DOEGrids enables federated identity and access interoperability with services provided by InCommon, ORCID, Internet2, and ESnet.

History and Development

The origins of the DOEGrids concept trace to early 2000s federated computing efforts linking DOE laboratories with academic partners like University of Chicago, Columbia University, Harvard University, and University of Michigan. Early pilots engaged middleware from the Globus Alliance and coordination with projects such as the Open Science Grid and TeraGrid, and leveraged networking by ESnet to prototype cross-site job execution for experiments like ATLAS experiment and computational programs in Oak Ridge National Laboratory and Argonne National Laboratory. Over successive phases, integration efforts incorporated identity frameworks used by InCommon and resource allocation models similar to those in XSEDE, while adopting security practices tested in collaborations with National Nuclear Security Administration facilities and programs overseen by the Office of Science (DOE). Partnerships expanded to include universities such as University of Texas at Austin and University of Illinois Urbana–Champaign to support distributed workflows for projects like NOAA-linked climate modeling and electric grid studies referencing California Independent System Operator datasets.

Architecture and Infrastructure

DOEGrids combines compute, storage, and network layers with middleware orchestration. Compute resources span heterogeneous systems from massively parallel supercomputers such as Summit (supercomputer) to campus clusters at University of California, San Diego and batch systems using Slurm Workload Manager, HTCondor, and scheduler integrations akin to those in Blue Waters. Storage and data movement rely on parallel and object stores comparable to implementations at NERSC and Oak Ridge Leadership Computing Facility, with data transfer using GridFTP and managed replication strategies inspired by Globus and SRM specifications. The network backbone leverages high-performance research networks such as ESnet, Internet2, and regional carriers connected to science gateways like SURA-affiliated campuses. Monitoring and operational tooling integrate standards from Open Grid Forum and logging systems used by National Energy Research Scientific Computing Center and other DOE facilities. Identity and access employ federated authentication approaches interoperable with InCommon and credential management influenced by practices at ORCID and CILogon.

Applications and Services

DOEGrids supports a wide spectrum of science domains and institutional services. Its platform hosts workflows for experimental collaborations including LIGO Scientific Collaboration, ATLAS experiment, and CMS (particle detector), as well as computational science in areas linked to National Renewable Energy Laboratory research and Princeton Plasma Physics Laboratory fusion studies. Services include job submission gateways used by researchers at Yale University, University of Wisconsin–Madison, and Duke University; data archival and curation for projects aligned with National Oceanic and Atmospheric Administration and NASA mission teams; and science gateways for communities represented by Genome Institute researchers and climate consortia tied to NOAA. Additional offerings mirror commercial cloud patterns but with governance for federally funded science, complementing clouds like Amazon Web Services and Microsoft Azure where needed for burst capacity.

Security and Access

Security in DOEGrids follows federal compliance models and laboratory practices including risk management frameworks similar to standards used by National Institute of Standards and Technology and directives aligned with the Office of Management and Budget. Authentication and authorization use federated identity through providers such as InCommon and credential broker systems analogous to CILogon; certificate-based mechanisms draw on Public Key Infrastructure approaches practiced at NERSC and DOE labs. Data confidentiality, integrity, and availability controls reflect procedures from Department of Energy directives and operational playbooks employed at Los Alamos National Laboratory and Lawrence Livermore National Laboratory. Incident response and threat intelligence coordination occur with organizations like US-CERT and regional computer emergency response teams associated with ESnet.

Governance and Funding

Governance of the DOEGrids ecosystem involves coordination among DOE program offices, laboratory management from Brookhaven National Laboratory, Argonne National Laboratory, and Oak Ridge National Laboratory, and academic partners such as University of California, Berkeley and Massachusetts Institute of Technology. Funding streams derive from DOE Office of Science allocations, cooperative agreements with institutions like University of Chicago and Princeton University, and occasionalsponsored projects involving agencies such as National Science Foundation and mission partners like NASA. Strategic decisions are informed by advisory panels drawing expertise from leaders of Open Science Grid, XSEDE, and representatives of large science collaborations including LIGO Scientific Collaboration and ATLAS experiment.

Category:United States Department of Energy