Grid computing

Grid computing
Name	Grid computing
Caption	Distributed computational resources connected across networks
Introduced	1990s
Designers	Ian Foster, Carl Kesselman
Developers	Globus Alliance, European Grid Infrastructure, Open Grid Forum

Grid computing is a distributed computing paradigm that coordinates heterogeneous computational, storage, and network resources across multiple administrative domains to solve large-scale problems. It emphasizes resource sharing, collaboration among institutions, and federated infrastructure to support computational science, engineering, and data-intensive research. Implementations often involve virtualization, scheduling, authentication, and high-performance networking to connect research centers, laboratories, and universities.

Overview

Grid computing integrates resources provided by institutions such as Argonne National Laboratory, CERN, Lawrence Berkeley National Laboratory, Oak Ridge National Laboratory, and National Center for Supercomputing Applications into shared infrastructures. Early projects included TeraGrid and European Grid Infrastructure, while coordination and policy were advanced by organizations like the Open Grid Forum and the Globus Alliance. Grid deployments rely on standards and protocols developed in collaboration with entities such as Institute of Electrical and Electronics Engineers, Internet Engineering Task Force, and national research and education networks like SURFnet and JANET.

Architecture and Components

Typical grid architectures separate concerns into layers involving resource providers such as IBM, HP, Dell EMC clusters and specialized facilities like Los Alamos National Laboratory supercomputers, middleware stacks from projects such as Globus Toolkit and UNICORE, and user-facing portals and science gateways developed by groups including TeraGrid partners and XSEDE. Architectural components include resource brokers and schedulers exemplified by software from Condor Project (now HTCondor), data management systems influenced by SRM specifications and tools like dCache, and information services inspired by LDAP directories and DNS naming. Networking underpinnings frequently use high-speed links provided by ESnet and GÉANT to support wide-area distributed execution across campuses and national labs.

Middleware and Standards

Middleware frameworks such as the Globus Toolkit, UNICORE, and gLite implement core services for job submission, data movement, security, and monitoring, interacting with standards from bodies including the Open Grid Forum, OASIS, and the World Wide Web Consortium. Security standards often rely on technologies like X.509 certificates and protocols from the IETF such as TLS for secure communication. Resource description and discovery use schemas and registries influenced by OGSA and specifications that integrate with identity federations such as Shibboleth and eduGAIN. Interoperability efforts have been coordinated with projects like Globus Alliance and regional infrastructures including Nordugrid and Swiss National Supercomputing Centre.

Applications and Use Cases

Grids have supported simulation and modelling in collaborations involving NASA centers, climate research teams at NOAA, and particle physics experiments at CERN (including Large Hadron Collider analyses). Bioinformatics pipelines run on federated clusters at institutions such as European Bioinformatics Institute and Broad Institute, while astronomy surveys from projects like LSST and Square Kilometre Array use distributed data processing across centers including National Radio Astronomy Observatory. Other use cases include virtual organizations coordinating among European Southern Observatory partners, seismic analysis for agencies like USGS, and multinational collaborations supported by frameworks used in ITER and Human Genome Project-related infrastructures.

Security, Privacy, and Trust

Security and trust models in grid environments employ credential management using X.509 certificates issued by certificate authorities such as DigiCert-style CAs in research networks, and federated identity systems like Shibboleth and eduGAIN for cross-institution authentication. Authorization mechanisms leverage role and attribute systems developed in projects linked to the Open Grid Forum, while secure data transfer uses protocols from the IETF and implementations from middleware such as Globus’s GridFTP. Governance and policy models have been informed by practices at European Grid Infrastructure, TeraGrid, and national laboratories like Argonne National Laboratory, addressing compliance with regional regulations encountered by organizations such as European Commission research programs.

Performance, Scalability, and Resource Management

Performance tuning and scalability strategies draw on scheduling algorithms from systems such as HTCondor, workload management approaches pioneered in TeraGrid and XSEDE, and data locality methods inspired by large-scale projects at CERN and Lawrence Livermore National Laboratory. Resource management integrates cluster managers like SLURM and Torque with grid-level brokers and meta-schedulers developed by Globus Alliance collaborators and research teams at University of Chicago consortiums. Network performance optimizations often use high-performance fabrics and QoS planning coordinated with providers like ESnet and GÉANT to enable low-latency, high-throughput transfers for workflows from experiments such as LIGO and computational efforts at Los Alamos National Laboratory.

History and Evolution of Grid Computing

Foundational ideas emerged in the 1990s from researchers including Ian Foster and Carl Kesselman and matured through projects like the Globus Toolkit, TeraGrid, and gLite. Subsequent evolution saw integration with cloud concepts advanced by companies and institutions such as Amazon Web Services, Google, and Microsoft Research, and coordination with regional initiatives like European Grid Infrastructure and Nordugrid. Standards bodies including the Open Grid Forum and collaborations with the IETF shaped interoperability, while the rise of virtualization and containerization technologies from organizations like Docker, Inc. and Kubernetes-related communities influenced modern distributed computing paradigms. Contemporary infrastructures blend grid principles with cloud-native platforms in efforts at XSEDE, national laboratories including Oak Ridge National Laboratory, and multinational science projects such as CERN collaborations.

Category:Distributed computing