Enabling Grids for E-sciencE

Enabling Grids for E-sciencE
Name	Enabling Grids for E-sciencE
Founded	2004
Dissolved	2010
Type	Research infrastructure project
Location	European Union
Key people	Ian Foster, Tony Hey, John Taylor
Area served	European Research Area
Focus	Grid computing, e-Science, distributed computing

Enabling Grids for E-sciencE was a European Union-funded research project that aimed to build and deploy a production-quality grid infrastructure for large-scale European Research Area scientific collaboration. It connected research organizations, research councils, and infrastructure projects across CERN, European Space Agency, Max Planck Society, and national laboratories to enable distributed computation and data sharing for projects in particle physics, bioinformatics, and earth observation. The project coordinated technical, organisational, and policy work to integrate middleware, standards, and operational procedures with major international initiatives.

Overview

Enabling Grids for E-sciencE (EGEE) sought to provide a persistent grid service by federating compute and storage resources across institutions such as CERN, European Space Agency, INFN, Centre National de la Recherche Scientifique, and Deutsches Elektronen-Synchrotron. EGEE bridged research programs including Large Hadron Collider, Human Genome Project, European Bioinformatics Institute, International Thermonuclear Experimental Reactor, and Copernicus Programme by offering a common middleware stack, operational centre, and user support model. Collaboration partners included GridPP, TERENA, PRACE, and national research networks. EGEE influenced follow-on initiatives like EGI and informed policy at the European Commission.

History and Development

EGEE originated from preparatory work by projects such as DataGrid and CrossGrid and was funded under successive Framework Programmes of the European Union starting in 2004. Early milestones included integration with the Globus Toolkit and deployment of production services to support Large Hadron Collider experiments and bioinformatics consortia. EGEE phases were coordinated with partner projects like OMII and gLite development led by INFN. Governance involved boards with representatives from CERN, national research councils, and industrial partners including IBM, HP, and Sun Microsystems. EGEE’s operational model evolved into the European Grid Infrastructure and influenced international collaborations with Open Science Grid and Worldwide LHC Computing Grid.

Architecture and Components

The EGEE architecture combined resource providers, regional operations centres, and central services. Core components included workload management systems, information systems, data management tools, and security services integrated across sites such as CERN and national centres. EGEE used catalogues, replica management, and transfer tools to connect storage elements at institutions like EMBL-EBI and computing elements at Max Planck Society data centres. Monitoring and accounting were provided via systems coordinated with TERENA and national network operators. EGEE’s federated model allowed local resource managers such as PBS, Torque (software), and LSF (software) to interoperate with grid services.

Middleware and Standards

EGEE adopted middleware stacks derived from projects including Globus Toolkit, gLite, and integrations with Unicore. Security and authentication relied on X.509 certificates and the European Grid Certification Authority framework, aligning with standards set by bodies like OGF and IETF. Data transfer and replication used protocols compatible with GridFTP and standards influenced by W3C and Dublin Core for metadata. Service discovery and information systems referenced schemas from GLUE Schema community work. Interoperability efforts engaged with Open Grid Forum and liaison with IEEE and national standards organisations.

Use Cases and Applications

EGEE supported high-energy physics workloads for the Large Hadron Collider experiments, bioinformatics pipelines for groups at European Bioinformatics Institute and Wellcome Trust Sanger Institute, and earth observation processing for European Space Agency missions. Applications included Monte Carlo simulations for ATLAS and CMS, sequence alignment used by EMBL-EBI, climate modelling for groups associated with ECMWF, and medical imaging research involving Karolinska Institute collaborators. EGEE also facilitated virtual organisations linking researchers from Max Planck Society, CNRS, INFN, and national institutes.

Deployment and Operational Challenges

Operational challenges included scaling authentication and authorization across federated domains involving European Commission policy constraints, coordinating regional operations centres such as those in Italy, France, and Germany, and providing reliable production services to experiments like ATLAS and CMS. Interoperability problems arose when integrating diverse middleware from vendors like IBM and open-source projects such as Globus Toolkit. Resource heterogeneity, network performance variations across research networks including GÉANT and national backbones, and accounting reconciliation across centres such as CERN and national data centres required extensive operational procedures and incident response coordination.

Impact and Future Directions

EGEE’s legacy includes the formation of European Grid Infrastructure, influence on federated research infrastructure models, and contributions to standards adopted by Open Grid Forum and international grids like Open Science Grid. It demonstrated models for cross-border scientific collaboration used later by initiatives such as PRACE and influenced cloud-oriented follow-ons including Helix Nebula and research cloud pilots. Future directions traceable to EGEE encompass hybrid cloud-grid integration, federated identity systems influenced by work with eduGAIN, and continuing support for large-scale collaboration in projects like Square Kilometre Array and Human Cell Atlas.

Category:Grid computing