UK National Grid Service

UK National Grid Service
Name	UK National Grid Service
Formation	2004
Type	Research infrastructure
Headquarters	United Kingdom
Region served	United Kingdom
Services	High-performance computing, distributed computing, data storage
Parent organization	None

UK National Grid Service The UK National Grid Service provided a distributed e‑infrastructure for computational science, data analysis, and collaborative research across the United Kingdom. It connected university clusters, supercomputing centres, and bespoke resources to support projects in astronomy, bioinformatics, climate science, particle physics, and chemistry. The Service integrated middleware, user support, and policy frameworks to enable researchers from institutions such as University of Oxford, University of Cambridge, University of Edinburgh, Imperial College London, and University of Manchester to run scalable workloads across resources.

History

The initiative emerged from collaborative efforts among organisations including the Jisc, the Engineering and Physical Sciences Research Council, and the Science and Technology Facilities Council during the early 2000s, influenced by international projects such as TeraGrid, Open Science Grid, and the European Grid Infrastructure. Initial prototypes built on middleware stacks developed in projects like Globus Toolkit, gLite, and Globus Alliance deployments, with pilot sites at centres such as the Manchester Computing Centre and the Oxford e‑Research Centre. Major milestones included formal service launch in 2004, membership expansions through the late 2000s, and integration with national facilities including the DiRAC consortium and the Hartree Centre. The Service evolved alongside complementary UK programmes like GridPP and EU initiatives such as EGEE, adapting governance and technical models in response to shifts toward cloud computing exemplified by projects at Amazon Web Services research collaborations and the European Open Science Cloud.

Architecture and Services

The Service architecture combined resource federation, identity management, and job scheduling layers drawn from middleware such as Globus Toolkit, UNICORE, and components from Apache Hadoop and HTCondor. Core services included secure authentication and authorisation via standards associated with Shibboleth, X.509, and integration with institutional credentials at universities like University College London. Data transfer capabilities relied on protocols from GridFTP and tools developed in projects like iRODS and integrations with storage systems at the Rutherford Appleton Laboratory. Compute provisioning supported batch scheduling through PBS Professional, Slurm Workload Manager, and Sun Grid Engine, while user portals and science gateways were influenced by work at myGrid, Taverna (software), and the UK e‑Science programme. The Service offered bespoke support for workflows in domains including LOFAR, LHCb, Human Genome Project downstream analyses, and Met Office climate model ensembles.

Membership and Governance

Membership comprised universities, research institutes, and national laboratories, with participating nodes from institutions such as University of Leeds, University of Glasgow, Queen Mary University of London, University of Oxford Department of Physics, and the STFC. Governance combined a stakeholder board with technical advisory groups drawing expertise from bodies like the Research Councils UK and advisory inputs from centres such as the UK Research and Innovation community. Policy frameworks for access, fair usage, and data stewardship referenced standards and guidance from organisations including the Digital Curation Centre and compliance models aligned with legal instruments like the Data Protection Act 1998. Operational decisions were informed by community forums mirrored on international exemplars like the Open Grid Forum.

Funding and Partnerships

Initial and sustaining funding derived from public research bodies including the Joint Information Systems Committee, the Higher Education Funding Council for England, and project grants from the European Commission under programmes such as Framework Programme 6 and Framework Programme 7. Strategic partnerships extended to national initiatives like the National e‑Infrastructure Service and collaborations with technology vendors and supercomputing centres including the CERN computing grid collaboration and partnerships resembling commercial research agreements with companies active in HPC procurement. The Service also coordinated with UK consortia such as HEFCE-backed infrastructure projects and contributed to pan-European research infrastructures including PRACE.

Usage and Impact

Researchers across disciplines used the Service for high-throughput and high-performance computing, enabling publications in journals associated with organisations like the Royal Society and datasets curated with assistance from the British Library. Scientific achievements supported by the infrastructure span analyses in cosmology, computational studies aligned with the Human Genome Project downstream tools, simulations used by the Met Office, and data processing workflows for facilities such as the Square Kilometre Array pathfinder projects. Training activities and summer schools linked to programmes at UK e‑Science All Hands meetings, and contributions to workforce development were noted by university computing centres and professional bodies including the Institute of Physics.

Technical Infrastructure and Security

The technical stack emphasised interoperability, reliability, and security, employing federated identity via Shibboleth and X.509 PKI issued in coordination with certificate authorities similar to those used by EGI. Monitoring and logging practices adopted tools inspired by Nagios and Ganglia, while incident response integrated processes comparable to those of national CERTs such as CERT‑UK. Security assessments considered threat models addressed in guidance from organisations like the National Cyber Security Centre and aligned with compliance expectations of funders including the EPSRC. Storage architectures implemented redundancy and tape backup strategies employed at national laboratories including the Rutherford Appleton Laboratory mass data facilities.

Category:Research infrastructure in the United Kingdom