OGSA-DAI

OGSA-DAI
Name	OGSA-DAI
Developer	UK e-Science Programme, National e-Science Centre
Released	2002
Programming language	Java
Operating system	Cross-platform
Genre	Data access, middleware, grid computing
License	Open source

OGSA-DAI is an open-source middleware project that provided distributed data access and integration services for grid and service-oriented infrastructures. Originally funded through the UK e-Science Programme and developed at the National e-Science Centre, the project focused on enabling applications to query, transform, and move data across heterogeneous sources using standardized service interfaces. OGSA-DAI influenced subsequent work on data fabrics and service-based data management in scientific, governmental, and commercial deployments.

Overview

OGSA-DAI was designed to bridge relational stores such as MySQL, PostgreSQL, Oracle Database, and Microsoft SQL Server with grid and service frameworks like Globus Toolkit and Apache Axis. The project emphasized standards from bodies including the Open Grid Forum and mechanisms compatible with Web Services Resource Framework and OGSA-style specifications. Through service-orientation, OGSA-DAI addressed needs observed in large-scale efforts such as Large Hadron Collider experiments, Human Genome Project collaborations, and environmental projects affiliated with European Space Agency missions. The software's Java implementation leveraged enterprise toolchains common to institutions such as CERN, NASA, and national laboratories in the United Kingdom and United States.

Architecture and Components

The architecture centered on a service container hosting distributed data services that exposed operations over SOAP and later RESTful bindings. Core components included the Data Service engine, the Resource Manager, and query processing modules interoperating with transaction coordinators like those in Apache Tomcat or JBoss. The design integrated with security infrastructures exemplified by Grid Security Infrastructure and relied on XML technologies such as XML Schema and XPath for payload descriptions. For workflow orchestration, OGSA-DAI worked alongside systems like Taverna and Kepler while participating in provenance initiatives related to Open Provenance Model and W3C PROV discussions. The component model accommodated adapters for storage systems used by projects at Los Alamos National Laboratory and research centers affiliated with Wellcome Trust.

Functionality and Features

OGSA-DAI provided remote SQL execution, data streaming, transformation services, and federation capabilities across disparate databases. It supported parameterised queries, server-side filtering, bulk data transfer, and staged extraction suitable for research infrastructures such as European Grid Infrastructure and National Grid Service. Features included result set slicing, format conversion into standards like CSV, XML, and early JSON profiles, and integration hooks for scripting languages used at institutions like EMBL-EBI and Sanger Institute. Metadata handling enabled interoperability with catalogues such as those associated with GBIF or DataCite-style registries, and connectors allowed linkage to file systems and archive services used by UK Research and Innovation funded projects.

Deployment and Use Cases

OGSA-DAI was deployed in scientific collaborations, bioinformatics consortia, and earth observation networks. Use cases encompassed federated queries across clinical trial databases in consortia involving Wellcome Trust partners, aggregation of sensor data for climate studies connected to European Space Agency programmes, and integration of archaeological datasets curated by museums like the British Museum. Grid portals and science gateways built on platforms such as gLite and UNICORE used OGSA-DAI as a backend for data provisioning. National-scale demonstrators, involving agencies such as JISC and infrastructure projects tied to EPSRC grants, showcased secure data sharing workflows and provenance capture for reproducible research.

Development History and Community

Initiated in the early 2000s under the auspices of the UK e-Science Programme, the project drew contributors from universities, research councils, and industrial partners including teams at the University of Edinburgh, University of Glasgow, and commercial collaborators acquainted with IBM and Oracle Corporation platforms. The community engaged through workshops at venues such as the International Supercomputing Conference and the IEEE International Conference on Web Services, with dissemination via technical reports and demonstrations at events like EGEE and SC Conference. Governance adopted open-source collaboration patterns, integrating feedback from user communities across medical research networks and high-energy physics collaborations.

Performance, Scalability, and Security

Performance engineering addressed query throughput, connection pooling, and result streaming to reduce memory pressure in high-throughput contexts exemplified by CERN-scale analyses and large genotyping datasets processed by facilities like Wellcome Trust Sanger Institute. Scalability strategies included horizontal replication of service instances, load balancing through web containers such as Apache HTTP Server with reverse proxies, and tuning of JDBC drivers for back-end databases including PostgreSQL and Oracle Database. Security considerations incorporated authentication and authorization using Grid Security Infrastructure credentials, X.509 certificates common in EGEE deployments, and integration with organizational identity providers modeled after Shibboleth and LDAP directories. Data confidentiality and auditing for clinical research complied with expectations set by bodies such as NHS Digital and ethics committees in academic medical centers.

Category:Distributed data management