OpenDAP

OpenDAP
Name	Open-source Project: OpenDAP
Developer	Open-source Community
Initial release	1990s
Latest release	ongoing
Written in	C, C++, Java, Python
Operating system	Cross-platform
License	Open-source

OpenDAP Open-source data access protocol and suite for remote scientific data access and delivery. Designed to enable interoperable access to remote datasets held on distributed servers hosted by institutions such as National Oceanic and Atmospheric Administration, NASA, NOAA, Scripps Institution of Oceanography, and Woods Hole Oceanographic Institution, OpenDAP has been used in workflows involving European Centre for Medium-Range Weather Forecasts, United States Geological Survey, National Center for Atmospheric Research, Met Office, and Japan Agency for Marine-Earth Science and Technology.

Overview

Open-source data-access middleware that provides a standardized network protocol and server/client libraries to retrieve metadata and subsets of scientific arrays and gridded data. It acts as a bridge between datasets stored in native formats at data centers like Lamont–Doherty Earth Observatory, Plymouth Marine Laboratory, Scripps Institution of Oceanography and analysis environments such as MATLAB, Python (programming language), R (programming language), IDL, and Ferret (software). The project supports interoperability with services run by European Space Agency, NOAA, National Aeronautics and Space Administration, United States Navy, and research consortia like Integrated Ocean Observing System.

History and Development

Originally developed in the mid-1990s by an academic consortium including scientists from University of Rhode Island, Woods Hole Oceanographic Institution, and Scripps Institution of Oceanography, the effort sought to address data distribution problems encountered by projects such as Argo (oceanography), TOGA, and World Ocean Database. Subsequent development involved contributions from government laboratories like NOAA, NASA, and USGS, and international partners including Met Office and European Centre for Medium-Range Weather Forecasts. Over time, integration work connected it to standards bodies and initiatives such as Open Geospatial Consortium, GFDR, and Global Ocean Observing System, while academic adopters including Columbia University, Massachusetts Institute of Technology, and University of Washington extended libraries for languages like C++, Java (programming language), and Python (programming language).

Architecture and Protocol

Client–server architecture with lightweight web-based protocols that mediate requests for metadata and data subsetting. Servers export dataset descriptions that can be consumed by client libraries in environments including MATLAB, Python (programming language), R (programming language), IDL, and ArcGIS; server implementations have been deployed on infrastructure provided by Amazon Web Services, Google Cloud Platform, and institutional clusters at SDSC, NCAR, and NERSC. The protocol defines endpoints for dataset discovery, constraint expressions for slicing arrays, and binary/ASCII encodings to serve data efficiently to consumers such as ESRI, QGIS, and Panoply. The architecture influenced or was integrated with standards from Open Geospatial Consortium and linked with metadata frameworks like ISO 19115 and Dublin Core.

Implementations and Software

Multiple server and client implementations exist in languages and toolchains used at institutions including NOAA, NASA, European Space Agency, and Scripps Institution of Oceanography. Open-source server projects have been packaged for deployment on platforms such as Apache HTTP Server and NGINX and bundled in analysis tools like THREDDS Data Server and ERDDAP. Client libraries and bindings are available for C++, Java (programming language), Python (programming language), Fortran, and R (programming language), and integration plugins exist for science portals run by ESIP Federation, Pangeo, and DataONE.

Data Models and Formats Supported

Supports array-oriented and gridded data models common to oceanography, atmospheric science, and remote sensing communities. Compatible formats and encodings include Network Common Data Form, netCDF Classic, netCDF-4, HDF5, GRIB, and plain binary arrays used by projects at NOAA, ECMWF, NASA, and USGS. The protocol exposes metadata structures that map to standards like CF (Climate and Forecast) metadata conventions and ISO 19115, enabling integration with catalogs maintained by Global Change Master Directory and repositories at National Centers for Environmental Information.

Use Cases and Applications

Widely used for remote subsetting and streaming of large observational and model datasets in scientific workflows at institutions such as Scripps Institution of Oceanography, NOAA, NASA, NCAR, and UCAR. Common applications include serving gridded forecasts from European Centre for Medium-Range Weather Forecasts, delivering satellite products produced by European Space Agency and NASA, and providing in situ arrays from observing systems like Argo (oceanography), GO-SHIP, and Global Ocean Observing System. It is embedded in analysis chains for climate model intercomparison projects such as Coupled Model Intercomparison Project and operational forecasting systems at Met Office and NOAA.

Security and Performance Considerations

Deployments must balance open access with controls used by institutions like NOAA, NASA, and European Space Agency; common practices include HTTPS/TLS termination via Let's Encrypt or institutional certificate authorities and authentication integrated with identity providers including ORCID, InCommon, and Globus. Performance strategies used by sites such as NERSC, SDSC, and NCAR include server-side caching, HTTP range requests, compression, and parallel data streams to accommodate high-throughput clients used by ESRI, Pangeo, and Google Cloud Platform. Security policies at data centers like NOAA and NASA often combine network controls, logging, and role-based access via systems such as OAuth 2.0 and institutional single sign-on federations.

Category:Data formats Category:Remote sensing