Singularity (software)

Singularity (software)
Name	Singularity
Developer	Singularity Project, Microsoft Research, Randy Katz
Released	2015
Programming language	Go, C
Operating system	Linux
License	BSD-derived

Singularity (software) Singularity is a container platform designed for reproducible, portable, and secure execution of applications on high-performance computing systems. It emphasizes user-controlled, immutable images and tight integration with research infrastructures operated by organizations such as NERSC, Oak Ridge, and LBNL. The project has influenced container use across institutions including CERN, Argonne, and Los Alamos.

Overview

Singularity provides a runtime for encapsulating applications and environments into single-file images that can be executed on clusters, supercomputers, and clouds such as Amazon Web Services, Google Cloud Platform, and Microsoft Azure. It targets scientific workloads originating from projects at UC Berkeley, MIT, and Stanford, enabling portability across systems like Cray and IBM POWER installations. The platform distinguishes itself from alternatives by preserving user identity, minimizing privileged daemons, and supporting formats interoperable with Docker and Open Container Initiative specifications.

History and Development

Development began in research settings influenced by computing efforts at LLNL and design principles from CHERI-style isolation work at Cambridge. Early contributors included researchers affiliated with UC Davis, Wisconsin, and developers from Microsoft Research. The project evolved through collaborations with national labs—NERSC, Argonne, Los Alamos—and through contributions from consortia such as Software Carpentry and the OpenStack community. Releases tracked adoption milestones at supercomputing conferences like Supercomputing Conference and workshops hosted by JOSS partners.

Architecture and Components

Singularity images are single-file containers combining filesystem layers, metadata, and optional digital signatures created by tools compatible with Docker Registry semantics and the OCI. Core components include the runtime, image builder, image signer, and libraries integrating with cluster managers such as Slurm, Torque, and PBS. The runtime invokes Linux kernel features like namespaces, cgroups, and capabilities via syscalls tied to distributions such as Debian, RHEL, and CentOS. Integration points exist for federated identity systems like CILogon and workload managers including Kubernetes gateways and scheduler plugins for HTCondor.

Use Cases and Applications

Singularity is widely used for reproducible pipelines in domains represented by institutions including Broad Institute, EMBL, and Max Planck Society. Common applications include genomics workflows driven by GATK, climate modeling from groups at NOAA, and astrophysics simulations originating at NASA Ames. It supports machine learning stacks developed at OpenAI, DeepMind, and academic labs leveraging TensorFlow and PyTorch within HPC contexts. Other deployments occur in bioinformatics cores, computational chemistry groups collaborating with Riken, and image processing teams linked to ESO.

Security and Isolation

Singularity emphasizes non-privileged execution: user processes run with user identities mapped to host accounts to comply with policies at Department of Energy, NIH, and civilian research facilities. Image signing and verification use public-key infrastructures similar to mechanisms used by GPG and enterprise certificate authorities such as Let's Encrypt. The project has been evaluated against security frameworks from NIST and auditing practices from SANS Institute. Singularity integrates with host-based security tools like SELinux, AppArmor, and kernel hardening from distributions such as Ubuntu LTS releases.

Performance and Benchmarking

Benchmarks performed on clusters operated by NERSC, OLCF, and ALCF compare Singularity to virtualization platforms like KVM, Xen, and container runtimes such as Docker and rkt. Results emphasize low overhead for MPI workloads using libraries such as Open MPI and MPICH, and near-native I/O performance when interfacing with parallel filesystems like Lustre and GPFS. HPC centers publish microbenchmarks for CPU-bound, memory-bound, and network-bound workloads presented at forums like SC.

Adoption and Community

Adoption is strongest among national laboratories, universities, and research consortia including ELIXIR, EGI, and Compute Canada. The community organizes workshops at meetings hosted by SC, USENIX, and domain conferences such as Bioinformatics Open Days. Contributions come from corporate partners like NVIDIA and cloud providers, plus open-source foundations such as Linux Foundation projects. Governance and roadmaps have been discussed in working groups affiliated with OpenStack Foundation and academic steering committees from NSF-funded centers.

Category:Containerization software