Sun Grid Engine

Sun Grid Engine
Name	Sun Grid Engine
Developer	Sun Microsystems; Oracle Corporation; community projects
Released	2001
Latest release version	varies by fork
Programming language	C, C++
Operating system	Solaris, Linux, FreeBSD, AIX
Genre	Batch-queuing system, job scheduler
License	Proprietary (original), open source (some forks)

Sun Grid Engine is a distributed resource management and batch-queuing system originally developed by Sun Microsystems and later maintained under different stewardship by Oracle Corporation and community projects. It provided job submission, scheduling, and accounting services for high-performance computing clusters used by institutions such as national laboratories, universities, and enterprises. The software influenced later cluster managers and grid middleware and appears in deployment narratives alongside technologies from companies like IBM, Hewlett-Packard, Microsoft, and Red Hat.

History

The project began at Sun Microsystems and evolved through milestones involving corporate actions by Oracle Corporation and community responses similar to forks following vendor transitions, paralleling events around projects like OpenSolaris, MySQL, and PostgreSQL. Adoption grew in contexts involving centers such as Lawrence Berkeley National Laboratory, CERN, and NASA, akin to how projects like Apache HTTP Server and NGINX spread across research and industry. Legal and stewardship changes prompted community forks and initiatives reminiscent of forking instances seen with OpenOffice, LibreOffice, and MariaDB.

Architecture and Components

The architecture used a master–execution paradigm comparable to designs in projects like Apache Hadoop, Kubernetes, and SLURM. Core components included a qmaster-like central manager, execution daemons, and client submission utilities, analogous in role to components in Torque, PBS Professional, and LSF. Integration points and interfaces resembled those in software from Red Hat, IBM Research, and Intel Parallel Studio, enabling hooks for accounting systems, authentication stacks like LDAP and Kerberos, and networked filesystems such as NFS and Lustre used at Oak Ridge National Laboratory and European Grid Infrastructure sites.

Installation and Configuration

Installation procedures paralleled practices for enterprise systems from SUSE, Ubuntu, and CentOS, requiring service management comparable to systemd, init, and SMF in Solaris. Configuration files and site policies were managed in ways similar to Ansible playbooks, Puppet manifests, and Chef cookbooks used by organizations like CERN, NASA Ames, and Los Alamos National Laboratory for reproducible deployments. Packaging and build workflows often mirrored conventions from GNU Autotools, CMake, and RPM/DEB ecosystems maintained by Debian, Fedora, and openSUSE contributors.

Scheduling and Resource Management

The scheduler implemented priority, fairshare, and queue-based policies analogous to mechanisms in SLURM, IBM Spectrum LSF, and HTCondor, supporting resource attributes comparable to CPU, memory, GPU, and license tokens as seen in deployments involving NVIDIA, AMD, and Intel accelerators. Backfill, preemption, and reservation behaved like features in Maui Scheduler and Moab Workload Manager used at centers such as Argonne National Laboratory and the National Energy Research Scientific Computing Center. Accounting and reporting integrated with tools similar to Ganglia, Nagios, and Prometheus for cluster telemetry and usage trends.

Administration and Monitoring

Administrative tasks used utilities and GUIs reminiscent of IBM Tivoli and Oracle Enterprise Manager for cluster health, job management, and user quotas. Monitoring stacks leveraged telemetry solutions from Graphite, Zabbix, and ELK Stack that are common at enterprises like Google, Amazon Web Services, and Microsoft Azure for log aggregation and metric dashboards. Security administration interfaced with directory services and compliance frameworks comparable to Active Directory, OpenLDAP, and FIPS-conformant deployments in government labs and defense contractors.

Use Cases and Integrations

Common use cases included batch processing for genomics pipelines in facilities like Broad Institute, computational fluid dynamics at aerospace firms such as Boeing and Lockheed Martin, and parameter sweep studies in academic groups at MIT, Stanford University, and University of California campuses. Integrations often tied to workflow managers and platforms like Pegasus, Nextflow, Galaxy, and Snakemake as well as container runtimes and orchestration frameworks from Docker, Singularity, and Kubernetes used in enterprise and research contexts by companies like Amazon, IBM, and Intel.

Forks and Legacy/Community Projects

Following stewardship transitions, community-led forks and reimplementations emerged similar to the community responses behind LibreOffice and MariaDB, with projects maintained by open-source foundations and academic consortia. Alternative projects and successors in the resource-management space include SLURM, HTCondor, Torque, Open Grid Scheduler, and Univa Grid Engine—each reflecting divergent governance and feature sets influenced by corporate contributors like Univa Systems and collaborative nodes such as European Open Science Cloud partners. Legacy deployments persist at research centers and commercial sites where migration paths involve modernization to Kubernetes, cloud-native batch services from Google Cloud and AWS Batch, or continued use of established schedulers like LSF and PBS Professional.

Category:Batch queuing systems Category:High performance computing Category:Cluster management