Univa Grid Engine

Univa Grid Engine
Name	Univa Grid Engine
Developer	Univa Corporation
Released	2011
Latest release	(discontinued commercial product)
Programming language	C, C++
Operating system	Linux, Solaris, AIX
Genre	Batch-queuing system, workload management system
License	Commercial (historical)

Univa Grid Engine is a distributed resource management and job scheduling system originally derived from open-source Grid Engine technology and later commercialized by Univa Corporation. It provided centralized queuing, resource allocation, and workload orchestration for high-performance computing clusters, enterprise datacenters, and cloud environments. Univa Grid Engine combined scheduling policies, accounting, and fault-tolerant components to support scientific computing projects, financial analytics, and media rendering workloads.

History

Univa Grid Engine traces lineage to the original Grid Engine project, the work of Sun Microsystems after acquisition of Gridware, and subsequent stewardship by Oracle Corporation and various open-source forks. Prominent organizations and events in its history include contributions from Sun Microsystems, the involvement of Oracle Corporation during the acquisition of Sun, and parallel projects such as the open-source Grid Engine fork maintained by the community and other vendors. Important entities linked to that ecosystem include Sun Microsystems, Oracle Corporation, IBM, Intel, Hewlett-Packard, Silicon Graphics International, and research centers like Lawrence Livermore National Laboratory and Los Alamos National Laboratory. Commercial efforts around Univa involved collaborations with cloud providers and enterprises such as Amazon Web Services, Microsoft Azure, and major financial services firms that deployed batch scheduling for risk analysis and quantitative research. The product evolved amid broader trends including the rise of Hadoop, OpenStack, and container orchestration efforts led by Docker and Kubernetes.

Architecture and Components

Univa Grid Engine implemented a master–worker architecture with distinct daemons and services responsible for job submission, scheduling, execution, and accounting. Key components paralleled names used across resource managers: the master daemon coordinated cluster state, execution daemons ran on compute hosts, and submit clients interfaced with users and workflow engines. Integration points and dependencies connected to operating systems and vendor platforms such as Red Hat Enterprise Linux, CentOS, SUSE Linux Enterprise Server, Oracle Solaris, and enterprise management tools produced by Puppet Labs and Red Hat. Fault-tolerance and scalability strategies mirrored approaches used in distributed systems research at institutions like Massachusetts Institute of Technology, Stanford University, and Lawrence Berkeley National Laboratory. Support for MPI stacks tied Univa Grid Engine to implementations from Open MPI, MPICH, and vendor libraries from Intel and Cray.

Features and Functionality

The system offered advanced scheduling policies, backfilling, reservations, complex resource requests, and consumable resource tracking to optimize utilization across heterogeneous hardware. It provided accounting and reporting suitable for chargeback and audit workflows used by enterprises and research centers such as National Aeronautics and Space Administration, European Organization for Nuclear Research, and Max Planck Society. Job submission interfaces supported popular scientific and engineering applications developed by organizations like ANSYS, MATLAB, GROMACS Consortium, and rendering pipelines used by studios such as Weta Digital and Industrial Light & Magic. Authentication and authorization integrated with directory services from Microsoft and Red Hat, and monitoring tied into tools from Nagios, Zabbix, and Splunk. The scheduler supported array jobs, parallel environments, and affinity policies adopted in large-scale campaigns at supercomputing facilities including Oak Ridge National Laboratory and Argonne National Laboratory.

Deployment and Administration

Administrators deployed Univa Grid Engine on-premises, in private clouds, and hybrid topologies, often alongside virtualization platforms from VMware and cloud services by Amazon Web Services and Google Cloud Platform. Configuration and automation used orchestration tooling from Ansible, Chef, and Puppet Labs, while CI/CD pipelines from Jenkins and GitLab integrated job submission for build and test farms. Capacity planning and performance tuning referenced benchmarking approaches from SPEC and collaborative projects at National Center for Supercomputing Applications. High-availability setups paralleled patterns documented by practitioners at Los Alamos National Laboratory and commercial deployments by technology consultancies such as Accenture and Deloitte.

Licensing and Commercialization

Univa packaged its Grid Engine offering under commercial licenses, providing enterprise support, proprietary enhancements, and value-added services tailored to sectors including finance, life sciences, and media production. Licensing models resembled those used by vendors such as Red Hat and SUSE, with options for subscription, support agreements, and professional services delivered by system integrators like IBM Global Services and Capgemini. The commercialization phase placed Univa in market discussions alongside other workload managers and schedulers produced by Slurm Workload Manager vendors, legacy products from Platform Computing (part of IBM), and cloud-native orchestration platforms driven by Google and Amazon Web Services.

Use Cases and Integrations

Univa Grid Engine was employed for high-throughput computing, batch-oriented scientific pipelines, rendering farms, and production workloads in finance for Monte Carlo simulations and risk analytics. Integrations spanned data platforms and software from Hadoop, Apache Spark, PostgreSQL, and message brokers such as Apache Kafka. Workflows orchestrated by scientific gateways and portals developed at University of California, San Diego, University of Cambridge, and Imperial College London used Grid Engine backends for job dispatch. Collaborative projects between universities, national labs, and enterprises often combined Univa deployments with scheduler-agnostic middleware from projects like Globus and workflow engines such as Pegasus and Apache Airflow.

Category:Batch-queuing systems