This article was accepted into the corpus but its outbound wikilinks were never NER-processed — typical at the deepest BFS hop or when the run's entity cap was reached. No expansion funnel to show.
| Cluster system | |
|---|---|
| Name | Cluster system |
| Type | Distributed computing |
Cluster system A cluster system is a coordinated collection of interconnected nodes organized to provide enhanced availability, compute capacity, and redundancy for workloads across diverse environments. Originating from early parallel projects such as Cray Research initiatives and NASA supercomputing efforts, cluster systems evolved alongside developments at MIT, UC Berkeley, and corporations like IBM and Intel to support scientific, commercial, and government use. Modern cluster systems integrate technologies from vendors and projects including Red Hat, Microsoft Azure, Google Cloud Platform, Amazon Web Services, and OpenStack to deliver scalable services for institutions such as CERN, Los Alamos National Laboratory, and Lawrence Livermore National Laboratory.
A cluster system is defined by multiple interconnected servers or workstations collaborating to present unified resources to users and applications, a concept central to projects at Bell Labs, DARPA, and SLAC National Accelerator Laboratory. Typical cluster paradigms trace to research at Stanford University, Carnegie Mellon University, and University of Oxford where systems were built to support experiments from High Energy Physics collaborations like ATLAS and CMS. Administratively, clusters are deployed by organizations such as NASA Ames Research Center, European Organization for Nuclear Research, and corporations like Hewlett-Packard to support workloads originating from NOAA, USGS, and Bloomberg L.P.. Cluster systems are contrasted with architectures promoted by Oracle Corporation, SAP SE, and VMware but often integrate middleware from Apache Software Foundation projects such as Hadoop and Spark.
Cluster architectures include high-performance computing clusters influenced by Beowulf designs from NASA and CESR, high-availability clusters used by J.P. Morgan, Goldman Sachs, and Federal Reserve System, and load-balancing clusters used by Facebook, Twitter, and Netflix. Grid computing models from European Grid Infrastructure and Open Science Grid overlap with cluster deployments at CERN and Fermilab. Cloud-native cluster types derive from orchestration frameworks such as Kubernetes (originating at Google), container ecosystems from Docker Inc., and service meshes like Istio. Hybrid clusters combine on-premises hardware from Dell EMC and HPE with public clouds like Alibaba Cloud and Microsoft Azure. Specialized architectures include GPU clusters using NVIDIA hardware for projects at DeepMind and OpenAI, storage clusters using Ceph and GlusterFS, and database clusters exemplified by Oracle Real Application Clusters and CockroachDB.
Core components of a cluster system include head nodes and compute nodes similar to configurations deployed at Los Alamos National Laboratory and Argonne National Laboratory, interconnects such as InfiniBand and Ethernet used by Intel and Mellanox Technologies, storage arrays from NetApp and Pure Storage, and management software like Slurm Workload Manager, HTCondor, and OpenPBS. Resource schedulers from Kubernetes and Apache Mesos coordinate jobs in environments used by Netflix and Airbnb, while file systems such as Lustre and GPFS serve clusters in research facilities like Oak Ridge National Laboratory. Networking fabrics and fabric management tools from Cisco Systems and Juniper Networks provide connectivity, and monitoring agents compatible with Prometheus, Nagios, and Zabbix collect telemetry for teams at Goldman Sachs and Morgan Stanley.
Cluster systems power scientific simulations at CERN, Los Alamos National Laboratory, and Argonne National Laboratory; financial risk models at J.P. Morgan and BlackRock; machine learning training at OpenAI, DeepMind, and Google Research; life-science genomics pipelines at Broad Institute and Wellcome Sanger Institute; and media rendering farms for studios such as Pixar and Industrial Light & Magic. Telecommunications providers like AT&T and Verizon Communications deploy clusters for signaling and subscriber services, while e-commerce platforms at Amazon.com and eBay use clusters for transaction processing and personalization. Public-sector uses include climate modeling for NOAA and Met Office and healthcare analytics for systems at NHS England and Centers for Disease Control and Prevention.
Performance tuning in cluster systems draws on practices developed at Sandia National Laboratories and Princeton Plasma Physics Laboratory, optimizing interconnect latency and throughput using technologies from Mellanox Technologies and Broadcom. Scalability models tested in deployments by Facebook and Google emphasize horizontal scaling with frameworks like Kubernetes and distributed databases such as Cassandra and CockroachDB. Benchmarks including LINPACK and domain-specific suites used by TOP500 participants at facilities like Oak Ridge National Laboratory measure floating-point performance, while I/O workloads are profiled with tools endorsed by SPEC and practiced by Netflix.
Cluster management strategies adopt orchestration and provisioning tools from Ansible, Puppet, and Chef used across enterprises like Capital One and Siemens. Monitoring and observability stacks combining Prometheus, Grafana Labs, and ELK Stack underpin operations at Spotify and Uber Technologies. Workflow managers such as Airflow and Nextflow schedule pipelines in biotech installations at EMBL and Broad Institute, and configuration management integrates with identity providers like Okta and Active Directory deployed by institutions including Harvard University and Stanford University.
Security models for cluster systems incorporate practices from NIST, compliance frameworks like HIPAA and GDPR when applied at NHS England and European Commission, and hardening guidelines followed by Department of Defense facilities. Fault-tolerance mechanisms inspired by work at Bell Labs and MIT Lincoln Laboratory use replication strategies seen in RAID arrays and consensus algorithms such as Paxos and Raft implemented in systems like etcd. Disaster recovery plans align with standards used by World Bank and IMF data centers, and secure enclaves and confidential computing features from Intel and AMD are increasingly adopted by research centers including Lawrence Livermore National Laboratory.
Category:Distributed computing systems