CloudLab

CloudLab
Name	CloudLab
Type	Research cloud testbed
Established	2011
Location	United States
Fields	Computer science, networking, distributed systems
Operator	National Science Foundation-funded consortium

CloudLab CloudLab is an open research testbed for experimentation in computer science and networking that provides infrastructure for researchers from institutions such as University of Utah, Princeton University, Carnegie Mellon University, University of Chicago, and University of California, San Diego. It supports experiments using hardware from vendors like Intel Corporation, NVIDIA, Dell Technologies, Hewlett Packard Enterprise, and integrates software from projects including OpenStack, Kubernetes, Terraform, Ansible. Funded initially through grants involving the National Science Foundation, CloudLab collaborates with initiatives such as GENI, XSEDE, Globus, ESnet, and Internet2.

Overview

CloudLab offers isolated slices of programmable compute, storage, and network resources across multiple sites including campuses and data centers affiliated with University of Utah, University of Wisconsin–Madison, University of Chicago, Princeton University, University of Utah School of Computing, and corporate partners. The platform enables reproducible experiments using technologies like virtualization provided by KVM, container orchestration by Kubernetes, and software-defined networking via Open vSwitch, P4, and ONOS. Researchers deploy custom operating systems such as Ubuntu, Fedora, FreeBSD, and experimental kernels like those from Linux Foundation projects. CloudLab interoperates with cloud APIs inspired by Amazon EC2, Google Compute Engine, and Microsoft Azure while emphasizing academic reproducibility found in repositories like GitHub and Zenodo.

History

CloudLab's origins trace to funding from the National Science Foundation and coordination with the Global Environment for Network Innovations (GENI) initiative. Key early collaborators included research groups at Princeton University, University of Utah, University of Wisconsin–Madison, University of Chicago, and Carnegie Mellon University. The project evolved alongside contemporaneous efforts like PlanetLab, Emulab, XSEDE, and later integration with networks such as ESnet and Internet2. Notable milestones involved hardware refreshes using platforms from Intel Corporation and NVIDIA and software transitions incorporating OpenStack releases and Kubernetes adoption. CloudLab supported experiments that linked to projects endorsed by agencies such as the Defense Advanced Research Projects Agency and partnerships with companies including Google, Amazon Web Services, Microsoft Research, and Cisco Systems.

Architecture and Infrastructure

CloudLab's distributed architecture uses rack-scale systems sited at universities and research centers including University of Utah, Princeton University, University of Wisconsin–Madison, University of Chicago, University of Utah School of Computing, and cloud partner locations. Hardware providers include Dell Technologies, Hewlett Packard Enterprise, Supermicro, Intel Corporation, and NVIDIA. The control plane leverages software such as OpenStack, Docker, Kubernetes, Ansible, and orchestration tools like Terraform and MAAS. Networking is implemented with programmable switches using P4, Open vSwitch, and controllers from ONOS and OpenDaylight. Storage options in experiments use systems like Ceph, Lustre, and NVMe arrays from Samsung Electronics. Time synchronization and monitoring integrate NTP, Prometheus, and Grafana, while identity and access management connects to federations like InCommon and authentication services such as CILogon.

Research and Educational Uses

CloudLab supports distributed systems research at institutions including MIT, Stanford University, University of California, Berkeley, Carnegie Mellon University, and Princeton University. Research domains include studies of software-defined networking with projects linked to ONOS, P4 Language Consortium, and Open vSwitch; container orchestration research interacting with Kubernetes and Docker Swarm; high-performance computing experiments referencing MPI implementations; and edge computing prototypes interfacing with initiatives such as OpenEdge, Fog Computing research groups, and EdgeX Foundry. Educational uses span courses at University of Utah, University of Wisconsin–Madison, University of Chicago, Princeton University, and University of California, San Diego for classes in distributed systems, operating systems, and networking, complementing curricula that rely on resources from ACM, IEEE, USENIX, and SIGCOMM.

Access and Account Management

Account provisioning uses institutional credentials via federations like InCommon and authentication services such as CILogon and Shibboleth. Users request experimental slices through web portals and APIs modeled after services from Amazon EC2 and OpenStack Horizon. Project administration follows policies similar to academic testbeds like Emulab and PlanetLab, with usage accounting compatible with tools from OpenStack Ceilometer and logging integrated into ELK Stack components such as Elasticsearch and Kibana. Training and onboarding are coordinated with workshops hosted by organizations including CRA, SIGCOMM, USENIX, and NSF-sponsored summer schools.

Projects and Case Studies

Notable projects conducted on the testbed include scalable network function virtualization experiments related to NFV efforts with Open Platform for NFV partners; research into congestion control building on protocols like TCP Cubic, QUIC (from IETF and Google), and comparisons to TCP BBR; distributed storage evaluations referencing Ceph and Lustre; and machine learning model training utilizing GPUs from NVIDIA with frameworks such as TensorFlow (from Google), PyTorch (from Meta Platforms, Inc.), and MXNet (from Apache Software Foundation). Case studies have been published in venues including SIGCOMM, NSDI, OSDI, ATC (USENIX Annual Technical Conference), and IEEE INFOCOM, often collaborating with corporate research groups at Google Research, Microsoft Research, Facebook AI Research, and IBM Research.

Governance and Funding

CloudLab is governed by a consortium model involving universities such as University of Utah, Princeton University, University of Chicago, University of Wisconsin–Madison, and Carnegie Mellon University, with programmatic oversight from agencies like the National Science Foundation. Funding sources include grants from the National Science Foundation and collaborative sponsorships with corporations like Intel Corporation, NVIDIA, Dell Technologies, Hewlett Packard Enterprise, and partnerships with network operators including ESnet and Internet2. Steering committees include academic principal investigators from institutions such as MIT, Stanford University, UC Berkeley, University of Illinois Urbana–Champaign, and industry liaisons from Cisco Systems, Google, Microsoft, and Amazon Web Services.

Category:Computer science testbeds