Whamcloud

Whamcloud
Name	Whamcloud
Type	Private
Founded	2010
Founder	William D. Gropp, Madhu Venugopal, Diede van Dijk
Headquarters	Seattle, Washington (state)
Industry	Computer storage, High-performance computing
Products	Lustre, BeeGFS (historical partnerships), Robinhood Policy Engine

Whamcloud was a software engineering organization focused on scale-out file systems and data management for large-scale computing. Founded in 2010, the company became prominent for sustaining and commercializing the development of the Lustre file system used in supercomputing centers, national laboratories, and enterprise data centers. Whamcloud worked with academic researchers, government laboratories, and technology vendors to advance storage performance, reliability, and tooling for scientific computing and cloud-scale deployments.

History

Whamcloud emerged from a collaboration among engineers and researchers active in the Lustre community, including contributors associated with Oak Ridge National Laboratory, Lawrence Livermore National Laboratory, and the European Organization for Nuclear Research. Early leadership included figures with ties to Argonne National Laboratory, Sandia National Laboratories, and academic groups at University of California, Santa Cruz and University of Edinburgh. The company gained visibility through participation in events such as the SC Conference and International Supercomputing Conference where it presented work alongside teams from Cray Inc., Intel Corporation, and IBM.

Throughout its existence, Whamcloud engaged with projects funded or influenced by agencies and consortia such as U.S. Department of Energy, National Science Foundation, and the European Grid Infrastructure. Strategic hires and community outreach expanded collaborations with institutions like Lawrence Berkeley National Laboratory and companies including Seagate Technology, NetApp, and Hewlett Packard Enterprise. In subsequent corporate developments, Whamcloud’s assets and team became integrated into larger commercial entities associated with storage vendors and systems integrators.

Products and Projects

Whamcloud’s work concentrated on products and projects that supported large-scale shared-storage deployments. The primary focus was the maintenance and enhancement of Lustre, with engineering contributions to upstream releases used by organizations such as NERSC, Oak Ridge Leadership Computing Facility, and Los Alamos National Laboratory. Complementary projects included metadata handling, high-availability tooling, and performance analysis utilities used in conjunction with systems from Cray Inc. and Dell EMC.

Other initiatives included development and stewardship of policy engines akin to Robinhood Policy Engine, integrations with cluster resource managers such as Slurm Workload Manager and Portable Batch System, and interoperability work with parallel file systems like BeeGFS and GPFS. Whamcloud engineers contributed to benchmarking efforts using testbeds from PRACE and collaborations with research programs at CERN and Max Planck Society.

Technology and Architecture

Whamcloud’s engineering emphasized the architectures that underpin distributed metadata services, scalable object storage backends, and high-throughput network stacks. Technical contributions addressed challenges in POSIX semantics at scale, metadata scalability that affected deployments at places like Argonne National Laboratory and Oak Ridge National Laboratory, and fault-tolerant designs reflecting lessons from deployments on systems built by Cray Inc. and Hewlett Packard Enterprise.

Workstreams included optimizations for interconnects such as InfiniBand, support for storage media ranging from Seagate Technology HDD arrays to emerging NVMe deployments promoted by Intel Corporation and Samsung Electronics, and integration pathways with clustered file-system management frameworks used by Red Hat and SUSE. Whamcloud software engineering practices mirrored those in academic projects at University of Illinois Urbana–Champaign and industrial projects at Microsoft Research, focusing on upstream open-source contributions and reproducible performance evaluation.

Community and Governance

Whamcloud played a central role in community stewardship of the Lustre ecosystem, organizing developer summits and contributing to governance dialogues between national labs, universities, and vendors such as NetApp and Seagate Technology. The company’s staff participated in working groups that included representatives from U.S. Department of Energy facilities, European research infrastructures like EGI (European Grid Infrastructure), and supercomputing centers including NERSC.

By sponsoring meetings at conferences such as SC Conference and International Supercomputing Conference, Whamcloud helped coordinate roadmaps that brought together stakeholders from Cray Inc., IBM, Hewlett Packard Enterprise, and research institutions. This governance role extended to code review, release management, and interoperability testing with projects at Lawrence Berkeley National Laboratory and Los Alamos National Laboratory.

Commercialization and Partnerships

Commercial engagements involved partnerships and contracts with major storage and high-performance computing vendors, systems integrators, and research consortia. Whamcloud collaborated with companies like NetApp, Cray Inc., Seagate Technology, and Hewlett Packard Enterprise to deliver support, integration, and engineering services around Lustre-based solutions for clients including Department of Energy laboratories and university clusters.

Partnerships also spanned technology providers for networking and hardware, including Intel Corporation and Mellanox Technologies (now part of NVIDIA), enabling performance tuning for InfiniBand fabrics and Ethernet deployments. Strategic alliances facilitated joint support models with channel partners and systems integrators accustomed to deploying at the scale of the Oak Ridge Leadership Computing Facility and other national supercomputing centers.

Category:Software companies Category:High-performance computing