SharcNet

SharcNet
Name	SharcNet
Established	2000
Type	Consortium
Location	Ontario, Canada
Service	High-performance computing

SharcNet is a Canadian high-performance computing consortium that provides centralized supercomputing resources to researchers across multiple universities and institutions. The consortium grew out of regional collaborations among academic institutions, provincial bodies, and national research initiatives to support computational science, engineering, and data-intensive scholarship. It has been associated with clusters, parallel file systems, and middleware that service projects in fields ranging from physics and chemistry to economics and digital humanities.

History

SharcNet originated from collaborations between researchers at University of Toronto, University of Waterloo, McMaster University, Queen's University, and York University in response to increasing demand for computational resources following initiatives like Compute Canada and provincial research strategies. Early development drew on expertise from groups involved with projects such as High Performance Computing Virtual Laboratory, NeSC and leveraged procurement models used by consortia like XSEDE and PRACE. Milestones included acquisition of clusters inspired by architectures used at Los Alamos National Laboratory, Lawrence Livermore National Laboratory, and design principles from Cray Research systems, enabling workloads similar to those run for Human Genome Project analyses and simulations akin to those performed for Large Hadron Collider studies.

Architecture and Services

SharcNet operated heterogeneous clusters composed of commodity servers, GPU nodes, and large-memory machines, integrating technologies from vendors and projects such as Intel Corporation, NVIDIA, AMD, Dell Technologies, IBM, and filesystems like Lustre and GPFS. The software stack incorporated schedulers and resource managers exemplified by Slurm Workload Manager, Torque/Maui Scheduler, and middleware associated with OpenMPI, MPICH, and development tools from GNU Project and Intel Parallel Studio. Services included batch scheduling, interactive access consistent with workflows used at National Institutes of Health, data management strategies reflecting methods in CERN collaborations, and portal interfaces inspired by systems like Galaxy Project and Jupyter. Networking relied on regional and national research networks comparable to CANARIE, Internet2, and Ontario Research and Education Optical Network.

Research and Academic Impact

Researchers supported by the consortium contributed to publications and grants connected to agencies such as Natural Sciences and Engineering Research Council of Canada, Canadian Institutes of Health Research, and international programs like Horizon 2020. Computational studies spanned disciplines linked to institutions such as University of British Columbia, McGill University, University of Alberta, Dalhousie University, and research centers like Perimeter Institute and Hospital for Sick Children. Outputs included simulations relevant to climate modeling projects similar to those produced by IPCC working groups, quantum chemistry calculations paralleling work at Argonne National Laboratory, and machine learning experiments in the style of research at Google DeepMind and OpenAI. Collaborations frequently intersected with large-scale data initiatives such as Human Connectome Project and genomics consortia resembling 1000 Genomes Project.

Governance and Funding

The consortium's governance involved representation from member universities, research institutes, and provincial partners, reflecting organizational practices similar to boards at Tri-Agency coordinated projects and advisory structures seen at NSERC. Funding models combined institutional contributions, grant funding from agencies like Canada Foundation for Innovation and matching funds analogous to programs run by Ontario Ministry of Colleges and Universities, as well as fee-for-service arrangements parallel to commercial services from Amazon Web Services and time allocation policies echoing principles used by National Science Foundation.

Membership and Access

Membership included faculty, graduate students, postdoctoral researchers, and staff from partner institutions such as University of Guelph, Ryerson University, Brock University, Laurentian University, and affiliated hospitals and national labs. Access policies balanced institutional allocations, peer-reviewed project proposals similar to access committees at PRACE and XSEDE, and training programs modeled on workshops by Software Carpentry and Data Carpentry. Security and identity management integrated systems comparable to Shibboleth and federated authentication architectures like those used by eduGAIN.

Notable Projects and Collaborations

Notable work supported by the consortium included computational chemistry campaigns reminiscent of studies at Pacific Northwest National Laboratory, astrophysics simulations comparable to efforts at Space Telescope Science Institute, and bioinformatics pipelines analogous to workflows at Broad Institute. Collaborations extended to national initiatives such as Compute Ontario and international partnerships reflecting exchanges with groups at University of Cambridge, Massachusetts Institute of Technology, Imperial College London, ETH Zurich, Princeton University, Harvard University, Stanford University, University of California, Berkeley, Columbia University, University of Chicago, Yale University, California Institute of Technology, Max Planck Society, Centre National de la Recherche Scientifique, Deutsches Elektronen-Synchrotron, Rutherford Appleton Laboratory, Commonwealth Scientific and Industrial Research Organisation, Australian National University, University of Tokyo, Peking University, Tsinghua University, Seoul National University, National University of Singapore, École Polytechnique Fédérale de Lausanne, Delft University of Technology, KU Leuven, University of Edinburgh, University of Manchester, King's College London, University of Sydney, Monash University, University of Melbourne, University of Auckland, McMaster University research groups and others.

Category:High-performance computing in Canada