PARMACS

PARMACS
Name	PARMACS
Type	Benchmark Suite
Developer	Parallel Computing Research Groups
First released	1990s
Latest release	2000s
Programming languages	Fortran, C
Operating system	Unix-like
License	Academic

PARMACS

PARMACS is a benchmark suite and collection of parallel computing kernels originally developed to evaluate message-passing performance, scalability, and microbenchmark behavior on distributed-memory systems. Designed to exercise communication patterns and synchronization primitives, it serves as a tool for comparative studies between architectures such as those from Cray Research, Intel Corporation, IBM, Sun Microsystems, and implementations of MPI and PVM.

Overview

PARMACS comprises a set of message-passing kernels and test programs that target latency, bandwidth, collective operations, and small-message behavior on systems like the Thinking Machines Corporation machines, Sequent Computer Systems SMPs, and clusters of DEC workstations. The suite influenced later benchmarks including NAS Parallel Benchmarks, SPEC MPI2007, and vendor suites from Hewlett-Packard and Oracle Corporation (Sun). Researchers from institutions such as Lawrence Livermore National Laboratory, Los Alamos National Laboratory, Sandia National Laboratories, and universities like Stanford University and Massachusetts Institute of Technology used PARMACS to compare implementations of MPI against alternatives including PVM and proprietary APIs on platforms such as IBM SP, SGI, and Cray T3D.

History and Development

PARMACS originated during the early 1990s amid efforts by parallel systems researchers at groups associated with projects like NAS and collaborative initiatives involving DARPA, NSF, and national laboratories. Influences and contemporaries included the NIST microbenchmark efforts, the LINPACK benchmarking lineage, and application-driven studies from centers such as CERFACS and EPCC. Developers adapted PARMACS to examine communication effects observed in distributed-memory experiments on systems from Fujitsu, Hitachi, NEC Corporation, and research clusters at University of California, Berkeley and University of Illinois at Urbana–Champaign. Papers presenting PARMACS results appeared at conferences like SC Conference, ACM SIGPLAN, and IEEE Supercomputing.

Architecture and Design

PARMACS is implemented in languages common to high-performance computing such as Fortran 77 and C and targets message-passing APIs like MPI-1, early MPI-2 features, and PVM; it exercises point-to-point sends, receives, nonblocking operations, and simple collectives. Test kernels include ping-pong latency tests, unidirectional and bidirectional bandwidth probes, and synthetic workloads used alongside profiling tools such as gprof, vendor tools from Intel Corporation and IBM, and tracing systems like Vampir and TAU. The design emphasizes portability across operating systems from Unix System V, BSD, and vendor-specific derivatives used on machines such as HP 9000, Sun SPARCserver, and AlphaServer. PARMACS also integrates with queueing systems and batch schedulers such as PBS and LSF for controlled runs on multi-node systems like Cray XT, IBM Blue Gene prototypes, and commodity clusters.

Applications and Use Cases

Researchers applied PARMACS to characterize interconnects such as Myrinet, Infiniband, Quadrics, and proprietary networks from Cray Research and IBM. Use cases included tuning communication libraries from vendors like Intel Corporation MPI, analyzing topology-aware mappings on systems like Blue Gene/L and Blue Gene/P, and comparing performance on clusters built with processors from AMD and Intel Xeon. PARMACS informed optimization strategies for scientific codes developed at institutions such as Argonne National Laboratory, Pacific Northwest National Laboratory, and research projects like NERSC workloads, aiding performance engineers working with application suites like LAMMPS, GROMACS, OpenFOAM, and WRF.

Performance and Evaluation

PARMACS measurements helped quantify metrics including round-trip latency, sustained bandwidth, and small-message throughput under varying process counts and topologies exemplified by systems at Oak Ridge National Laboratory and Lawrence Berkeley National Laboratory. Comparative studies reported in venues such as IEEE Transactions on Computers and ACM Transactions on Computer Systems used PARMACS to evaluate effects of network contention, rendezvous protocols, eager protocols, and NIC offload features in hardware from Mellanox Technologies and Broadcom. Results guided improvements in implementations of MPI libraries from organizations like Argonne National Laboratory (MPICH) and Open MPI Project.

Limitations and Criticism

Critics noted PARMACS's focus on synthetic kernels rather than full applications, drawing comparisons to criticisms leveled at LINPACK and prompting calls for complementary suites like SPEC and the NAS Parallel Benchmarks. The benchmark's emphasis on small-message behavior limits its ability to model complex communication patterns found in codes such as PETSc-based solvers, Trilinos packages, and large-scale multiphysics frameworks used in projects at Sandia National Laboratories and Los Alamos National Laboratory. Additionally, updates lagged relative to rapid changes in interconnect hardware and APIs from vendors such as NVIDIA (GPUDirect) and cloud providers like Amazon Web Services implementing HPC offerings, leading to efforts to extend or replace PARMACS with more contemporary microbenchmark suites.

Category:Parallel computing benchmarks