This article was accepted into the corpus but its outbound wikilinks were never NER-processed — typical at the deepest BFS hop or when the run's entity cap was reached. No expansion funnel to show.
| CrayPat | |
|---|---|
| Name | CrayPat |
| Developer | Cray Research |
| Released | 1990s |
| Latest release | legacy / integrated into profiling toolchains |
| Programming language | Fortran, C, Fortran 77, C++ |
| Operating system | UNICOS, UNICOS/mk, Linux (via ports) |
| Genre | Performance analysis, profiling, tracing |
CrayPat
CrayPat is a performance analysis and profiling toolset originally developed for Cray supercomputers and later adapted to other high-performance computing environments. It provides profiling, event tracing, and call-path analysis tailored to applications written in Fortran, C, and C++ running on architectures derived from Cray Research systems and related parallel platforms. CrayPat has been used in production and research settings to optimize computational kernels, parallelization strategies, and I/O behavior for scientific workloads produced by collaborations involving institutions such as Argonne National Laboratory, Lawrence Livermore National Laboratory, and national centers like NERSC.
CrayPat bundles a lightweight instrumentation and sampling framework with post-processing utilities for visualizing time and hardware-event distributions across processes and threads. It supports profiling of applications compiled with compilers from vendors such as Cray Inc., GCC compilers, and vendors like Intel and PGI for heterogeneous platforms. The tool interoperates with system-level performance counter systems such as PAPI and vendor telemetry services used at centers including Oak Ridge National Laboratory and Sandia National Laboratories. CrayPat’s output formats influenced formats used by tools from HPCToolkit, TAU (Tuning and Analysis Utilities), and VampirTrace.
CrayPat comprises several modular components: an application-side instrumentation layer, a sampling daemon and low-overhead timers, a tracer for event recording, and a set of post-processing utilities for aggregation and visualization. The instrumentation hooks leverage compiler-generated metadata from Fortran 90, Fortran 77, and C++ symbol tables and use runtime libraries that interact with thread and process managers such as MPI implementations including MPICH, Open MPI, and vendor-specific MPI stacks used on systems like Cray XC series. Hardware counter integration is mediated through interfaces compatible with Perf (Linux tool), libpfm, and PAPI where available. CrayPat’s trace and profile files are consumed by visualization front ends such as Paraver, Jumpshot, and bespoke tools developed at computational centers.
Users typically enable CrayPat via compiler flags or runtime preload mechanisms that insert lightweight probes and sampling timers into application binaries. Typical workflows invoke CrayPat from batch systems using schedulers like Slurm Workload Manager or PBS Professional on machines managed by site administrators at facilities referencing NERSC or XSEDE allocations. After a run, data files are processed by utilities that produce call-path inclusive and exclusive metrics, flame graphs compatible views, and per-thread or per-process breakdowns useful when analyzing applications that employ OpenMP and MPI hybrid parallelism. Integration with debuggers such as GDB and performance suites like Valgrind is common in performance engineering cycles involving teams at Lawrence Berkeley National Laboratory and industrial partners like IBM.
CrayPat reports wall-clock exclusive and inclusive times, hardware counter-derived metrics (cache misses, branch mispredictions), MPI communication statistics (time in MPI_Send, MPI_Recv), I/O timing, and application-level counters exposed via user APIs. Its sampling model balances overhead against statistical fidelity and is suitable for large-scale runs on supercomputers including models developed by Cray Research successors and competitor designs from Fujitsu and HPE. Post-processing enables hotspot identification, load imbalance detection across processes on systems such as Titan (supercomputer) or Summit (supercomputer), and microarchitecture-specific tuning when matched with processor families from AMD or Intel Corporation. Analysts combine CrayPat outputs with roofline models and tools like Intel VTune Amplifier for cross-validation.
CrayPat integrates into broader performance engineering ecosystems. Outputs can be converted for consumption by visualization tools like Scalasca, ParaView, and Grafana dashboards when combined with telemetry collectors. It is often scripted into continuous integration pipelines for scientific software projects hosted on platforms such as GitHub or GitLab to detect regressions. Workflows at HPC centers harness job monitoring systems such as Ganglia and Prometheus to correlate CrayPat-derived application metrics with system telemetry from resource managers like Slurm and hardware management frameworks from HPE or Dell EMC.
CrayPat originated in the 1990s as part of performance tooling for Cray vector and parallel supercomputers produced by Cray Research. Development followed architecture transitions from vector processors to massively parallel processors and multithreaded nodes, with successive adaptations for UNICOS, UNICOS/mk, and eventually Linux environments. The tool evolved alongside influential projects and standards such as MPI and OpenMP, and through collaborations with national laboratories including Argonne National Laboratory and Sandia National Laboratories. Over time features were back-ported and inspired integration with community projects including PAPI and influenced commercial offerings from HPE and performance engineering suites used at centers like NERSC.
CrayPat has been criticized for limited portability outside legacy Cray ecosystems and for a learning curve associated with interpreting detailed call-path profiles and trace files. Comparisons with modern sampling frameworks such as HPCToolkit and event-based systems like eBPF highlight trade-offs in overhead, granularity, and ease of use. Some centers have moved to unified telemetry stacks and visualization ecosystems that favor open formats pioneered by projects such as OpenTelemetry and community tools like Scalasca and TAU (Tuning and Analysis Utilities), reducing reliance on vendor-specific tools. Nonetheless, CrayPat remains referenced in historical performance studies and as a model for profiling designs in HPC toolchains.
Category:Performance analysis tools