polyhedral compilation

polyhedral compilation
Name	Polyhedral compilation
Discipline	Computer science
Subdiscipline	Compiler optimization
Firstappeared	1990s
Practitioners	Paul Feautrier, Francois Irigoin, Olivier Temam, Christophe Alias, Guy Streeter
Notableworks	Loop tiling, Loop fusion, Loop interchange, Dependence analysis

polyhedral compilation is a compiler optimization framework that models loop nests and array accesses using geometrical and algebraic structures to enable advanced transformations. It integrates concepts from Linear algebra, Integer linear programming, Affine transformations, Theoretical computer science, and Computational geometry to optimize programs for modern architectures like Intel Corporation processors and NVIDIA GPUs. The approach has been developed and applied by researchers from institutions such as INRIA, École Polytechnique, Stanford University, Massachusetts Institute of Technology, and companies including IBM and Google.

Overview

Polyhedral compilation represents iteration spaces and data accesses as integer polyhedra and uses algebraic operations to perform semantics-preserving transformations. Pioneering work by Paul Feautrier, Francois Irigoin, and collaborators at INRIA and École Polytechnique established the theoretical basis, while systems from IBM Research and academic groups at University of California, Berkeley and University of Illinois Urbana-Champaign demonstrated practical benefits. Tools like those developed at Stanford University and ETH Zurich integrate with compilers such as LLVM and GCC to apply transformations like Loop tiling, Loop interchange, and Loop fusion for improved cache locality and parallelism.

Mathematical Foundations

The method relies on representing iteration domains as convex sets defined by affine inequalities drawn from Linear programming and solved with techniques related to Integer linear programming and Polyhedral theory. Dependence constraints are expressed using affine relations and solved via algorithms connected to Fourier–Motzkin elimination and the Smith normal form. Reasoning about schedules uses integer matrices and concepts from Lattice (group), Matrix (mathematics), and Convex polytope theory, while optimization objectives may invoke results from Graph theory and Combinatorial optimization for minimizing communication or makespan.

Dependence Analysis and Representation

Dependences between array accesses are modeled as relations over integer tuples and captured with tools inspired by Presburger arithmetic and Automata theory. The exact representation often employs integer affine relations and uses dependence tests that trace lineage back to work at IBM Research and by researchers at École Normale Supérieure and University of Paris. Dependence graphs interact with scheduling through representations akin to those developed in Parallel computing research at Los Alamos National Laboratory and Lawrence Livermore National Laboratory.

Transformation Techniques

Transformations are computed by synthesizing new affine schedules and applying unimodular transformations, skewing, and tiling strategies rooted in the literature of Numerical linear algebra and Discrete geometry. Popular strategies include multi-level tiling influenced by cache hierarchies in Intel Xeon systems and wavefront scheduling used in efforts from Cray Research and Oak Ridge National Laboratory for stencil codes. Legality of transformations is ensured via dependence-preserving prescriptions attributed to Paul Feautrier and further extended by teams at University of Toronto and Tokyo Institute of Technology.

Implementation and Tools

Implementations exist as standalone frameworks and compiler passes, integrating with ecosystems like LLVM and GCC and research platforms from Rice University and University of California, San Diego. Prominent tools and frameworks originate from INRIA, Riken, ETSI, and corporate research labs at Intel Corporation and NVIDIA, and academic projects at École Polytechnique Fédérale de Lausanne and University of Cambridge. These tools often interface with solvers and libraries developed by groups at Max Planck Society and Los Alamos National Laboratory to handle integer programming and polyhedral manipulations.

Applications and Performance

Polyhedral techniques have been applied to dense linear algebra kernels studied at Lawrence Berkeley National Laboratory and Argonne National Laboratory, to stencil computations developed in projects at Sandia National Laboratories, and to high-performance computing codes used at CERN. They enable optimizations for vector units in processors from AMD and Intel, for manycore accelerators from NVIDIA and AMD Radeon Technologies Group, and for distributed systems researched by Microsoft Research and Google DeepMind. Empirical performance studies compare polyhedral-optimized codes against hand-tuned libraries like BLAS and implementations from teams at NVIDIA and Intel Math Kernel Library showing improvements in cache reuse, parallel scalability, and communication reduction reported by groups at University of Illinois Urbana-Champaign and University of Texas at Austin.

Limitations and Future Directions

Limitations include scaling to irregular programs and handling dynamic data structures, challenges echoed in work from Bell Labs Research and by researchers at Carnegie Mellon University. Future directions point to integration with machine learning models from OpenAI and DeepMind for heuristic selection, extension to heterogeneous systems investigated at Argonne National Laboratory and Oak Ridge National Laboratory, and tighter coupling with runtime systems developed at Netflix and Amazon Web Services for adaptive scheduling. Ongoing research at INRIA, ETH Zurich, Stanford University, and Massachusetts Institute of Technology aims to broaden applicability, reduce analysis cost, and combine polyhedral strategies with domain-specific languages such as those from TensorFlow and PyTorch.

Category:Compiler optimization