PEP 517

PEP 517
Title	PEP 517
Status	Final
Authors	Some Python core contributors
Created	2018
Python version	"Any"

PEP 517

PEP 517 defines a build-system interface for packaging in the Python ecosystem. It specifies how build frontends interact with build backends to produce distributable artifacts, and complements other packaging specifications and tools. The proposal influenced tools and projects across the Python landscape and interoperability efforts among packaging stakeholders.

Overview

PEP 517 introduces a standardized build-backend API to decouple build frontends like pip and installers such as setuptools from build backends like setuptools, flit, and poetry. The specification sits alongside other packaging efforts and aligns with work by core contributors, packaging working groups, and standards efforts influenced by projects such as Django, NumPy, and SciPy. The goal was to create a clear separation between projects maintained by organizations like Python Software Foundation, real-world ecosystems found in projects like Jupyter, and independent maintainers contributing to packaging infrastructure.

Motivation and Design

The motivation arose from incompatibilities between legacy build flows driven by setuptools and emergent projects such as poetry, flit, and distutils2. Stakeholders included maintainers from CPython, the Python Packaging Authority, maintainers of pip, and ecosystem projects including Twisted, Pyramid, and Zope. Design discussions referenced practices in continuous-integration contexts used by organizations like Mozilla and companies such as Dropbox, and drew on interoperability patterns present in projects like Kubernetes and Docker for modular tooling. The design emphasizes a minimal and explicit API surface, intended to enable alternative backends maintained by communities around projects like SQLAlchemy, Requests, and Matplotlib, while allowing frontends from teams responsible for pip and virtualenv to operate without backend-specific hacks.

Specification

The specification defines a number of backend entry points and build hooks such as get_requires_for_build_wheel, prepare_metadata_for_build_wheel, and build_wheel, described as callable interactions between frontends and backends. It specifies metadata handling that complements standards discussed in other proposals and repositories maintained by packaging authorities and organizations such as PyPI, Read the Docs, and GitHub. Error handling and lifecycle expectations are framed to support reproducible builds required by projects like CPython, TensorFlow, and PyTorch. The API contract aims to be stable so that tooling produced by large projects like Ansible, SaltStack, and Homebrew for Python packaging can rely on predictable behavior.

Implementation and Tooling

Multiple tooling stacks implemented the specification, including backends like setuptools_scm, flit, and poetry-core, and frontends like pip and build. Tool integration efforts involved maintainers from projects such as virtualenv, tox, and nox, and continuous-integration providers like Travis CI, CircleCI, and GitHub Actions. Packaging-heavy libraries such as pandas, scikit-learn, and Pillow adopted or adapted build metadata flows to interoperate with build backends. Tooling ecosystems maintained by companies such as Microsoft, Google, and Facebook incorporated compatibility layers to support the API in their developer tools and IDE integrations like PyCharm and Visual Studio Code.

Adoption and Impact

Adoption by major tools and repositories ensured interoperable build flows across projects like Django, Flask, Pyramid, and TurboGears, and in scientific stacks including NumPy, SciPy, and pandas. PyPI and package index mirrors adjusted ingestion workflows to rely on artifacts produced via the standardized interface. Enterprises and open-source foundations—such as Wikimedia Foundation, CERN, and NASA—saw benefits in reproducible packaging for projects like Odoo, Plone, and OpenStack. The specification influenced subsequent packaging proposals and discussions among groups involved with the Python Packaging Authority and the broader open-source ecosystem including Linux distributions like Debian and Fedora.

Compatibility and Migration Considerations

Migration from legacy systems required maintainers of projects such as setuptools, distutils, and legacy installer integrations to implement backend hooks or provide shim layers. Tooling transitions affected projects with complex build steps like Cython-based extensions in SciPy, compiled-extension workflows in Pillow, and binary wheels produced by projects including TensorFlow and PyTorch. Organizations operating large codebases—examples include Google, Dropbox, and Instagram—needed migration plans for CI pipelines and build infrastructures, while downstream packagers in distributions like Ubuntu and CentOS coordinated with upstream maintainers to ensure seamless transitions.

Category:Python packaging