HPACK

HPACK
Name	HPACK
Developer	Internet Engineering Task Force (IETF)
First published	2015
Status	Final
Related	HTTP/2, QUIC

HPACK is a header compression format designed for use with HTTP/2 to reduce overhead and improve performance between client (computing), server (computing), and intermediary proxies. It specifies a static table of common header fields and a dynamic table that adapts to runtime traffic patterns, balancing compression ratio with latency and memory constraints. HPACK aims to minimize redundant header bytes while avoiding vulnerabilities known from earlier compression schemes, enabling safer transport in multipath and persistent connection scenarios.

Overview

HPACK was developed within the Internet Engineering Task Force working group alongside protocols such as HTTP/2 and later influenced components of QUIC header compression. The specification defines two core operations: representation of headers using indexed and literal forms, and stateful management of a dynamic table shared per-connection between client (computing) and server (computing). Implementers include major projects like nginx, Apache HTTP Server, Google Chromium, Mozilla Firefox, Microsoft Edge, cURL, and libraries such as nghttp2 and libnghttp2. Operational deployment touches large-scale infrastructures operated by organizations like Google, Facebook, Amazon (company), Cloudflare, Akamai Technologies, and Fastly.

Compression Mechanism

HPACK uses a combination of a predefined static table and a per-connection dynamic table to encode header fields. The static table mirrors common fields seen in requests and responses handled by Nginx, Apache HTTP Server, and Microsoft IIS, enabling compact indexed representations. Encodings include indexed representation, literal representation with incremental indexing, and literal without indexing, choices that influence behavior in stacks such as Node.js, Go (programming language), Rust (programming language), and Java (programming language). The integer representation employs a variable-length prefix scheme inspired by techniques used in Extensible Binary Meta Language and similar binary protocols. Huffman coding in HPACK reduces size for repetitive strings and is influenced by canonical Huffman techniques used in Deflate and RFC 7541 derivations, impacting implementations in OpenSSL, BoringSSL, and LibreSSL.

Dynamic Table and Eviction

The dynamic table holds recently seen header field name/value pairs and is managed using insertions and evictions to respect a maximum size setting. Size updates are negotiated during connection setup and may be adjusted by endpoints including Chrome (web browser), Firefox (web browser), and Edge (web browser). Eviction follows a least-recently-added policy that affects memory usage in servers run by Netflix, Yahoo, Spotify, and GitHub, and influences caching strategies in reverse proxies like Varnish, Traefik, and HAProxy. The dynamic table model requires careful coordination during stream multiplexing in HTTP/2 sessions across services hosted on platforms such as Amazon Web Services, Microsoft Azure, and Google Cloud Platform.

Security Considerations

HPACK was designed to mitigate attacks exemplified by compression-related vulnerabilities observed in contexts involving Transport Layer Security and protocols that compress plaintext before encryption. The dynamic table and choices such as "never indexed" help prevent side-channel leakage exploited in cases analogous to attacks discussed in IETF security analyses and incident responses involving Heartbleed. Implementers in projects like OpenBSD, FreeBSD, NetBSD, and operating systems from Apple Inc. and Canonical (company) must consider bounds checking and resource exhaustion to avoid denial-of-service vectors. HPACK influenced later designs in QPACK for use with HTTP/3 on QUIC to address head-of-line blocking and speculative stream decoding vulnerabilities that affected concurrent systems in high-throughput environments used by Twitter, LinkedIn, and Pinterest.

Implementations and Performance

Widely used implementations include protocol stacks in nghttp2, Envoy (software), HAProxy, Traefik, LiteSpeed Technologies, Caddy (web server), and language-native libraries for Python (programming language), Ruby (programming language), PHP, Perl, Erlang, and Elixir (programming language). Performance studies have compared HPACK behavior under workloads seen by content delivery networks operated by Akamai Technologies and Cloudflare as well as large-scale services by Google and Amazon (company), showing trade-offs between compression gains and CPU cost similar to those examined in benchmarking tools like wrk and ab (ApacheBench). Profiling in environments using Linux, FreeBSD, and Windows Server demonstrates that Huffman decoding can be a CPU hotspot in constrained devices such as those running OpenWrt or embedded stacks produced by Espressif Systems.

History and Standardization

HPACK was standardized by the Internet Engineering Task Force as part of the HTTP/2 suite and published alongside RFCs governing binary framing and header semantics. Key contributors came from organizations including Google, Mozilla Foundation, Microsoft, Akamai Technologies, and Fastly, and work progressed through mailing list discussions and meetings at IETF events such as IETF 86 and IETF 87. Subsequent evolution informed QPACK for HTTP/3 within the IETF QUIC working group, reflecting lessons learned from deployment at scale by companies like Facebook, Netflix, and Twitter. The specification remains a critical component of modern web protocol stacks maintained across open-source projects and commercial products.

Category:Internet protocols