Channel capacity

Channel capacity
Name	Channel capacity
Field	Information theory
Introduced	1948
Introduced by	Claude Shannon
Related	Entropy, Mutual information, Noisy-channel coding theorem

Channel capacity is the maximum achievable rate at which information can be transmitted over a communication channel with arbitrarily low probability of error. It sits at the core of Claude Shannon's information theory and links foundational results such as the Noisy-channel coding theorem to practical designs in Bell Labs, AT&T, and modern digital infrastructures like Internet backbone systems. Channel capacity connects with concepts in Harry Nyquist's sampling theory, Nyquist–Shannon sampling theorem, and the work of Richard Hamming on error-correcting codes.

Definition and Basic Concepts

Channel capacity quantifies an information channel's ultimate throughput in bits per use or bits per second under specified constraints. The definition relies on entropy, mutual information, and probabilistic channel models such as the binary symmetric channel and the additive white Gaussian noise channel. Capacity is distinct from achievable rates in practical systems implemented by organizations like IEEE and standards bodies such as 3GPP or IETF; it is an ideal limit that informs design of turbo codes, low-density parity-check codes, and modulation schemes used by Wi-Fi and LTE.

Mathematical Formulation

Formally, for a discrete memoryless channel described by conditional probabilities P(y|x), capacity C is the supremum over input distributions P(x) of the mutual information I(X;Y). This optimization appears in work by Shannon and in textbooks by Thomas M. Cover and Joy A. Thomas and by David MacKay. For continuous channels like the additive white Gaussian noise channel, capacity expressions incorporate power spectral density constraints and use integrals rather than sums; famous derivations involve Gaussian distribution properties and Lagrange multipliers familiar from the calculus of variations and constrained optimization literature associated with John von Neumann's mathematical physics studies.

Classical Channel Capacity Theorems

The Noisy-channel coding theorem, proved by Claude Shannon in 1948, establishes that rates below capacity are achievable with vanishing error probability and rates above are not. The channel coding converse and achievability proofs use techniques such as typical sequences and random coding, tools that also appear in the works of Richard Gallager and Imre Csiszár. Extensions include the strong converse proven in contexts linked to researchers like Wolfgang Hoeffding and Sergey Verdú. Classical results tie into combinatorial coding bounds by Hamming and asymptotic equipartition properties developed in the post-war mathematical community including Andrey Kolmogorov's probability theory.

Examples and Applications

Canonical examples include the binary symmetric channel whose capacity is 1 − H2(p) with H2 the binary entropy, and the additive white Gaussian noise channel yielding C = 1/2 log(1+SNR) (per dimension) in analyses used by Claude Shannon and later by engineers at Bell Labs and Nokia. Applications span satellite communication operated by agencies like NASA, cellular networks designed by firms such as Ericsson, digital storage technologies from Hewlett-Packard and Seagate, and optical fiber systems advanced by Corning Incorporated and research groups at Bell Labs. Capacity limits guide spectrum allocation policies influenced by regulators like the Federal Communications Commission.

Computing and Estimating Capacity

Exact capacity computation often reduces to convex optimization; algorithms include the Blahut–Arimoto algorithm and numerical methods used in research at institutions like MIT and Stanford University. For channels with memory or feedback, techniques draw on state-space models familiar in the work of Norbert Wiener and estimation methods from Kalman filtering theory. Empirical estimation appears in experimental platforms deployed at CERN and national laboratories where measurement noise, sampling, and model selection affect convergence and bias.

Extensions and Generalizations

Extensions include multi-user capacities such as the multiple-access channel, broadcast channel, and relay channel, with seminal contributions from researchers like Thomas Cover and El Gamal. Quantum generalizations lead to quantum channel capacities studied by Peter Shor, Alexander Holevo, and institutions like IBM Research and Microsoft Research. Network information theory blends graph-theoretic perspectives from work at Bell Labs and algorithmic motifs explored at Carnegie Mellon University and Princeton University.

Practical Considerations and Limitations

Real-world systems face constraints not captured by ideal capacity: finite blocklength effects analyzed by researchers including Yury Polyanskiy and Vladimir V. Poor; implementation complexity in devices by companies like Qualcomm; regulatory spectrum limits enforced by bodies such as the International Telecommunication Union; and channel nonstationarity encountered in mobile environments studied by teams at Ericsson and Nokia. Trade-offs between latency, reliability, and throughput motivate practical codes like LDPC codes and turbo codes whose performance approaches capacity only under engineering compromises highlighted in standards from 3GPP and IEEE 802.11.

Category:Information theory