This article was accepted into the corpus but its outbound wikilinks were never NER-processed — typical at the deepest BFS hop or when the run's entity cap was reached. No expansion funnel to show.
| Shannon's theorem | |
|---|---|
| Name | Shannon's theorem |
| Field | Information theory |
| Statement | Maximum reliable communication rate over a noisy channel |
| Author | Claude E. Shannon |
| Date | 1948 |
Shannon's theorem Claude E. Shannon's 1948 result establishes the fundamental limit on reliable information transmission over a noisy communication channel, asserting that for a given channel there exists a maximum rate—called channel capacity—below which arbitrarily small error probability is achievable and above which reliable communication is impossible. The theorem sits at the nexus of Claude Shannon, Bell Labs, 1948 in science, and the modern disciplines of electrical engineering, computer science, and cryptography, and it underpins technologies developed by AT&T, IBM, Bell Telephone Laboratories, and contemporary firms such as Intel and Google.
Shannon's theorem states that for a specified stochastic channel model—such as the binary symmetric channel, the additive white Gaussian noise channel, or a general discrete memoryless channel—there exists a nonnegative real number C, the channel capacity, with the following property: for any information rate R < C there exist coding and decoding schemes (invented by engineers affiliated with Bell Labs, MIT, Princeton University, Harvard University) enabling transmission with arbitrarily small error probability, whereas for R > C no such schemes exist. The theorem connects concepts from probability theory, measure theory, ergodic theory, and statistics, and it uses constructs related to Markov chains, stochastic processes, typicality, and entropy as developed in the milieu of Norbert Wiener, Andrey Kolmogorov, and John von Neumann.
The result emerged from Shannon's landmark 1948 paper at Bell Telephone Laboratories published in the Bell System Technical Journal during an era shaped by World War II, the Cold War, and advances from institutions such as MIT Radiation Laboratory and Harvard Radio Research Laboratory. Shannon synthesized earlier work on thermodynamics analogies to entropy by Ludwig Boltzmann and Josiah Willard Gibbs, mathematical foundations by Kolmogorov and Borel, and channel considerations influenced by engineers at AT&T and theorists like Ralph Hartley. Follow-up development involved researchers at Princeton University, Cambridge University, ETH Zurich, and University of California, Berkeley who extended Shannon's ideas into coding theory, including contributions by Richard Hamming, Marcel Golay, Andrew Viterbi, Gottfried Ungerboeck, and later work at Bell Labs that impacted standards promoted by IEEE and ITU.
The formal statement considers a discrete memoryless channel defined by a finite input alphabet and conditional distributions P(y|x). Shannon introduced the information-theoretic quantity mutual information I(X;Y) and defined capacity C = max_{P_X} I(X;Y). The direct part (achievability) constructs block codes using random coding arguments and the concept of typical sets, leveraging the law of large numbers, Asymptotic Equipartition Property, and concentration inequalities from Kolmogorov and Paul Lévy. The converse part (optimality) uses Fano's inequality and information inequalities to show that rates above C lead to nonvanishing error probabilities. Variations of proofs draw on methods from Markov chain theory, Large deviations theory via S. R. Srinivasa Varadhan and Cramér, and techniques from functional analysis and convex optimization in the spirit of John von Neumann and L. N. Trefethen.
Shannon's theorem underlies digital communications systems engineered by AT&T, Nokia, Ericsson, and Qualcomm; it informs compression standards by MPEG, ITU-T, and ISO; and it guides design of error-correcting codes implemented in technologies by Intel, Samsung, and Broadcom. The theorem has conceptual impact on debates involving Alan Turing-related computation limits, on statistical inference developments at Bell Labs and Microsoft Research, and on cryptographic protocols investigated at NSA and academic centers like Stanford University and MIT. It also motivated advances in coding families such as Reed–Solomon codes, turbo codes, LDPC codes, and polar codes developed by researchers at École Polytechnique, University of Cambridge, and Tsinghua University, with practical adoption in 3G, 4G, and 5G standards overseen by 3GPP.
Generalizations include channel models with memory studied by Shannon and later by Jorma Rissanen and Thomas Cover, continuous-time channels analyzed by Norbert Wiener methods, multiuser extensions like the multiple access channel and broadcast channel researched at Bell Labs and Princeton University, and quantum analogues developed in quantum information theory by researchers at IBM Research, Caltech, and Perimeter Institute. Network information theory, spearheaded by contributors such as El Gamal and Thomas Cover, expands capacity concepts to relay channels, interference channels, and wiretap channels explored by Wyner and Csiszár, while rate-distortion theory links to lossy compression problems tackled at Bell Labs and AT&T Bell Labs.
For the binary symmetric channel with crossover probability p, capacity equals 1 - H_2(p) where H_2 is the binary entropy used in analyses at Bell Labs and Princeton University; for p = 0.1 the capacity is approximately 1 - H_2(0.1) ≈ 0.531 bits per channel use, a figure relevant to system designers at Qualcomm and Ericsson. For the additive white Gaussian noise channel with signal-to-noise ratio SNR, capacity equals (1/2) log2(1+SNR) bits per channel use, a formula employed in link budgeting by Nokia and Huawei engineers. Practical code families such as LDPC codes and turbo codes approach these limits in deployments by Samsung and Intel across standards like Wi-Fi and LTE. Category:Information theory