This article was accepted into the corpus but its outbound wikilinks were never NER-processed — typical at the deepest BFS hop or when the run's entity cap was reached. No expansion funnel to show.
| Cramér–Rao bound | |
|---|---|
| Name | Cramér–Rao bound |
| Field | Statistics |
| Introduced | 1946 |
| Named after | Harald Cramér; C. R. Rao |
Cramér–Rao bound The Cramér–Rao bound is a fundamental result in statistical estimation theory that gives a lower bound on the variance of unbiased estimators for a parameter, connecting concepts in probability, information, and inference. Developed from work by Harald Cramér and Calyampudi Radhakrishna Rao, it unifies ideas used across applications in signal processing, econometrics, and physics. The bound links the Fisher information to estimation efficiency and appears alongside other limits such as the Bhattacharyya bound and the Chapman–Robbins bound.
The Cramér–Rao bound arises in the context of parametric models such as those studied by Andrey Kolmogorov, Ronald Fisher, and Jerzy Neyman, situating it among classical results like the Neyman–Pearson lemma and the Rao–Blackwell theorem. It is applied when one models data using families like the exponential family of distributions described by S. N. Bhattacharyya and is central to methods developed by researchers at institutions like Bell Labs and Princeton University. The bound informs design and analysis in domains influenced by figures such as Claude Shannon, John von Neumann, and Norbert Wiener.
Formally, consider a parametric probability density or mass function p(x; θ) with parameter θ as in works by Egon Pearson and Jerzy Neyman. Let T(X) be an unbiased estimator for θ under regularity conditions of the likelihood, concepts refined by R. A. Fisher and C. R. Rao. The Cramér–Rao inequality states Var[T(X)] ≥ 1 / I(θ), where I(θ) is the Fisher information introduced by Ronald Fisher and expanded by C. R. Rao and H. Cramér. Multivariate generalizations use the Fisher information matrix studied in texts by Andrey Kolmogorov and Thomas Bayes-related scholarship, and are expressed as covariance matrices bounded by the matrix inverse of the information, an approach used in work by Herman Chernoff and Z. W. Birnbaum.
The usual regularity conditions require interchange of differentiation and integration as in treatments by William Feller and Joseph L. Doob, and assume support of p(x; θ) does not depend on θ, a constraint also discussed by Persi Diaconis and David Cox. Additional smoothness and identifiability assumptions echo themes in the work of Kolmogorov and Andrey Markov. These conditions are connected to asymptotic theory developed by Jacob Wolfowitz, Lehmann and Scheffé, and researchers at Harvard University and Stanford University.
Standard derivations follow by applying the Cauchy–Schwarz inequality as used in proofs by Émile Borel and Stefan Banach, relating score functions to unbiased estimators, a method refined by C. R. Rao and Harald Cramér. Alternative proofs use information geometry pioneered by Shun'ichi Amari and follow ideas of Bernard van der Waerden and Norbert Wiener. Connections to the Kullback–Leibler divergence popularized by Solomon Kullback and Richard Leibler provide an information-theoretic perspective echoed by David Hilbert-style functional analysis and work in asymptotic efficiency by Lucien Le Cam.
The bound’s properties include invariance under reparameterization, echoing principles in Élie Cartan-inspired differential geometry and in the Fisher information metric developed by Shun'ichi Amari. It yields insight into the Cramér–Rao lower bound’s role as an efficiency benchmark akin to bounds studied by Harold Hotelling and C. R. Rao. Interpretations relate to limits in estimation precision appearing in engineering contexts influenced by Claude Shannon and Norbert Wiener, and to quantum analogues explored by Hans Bethe-adjacent quantum information researchers and institutions such as MIT and Caltech.
Attainability conditions tie to the exponential family where maximum likelihood estimators (MLEs) by Ronald Fisher and Sir David Cox achieve the bound asymptotically under regularity, a result developed further by Hendrik C. Tijms and John A. Nelder. Rao–Blackwellization and Lehmann–Scheffé theory, championed by E. L. Lehmann and Henry Scheffé, describe improvements toward efficiency. Finite-sample attainment occurs in specific models like the normal family investigated by Carl Friedrich Gauss and Adrien-Marie Legendre via least squares; in other settings, bounds like the Barankin bound and Hájek–Le Cam convolution theorem by Jaroslav Hájek and Lucien Le Cam describe limits to attainability.
Concrete examples include location and scale families such as the normal distribution used by Carl Friedrich Gauss and Pierre-Simon Laplace in regression and time-series contexts influenced by George Udny Yule and Norbert Wiener. Signal processing applications at Bell Labs and in radar influenced by Solomon Kullback use the bound to assess estimator performance; econometric applications trace to Trygve Haavelmo and James Heckman in model identification. In experimental physics, high-energy experiments at CERN and observational astronomy at Mount Wilson Observatory employ Fisher information approaches; in machine learning, methods at Google, OpenAI, and DeepMind relate the bound to gradient-based estimation and variants like the Bayesian Cramér–Rao bound studied by Harold Jeffreys and Dennis Lindley.
Category:Statistical theory