Neyman–Pearson lemma

Neyman–Pearson lemma The Neyman–Pearson lemma is a foundational result in statistical hypothesis testing that characterizes most powerful tests for distinguishing between two simple hypotheses. It gives a precise decision rule based on likelihood ratios that optimizes Type II error for a fixed Type I error, and it underpins methods used across statistical inference, signal processing, and econometrics. The lemma has influenced the development of modern testing theory associated with figures and institutions across 20th‑century mathematics and statistics.

Statement and interpretation

The lemma states that for testing a simple null hypothesis against a simple alternative, the most powerful test at a given significance level rejects the null precisely when the likelihood ratio of the observed data favors the alternative beyond a threshold. This result is central to the work of Jerzy Neyman, Egon Pearson, Ronald Fisher, Andrey Kolmogorov, and Harald Cramér in formalizing testing procedures and relates to constructions used at institutions like University of California, Berkeley, London School of Economics, University of Cambridge, University of Warsaw, and University of Chicago. In practice the rule is implemented by comparing probability densities or mass functions associated with the hypotheses; applications draw on theories developed in the communities around Princeton University, Columbia University, Harvard University, Yale University, and Stanford University. The likelihood‑ratio form connects to optimality concepts used by John von Neumann, Alfréd Rényi, and Norbert Wiener in decision and information theory.

Proof

The standard proof constructs tests by ordering sample outcomes according to the likelihood ratio and shows that any other test of the same size cannot achieve a lower Type II error. This argument uses convexity and variational principles reminiscent of methods employed by Andrey Kolmogorov, Richard von Mises, Emil Post, and Alonzo Church in foundational work on measure and decision problems. A common presentation invokes measure theory as in texts associated with Paul Halmos, Shizuo Kakutani, and I. M. Gelfand, and can be formalized using tools from functional analysis developed at places like Massachusetts Institute of Technology, University of Göttingen, and Princeton University. Alternative proofs exploit Neyman‑Pearson ordering and employ inequalities studied by Issai Schur, G. H. Hardy, and J. E. Littlewood.

Applications and examples

The lemma is applied to derive uniformly most powerful tests for parametric families in contexts considered by Karl Pearson, Florence Nightingale, William Sealy Gosset, and practitioners at Rothamsted Experimental Station. Classical examples include testing means in normal samples (used in work by Jerzy Neyman and Egon Pearson), binomial tests connected to studies by Thomas Bayes, Pierre-Simon Laplace, and Simeon Denis Poisson, and exponential family problems treated in monographs from Princeton University Press and Wiley. In engineering, the likelihood‑ratio rule underpins detectors in radar and communications developed by Claude Shannon, Harry Nyquist, Alan Turing, and Norbert Wiener; in econometrics it guides hypothesis testing in models advanced by Trygve Haavelmo, Lawrence Klein, and James Tobin; in biostatistics it informs trial design work associated with Austin Bradford Hill, Samuel Wilks, and William Gosset. Examples include simple versus simple tests yielding Neyman‑Pearson critical regions in coin‑toss experiments historically studied by John Venn, Augustin Cournot, and Thomas Simpson.

Extensions and generalizations

Generalizations expand the lemma to composite hypotheses, leading to uniformly most powerful tests and generalized likelihood ratio tests developed by Jerzy Neyman, Egon Pearson, Ronald Fisher, and later refined by researchers at Bell Labs, Bellcore, and AT&T. The Wald sequential testing framework of Abraham Wald and the decision‑theoretic formulations of Lehmann and Romano build on the lemma; extensions involve invariance principles as in work by Jerzy Neyman's contemporaries and modern asymptotic results associated with Lucien Le Cam, Erich Lehmann, Jack Kiefer, and J. Neyman's school. Connections to information criteria and model selection draw on ideas from Hirotugu Akaike, A. C. Aitken, and David Cox; nonparametric and robust adaptations relate to contributions by John Tukey, Peter Huber, and Jerome Friedman. In high‑dimensional contexts links are made to compressed sensing advances by Emmanuel Candès, David Donoho, and Terence Tao.

Historical background and origins

The lemma originated in a 1928 collaboration between Jerzy Neyman and Egon Pearson at a time when statistical foundations were being debated by Ronald Fisher, Karl Pearson, and William Sealy Gosset. Their formulation emerged amid institutional developments at University of Warsaw, University College London, and University of Cambridge, and it influenced statistical curricula at London School of Economics and University of Chicago. Theoretical lineage traces through probability theory contributions of Andrey Kolmogorov, Émile Borel, and Paul Lévy, and through decision theory lines that later involved John von Neumann and Oskar Morgenstern. The lemma's adoption across disciplines—from the numerical work at Bell Labs to clinical trials at Johns Hopkins University and economic modeling at Cowles Commission—reflects its broad impact on 20th‑century quantitative science.

Category:Statistical theorems