Bayesianism

Bayesianism
Name	Bayesianism
Caption	Thomas Bayes (attributed)
Region	England
Era	18th century–present
Main figures	Thomas Bayes; Pierre-Simon Laplace; Thomas Henry Bayes; Ronald A. Fisher; Jerzy Neyman; Bruno de Finetti; Harold Jeffreys; Andrey Kolmogorov; Leonard J. Savage; Alan Turing; I. J. Good; David Cox; John Maynard Keynes; Edward Tufte; Karl Pearson; George Box; Bradley Efron; Andrew Gelman; Donald Rubin
Influences	Isaac Newton; Gottfried Wilhelm Leibniz; Pierre de Fermat; Blaise Pascal; John Maynard Keynes; Frank Ramsey; Richard von Mises
Notable works	An Essay towards solving a Problem in the Doctrine of Chances; Laplace's Théorie analytique des probabilités; Harold Jeffreys's Theory of Probability; Thomas Bayes's essay

Bayesianism is a framework for probabilistic inference that updates degrees of belief using conditional probability and Bayes's rule. It integrates prior information and new data to produce posterior probabilities, informing decision making, prediction, and hypothesis testing across scientific, medical, industrial, and policy domains. Proponents emphasize coherence and normative consistency while critics debate objectivity, computation, and choice of priors.

Overview and Key Principles

Bayesianism centers on Bayes's rule as a normative axiom connecting prior beliefs and likelihoods to posterior beliefs, drawing on foundations set by Thomas Bayes, Pierre-Simon Laplace, and later formalizers like Bruno de Finetti and Leonard J. Savage. Core principles include probabilistic coherence influenced by Andrey Kolmogorov's axioms, subjective priors articulated by Frank Ramsey and John Maynard Keynes, and decision-theoretic prescriptions developed by Savage and Frank P. Ramsey. Related methodological pillars feature model comparison via marginal likelihoods and Bayes factors employed by Harold Jeffreys and extended in modern software by teams at Stanford University, Princeton University, and Harvard University. Practitioners often reference algorithmic developments from Alan Turing, I. J. Good, and contemporary work at Massachusetts Institute of Technology and University of Cambridge to implement Monte Carlo methods such as Markov chain Monte Carlo popularized in part by research from Andrew Gelman and Donald Rubin.

History and Development

The historical arc begins with early probability work by Blaise Pascal, Pierre de Fermat, and mathematical consolidation by Isaac Newton and Gottfried Wilhelm Leibniz. The eponymous essay attributed to Thomas Bayes was extended by Pierre-Simon Laplace in Napoleon-era France and later synthesized by Harold Jeffreys in responses to debates involving Ronald A. Fisher and Jerzy Neyman on inference paradigms. Twentieth-century developments involved contributions from Karl Pearson, George Box, and John Tukey alongside foundational critiques by Bruno de Finetti and formal probability axiomatization by Andrey Kolmogorov. Mid-century computational and philosophical expansions were influenced by Alan Turing, I. J. Good, and Leonard J. Savage, while late twentieth- and early twenty-first-century advances emerged from groups at University of California, Berkeley, Columbia University, and Imperial College London focused on hierarchical modeling, empirical Bayes, and Bayesian nonparametrics.

Formal Foundations and Methods

Formal foundations rest on conditional probability and updating rules linked to work by Andrey Kolmogorov and likelihood principles debated by Ronald A. Fisher and Jerzy Neyman. Key methods include prior elicitation approaches from Bruno de Finetti and objective-prior frameworks proposed by Harold Jeffreys and operationalized by Jose M. Bernardo and James O. Berger. Computational strategies rely on Markov chain Monte Carlo algorithms such as Metropolis–Hastings, with roots traceable to Nicholas Metropolis and refined by W. K. Hastings; Hamiltonian Monte Carlo innovations relate to developments at Stanford University and software from Google and MC3. Model selection techniques include Bayes factors (Jeffreys), Bayesian information criterion comparisons influenced by Gideon E. Schwarz, and cross-validation practices linked to Bradley Efron and Herman Chernoff.

Applications and Case Studies

Bayesian methods underpin analyses in clinical trials at institutions like Mayo Clinic and Johns Hopkins University, adaptive designs in pharmaceutical work involving Food and Drug Administration review processes, and diagnostic systems used in radiology units at Massachusetts General Hospital. In ecology they inform species distribution modeling practiced at Smithsonian Institution and Nature Conservancy projects; in economics they support forecasting at Federal Reserve branches and policy evaluation at International Monetary Fund. Engineering deployments include reliability assessments by NASA and safety analyses for Boeing programs. Case studies span genetics and genomics research at Broad Institute, machine learning applications in industry by Google and Amazon, and astronomy surveys at European Southern Observatory and National Aeronautics and Space Administration telescopes.

Philosophical Debates and Criticisms

Philosophical debates revolve around subjectivity versus objectivity, with critics such as Ronald A. Fisher and proponents like Bruno de Finetti arguing about personalist interpretation and coherence criteria defended by Leonard J. Savage. Critics raise concerns about prior sensitivity exemplified in controversies involving high-profile medical meta-analyses at Cochrane Collaboration and policy modeling for World Health Organization scenarios. Debates also involve falsifiability discussions linked to viewpoints expressed in venues like Philosophy of Science and methodological exchanges at Royal Statistical Society meetings. Computational tractability and model misspecification issues have been highlighted in technical forums at Institute of Electrical and Electronics Engineers conferences and journals associated with American Statistical Association.

Variants and Related Approaches

Variants include objective Bayesianism advocated by Harold Jeffreys and Jose M. Bernardo, empirical Bayes methods developed by Bradley Efron and applied in genomics at Stanford School of Medicine, and hierarchical Bayes popularized by Andrew Gelman and used in political science at Princeton University. Related approaches encompass Bayesian nonparametrics advanced by researchers at University College London and University of Oxford, approximate Bayesian computation used in population genetics at University of Chicago, and Bayesian machine learning methodologies deployed at Carnegie Mellon University and DeepMind. Connections exist with decision theory from John von Neumann and Oskar Morgenstern and with information-theoretic perspectives discussed in outlets tied to Institute for Advanced Study and Santa Fe Institute.

Category:Probability theory