Markov random field

Markov random field
Cmglee · CC BY-SA 4.0 · source
Name	Markov random field
Discipline	Probability theory, Statistical mechanics, Machine learning
Introduced	20th century

Markov random field

A Markov random field is a probabilistic model for spatially or relationally indexed variables used in Andrey Markov-inspired contexts in Statistical mechanics, Probability theory, Computer vision, Machine learning and Spatial statistics. It encodes conditional independence via an undirected graph relating random variables, linking ideas from Ising model, Gibbs measure, Hammersley–Clifford theorem and developments in Alan Turing-era computation, enabling connections to algorithms developed at Bell Labs, Microsoft Research, Google Research and theoretical work from institutions like Princeton University and Cambridge University.

Definition and basic properties

A Markov random field is defined on an undirected graph where each vertex corresponds to a random variable and edges encode conditional dependence; this allows the model to express local Markov properties related to the work of Andrey Kolmogorov, Emanuel Parzen, Norbert Wiener, and the probabilistic foundations advanced at University of Chicago and Harvard University. The local, pairwise and global Markov properties relate to conditional independence statements studied by Thomas Bayes-inspired scholars and formalized alongside results from Oskar Morgenstern and John von Neumann. The positivity condition that underlies equivalence to Gibbs distributions connects to results proven by mathematicians associated with Institute for Advanced Study and University of Cambridge.

Mathematical formulation

Formally, given an undirected graph with vertex set V and edge set E, the joint distribution factors according to cliques via clique potentials in the spirit of the Gibbs distribution and the Hammersley–Clifford theorem first articulated following foundational work at University of Oxford and École Normale Supérieure. Parameterizations include exponential family forms related to results from Jerzy Neyman and Eğer Etem Özalp-style statistical theory; maximum likelihood and pseudo-likelihood estimators relate to asymptotic theory treated in the traditions of Ronald Fisher and later statisticians at Columbia University and Stanford University. Connections to partition functions recall partition problems studied in Ludwig Boltzmann's era and analytic techniques used at Max Planck Institute and Princeton Plasma Physics Laboratory.

Examples and common models

Common instances include the Ising model prominent in Wilhelm Lenz's lineage and studied extensively by researchers at Los Alamos National Laboratory and CERN, the Potts model which generalizes binary interactions in statistical physics centers like École Polytechnique, and pairwise binary fields used in early ImageNet-era Computer vision research at Stanford Vision Lab and MIT Computer Science and Artificial Intelligence Laboratory. Other structured models such as conditional random fields were advanced by teams at Johns Hopkins University and Carnegie Mellon University and are applied in sequence labeling tasks tied to work at Google and Microsoft.

Inference and learning algorithms

Inference methods for these fields draw on algorithms like belief propagation developed in connection with work at AT&T Bell Laboratories and later applied by researchers at Facebook AI Research and DeepMind, while Markov chain Monte Carlo techniques trace to milestones at Los Alamos National Laboratory and theoretical advances at University of California, Berkeley. Learning approaches include maximum likelihood training, contrastive divergence associated with research from Geoffrey Hinton and collaborators at University of Toronto, and variational methods influenced by methods from Yann LeCun's groups and the statistical learning theory community at Massachusetts Institute of Technology and University of Cambridge.

Applications

Applications span image restoration and segmentation pioneered at Toyota Technical Institute, natural language processing influenced by projects at IBM Research and Microsoft Research, spatial statistics problems investigated by teams at United States Geological Survey and National Aeronautics and Space Administration, and networked data modeling explored at Facebook and Twitter. In bioinformatics, models related to these fields have been used in structural prediction research at Cold Spring Harbor Laboratory and European Molecular Biology Laboratory, while econometric-style spatial dependence problems have been studied in contexts involving World Bank-backed research and policy groups at International Monetary Fund.

Relation to other probabilistic models

These undirected models contrast with directed graphical models such as Bayesian networks developed by researchers at Carnegie Mellon University and Stanford University; they relate to factor graphs used in coding theory research at École Polytechnique Fédérale de Lausanne and to energy-based models studied by groups at Facebook AI Research and Google DeepMind. Connections to latent-variable models touch on topics explored at Princeton University and Columbia University, while theoretical correspondences to statistical physics link back to contributions by Ludwig Boltzmann and computational frameworks advanced at Los Alamos National Laboratory and CERN.

Category:Probabilistic models