SOM

SOM
Name	SOM
Field	Neural networks
Invented	1982
Inventor	Teuvo Kohonen
Related	Artificial neural network, Unsupervised learning, Dimensionality reduction

SOM

A self-organizing map (commonly abbreviated SOM) is an unsupervised artificial neural network used for dimensionality reduction and data visualization. It projects high-dimensional data onto a usually two-dimensional lattice preserving topological relationships, enabling clustering and pattern discovery across datasets from domains such as bioinformatics, remote sensing, finance, and image processing.

Overview

SOMs were introduced by Teuvo Kohonen and belong to a family of neural models alongside Hopfield network, Boltzmann machine, and Kohonen network variants. They operate using competitive learning similar to mechanisms in Hebbian theory and align with manifold learning approaches like t-distributed stochastic neighbor embedding and principal component analysis. Architecturally, a SOM maps inputs to discrete units on a grid, with neighborhood functions inspired by kernels used in Gaussian process models and smoothing methods in Kernel density estimation.

History

The concept was formalized in 1982 by Teuvo Kohonen during research at the University of Helsinki and later developed in publications across venues such as the IEEE Transactions on Neural Networks and proceedings of the Neural Information Processing Systems conference. Early applications appeared in ecological studies and speech analysis, paralleling developments in self-organization theory influenced by work at institutions like the Santa Fe Institute and research by figures associated with Complexity science. Over subsequent decades, SOMs were integrated into toolboxes alongside algorithms from Signal Processing labs and used in industry by companies including Siemens and Hitachi for pattern recognition.

Structure and Function

A SOM consists of an input layer connected to a typically two-dimensional output lattice of neurons; learning updates are localized via a neighborhood function analogous to kernels in Gaussian blur operations used by Adobe Systems software and image analysis in NASA missions. During training, competitive selection identifies a best-matching unit, then weight vectors of that unit and neighbors are adjusted using a learning rate schedule similar to annealing schemes from Simulated annealing and training strategies seen in Stochastic gradient descent literature. Common neighborhood topologies include rectangular and hexagonal grids reminiscent of tiling problems studied by scholars at Princeton University and tiling research in Mathematics departments. Implementation often leverages numerical libraries originating from projects such as LAPACK and BLAS for vector operations.

Applications

SOMs have been applied widely: in bioinformatics for microarray analysis by groups at Harvard Medical School and European Bioinformatics Institute; in remote sensing for land-cover classification used by European Space Agency and United States Geological Survey; in finance for portfolio clustering by analysts at firms like Goldman Sachs and Morgan Stanley; and in image compression and color quantization in products developed by Google and Apple Inc.. Other deployments include fault detection in industrial plants maintained by General Electric and ABB, document clustering in projects at IBM Research and Microsoft Research, and customer segmentation in marketing operations at Procter & Gamble.

Variants and Extensions

Extensions include the Growing Grid and Growing Neural Gas proposed by researchers such as Bernd Fritzke, integration with supervised layers as in hybrid architectures evaluated at MIT, and temporal extensions like the Temporal Kohonen Map used in speech labs at Bell Labs. Combining SOMs with clustering algorithms like k-means or hierarchical clustering has been explored by teams at Stanford University and University of California, Berkeley. Other notable modifications include batch training algorithms often used in implementations from the R Project for Statistical Computing and incremental variants employed in robotics research at Carnegie Mellon University.

Criticisms and Limitations

Critiques of SOMs highlight issues such as sensitivity to initialization and hyperparameters noted in comparative studies from University College London and ETH Zurich, and difficulties scaling to extremely high-dimensional data compared to deep learning models developed at OpenAI and DeepMind. Topology preservation can be imperfect for complex manifolds, a problem discussed alongside limitations of multidimensional scaling and other projection techniques in reviews by the Royal Society. Additionally, interpretability of map units can be challenging in domains regulated by laws like the General Data Protection Regulation where explainability is required, leading practitioners to prefer alternative methods from institutions such as NIST.

Category:Artificial neural networks