This article was accepted into the corpus but its outbound wikilinks were never NER-processed — typical at the deepest BFS hop or when the run's entity cap was reached. No expansion funnel to show.
| Beam search | |
|---|---|
| Name | Beam search |
| Type | Heuristic search |
| Inventors | Henry Kautz and Judea Pearl |
| Year | 1977 |
| Domain | Natural language processing, speech recognition, machine translation |
| Related | A*, greedy search, breadth-first search, depth-first search |
Beam search
Beam search is a heuristic graph search algorithm that explores a graph by expanding the most promising nodes within a fixed-width set called the beam. It trades optimality for tractability by maintaining a bounded frontier of partial solutions, making it widely used in large-scale sequence prediction tasks in IBM, Google, Microsoft, OpenAI, and research groups at Stanford University and Massachusetts Institute of Technology. The method underpins practical systems in automatic speech recognition, statistical machine translation, and modern neural sequence models developed at laboratories such as DeepMind and Facebook AI Research.
Beam search was introduced in the late 20th century as a practical compromise between exhaustive search algorithms like A* and simple heuristics such as greedy best-first strategies. It restricts exploration to a fixed number of best-scoring partial hypotheses at each step, where scoring functions often derive from models trained on corpora collected by institutions like Carnegie Mellon University, University of California, Berkeley, and University of Toronto. The algorithm became central to systems deployed by companies including Apple Inc., Amazon, and Baidu for tasks where latency and memory constraints preclude exact inference.
The core algorithm initializes a set containing a start state and iteratively expands all candidates, scores successor states using a domain model, and prunes to retain the top-k candidates, where k is the beam width. Scoring frequently uses log-probabilities from statistical models such as n-gram language models developed in work at Google Research or probability estimates from neural architectures pioneered at University of Oxford and University College London. At each timestep the algorithm applies domain-specific constraints derived from resources like the Penn Treebank or the ImageNet dataset when applied to structured prediction. Variants of scoring incorporate length normalization inspired by methods in research from Microsoft Research and reranking often employs discriminative models inspired by techniques from Johns Hopkins University.
Numerous adaptations improve diversity, quality, and correctness. Diverse beam search introduces mechanisms to encourage varied hypotheses, drawing motivation from ensemble methods used at Los Alamos National Laboratory and ideas explored in the context of NeurIPS and ICML conferences. Beam search with constraints integrates finite-state transducers and weighted automata studied at AT&T Labs Research and in work by scholars affiliated with Columbia University. Algorithms combining beam search with dynamic programming borrow techniques related to algorithms from Bell Labs and classical treatments in texts by authors associated with Princeton University. Improvements like length-penalty adjustments, coverage penalties, and stochastic beam search reflect innovations reported at ACL and EMNLP workshops.
Beam search is ubiquitous across applications that require sequential decision making under resource limits. In automatic speech recognition systems deployed by Nuance Communications and companies such as Google, beam search decodes acoustic model outputs into word sequences using vocabularies drawn from datasets like the Switchboard corpus. In machine translation, systems developed at SYSTRAN and research programs at Microsoft Translator rely on beam decoding to generate candidate translations. In image captioning engines used by Pinterest and research at Adobe Systems, beam search composes natural language descriptions from convolutional or transformer-based encoders originating in work at University of Toronto. Robotics planners in laboratories like MIT CSAIL and Carnegie Mellon University use beam-style heuristics for trajectory generation under real-time constraints.
Evaluating beam search involves trade-offs among speed, memory, and output quality. Larger beam widths often improve scores on automatic metrics popularized by initiatives at BLEU-related workshops and benchmarks established by WMT and GLUE, but can lead to diminishing returns and longer latency noted in industrial reports from Amazon Web Services and Google Cloud. Beam search is not guaranteed to find globally optimal solutions; examples contrasting it with exact inference appear in theoretical treatments from Stanford University and counterexamples discussed at COLT. Additionally, deterministic pruning can reduce output diversity, an issue analyzed in studies presented at NAACL and addressed by diversity-promoting variants.
Practical implementations optimize beam operations for throughput on hardware from NVIDIA and Intel Corporation, leveraging batch processing and efficient memory layouts used in toolkits like TensorFlow, PyTorch, and libraries originating from Theano. Time complexity per timestep is O(k · b · log k) for beam width k and average branching b when using priority queues, while memory scales as O(k · L) for sequence length L. Engineering challenges include parallelization across accelerators in data centers run by Google Cloud Platform and correctness when integrating with finite-state decoders designed by teams at Lucent Technologies and Siemens AG.
Category:Search algorithms