GPT (language model)
Generated by GPT-5-miniExpansion Funnel Raw 94 → Dedup 0 → NER 0 → Enqueued 0
| GPT (language model) | |
|---|---|
.png) | |
| Name | GPT |
| Type | Autoregressive language model |
| Developer | OpenAI |
| First release | 2018 |
| Latest release | 2023 |
| Primary language | English (multilingual capabilities) |
| License | Proprietary (various) |
| Website | OpenAI |
GPT (language model) is a family of autoregressive transformer-based models developed to generate human-like text and perform language understanding tasks. It was introduced by OpenAI and has influenced research and deployment across industry and academia, impacting projects associated with Microsoft, Google, Meta Platforms, NVIDIA, Amazon Web Services, and numerous universities. The models have been compared and contrasted with systems from DeepMind, IBM Research, Stanford University, and other research labs in the fields of artificial intelligence, natural language processing, and computational linguistics.
Overview
GPT models implement the transformer architecture introduced by researchers at Google Research and popularized in the paper "Attention Is All You Need" authored by scientists affiliated with Google Brain and Google DeepMind collaborators. The lineage of GPT is often discussed alongside other influential systems such as BERT (from Google Research), T5 (from Google Research and Google Brain), and sequence models studied at institutions like MIT, Harvard University, University of California, Berkeley, and Carnegie Mellon University. Industrial ecosystems that adopted GPT-like models include products and services from Microsoft Azure, Amazon, Salesforce, SAP, and Adobe. Governmental and regulatory bodies including the European Commission, United Kingdom Information Commissioner's Office, and United States Federal Trade Commission have examined implications of large pretrained models in policy contexts alongside academic ethicists at Oxford University and Cambridge University.
Architecture and Training
GPT's architecture is rooted in the transformer decoder stack, featuring self-attention, layer normalization, and position-wise feed-forward networks developed from the original transformer research by Google Research teams like Ashish Vaswani et al. Training regimes involve large-scale unsupervised pretraining on web text corpora and subsequent fine-tuning or reinforcement learning with human feedback, practices refined in collaborations between OpenAI and partners such as Microsoft Research and compute providers like NVIDIA (GPU) and Google Cloud. Datasets and pretraining pipelines referenced in the literature often include corpora assembled by groups at Common Crawl, The Allen Institute for AI, Stanford NLP Group, and datasets curated at Hugging Face. Optimization techniques include Adam variants introduced by researchers at Diederik P. Kingma's groups and scaling laws characterized in papers by teams at OpenAI and DeepMind. Infrastructure for training has drawn on supercomputing efforts at facilities associated with Argonne National Laboratory, Lawrence Livermore National Laboratory, and high-performance clusters maintained by corporations like Microsoft and Google.
Model Versions and Evolution
Early generative pretrained transformers were released in iterations with increasing parameter counts and capabilities. Notable milestones include model families and releases that prompted comparisons with work from groups at Facebook AI Research (FAIR), DeepMind (e.g., Gopher), and academic projects at University of Toronto and ETH Zurich. Industry partnerships between OpenAI and Microsoft produced integrated deployments within products such as Microsoft Word and GitHub Copilot. The evolution involved changes inspired by research from Stanford's Human-Centered AI center, architecture refinements paralleling work at Carnegie Mellon University, and safety-oriented updates influenced by policy teams at The Brookings Institution and Center for Security and Emerging Technology.
Capabilities and Applications
GPT models have been applied to tasks spanning code generation, content summarization, machine translation, question answering, dialogue systems, and creative writing, often compared to benchmarks and systems from Google DeepMind, IBM Watson, Microsoft Research, and startups incubated at Y Combinator. Industry applications extend into domains served by McKinsey & Company, Accenture, Deloitte, PwC, and KPMG for business intelligence, as well as integration into products by Salesforce, Slack Technologies, and Zoom Video Communications. Research deployments appear in collaborations with hospitals and medical research centers such as Mayo Clinic and Johns Hopkins University for prototyping clinical note summarization, while legal technology firms working with courts and law schools like Yale Law School and Harvard Law School have studied document review use cases. Education platforms at organizations like Coursera, edX, and Khan Academy have experimented with tutoring assistants leveraging such models.
Limitations, Risks, and Ethics
Limitations of GPT include susceptibility to hallucinations, sensitivity to prompt phrasing, and biases reflecting training data; these concerns have been examined by ethicists at MIT Media Lab, AI Now Institute, Harvard Berkman Klein Center, and policy researchers at RAND Corporation. Risks include misuse for disinformation campaigns studied in contexts involving incidents linked to platforms like Twitter (now X), Facebook (Meta), and regulatory scrutiny by the European Commission and national agencies such as the U.S. Department of Justice. Ethical discussions reference frameworks and guidelines from bodies including IEEE Standards Association, Partnership on AI, OpenAI Ethics Team, and advisory reports from UNESCO, OECD, and the Council of Europe. Mitigations have involved methods originating in research at Stanford HAI, safety audits paralleling practices at DeepMind Ethics & Society, and legal analyses by scholars associated with Columbia Law School and NYU School of Law.
Evaluation and Benchmarks
Evaluation of GPT models uses standard benchmarks and tasks developed by communities at GLUE, SuperGLUE, SQuAD (Stanford), COCO (Microsoft/University of North Carolina collaborations), and specialized leaderboards maintained by Hugging Face and academic consortia at Papers with Code. Comparative evaluations include studies with models from Google Research, DeepMind, Meta AI Research, and open-source initiatives such as those at EleutherAI and BigScience. Performance metrics are supplemented by human evaluations conducted in partnerships with organizations such as Prolific, Amazon Mechanical Turk, and institutional review boards at research universities including Princeton University and University of Cambridge.