GPT

GPT.

Overview

GPT is a family of autoregressive large language models developed by OpenAI that generate human-like text across diverse domains. Influenced by research from Google Research, DeepMind, Stanford University, Carnegie Mellon University, Massachusetts Institute of Technology, and University of California, Berkeley, GPT implementations intersect with projects from Microsoft, Amazon, NVIDIA, Intel, and IBM. The model family follows trends established by architectures such as Transformer from Google Brain, and has been discussed in policy fora including United States Congress, European Commission, United Kingdom Parliament, United Nations panels, and industry events like NeurIPS and ICLR.

Architecture and Training

GPT uses a decoder-only variant of the Transformer architecture popularized by Google Brain researchers. Training has relied on large-scale datasets curated from sources associated with Common Crawl, Wikipedia, Project Gutenberg, arXiv, and corpora assembled in collaboration with corporate partners including Microsoft and cloud platforms such as Amazon Web Services, Google Cloud Platform, and Microsoft Azure. Optimization leverages practices from papers by teams at OpenAI, DeepMind, Google Research and employs hardware from NVIDIA GPUs and Google TPU accelerators. The development cycle involved evaluation methodologies inspired by benchmarks like GLUE, SuperGLUE, SQuAD, LAMBADA, and adversarial evaluations discussed at ICLR and NeurIPS. Safety and alignment research referenced frameworks from Allen Institute for AI, Partnership on AI, Future of Life Institute, and regulatory guidance by European Commission and United States Department of Commerce.

Variants and Implementations

Multiple commercially and academically distinct variants exist, produced by organizations including OpenAI, Microsoft, Anthropic, Cohere, Meta, Google DeepMind, and research groups at Stanford University and MIT. Implementations have been integrated into products from Microsoft Office, GitHub (owned by Microsoft), Salesforce, Zoom Video Communications, and startups incubated by Y Combinator. Open-source reimplementations and forks have been published by communities around Hugging Face, EleutherAI, BigScience Workshop, and research labs at Tsinghua University, Peking University, University of Toronto, and University of Washington.

Applications

GPT-style models power features in tools developed by Microsoft, Google, Apple Inc., Adobe Inc., and Meta for tasks including content generation for platforms like WordPress, code assistance in GitHub Copilot (a collaboration between GitHub and OpenAI/Microsoft), automated summarization for publishers such as The New York Times and The Washington Post, customer support solutions used by Salesforce and Zendesk, educational tools built by companies collaborating with Khan Academy, and research assistance in workflows at Elsevier and Springer Nature. Specialized deployments appear in healthcare pilot projects with institutions like Mayo Clinic, Mount Sinai Health System, and pharmaceutical collaborations involving Pfizer and Johnson & Johnson; legal tech integrations have been trialed by firms including DLA Piper and Baker McKenzie. GPT-driven features are also present in entertainment and gaming products from Electronic Arts, Netflix, and Unity Technologies.

Limitations and Risks

Limitations and risks have been examined by academics at Harvard University, Yale University, Princeton University, University of Oxford, and policy researchers at RAND Corporation and Brookings Institution. Known failure modes include hallucination and factual errors documented in papers from OpenAI, DeepMind, and Anthropic; biases reflecting training data distribution discussed in studies involving AI Now Institute and Data & Society Research Institute; and adversarial and misuse vectors explored by teams at MITRE Corporation and Sandia National Laboratories. Governance and regulatory responses have been proposed by bodies such as the European Commission, United States Federal Trade Commission, and national agencies in United Kingdom, Canada, and Australia; these discussions reference frameworks like the AI Act and guidance from Organisation for Economic Co-operation and Development. Technical mitigations under investigation include alignment research at OpenAI and DeepMind, red-teaming efforts promoted by Partnership on AI, differential privacy techniques rooted in work from Duke University and University of Pennsylvania, and auditing approaches advocated by Transparency International and Access Now.

Category:Artificial intelligence