Unicode (standard)

Unicode (standard)
Name	Unicode
Status	Active standard
First published	1991
Owner	Unicode Consortium
Latest version	Unicode 15.1
Website	unicode.org

Unicode (standard) Unicode is a computing industry standard for the consistent encoding, representation, and handling of text expressed in most of the world's writing systems. It provides a unique number for every character, enabling interoperable text processing across platforms and applications such as Apple Inc., Microsoft, Google, IBM, and Adobe Systems. Unicode underpins internet protocols and file formats used by organizations like W3C, IETF, ISO and governments such as Government of India and United States Federal Government.

History

The genesis of the standard arose in the late 1980s when engineers at companies including Xerox, Sun Microsystems, NeXT and Apple Inc. confronted incompatible character sets like ASCII, ISO/IEC 8859, Shift JIS and EBCDIC. Key figures and groups such as the Unicode Consortium and contributors from Ricoh, HP, Novell, and Softbank coalesced to reconcile disparate encodings; early meetings involved participants from University of California, Berkeley and Bell Labs. The release of the first Unicode specification in 1991 followed parallel international efforts such as ISO/IEC 10646, leading to an enduring liaison between the two standards bodies. Major milestones include the merger with ISO/IEC JTC 1/SC 2 activities, the expansion to cover historic scripts represented at institutions like the British Library and Library of Congress, and the addition of emoji driven in part by companies such as NTT DoCoMo and Facebook.

Design and Principles

Unicode is guided by principles of universality, stability, and unambiguous character identity, developed by committees drawing on expertise from MIT, Stanford University, Brown University, and national standards bodies including ANSI and DIN. It separates abstract characters from glyphs, allowing font vendors like Monotype Imaging and Google's Noto Fonts project to render characters differently in contexts such as Microsoft Word, LibreOffice, and Adobe InDesign. The standard defines canonical equivalence and normalization to manage combining marks used in scripts such as Devanagari, Arabic script, Hebrew alphabet, and Hangul; it prescribes properties like bidirectional behavior for mixing Hebrew alphabet and Latin script. Security and stability considerations reference work by researchers at University College London, Carnegie Mellon University, and corporations including Cisco Systems and Mozilla Foundation.

Code Points and Planes

Unicode assigns each abstract character a code point written as U+XXXX, arranged into 17 planes from Plane 0 (the Basic Multilingual Plane) to Plane 16 (Supplementary Private Use Area‑B). The BMP contains widely used scripts such as Latin alphabet, Cyrillic script, Greek alphabet, and modern ranges used by products from Microsoft and Apple Inc.. Supplementary planes host historic and rare scripts studied at Yale University and University of Oxford, as well as emoji popularized by Google and Twitter. Private Use Areas are relied upon by software projects like Emacs and regional standards bodies in places like Japan and Korea for vendor-specific glyphs. Special code points include control codes inherited from ISO/IEC 6429 and surrogate code points used by UTF-16 implementations in systems such as Windows NT.

Character Encodings and Transformation Formats

Unicode text is serialized by transformation formats including UTF-8, UTF-16, and UTF-32. UTF-8, created by researchers including those at Xerox and popularized by Tim Berners-Lee's World Wide Web Consortium implementations, is the dominant encoding on the Internet and projects like Apache HTTP Server and nginx. UTF-16 is common in programming environments such as Java and Microsoft .NET, while UTF-32 appears in certain academic tools from GNU Project. The standard specifies mechanisms for byte order via byte order mark usage in applications like Notepad and for error handling strategies adopted by Python and Perl. Collation and locale-aware operations reference standards and libraries maintained by groups such as ICU and projects at Mozilla Foundation.

Implementation and Adoption

Adoption spans operating systems, software libraries, and hardware: Linux, Windows NT, macOS, mobile platforms from Apple Inc. and Google use Unicode for filenames, user interfaces, and input methods developed by vendors like Microsoft and Samsung Electronics. Typeface vendors and font formats supported by OpenType enable rendering across applications such as Adobe Photoshop and Sublime Text. Web standards by W3C and protocols by IETF mandate Unicode for interoperable HTML, XML, and JSON content consumed by services including Facebook, Twitter, Amazon Web Services and Alibaba Group. Internationalization suites from Oracle Corporation and IBM incorporate Unicode-aware APIs for databases like MySQL and PostgreSQL.

Governance and Development Process

The Unicode Consortium, a nonprofit with corporate and organizational members including Google, Apple Inc., Microsoft, IBM, and Facebook, manages the standard through committees and technical working groups. Proposals are submitted by scholars, corporations, and cultural institutions such as Smithsonian Institution and UNESCO; encoding decisions consider evidence from linguists at University of California, Berkeley, University of Cambridge, and activists representing communities like the Māori Council and Sámi Council. The Consortium coordinates releases in concert with ISO/IEC JTC 1/SC 2 to maintain cross-reference with ISO/IEC 10646 and publishes data files and algorithms used by implementers including ICU and open source projects on platforms like GitHub.

Category:Character encoding standards