ONNX

ONNX
Name	ONNX
Developer	Microsoft Corporation, Facebook, Inc., Linux Foundation
Initial release	2017
Programming language	C++, Python (programming language), Protobuf
Operating system	Linux, Microsoft Windows, macOS
License	Apache License

ONNX ONNX is an open standard for representing machine learning models that enables interoperability among TensorFlow, PyTorch, Caffe2, MXNet, Scikit-learn, Keras (software), and many other frameworks. It provides a common serialization format and operator specification to permit models trained in one environment to be run in another or optimized using external tooling like Intel Corporation toolchains, NVIDIA Corporation runtimes, or cloud services such as Amazon Web Services, Microsoft Azure, and Google Cloud Platform. The project is governed and developed collaboratively by companies and contributors from across the industry including bodies associated with the Linux Foundation and partner projects in the Open Neural Network Exchange (ONNX) ecosystem.

Overview

ONNX defines a graph-based intermediate representation for neural networks and other machine learning models that standardizes operator semantics and tensor formats to facilitate cross-platform deployment across devices like NVIDIA Jetson, Intel Nervana, Google Edge TPU, and embedded platforms such as Raspberry Pi. It aims to bridge ecosystems that include projects like Apache MXNet, Theano, CNTK, Chainer, and tools such as TensorRT, OpenVINO, TVM (compiler) to enable portable model exchange, optimization, and inference acceleration. Major technology companies—Facebook, Inc., Microsoft Corporation, Amazon.com, Inc., IBM, and Google LLC researchers—have contributed operators, tooling, and converters to extend support across hardware accelerators like AMD, NVIDIA, and Intel Corporation devices.

History and Development

ONNX emerged from a collaboration between Microsoft Corporation and Facebook, Inc. announced in 2017 to address fragmentation caused by competing formats from projects such as TensorFlow, Caffe, and Torch (machine learning) variants. The early roadmap incorporated lessons from academic projects at institutions like Stanford University and MIT and leveraged serialization technologies similar to Protocol Buffers popularized by Google LLC. Governance moved towards a community-driven model under the auspices of the Linux Foundation, attracting contributors from Amazon.com, Inc., IBM, NVIDIA Corporation, Intel Corporation, Alibaba Group, and research groups at Carnegie Mellon University and University of California, Berkeley. Subsequent milestones included the formation of ONNX operator specifications, the release of the ONNX Runtime by Microsoft Corporation, and integrations with compiler projects such as Apache TVM and optimizers like Model Optimizer (OpenVINO).

Architecture and File Format

ONNX specifies a protobuf-based file format that encodes computational graphs composed of nodes (operators), tensors, and attributes; it uses concepts similar to computational graphs from TensorFlow and static graph IRs used in compiler projects like LLVM. The format includes operator sets ("opsets") that version operator semantics, enabling projects such as PyTorch, Caffe2, MXNet, and Keras (software) to map native operations to standardized ONNX operators. ONNXRuntime, developed by Microsoft Corporation, consumes these protobuf files and integrates with backends like TensorRT, OpenVINO, and DirectML to execute graphs with hardware acceleration. File metadata can reference provenance and tooling such as Weights & Biases, MLflow, and experimentation platforms used by teams at Google LLC and Facebook, Inc..

Supported Frameworks and Tools

A broad ecosystem of frameworks provides import/export support for ONNX, including PyTorch, TensorFlow, Keras (software), Scikit-learn, Apache MXNet, Caffe2, Chainer, and Theano forks. Tooling and runtimes that operate on ONNX artifacts include ONNX Runtime (by Microsoft Corporation), TensorRT (by NVIDIA Corporation), OpenVINO (by Intel Corporation), Apache TVM (by the Apache Software Foundation community), Glow (machine learning compiler) (by Facebook, Inc.), and vendor SDKs from Qualcomm Incorporated and Arm Ltd.. Model conversion utilities and ecosystem services often integrate with versioning and CI/CD systems from GitHub, Inc., GitLab Inc., and enterprise offerings from Atlassian.

Model Interchange and Runtime Execution

ONNX enables model interchange by mapping framework-specific graphs onto standardized operator definitions and serializing them into ONNX files that can be consumed by runtimes such as ONNX Runtime, TensorRT, OpenVINO, and embedded inference engines for Android (operating system) and iOS. Execution workflows typically combine ONNX conversion from training frameworks like PyTorch or TensorFlow with optimizers such as ONNX Optimizer and accelerators like NVIDIA CUDA or AMD ROCm to deploy inference at scale in cloud infrastructures operated by Amazon Web Services, Microsoft Azure, and Google Cloud Platform. Profiling and debugging workflows intersect with observability tools from Datadog, New Relic, and research tooling developed at Berkeley AI Research.

Adoption and Use Cases

ONNX is used across industries for model portability, edge deployment, and inference acceleration in products from Microsoft Corporation, Facebook, Inc., Amazon.com, Inc., Alibaba Group, Baidu, Inc., and startups leveraging accelerators from NVIDIA Corporation and Intel Corporation. Common use cases include computer vision pipelines integrating pre-trained models from ImageNet, natural language processing models like transformer variants developed in academic groups at Google Research, OpenAI, Facebook AI Research, and recommendation systems deployed by Netflix, Inc. and Spotify Technology S.A.. Enterprises adopt ONNX for scenarios involving hybrid cloud strategies with providers such as Oracle Corporation and IBM to standardize ML deployment, while research labs at MIT, Stanford University, and Carnegie Mellon University use ONNX to share reproducible artifacts.

Limitations and Criticisms

Critics point to gaps in operator coverage and semantic mismatches when mapping research prototypes from labs at DeepMind or experimental layers from OpenAI into the ONNX operator set, requiring custom operators or exporter patches maintained by vendors like NVIDIA Corporation or Intel Corporation. Versioning complexity in opsets and differences between dynamic and static graph semantics similar to debates between TensorFlow and PyTorch communities can complicate reproducibility for teams at Google Research and enterprise groups at Facebook, Inc. and Microsoft Corporation. Performance variability across runtimes—observed in benchmarks by organizations such as MLPerf and corporate research groups at Amazon.com, Inc.—sometimes necessitates per-backend tuning using projects like Apache TVM or vendor-specific SDKs. Additionally, governance and contribution processes under the Linux Foundation have drawn commentary from contributors at GitHub, Inc. and academic collaborators regarding the pace of operator standardization and long-term roadmap alignment.

Category:Machine learning standards