ARKit

ARKit is a software framework developed by Apple Inc. that enables developers to create augmented reality applications for iPhone and iPad. First introduced with iOS 11 in 2017, it provides a suite of tools for blending digital objects and information with the real world through the device's camera. The framework leverages advanced hardware like the Apple A11 Bionic chip and sophisticated sensors to understand and interact with environments. This technology has positioned Apple as a major player in the mobile AR space, fostering a new wave of interactive apps.

Overview

ARKit operates by utilizing a technique known as visual-inertial odometry, which combines data from the device's motion sensors and camera to accurately track the world without requiring additional hardware. This core functionality allows digital content to appear anchored to real-world surfaces. The framework is deeply integrated with Apple's ecosystem, working seamlessly with graphics technologies like Metal (API) and SceneKit, and is supported across a wide range of devices from the iPhone 6S onward. Its development is closely tied to advancements in Apple's custom system on a chip designs and the capabilities of the TrueDepth camera system.

Features and capabilities

Key features include robust plane detection for identifying horizontal and vertical surfaces like floors and walls, enabling virtual objects to be placed realistically. World tracking maintains a persistent understanding of a space, allowing for experiences that can be left and returned to. Later versions introduced people occlusion, where the framework can recognize and allow virtual content to pass behind human figures captured by the camera. The Face Tracking API, powered by the TrueDepth camera on devices like the iPhone X, enables detailed facial expression analysis for apps and Animoji. Image tracking and object detection allow apps to recognize specific pictures or items and trigger AR interactions.

Development and tools

Developers primarily use Xcode and the Swift (programming language) or Objective-C programming languages to build applications. ARKit is accessed through APIs within the broader iOS SDK, and projects often incorporate 3D authoring tools like Blender (software) or Autodesk Maya for creating assets. For visualization and scene management, many developers use Reality Composer or RealityKit, Apple's higher-level frameworks designed to work in tandem. The ARKit 3 update brought powerful features like motion capture and simultaneous front and back camera access, greatly expanding creative possibilities. Testing and deployment are managed through the App Store Connect portal.

Version history

The initial release, now referred to as ARKit 1.0, debuted at the WWDC 2017 keynote and was part of iOS 11, offering basic world tracking and plane detection. ARKit 2.0 arrived with iOS 12, introducing shared experiences allowing multiple users to view the same AR scene on different devices, along with persistent world maps. ARKit 3 was unveiled at WWDC 2019 alongside iOS 13, bringing the significant additions of people occlusion and motion capture. ARKit 4, released with iOS 14, integrated LiDAR Scanner data from the iPad Pro (4th generation) for faster and more accurate environment understanding, including a new Depth API. Subsequent updates have continued to refine performance and add support for newer hardware like the Apple A15 Bionic.

Applications and impact

ARKit has enabled diverse applications across numerous fields, from interactive games like Pokémon Go to practical tools in IKEA Place for visualizing furniture. In education, apps allow students to explore detailed models of the human heart or historical artifacts. The retail and design sectors use it for virtual try-ons and architectural previews. Major cultural institutions, including the Smithsonian Institution and the British Museum, have created AR exhibits. The framework has also spurred innovation in professional fields, aiding in complex visualization for engineering and medicine, and has influenced the development of competing platforms like Google ARCore. Its proliferation is considered a foundational step toward future wearable technology such as the anticipated Apple Vision Pro.

Category:Augmented reality Category:Apple Inc. software Category:IOS