Acme (DeepMind)

Acme (DeepMind)
Name	Acme
Developer	DeepMind
Released	2019
Programming language	Python
License	Apache License 2.0
Repository	GitHub

Acme (DeepMind) Acme is a software framework for building distributed reinforcement learning agents developed by DeepMind. It provides a set of libraries and abstractions to connect algorithms, environments, and data pipelines, intended for research and production at scale. Acme emphasizes modularity, reproducibility, and performance, enabling integration with a range of tools and platforms from TensorFlow to PyTorch, and deployment on clusters such as Kubernetes and cloud services like Google Cloud Platform.

Overview

Acme was introduced by DeepMind to standardize agent construction across projects including experiments similar to those in AlphaGo, AlphaZero, AlphaStar, MuZero, and Agent57. The framework leverages concepts and components familiar to engineers working with TensorFlow and JAX and interacts with environments such as OpenAI Gym, DeepMind Lab, VizDoom, Mujoco, and Unity ML-Agents. Acme's goals align with reproducible practices promoted by institutions like OpenAI, Carnegie Mellon University, MIT, Stanford University, and University of Oxford. It supports integration with orchestration and monitoring tools like Docker, Prometheus, Grafana, and Ray.

Architecture and Components

Acme's architecture separates actors, learners, replay, and environment interfaces, inspired by systems architecture used in projects at DeepMind and design patterns from Google engineering. Key components include replay buffers influenced by research at University of Toronto and DeepMind labs, policy modules compatible with TensorFlow Agents and Stable Baselines3, and dataset pipelines interoperable with TFDS and HDF5 artifacts. Networking and RPC layers mirror designs from gRPC and gRPC's use in Borg and Kubernetes. Acme interfaces with accelerators like NVIDIA GPUs, TPU accelerators developed by Google and makes use of linear algebra libraries such as Eigen and numerical backends used by NumPy and SciPy.

Research and Development

Acme has supported research across areas explored by teams collaborating with DeepMind and academic groups from University College London, ETH Zurich, Princeton University, University of California, Berkeley, and University of Washington. It has been used in experiments related to algorithms comparable to DQN, DDPG, PPO, SAC, IMPALA, and RTRL, extending work from laboratories including DeepMind's own Alpha projects and field advancements stemming from conferences like NeurIPS, ICML, ICLR, AAAI, and UAI. Development follows engineering practices common at Google Research and DeepMind involving code review, continuous integration tools such as Jenkins, and collaboration with platforms like GitHub and Phabricator.

Applications and Use Cases

Acme has been employed in domains paralleling applications seen in projects by DeepMind, OpenAI, and corporate labs at Microsoft Research, Facebook AI Research, and IBM Research. Use cases include simulated robotics tasks benchmarked in Mujoco and Roboschool, game-playing scenarios in Atari 2600 and StarCraft II reminiscent of AlphaStar, as well as control problems researched at ETH Zurich and Caltech. Industry deployments integrate Acme-based agents with orchestration platforms such as Kubernetes and monitoring stacks like Prometheus for production services at organizations including Google and cloud providers like Amazon Web Services and Microsoft Azure.

Performance and Benchmarks

Evaluations with Acme align with benchmark suites used across the field, including Atari benchmarks, DeepMind Control Suite, and Procgen; these suites are commonly cited by groups at OpenAI, DeepMind, Google Brain, and universities like Columbia University and Harvard University. Performance comparisons often reference algorithms such as IMPALA, Rainbow, R2D2, and MuZero. Profiling and optimization employ tools used across industry and academia like nvprof, TensorBoard, FlameGraph, and cluster telemetry systems from Google SRE practices.

Adoption and Community

Acme's community includes contributors from corporate labs such as DeepMind, Google Brain, OpenAI, Microsoft Research, and academic institutions including University of Oxford, MIT, Stanford University, Imperial College London, EPFL, University of Toronto, and University of Cambridge. Discussions and collaborations occur on platforms like GitHub, , and at conferences including NeurIPS, ICML, ICLR, and workshops organized by ACL and CVPR. Educational use spans courses at Stanford University and MIT where frameworks like Acme are taught alongside TensorFlow and PyTorch in labs following curricula influenced by textbooks from MIT Press authors.

History and Timeline

Acme was released by DeepMind in 2019 with open-source artifacts on GitHub, following internal tooling and precedents from earlier projects at DeepMind and Google DeepMind. Subsequent updates expanded interoperability with JAX, PyTorch, and containerization workflows using Docker and Kubernetes. The project evolved in parallel with research milestones at DeepMind such as AlphaGo Zero and community efforts from OpenAI and academic partners at UC Berkeley and Carnegie Mellon University, and it has been presented at venues like NeurIPS and ICLR.

Category:DeepMind software