NoteDb

NoteDb
Name	NoteDb
Developer	Google
Initial release	2016
Programming language	Java (programming language)
Repository	Gerrit Code Review
License	Apache License
Operating system	Linux, Windows, macOS

NoteDb NoteDb is a data storage and event-sourcing subsystem used to persist code review metadata for the Gerrit Code Review project. It provides a structured way to record comments, approvals, patch set metadata, reviewer actions, and change messages in a version-controlled repository, enabling auditability, replication, and integration with continuous integration systems such as Jenkins (software), Zuul (software), and Buildbot. Initially developed within Google engineering culture and later integrated with the wider open source community, NoteDb complements change storage models like ReviewDb by leveraging Git as a durable source of truth.

Overview

NoteDb records review-related events as serialized objects stored in git notes and ref namespaces under the control of Gerrit Code Review. It replaces or augments legacy relational backends, providing a distributed, git-centric representation that integrates with workflows involving GitHub, GitLab, and enterprise deployments like Google Cloud Platform or Amazon Web Services. By design it enables features such as change history reconstruction, offline inspection via standard git tooling, and easier replication across geographically distributed instances in conjunction with systems like Apache Kafka and Google Cloud Storage.

History and Development

The project emerged during efforts to migrate metadata away from ReviewDb—a relational schema originally used by Gerrit Code Review—toward a git-based model. Key milestones include early design proposals discussed at Gerrit Code Review hackathons and implemented in incremental migrations influenced by practices from Google's internal code review systems and academic work on event sourcing. Contributors from organizations including Google, Red Hat, and independent maintainers in the Gerrit Code Review community refined the model through RFCs and code reviews, with adoption accelerating as major deployments standardized on git-native metadata storage. The transition intersected with broader ecosystem events such as the rise of continuous integration services and widespread adoption of distributed version control best practices.

Architecture and Design

NoteDb's architecture centers on storing typed entities as structured notes attached to refs in a git repository, using serialization formats compatible with the host platform. It leverages Git primitives—commits, trees, and refs—to encode and version events, allowing linearizable reconstruction of change state. Components include a writer that converts higher-level change events into note objects, a reader that reconstructs change models for presentation in Gerrit Code Review's UI, and migration tooling to map between legacy schemas and note representations. The design emphasizes immutability, provenance, and compatibility with existing Gerrit Code Review APIs, facilitating integration with tools such as Jenkins (software), Zuul (software), and Travis CI.

Data Model and Storage

The data model encodes entities such as approvals, reviewers, comments, and patch set metadata as typed records serialized into git objects. Storage uses dedicated refs and namespaces so that each change and patch set corresponds to a predictable location within the repository. This approach enables atomic updates via git commits and leverages existing replication mechanisms used by Git hosting platforms like GitLab and GitHub. The model supports rich metadata: reviewer identity from providers like LDAP, OAuth (protocol), and OpenID Connect, timestamps, action provenance, and message bodies to support compliance requirements in organizations such as NASA or European Space Agency that demand auditable trails.

Integration with Gerrit

NoteDb is tightly integrated with Gerrit Code Review as both a storage backend alternative and an augmentation to legacy systems. Integration points include the change lifecycle management, UI rendering of comments and approvals, hooks for plugins, and repair tools used during repository maintenance. Plugin ecosystems for Gerrit Code Review—maintained by vendors like Sony, contributors from Red Hat, and individual maintainers—use NoteDb APIs to persist custom metadata. The integration also interacts with authentication providers such as LDAP, SAML, and OAuth (protocol) implementations used by enterprise installations.

Use Cases and Adoption

Adopters use NoteDb for audit logging, cross-datacenter replication, offline analysis with Git tooling, integration with continuous delivery pipelines including Spinnaker (software), and forensic reconstruction for compliance audits by organizations like European Commission agencies or private firms. Large-scale deployments at companies leveraging Gerrit Code Review have migrated to NoteDb to reduce operational complexity and improve replication fidelity. Community projects and foundations that coordinate distributed development use NoteDb to centralize review artifacts while remaining compatible with collaborative platforms like GitHub and GitLab.

Performance and Scalability

NoteDb scales by leveraging git's native storage and packfile mechanisms, allowing repositories to handle large volumes of metadata without central database contention. Performance characteristics depend on git hosting infrastructure, object packing, and network replication strategies; integration with systems like Redis for caching or Apache Cassandra for complementary data stores can mitigate hotspot concerns. Benchmarks performed by operators consider commit throughput, read latency for reconstructing change state, and garbage collection impacts from git maintenance tasks. Techniques such as sharding refs, periodic repacking, and using dedicated metadata repositories help maintain throughput in deployments by organizations like Google and large telecom vendors.

Security and Access Control

Access control in NoteDb aligns with Gerrit Code Review's permission model and underlying git server authorization. Security considerations include preventing unauthorized ref writes, ensuring signature and provenance verification when integrated with GPG signing or S/MIME, and auditing via immutable git history. Enterprises often combine NoteDb with identity providers such as LDAP, SAML, and OpenID Connect to enforce role-based policies, and integrate with secrets management systems including HashiCorp Vault for credential handling. Operational best practices include restricting shell access to git repositories, enforcing server-side hooks, and periodic integrity checks to detect tampering.

Category:Software