Etcd (software)

Etcd (software)
Name	etcd
Developer	CoreOS, Red Hat, CNCF
Initial release	2013
Programming language	Go
License	Apache License 2.0
Repository	GitHub

Etcd (software) Etcd is a distributed, reliable key-value store used for shared configuration and service discovery in clustered environments. It provides strongly consistent storage using the Raft consensus algorithm and integrates with orchestration systems for high-availability workloads. Etcd is implemented in Go (programming language) and is maintained by contributors from CoreOS, Red Hat, and the Cloud Native Computing Foundation.

Overview

Etcd originated at CoreOS to serve configuration needs for distributed systems such as Kubernetes, Fleet (software), and Container Linux. Its design emphasizes consistency and simplicity influenced by research like the Paxos and Raft (computer science) papers and operational practices from projects at Google and Amazon Web Services. Etcd is widely packaged by vendors including Red Hat, Canonical (company), and Rancher Labs, and is used alongside orchestration projects such as Kubernetes, Mesos, and Docker Swarm.

Architecture

Etcd employs a cluster architecture built on the Raft (computer science) consensus protocol to elect a leader and replicate logs across members. A typical deployment involves odd-numbered clusters across datacenters operated by providers like Amazon Web Services, Google Cloud Platform, or Microsoft Azure to tolerate failures. The storage backend uses a B-tree-like structure and a write-ahead log to persist data on filesystems such as ext4 and XFS. Clients interact via a gRPC API and an HTTP/JSON gateway modeled after RESTful services in projects like Consul (software) and Zookeeper. For bootstrapping and membership changes, etcd integrates with discovery services and orchestration tools including systemd, Ansible, and Terraform.

Features and Functionality

Etcd offers strong consistency, linearizable reads, and serializable snapshots enabling features appreciated in Kubernetes controller loops, Prometheus service discovery, and Envoy (software) control planes. It supports watch streams for change notifications, lease mechanisms for ephemeral keys used by systems like Nomad (software), and transactions (compare-and-swap semantics) similar to conditional updates in Consul (software) and Zookeeper. Snapshotting and compaction manage storage growth, and TLS-based mutual authentication aligns with practices from OpenSSL and Let’s Encrypt for secure transport. Integration libraries and client SDKs exist across ecosystems including C#, Python (programming language), Java, and Node.js.

Use Cases and Adoption

Etcd is a foundational component of Kubernetes control planes, storing cluster state, leader election metadata, and configuration for controllers like kube-scheduler and kube-controller-manager. Service discovery use cases appear in microservices deployments managed by Docker (software), HashiCorp Nomad, and Rancher (software). Operators in enterprises such as Red Hat, cloud providers like Google and Amazon Web Services, and platforms like OpenShift rely on etcd for high-availability state storage. Other adopters include projects in the CNCF landscape such as Helm (software), Istio, and Linkerd.

Security and Reliability

Etcd implements role-based access control and mutual TLS to secure communication between peers and clients, following security practices propagated by organizations like CIS and NIST. Reliability is ensured by quorum-based writes and leader election mechanisms derived from Raft (computer science); operators are advised to run odd-sized clusters and cross-zone replicas in providers including Amazon Web Services and Google Cloud Platform. Backup and restore workflows use snapshots compatible with tooling from Velero and Restic (software), while monitoring integrations with Prometheus and alerting via Grafana help detect degradation. Vulnerability disclosures and mitigations often involve collaborations among CoreOS, Red Hat, and the CNCF security teams.

Development and Governance

Development is hosted on GitHub with contributions from maintainers affiliated with Red Hat, CoreOS, and independent contributors across the Cloud Native Computing Foundation ecosystem. The project follows open-source processes similar to other CNCF projects such as Kubernetes, with governance and community meetings mirroring practices used by Linux Foundation projects. Release cycles, CHANGELOGs, and issue tracking are managed through GitHub, and interoperability testing occurs in CI systems used by vendors like Travis CI and GitHub Actions. The project participates in events such as KubeCon and collaborates with adjacent projects in the CNCF landscape.

Category:Distributed key-value stores Category:Cloud Native Computing Foundation