Facebook MyRocks

Facebook MyRocks
Name	MyRocks
Developer	Facebook
Released	2016
Programming language	C++
Operating system	Linux
Genre	Storage engine

Facebook MyRocks MyRocks is a storage engine originally developed at Facebook for use with MySQL and MariaDB that optimizes write performance and storage efficiency by leveraging a Log-Structured Merge-tree design and RocksDB foundations. It was engineered to meet the needs of large-scale services at Facebook such as Facebook Messenger, Instagram, WhatsApp, and Oculus VR, aiming to reduce hardware costs while improving latency under heavy write workloads. MyRocks influenced practices in cloud providers like Amazon Web Services, Google Cloud Platform, and Microsoft Azure and drew attention from database projects including PostgreSQL, MongoDB, and MariaDB Corporation.

Overview

MyRocks integrates the high-level server features of MySQL with the low-level storage characteristics of RocksDB to provide a disk-efficient, write-optimized engine suitable for SSD-backed deployments. It targets workloads similar to those handled by Apache Cassandra, HBase, and ScyllaDB, competing on metrics such as write amplification, read amplification, and space amplification. The project entered public discussion alongside storage engines like InnoDB and influenced comparisons in benchmarking efforts by organizations such as Yahoo!, Twitter, LinkedIn, and Pinterest.

Architecture and Design

MyRocks implements a column-family-like layout atop RocksDB using a C++ codebase influenced by systems developed at Facebook Research and engineering teams that previously contributed to projects such as TAO (Facebook), Presto, and Hive (software). The design emphasizes LSM-tree mechanics similar to those in LevelDB and Google Bigtable, relying on SSTables, compaction, bloom filters, and write-ahead logging strategies comparable to implementations by Apache HBase and Cassandra (database). Integration with MySQL Server requires careful handling of transactional semantics, crash recovery, and MVCC patterns seen in engines like InnoDB and storage integrations at Oracle Corporation.

Performance and Benchmarks

Benchmarks for MyRocks were published by teams at Facebook comparing it against InnoDB under workloads derived from production traces at services including Messenger, Instagram, and Workplace (software). Independent evaluations by engineers from Uber Technologies, Airbnb, Dropbox, and Square (company) measured throughput, tail latency, and storage footprint, often leveraging benchmarking suites such as sysbench, YCSB, and custom trace-driven tools used in Netflix research. Results highlighted improvements in write amplification and storage efficiency versus InnoDB on SSDs, while read-heavy or random point-read workloads sometimes favored alternatives like MySQL Cluster or MariaDB Galera Cluster depending on access patterns.

Deployment and Integration

Deployments of MyRocks were coordinated with operations teams at Facebook and adopted by other large-scale operators including Alibaba Group, Baidu, Tencent, and several cloud-native companies using Kubernetes, Docker, and container orchestration stacks from Red Hat and Canonical (company). Integration required orchestration with backup systems such as Bacula (software), snapshotting mechanisms from LVM and ZFS, and monitoring via stacks like Prometheus (software), Grafana, and StatsD. Commercial distributions and forks were discussed in communities around Percona, MariaDB Foundation, and EnterpriseDB for enterprises running mixed OLTP/OLAP workloads.

Use Cases and Adoption

MyRocks was particularly well-suited for heavy-write OLTP use cases at scale, including social graph updates, messaging systems, and telemetry ingestion used by projects at Facebook AI Research, OpenAI collaborations, and analytics pipelines at Snowflake Inc. and Databricks. Adoption patterns mirrored migrations seen in companies such as Pinterest, Medium, and Etsy where storage efficiency translated to reduced operational costs on platforms like Amazon EC2, Google Compute Engine, and Microsoft Azure Virtual Machines.

Development History and Maintenance

Development began at Facebook around 2015–2016 with contributions from engineers who had previously worked on storage and database projects at Facebook, Google, and Yahoo!. The codebase and design discussions were publicized alongside other open-source initiatives like HHVM, Presto, and Osquery, influencing community efforts around RocksDB maintained by contributors from Facebook and Google. Maintenance responsibilities evolved over time with patches and performance improvements submitted by engineers from organizations including Percona, MariaDB Corporation, and independent contributors from GitHub ecosystems. The lifecycle saw debates about long-term support, compatibility with upstream MySQL releases maintained by Oracle Corporation, and coordination with distributors like Debian and Ubuntu package maintainers.

Category:Database engines