MyRocks

MyRocks
Name	MyRocks
Developer	Facebook, MariaDB Corporation, Percona
Released	2016
Programming language	C++
Operating system	Linux, FreeBSD
License	BSD-style
Repository	Facebook Open Source

MyRocks

MyRocks is a storage engine designed for high write throughput and space-efficient storage in relational workloads. It was developed by Facebook and later maintained and enhanced by vendors and projects that include MariaDB Corporation, Percona, and the broader open source community. The engine bridges NoSQL-style log-structured merge (LSM) tree design with SQL features from MySQL, MariaDB, and storage optimizations from RocksDB.

Overview

MyRocks originated at Facebook to address large-scale write amplification and storage costs encountered by services such as Facebook Messages, Instagram, WhatsApp, and Messenger. The project leveraged research and production practices from teams at Facebook and incorporated components inspired by RocksDB—itself a fork of LevelDB created by engineers from Google for use in systems like Chrome and Bigtable-adjacent projects. MyRocks integrates with database ecosystems including MySQL, MariaDB, and forks maintained by vendors such as Percona Server to offer transactional semantics familiar to users of these systems. Contributors have included engineers associated with companies like Facebook, Oracle Corporation, MariaDB Corporation, and Percona, as well as community contributors connected to institutions such as University of California, Berkeley and firms like Pinterest.

Architecture and Storage Engine

MyRocks embeds an LSM-tree storage engine that derives from RocksDB's design, using levels, compaction, and write-ahead logging similar to systems like Cassandra and HBase. It runs inside server processes such as mysqld from MySQL and MariaDB Server, mapping SQL constructs like tables and indexes to LSM components. MyRocks uses column families and block-based table formats analogous to implementations in RocksDB and LevelDB, and it interacts with operating system facilities provided by Linux kernels, I/O schedulers such as CFQ, and storage hardware from vendors like Samsung Electronics and Intel Corporation. The engine supports transactions and crash recovery via mechanisms compatible with InnoDB's redo logging model while differing in storage layout and compaction strategies influenced by research from Google and academic work from MIT.

Performance and Use Cases

MyRocks targets workloads similar to those faced by large-scale online services including Facebook's feed, Twitter timelines, LinkedIn activity streams, and storage layers for Netflix recommendation metadata. It excels in write-heavy, append-oriented workloads comparable to use cases for Apache Kafka, Apache Cassandra, and ScyllaDB, and it reduces space usage relative to row-store engines used in MySQL and MariaDB deployments at companies like Dropbox, Airbnb, Uber Technologies, and Spotify. Benchmarks published by contributors compare MyRocks to engines such as InnoDB and TokuDB across metrics familiar to engineers at Amazon Web Services, Google Cloud Platform, and Microsoft Azure design teams. Operational adopters have included technology firms and research groups from Stanford University and Princeton University running analytic frontends where cost per gigabyte and write amplification metrics were critical.

Configuration and Tuning

Tuning MyRocks involves parameters inherited from RocksDB and exposing them via configuration interfaces present in MySQL and MariaDB. Administrators tune compaction settings, write buffer sizes, and block cache parameters similarly to tuning performed in deployments of Cassandra, HBase, and MongoDB by teams from Yahoo! and eBay. Performance-sensitive configurations may consider storage media produced by Western Digital, Seagate Technology, Micron Technology, and NVMe devices used by operations teams at GitHub and Stripe. Monitoring and observability integrate with tooling like Prometheus, Grafana, and log collectors used by Splunk and ELK Stack (Elasticsearch, Logstash, Kibana) in enterprises such as NASA and Intel for capacity planning and compaction tuning.

Compatibility and Integration

MyRocks is integrated into server binaries like MySQL and MariaDB and supported in maintained forks such as Percona Server for MySQL. This compatibility allows it to interoperate with client ecosystems including connectors for Python (used at Dropbox), Java ecosystems in companies like Spotify, and frameworks used by Facebook such as internal RPCs and caching layers like Memcached and Redis. Replication and backup strategies leverage tooling familiar to operators of MySQL replication, Galera Cluster, and cloud services like Amazon RDS and Google Cloud SQL where operational teams at Salesforce and Shopify require predictable behavior. Integration with storage solutions and orchestration systems used by Kubernetes and Docker enables containerized deployments favored by engineering teams at Netflix, Spotify, and Airbnb.

Adoption and Maintenance

Adoption of MyRocks has been driven by large-scale web companies and service providers balancing storage cost and write performance; adopters and contributors have included engineers from Facebook, Percona, MariaDB Corporation, and cloud providers like Alibaba Group and Tencent. Maintenance is carried out in open source repositories with contributions from corporate engineers and independent developers affiliated with projects and institutions including GitHub, Bitbucket, OpenStack Foundation, and academic collaborators from Carnegie Mellon University. The engine's lifecycle intersects with commercial offerings and support from firms such as Percona and MariaDB Corporation and has influenced design choices in other storage engines and systems developed by teams at Google, Amazon, and Microsoft Research.

Category:Database engines