Presto

Presto
Name	Presto
Developer	Facebook (initial), later contributors
Released	2012
Programming language	C++
Operating system	Linux, macOS, Windows (client)
License	Apache License 2.0 (originally)

Presto Presto is a distributed SQL query engine designed for interactive analytic queries against large datasets. It was created to enable low-latency, high-concurrency queries across heterogeneous data sources and integrates with data warehouses, object stores, and metadata services. The project influenced and interacted with an ecosystem of data-processing technologies, engines, query planners, and cloud platforms.

Overview

Presto is a massively parallel processing (MPP) engine that executes SQL queries using a distributed architecture combining a coordinator and multiple workers. It connects to storage layers like Hadoop Distributed File System, Amazon S3, Google Cloud Storage and metadata systems such as Apache Hive Metastore and AWS Glue. The engine supports ANSI SQL extensions, columnar formats like Apache Parquet and Apache ORC, and integrates with query tools and visualization platforms including Apache Superset, Tableau, Looker, and Power BI.

History

Development began at Facebook in 2012 to address interactive analytics workloads that could not be satisfied by Apache Hive or traditional data warehouses like Teradata and Oracle Database. Early design and prototypes referenced academic systems such as the Volcano (query processing system) model and research from UC Berkeley’s AMPLab. The open-source project rapidly attracted contributors from companies including Airbnb, Uber, Netflix, LinkedIn, and Twitter. In 2019 a divergent community led to forks and reprojects, with organizations like Starburst Data commercializing aspects and other groups continuing community releases.

Architecture and Features

Presto employs a separation between a single coordinator and multiple worker nodes; the coordinator parses SQL, plans distributed execution, and schedules tasks while workers execute plan fragments. The execution engine supports pipelined operators, vectorized processing, runtime filters, and memory-management strategies influenced by systems such as Apache Arrow and Vectorized execution research. Connectors implement access to sources like MySQL, PostgreSQL, Microsoft SQL Server, Apache Cassandra, MongoDB, and cloud-native stores. Security features include authentication integrations with Kerberos, LDAP, OAuth, and authorization with systems like Apache Ranger and AWS IAM.

Use Cases and Implementations

Presto is used for interactive analytics, ad-hoc exploration, ETL offload, and federated querying across transactional and analytical stores. Deployments span on-premises clusters managed with orchestration tools like Kubernetes, Apache Mesos, and Docker Swarm, to managed services from cloud providers such as Amazon EMR, Google Cloud Dataproc, and Azure HDInsight. Organizations in advertising, e-commerce, streaming, and finance—examples include Spotify, Pinterest, Stripe, and Comcast—have used Presto or derivatives to serve dashboards, recommendation pipelines, and billing analytics. Integrations with metadata catalogs like Apache Atlas and lineage tools such as OpenLineage support governance and compliance workflows.

Performance and Evaluation

Benchmarking compares Presto to engines like Apache Spark SQL, Dremio, Trino, ClickHouse, and proprietary data warehouses such as Snowflake and Google BigQuery. Strengths include low latency for interactive queries, efficient single-stage query planning, and broad connector support; limitations historically involved memory pressure under large joins and complex aggregations requiring careful tuning of spill-to-disk and shuffle strategies. Performance improvements derived from techniques in distributed databases research at institutions like MIT and Stanford informed adaptive query execution, cost-based optimization, and dynamic resource allocation enhancements.

Adoption and Community

The ecosystem includes corporate adopters, independent vendors, and open-source contributors from projects such as Trino (a community fork), Starburst Data (commercial distributions), and multiple cloud providers offering managed deployments. Community activity occurs in mailing lists, issue trackers, and conferences including Strata Data Conference, KubeCon, AWS re:Invent, and Google Cloud Next. Academic citations appeared in research on federated query processing, data lake architectures, and interactive analytics; institutions involved include Carnegie Mellon University and ETH Zurich.

Legal and Licensing Issues

The original project was distributed under the Apache License 2.0, allowing broad commercial use, modification, and redistribution. Forks and commercial distributions led to trademark and contribution governance discussions similar to those seen in other open-source ecosystems, involving corporate contributor license agreements and trademark policing practiced by entities such as The Linux Foundation and Apache Software Foundation. Compliance considerations in regulated industries prompted organizations to integrate Presto with data governance and audit tooling from vendors like Collibra and Informatica to meet standards such as HIPAA and GDPR.

Category:Query engines Category:Distributed computing