Amazon Redshift

Amazon Redshift
Name	Amazon Redshift
Developer	Amazon Web Services
Released	2012
Written in	C++
Platform	Cloud
License	Proprietary

Amazon Redshift is a cloud-based data warehousing service designed for large-scale analytic workloads, combining columnar storage, Massively Parallel Processing (MPP), and cloud-native infrastructure. It is offered by Amazon Web Services alongside services such as Amazon S3, Amazon EC2, Amazon RDS, AWS Lambda, and AWS Glue, and is used in conjunction with analytics and business intelligence platforms including Tableau, Looker, Qlik, Microsoft Power BI, and SAP BusinessObjects. Enterprises integrate it with data integration tools and services such as Talend, Informatica, Fivetran, Stitch (software), and Matillion for ETL and ELT pipelines.

Overview

Amazon Redshift is positioned within the Amazon Web Services portfolio alongside Amazon Aurora, Amazon DynamoDB, Amazon EMR, Amazon Kinesis, and Amazon Athena to support petabyte-scale analytics. It competes with products like Google BigQuery, Microsoft Azure Synapse Analytics, Snowflake (computing) and Teradata, and is used by organizations across industries including Netflix, Airbnb, Comcast, Yelp, and Nasdaq. Typical use cases include data warehousing, business intelligence, ad-hoc analytics, and machine learning feature stores integrated with Amazon SageMaker, Databricks, and H2O.ai.

Architecture

The architecture is based on a leader node coordinating multiple compute nodes in an MPP cluster topology similar to architectures from Greenplum, Vertica, Netezza, and Exadata (Oracle) origins. Data is stored in columnar format on local SSDs or networked storage backed by Amazon EBS and Amazon S3 for snapshot and UNLOAD operations, and distribution styles (KEY, ALL, EVEN) govern data placement akin to strategies used in PostgreSQL and Apache HBase. Redshift integrates SQL compatibility influenced by PostgreSQL and supports JDBC and ODBC drivers used by clients such as SQL Workbench/J, DBeaver, and SQuirreL SQL Client.

Performance and Scalability

Performance features include zone maps, compression encodings, sort keys, and adaptive query optimization comparable to techniques from Query optimization research labs at Carnegie Mellon University, University of California, Berkeley, and Massachusetts Institute of Technology. Concurrency scaling, materialized views, result caching, and spectrum integration enable scaling for mixed workloads similar to hybrid approaches in Snowflake (computing) and Google BigQuery. Workload management (WLM) queues and concurrency controls mirror practices from Teradata operations and IBM Db2 tuning, while materialized views and late binding views support integration patterns used by Looker and Mode Analytics.

Security and Compliance

Security features encompass network isolation with Amazon VPC, encryption at rest with AWS KMS, and auditing with AWS CloudTrail and AWS CloudWatch, aligning with compliance programs like ISO 27001, SOC 2, PCI DSS, HIPAA, and FedRAMP. Identity and access control integrate with AWS IAM and federated single sign-on solutions such as Okta, Azure Active Directory, Ping Identity, and OneLogin. Redshift participates in cloud security architectures discussed in publications from NIST and is frequently evaluated alongside controls recommended by CIS (Center for Internet Security).

Integration and Ecosystem

Redshift’s ecosystem includes data ingestion and orchestration tools such as Apache Kafka, Amazon Kinesis Data Streams, AWS Data Pipeline, Apache NiFi, and Apache Airflow; analytics and visualization tools like Tableau, Microsoft Power BI, QlikView, Looker, and MicroStrategy; and machine learning integrations with Amazon SageMaker, Databricks, and TensorFlow. It interoperates with data catalogs including AWS Glue Data Catalog, Apache Hive Metastore, and governance platforms such as Alation and Collibra. Connectivity extends through JDBC/ODBC drivers, the psql client lineage from PostgreSQL, and partners like Matillion and Fivetran that provide connectors to sources like Salesforce, Google Analytics, Stripe, and Marketo.

Pricing and Deployment Options

Pricing models include managed cluster instances billed per hour similar to Amazon EC2 instance families (RA3, DC2, DS2 legacy), managed storage with billing per TB-month, and on-demand, reserved, and spot instance-like purchasing options analogous to EC2 Reserved Instances and Savings Plans. Deployment variants include single-cluster, multi-cluster with concurrency scaling, and spectrum-enabled queries that run against data in Amazon S3. Organizations often compare total cost of ownership with alternatives such as Snowflake (computing), Google BigQuery, and Microsoft Azure Synapse Analytics when evaluating long-term commitments.

History and Development

Redshift was announced by Amazon Web Services in 2012 during a period of cloud data platform expansion alongside offerings from Google, Microsoft, IBM, and Oracle Corporation. Its early design drew on columnar store research from academic groups at Stanford University and University of Wisconsin–Madison and commercial precedents set by Teradata and Vertica. Over time, features such as Redshift Spectrum, RA3 instances, AQUA cache, and integration with AWS Lake Formation were introduced, following industry trends set by Snowflake (computing) and Google BigQuery. Key milestones occurred at AWS re:Invent events where enhancements to performance, security, and integrations were announced, reflecting ongoing competition with Google Cloud Next, Microsoft Ignite, and industry adoption by companies including Pinterest, Yelp, HubSpot, and Expedia Group.

Category:Cloud data warehousing