Concourse (software)

Concourse (software)
Name	Concourse
Developer	Pivotal Software; later Concourse CI, Inc.
Released	2013
Programming language	Go (programming language)
Operating system	Linux, macOS, Windows (via Docker)
Platform	Cloud Foundry, Kubernetes, Amazon Web Services, Microsoft Azure, Google Cloud Platform
License	Apache License

Concourse (software) is an open-source continuous integration and continuous delivery system designed for pipeline-centric automation of software delivery. It emphasizes reproducibility, minimalism, and container-based isolation, using a declarative pipeline model that integrates with modern infrastructure such as Kubernetes, Docker, and major cloud providers like Amazon Web Services, Google Cloud Platform, and Microsoft Azure. Originating at Pivotal Software and later stewarded by an independent company, Concourse has influenced practices in continuous integration, continuous delivery, and infrastructure automation across enterprises and research projects.

History

Concourse was initiated within Pivotal Software as part of efforts to improve Cloud Foundry platform delivery workflows and was publicly introduced in the mid-2010s alongside trends driven by DevOps advocates and events such as DockerCon and KubeCon. Early development followed patterns used by projects like Jenkins and Travis CI but diverged toward container-first designs influenced by Docker and orchestration platforms like Kubernetes. The project attracted contributors from organizations including Pivotal Labs and independent maintainers; later stewardship moved to Concourse CI, Inc. while the codebase remained open under an Apache License. Over time Concourse integrated community work inspired by concepts from GitHub Actions, CircleCI, and Drone (software), participating in ecosystem discussions at conferences such as Velocity Conference and All Day DevOps.

Architecture

Concourse is built on a distributed architecture with distinct components that enable scalable pipeline execution. The core components include a centralized web UI and API server, a scheduler, and worker processes that execute isolated tasks inside Docker containers or garden-runc-style containers. The architecture relies on a resource model for inputs and outputs, aligning with artifact flows used by Artifact repositorys, and a declarative pipeline language stored in Git repositories. Concourse implements worker isolation reminiscent of patterns from Linux namespaces and cgroups and integrates with orchestration systems like Kubernetes and cloud platforms such as Amazon Web Services and Google Cloud Platform to provision execution environments. The system also uses a lightweight database and message coordination that can be backed by technologies similar to PostgreSQL and Redis in production deployments.

Features

Concourse provides features that support reproducible builds, auditability, and composable automation. Key features include a declarative pipeline syntax for jobs and resources, resource types for Git and container registries, and reusable tasks packaged as container images—concepts comparable to patterns found in Docker Compose and Helm charts. The web-based interface offers visual pipeline graphs and build logs akin to interfaces in Jenkins Blue Ocean and GitLab CI/CD. Concourse emphasizes immutability of build artifacts and task isolation, following principles advanced by HashiCorp tools and influenced by Infrastructure as Code practices exemplified by Terraform and Ansible. Advanced features include manual approvals, parameterized pipelines, and resource check intervals for integration with external systems like Artifactory and Nexus Repository.

Configuration and Pipelines

Pipelines in Concourse are defined declaratively using YAML files that describe resources, jobs, and tasks, similar in role to .travis.yml and .gitlab-ci.yml configurations. Pipelines reference external resources such as Git repositories, container registries like Docker Hub, and object storage services from Amazon S3 or Google Cloud Storage. Task steps run commands inside container images, allowing reuse of images published to registries used by projects at GitHub or Docker Hub. Pipelines support triggers on commits, scheduled intervals, and manual interventions, enabling workflows comparable to those in CircleCI and GitHub Actions. Concourse also provides a command-line tool for configuring pipelines and interacting with the server, aligning with dev toolchains used in Continuous Integration ecosystems.

Security and Authentication

Concourse supports multiple authentication backends and security models to integrate with enterprise identity systems. Built-in and pluggable authentication options include OAuth 2.0 providers, integration with GitHub OAuth apps, LDAP directories, and SAML-based single sign-on used by organizations leveraging Okta or Azure Active Directory. Role-based access controls and team scoping enable separation of duties similar to controls found in HashiCorp Vault and Kubernetes RBAC. Concourse worker isolation reduces attack surface by executing tasks within container boundaries, and pipelines can be configured to use secrets managers and credential stores like Vault (software) to avoid embedded secrets in configuration files.

Integrations and Plugins

Concourse integrates with a broad set of platforms and services through resource types, webhooks, and community-contributed extensions. Native integrations cover GitHub, GitLab, Bitbucket, container registries such as Docker Hub and Amazon Elastic Container Registry, and cloud provisioning APIs from Amazon Web Services, Microsoft Azure, and Google Cloud Platform. The resource model enables custom resource types implemented by projects or organizations, similar to plugin ecosystems in Jenkins and SonarQube, and community plugins provide connectivity to artifact stores like JFrog Artifactory, notification systems such as Slack, and issue trackers including JIRA.

Use Cases and Adoption

Concourse is used across enterprises, startups, and open-source projects for CI/CD pipelines, continuous deployment to Kubernetes clusters, and build pipelines for containerized microservices influenced by 12-factor apps principles. Adoption scenarios include automated testing, artifact promotion workflows, and platform engineering within organizations using Cloud Foundry or cloud-native stacks on Amazon Web Services and Google Cloud Platform. Notable adopters often appear at conferences and in case studies alongside other CI/CD tools such as Jenkins, GitLab CI/CD, and CircleCI, choosing Concourse for its emphasis on reproducibility, pipeline visualization, and container isolation.

Category:Continuous integration