Tidyverse

Tidyverse
Name	Tidyverse
Developer	RStudio
Released	2016
Operating system	Cross-platform
License	MIT

Tidyverse The Tidyverse is a collection of packages for the R programming language that emphasizes a coherent grammar for data manipulation, visualization, and analysis. It was introduced and promoted by RStudio and notable contributors to the R ecosystem, and has influenced workflows across academic, governmental, and industrial organizations. The collection integrates packages for data import, transformation, modeling, and presentation, and is commonly used alongside tools and projects from institutions and projects within the data science landscape.

Overview

The project emerged in a period when Hadley Wickham and colleagues sought to standardize tools used in data science practice, aligning with efforts by organizations such as RStudio and influenced by work from research groups at Imperial College London, Massachusetts Institute of Technology, and University of Washington. It interoperates with the base R (programming language) distribution and has been adopted in contexts ranging from the World Health Organization to technology companies like Google and Microsoft. The ecosystem interacts with file formats and services developed by groups such as Apache Software Foundation projects and with containerization platforms like Docker for reproducible deployment. The design reflects conventions used in statistical communities represented by societies such as the American Statistical Association and conferences like UseR!.

Core packages

Core components include packages originally authored and maintained by contributors associated with RStudio and Posit (company), collectively forming a consistent API. Prominent packages in the suite are widely cited in tutorials and textbooks authored by figures like Garrett Grolemund and Hadley Wickham, and are used in courses at institutions such as Stanford University, Harvard University, and University of California, Berkeley. The core set often includes packages for data manipulation, tidying, and visualization created alongside companion projects used in domains like epidemiology at Centers for Disease Control and Prevention and econometrics at National Bureau of Economic Research. Many packages are distributed via repositories maintained by groups such as CRAN and mirrored by platforms like GitHub and GitLab.

Design principles and data structures

The collection embodies principles influenced by software engineering and statistical computing traditions found in publications from venues such as The R Journal and Journal of Statistical Software. It prescribes tidy data conventions that align conceptually with data modeling standards used in organizations like OECD and with data formats exchanged in projects by European Space Agency. Core data structures promoted by the ecosystem have analogues in tabular formats handled by libraries in languages developed at institutions like Bell Labs and influenced by paradigms appearing in projects such as dplyr, tidyr, and ggplot2 (noting those are package names, not an exact citation here). The approach draws on grammar-of-graphics ideas propagated by researchers connected to conferences like UseR! and journals such as Journal of Computational and Graphical Statistics.

Typical workflow and examples

Typical workflows integrate data import, cleaning, transformation, visualization, modeling, and reporting and are taught in workshops hosted by organizations like Carpentries, DataCamp, and Coursera. Practitioners commonly use interactive environments such as RStudio IDE, literate programming via R Markdown, and deployment through services like Shiny and cloud platforms offered by Amazon Web Services and Google Cloud Platform. Example pipelines are demonstrated in textbooks and materials from O'Reilly Media, university syllabi at Massachusetts Institute of Technology and University of Oxford, and training programs at enterprises like Facebook and Airbnb. Reproducible reports produced using this workflow are shared in repositories on GitHub and archived in data journals associated with organizations like PLOS.

Adoption and community

The ecosystem has a large community of users and contributors spanning academic groups at Stanford University School of Medicine, governmental agencies like NASA, and companies including Spotify and Uber Technologies. Community activity is visible in discussion forums hosted by organizations such as Stack Overflow, package development on GitHub, and conference presentations at events like rstudio::conf and UseR!. Educational resources have been produced by publishers including Springer and Wiley, and certification or training courses are offered by providers like DataCamp and corporate training teams at Microsoft. The contributor base includes academics and engineers affiliated with institutions such as Yale University and University of Cambridge.

Criticisms and limitations

Critiques of the approach have been raised in academic and practitioner forums including articles in The American Statistician and blog posts by data practitioners at companies like Netflix and Airbnb. Concerns include performance on very large datasets relative to tools from projects such as Apache Spark and Hadoop, interoperability limits with strict typing systems used in languages developed at Bell Labs and by teams at Google, and learning-curve issues noted in course feedback from universities such as University of California, San Diego. Some organizations advocate alternative stacks based on ecosystems like pandas in the Python (programming language), or distributed computing frameworks developed by Databricks. Discussions about software licensing, maintainability, and governance reference practices from foundations such as the Apache Software Foundation and working groups within The Carpentries.

Category:Software