CART

CART
Name	CART
Developer	Leo Breiman; Jerome H. Friedman; Richard A. Olshen; Charles J. Stone
Introduced	1984
Genre	Decision tree learning; Predictive modeling
Programming languages	Fortran; C; Python; R; Java
License	Various

CART

CART is a tree-based predictive modeling technique introduced in the 1980s for classification and regression tasks. It produces binary decision trees via recursive partitioning, yielding interpretable models used across applied fields such as medicine, finance, and remote sensing. The method underpins numerous later algorithms and toolkits in statistical learning and data mining.

Overview

CART constructs binary trees by splitting a dataset into subsets using feature-value tests, producing terminal nodes that output class labels or numeric predictions; key concepts were formalized by Leo Breiman, Jerome H. Friedman, Richard A. Olshen, and Charles J. Stone in a seminal monograph. The approach uses impurity measures and split-search strategies to choose partitions, then applies pruning procedures to avoid overfitting, ideas referenced in later work by Vladimir N. Vapnik and Robert Tibshirani. Implementations appear in statistical environments such as R, machine-learning libraries like scikit-learn, and platforms maintained by organizations including National Institute of Standards and Technology and companies like IBM and Microsoft.

History

The method’s formal presentation appeared in a 1984 book by the four authors, building on earlier decision tree concepts from researchers at Bell Labs and theoretical developments by scholars such as Jerome H. Friedman and Leo Breiman during the 1970s and 1980s. CART coexisted with contemporaneous algorithms like those from Ross Quinlan and the ID3/C4.5 lineage, and influenced ensemble methods developed later by Leo Breiman (bagging, random forests) and by Yoav Freund and Robert E. Schapire (boosting). Adoption accelerated with increases in computing capacity at institutions including Lawrence Berkeley National Laboratory and with software packages from groups at AT&T and academic labs at Stanford University and University of California, Berkeley.

Methodology

CART grows a binary tree via recursive partitioning: at each node it evaluates candidate splits on predictor variables and selects the one that optimizes an impurity criterion—Gini impurity for classification or squared-error loss for regression—concepts discussed in statistical texts by Trevor Hastie and Robert Tibshirani. The split search can handle continuous and categorical predictors, including missing-value strategies inspired by work at Harvard University and Massachusetts Institute of Technology. After full growth, CART applies cost-complexity pruning derived from complexity-penalization ideas in the literature of Vladimir N. Vapnik and model selection frameworks of George E. P. Box. Cross-validation procedures popularized in applied statistics by researchers at CERN and Bell Labs are commonly used to select pruning parameters.

Applications

Practitioners have applied the technique to clinical risk stratification in studies at institutions like Mayo Clinic, Johns Hopkins Hospital, and Cleveland Clinic, to credit scoring in firms such as FICO and American Express, and to remote sensing classification in projects by NASA and European Space Agency. Public-health surveillance systems at Centers for Disease Control and Prevention and ecological modeling efforts at US Geological Survey have used decision-tree models derived from CART methodology. In engineering, groups at General Electric and Siemens deploy tree-based models for predictive maintenance; in marketing, teams at Amazon (company) and Procter & Gamble employ such models for segmentation and churn prediction.

Variants and Extensions

Extensions include ensemble adaptations—bagging and random forests by Leo Breiman; boosting frameworks exemplified by AdaBoost from Yoav Freund and Robert E. Schapire; and gradient-boosted trees implemented in systems like XGBoost, LightGBM, and CatBoost originating from teams at DMLC and Microsoft. Hybrid approaches combine trees with linear models (model trees) developed in academic work at University of Waikato and others. Probabilistic and Bayesian variants draw on research by groups at University of Cambridge and Harvard University; survival-tree adaptations appear in clinical-statistics literature from Memorial Sloan Kettering Cancer Center.

Performance and Evaluation

Performance assessment uses metrics and techniques standard in machine-learning research from communities around NeurIPS, ICML, and KDD: accuracy, precision-recall, area under the ROC curve, mean-squared error, and calibration plots. Benchmarking on datasets curated by UCI Machine Learning Repository and competitions hosted by Kaggle informs practitioner choices. Comparative studies by researchers at Stanford University and Carnegie Mellon University examine bias-variance trade-offs and interpretability versus predictive power, while software toolkits provide cross-validation and bootstrap utilities inspired by foundational work at Bell Labs.

Limitations and Criticisms

Critiques focus on instability to data perturbations noted by Leo Breiman, potential for overfitting without careful pruning as highlighted in reviews from Royal Statistical Society, and reduced predictive accuracy relative to ensemble methods reported by research groups at MIT and University of Toronto. Additional concerns include challenges handling high-dimensional sparse data encountered in projects at Google and Facebook (company), and interpretability limits when trees become large—issues addressed in model-interpretability research at Alan Turing Institute and University College London.

Category:Machine learning algorithms