Module 5: Tree-Based Algorithms
Decision trees, and models that combine many of them, are among the strongest models for tabular data. In this module you see how a single tree learns, and how random forests and gradient boosting combine many trees into a better model.
What you will be able to do
After this module you can:
- explain how a decision tree chooses its splits and why it overfits;
- explain how random forests and gradient boosting combine trees, and tune their main hyperparameters;
- interpret feature importances with care;
- use a gradient boosting library with early stopping on a validation set.
Self-study
Work through the activities in order:
- Decision Trees: a decision tree by hand and with scikit-learn, how a tree learns, overfitting, hyperparameters, and regression trees.
- Random Forests: many trees trained on random parts of the data, their hyperparameters, voting and extra-trees.
- Gradient Boosting: trees that each correct the errors of the previous ones, their hyperparameters, XGBoost and LightGBM, AdaBoost and stacking.
- Review: a summary of the module and a short quiz.
DataLab
After the Self-study, apply what you learned in the DataLab project: Iteration 5.