Gradient Boosting and Stacking

Estimated time: 1.5 hours

Continue reading Chapter 7: Ensemble Learning and Random Forests, starting with the Boosting section and continuing to the end of the chapter.

In the course edition, this corresponds approximately to pp. 222–235.

The main focus is gradient boosting: building trees sequentially so that each new model attempts to correct errors made by the models that came before it.

The chapter also introduces stacking. You do not need to master stacking at this stage, but you should understand the basic idea and recognize it as another ensemble-learning strategy.

First Pass: Get the Big Picture — 10 minutes

• Skim the remaining section titles.
• Look at the plots and figures.
• Read exercise questions 6–7.

Second Pass: Read for Understanding — 60 minutes

Read the remainder of the chapter carefully.

Focus especially on:

• the general idea of boosting;
• how sequential learning differs from the independent trees used in a random forest;
• how gradient boosting improves a model iteratively;
• the basic idea behind stacking.

Third Pass: Review — 20 minutes

Answer exercise questions 6–7.

Return to the relevant sections when needed and make sure you can explain the main ideas in your own words.