# IMPORT WHAT YOU NEED HERERegression
Before you start. Needs:
data/002_processed_data/xy_df_hopv.csv, which you create in Data Preparation of Iteration 2. Creates: your models in themodels/folder.Paths in this notebook are relative to its folder,
modules/3_modeling_and_evaluation/2_datalab/, so the data folder is../../../data/. If a file is not found, runfrom pathlib import Path; print(Path.cwd())to see where the notebook runs, and check that the earlier notebook saved its file indata/002_processed_data/.
In the Baseline notebook of Iteration 2, you trained the Iteration 1 regression model(s) on xy_df_hopv.csv with:
- a simple train/test split
- default model parameters
Your goal in this notebook is to improve that baseline in a controlled way. We will change one thing at a time and track what happens to the performance metric.
Repeat what you did in the Classification notebook of this iteration.
- Load
xy_df_hopv.csvintoxy_df. As in the baseline, for each target, remove only the rows where that target is missing. - Add cross-validation as you did for classification, but with
KFold(n_splits=5, shuffle=True, random_state=42): stratification needs classes, and HOMO and LUMO are continuous values. - Tune the model as you did for classification, with
scoring="neg_mean_absolute_error"(scikit-learn maximises scores, so errors are negated).
And finally, as you did for classification, put all results together and reflect, the numbers can be different:
| Step | HOMO (MAE) | LUMO (MAE) |
|---|---|---|
| starter_dataset.xlsx, simple train test split, default model parameters | 0.103 | 0.128 |
| xy_df, simple train test split, default model parameters (Iteration 2 baseline) | 0.138 | 0.142 |
| xy_df, 5-fold CV, default model parameters | 0.097 ± 0.020 (5-fold CV) | 0.124 ± 0.012 (5-fold CV) |
| xy_df, 5-fold CV, tuned model parameters | 0.097 ± 0.019 (5-fold CV) | 0.123 ± 0.012 (5-fold CV) |
Why is the first row better than the second, although the second model learned from about five times more molecules? Look at the size of the starter test set: the two numbers come from different test sets, so they cannot be compared directly. Only the cross-validated rows share one protocol.
The test set scores HOMO and LUMO with the MAE over all HOMO and LUMO predictions together, which is the average of the HOMO MAE and the LUMO MAE.
In Iteration 1, we trained separate models for HOMO and LUMO predictions. An alternative approach is to use a single model that predicts both simultaneously. For the next steps, it is up to you to decide which strategy to pursue. Make sure you can clearly justify your choice.
Step 1 — Cross-validation
# YOUR CODE HEREStep 2 — Hyperparameter tuning
# YOUR CODE HERESave your model(s) into the models folder.
# YOUR CODE HERE
# if you trained two models per target
homo_model_name = 'homo_iteration3.joblib'
lumo_model_name = 'lumo_iteration3.joblib'
# if you trained one model to predict both targets
model_name = 'homolumo_iteration3.joblib'