# IMPORT WHAT YOU NEED HEREBaseline
Before you start. Needs:
data/002_processed_data/xy_df_hopv.csv, which you create in Data Preparation. Creates: nothing that later notebooks need, but write down your scores: Iteration 3 starts from them.Paths in this notebook are relative to its folder,
modules/2_data_preparation/2_datalab/, so the data folder is../../../data/. If a file is not found, runfrom pathlib import Path; print(Path.cwd())to see where the notebook runs, and check that the earlier notebook saved its file indata/002_processed_data/.
In Iteration 1 you trained a classification, a regression and a clustering model on starter_dataset.xlsx, with 80 molecules. Now you have your own dataset of the HOPV molecules, xy_df_hopv.csv, with 347 molecules.
In this notebook you train the Iteration 1 models again, in the same way, on the new dataset. The only change is the data. The result is your baseline for this dataset: in Iteration 3 you improve the models one step at a time and compare every step with this baseline.
The metric
Stability predictions for the test molecules are scored with macro-F1. The F1 score of a class combines its precision (of the molecules predicted as that class, the share that really belongs to it) and its recall (of the molecules of that class, the share that the model finds). Macro-F1 is the average of the F1 scores of low, medium and high, with equal weight. You learn these metrics in detail in section 8 of Modeling & Evaluation, the first activity of the Module 3 Self-study. Accuracy is easy to read, but it can look good while a model ignores the smaller classes. In this notebook, report accuracy and macro-F1 side by side (f1_score(y_test, y_pred, average="macro")), and use macro-F1 to make decisions. Always compare with a majority-class baseline, a “model” that always predicts the most frequent class: DummyClassifier(strategy="most_frequent") from sklearn.dummy.
Classification
Load xy_df_hopv.csv into xy_df and use this processed dataset of 347 molecules. As in Iteration 1, remove only the rows where stability is missing. Train and test the model exactly as you did in Iteration 1, so that the only change is the dataset. (One HOPV molecule has missing descriptor values. A random forest trains anyway: since scikit-learn 1.4, tree-based models handle NaN themselves. Most other models stop with an error, as in the Data Preparation notebook of the Module 2 Self-study.)
Report accuracy and macro-F1 for your model and for the majority-class baseline on the same test set, and plot the confusion matrix (ConfusionMatrixDisplay.from_predictions from sklearn.metrics). Which classes does the model confuse?
# YOUR CODE HERECompare the test scores to the ones you obtained in Iteration 1 (Classification). Did they improve, stay the same, or decrease? Provide a possible explanation for the outcome.
Careful: the two scores come from different test sets (about 15 starter molecules versus about 60 HOPV molecules). The difference is not only the effect of more training data, so this is a learning comparison: it tells you something about the data, but it does not prove that one model is better. From Iteration 3 on, you compare changes under one fixed protocol: the same data, folds and metric.
Also count the classes after removing the rows with a missing stability, and compare with what you would get if you removed every row with any missing target (homo, lumo or stability), as many first attempts do. How many molecules would you lose, and why does the class mix change?
Regression
Train the regression model(s) of Iteration 1 again on xy_df, in the same way: for each target, remove only the rows where that target is missing, and use the same simple train/test split and default parameters. Report the MAE of homo and lumo on the test set, next to a baseline that always predicts the mean of the training targets (DummyRegressor from sklearn.dummy).
Compare with your Iteration 1 scores. As for classification, the test sets differ, so this is a learning comparison.
# YOUR CODE HEREClustering
- Load
xy_df_hopv.csvintoxy_df - Apply the same clustering procedure from Iteration 1 to
xy_df. - Note: in
hdb.labels_,-1marks noise (molecules that do not belong to any cluster), and HDBSCAN gives the label-3to a molecule with missing feature values. Check for missing values and decide how to handle them.
Once you obtain the clusters:
- Check whether the pattern you discovered earlier still holds with the larger dataset.
- Do you observe more clusters, fewer clusters, or no change?
- Investigate if new patterns emerge now that you have more molecules.
# YOUR CODE HEREWrite down your baseline
Note your scores and how you got them: the data (and how you filtered it), the features, the model and its parameters, the split and the scores. Iteration 3 starts from these numbers.