Data Preparation

Data preparation is the process of cleaning, transforming, and organizing raw data into a suitable format for machine learning models, ensuring that the algorithms can learn effectively and produce reliable results.

Outline:

Motivation
Feature Scaling
Feature Transformations
Missing Data
Merging Data
Encoding Data
Exercise

1. Motivation

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

Before diving into the details, I want you to avoid a common misconception: that improving machine learning pipeline performance is mainly about choosing or tuning models. In reality, proper data preparation, even with the exact same model, can be the key difference between weak and strong results. Every phase of the CRISP-DM cycle is important.

Let’s load our housing data and see an example:

from sklearn.datasets import fetch_california_housing
X, y = fetch_california_housing(as_frame=True, return_X_y=True)

I’ll load a model we haven’t studied yet, a Support Vector Machine. By now, you’ve seen that loading different models in scikit-learn follows the same simple pattern. You don’t need to understand how this particular algorithm works; just think of it as another machine learning model for demonstration purposes.

from sklearn.svm import SVR
model = SVR()

and train/test it on the dataset:

from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_absolute_error

# split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)
# fit
model.fit(X_train, y_train)
# test
y_pred = model.predict(X_test)
mae = mean_absolute_error(y_test, y_pred)
print(mae)

That’s quite a large error compared to our initial results.

Now let’s scale the data. We’ll discuss what scaling means in a moment, for now, just understand that we’re applying a data preparation step to improve the model’s performance.

# scale the data
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)

# repeat

# call the model again
model = SVR()
# fit
model.fit(X_train, y_train)
# test
y_pred = model.predict(X_test)
mae = mean_absolute_error(y_test, y_pred)
print(mae)

The error dropped from $86K to $39K, almost a $50K reduction in house price prediction error, just by applying one data preparation step. This shows how critical proper data preparation is, as models can perform poorly without producing any error messages to indicate what went wrong.

2. Feature Scaling

What we applied above is a feature scaling technique called standardization. Feature scaling is one of the most common data preparation steps in both machine learning and deep learning. Rather than defining it right away, let’s first observe its effect and discuss what it means afterward.

scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

It’s important to note that the output of any scikit-learn transformation is a NumPy array, regardless of whether the input was originally a NumPy array or a pandas DataFrame.

type(X), type(X_scaled)

X.head()

# similar to .head()
X_scaled[:5, :]

As a result, the transformed data no longer retains the original column names or index, though the column order remains the same. If we want to keep the familiar structure, we can easily recreate a DataFrame using the original column names.

X_scaled = pd.DataFrame(X_scaled, columns=X.columns)
X_scaled.head()

Now let’s examine one of the columns, HouseAge, to see how it changes before and after scaling.

fig, ax = plt.subplots(1, 2, figsize=(10, 5))
ax[0].hist(X['HouseAge'])
ax[0].set_xlabel('HouseAge')
ax[1].hist(X_scaled['HouseAge'])
ax[1].set_xlabel('HouseAge (Scaled)')

Everything looks the same except for the values on the x-axis: the overall shape of the distribution is identical, but after scaling it is centered around zero with a standard deviation close to one.

This is the case for all the columns:

X_scaled.mean()

X_scaled.std()

another type of feature scaling is min-max scaling also called normalization.

from sklearn.preprocessing import MinMaxScaler
scaler = MinMaxScaler()
X_scaled = scaler.fit_transform(X)

X_scaled = pd.DataFrame(X_scaled, columns=X.columns)

fig, ax = plt.subplots(1, 2, figsize=(10, 5))
ax[0].hist(X['HouseAge'])
ax[0].set_xlabel('HouseAge')
ax[1].hist(X_scaled['HouseAge'])
ax[1].set_xlabel('HouseAge (Scaled)')

What is the difference? The feature values have been rescaled to lie between 0 (the minimum) and 1 (the maximum), which is why this method is called min-max scaling. You can adjust the scaler’s parameters to set a different minimum and maximum range if needed.

scaler = MinMaxScaler(feature_range=(-1, 1)) # let's change the range
X_scaled = scaler.fit_transform(X)
X_scaled = pd.DataFrame(X_scaled, columns=X.columns)
fig, ax = plt.subplots(1, 2, figsize=(10, 5))
ax[0].hist(X['HouseAge'])
ax[0].set_xlabel('HouseAge')
ax[1].hist(X_scaled['HouseAge'])
ax[1].set_xlabel('HouseAge (Scaled)')

X_scaled.min()

X_scaled.max()

Now we can define what feature scaling is.

Most machine learning algorithms perform poorly when numerical features have very different scales, with a few exceptions. In 002_classification.ipynb, we mentioned that some models require normalization of the input data, while models such as RandomForest and DecisionTree do not. This is precisely why we started with Random Forest, it works well even without scaling. This is also why I selected SVM in the motivation section, because it is sensitive to the scale of the feature values.

There are two common techniques to bring all features to a similar scale: standardization and min-max scaling. Standardization rescales the data so that each feature has a mean of zero and a standard deviation of one. Min-max scaling (also called normalization) rescales the data to a fixed range, typically between 0 and 1.

`fit_transform` vs `transform`

Before we conclude this section, there is one important detail to understand. Earlier, in the motivation section, we standardized the data using the following code:

from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)

YOUR TURN

What is the difference between fit_transform and transform, and why do we apply the first to X_train and the second to X_test?

The answer will be given below but first try to find the answer yourself. It is briefly discussed in the course textbook Chapter 2 under Feature Scaling and Transformation.

# YOUR ANSWER HERE
# If you prefer write it in markdown.

SOLUTION

When you call fit_transform, the scaler calculates the mean and standard deviation of each feature (this is the “fit” step) and then uses those values to scale the data (the “transform” step). We do this only on the training data, because the model should learn scaling parameters only from the data it has “seen” during training.

When you call transform on the test data, the scaler applies the same mean and standard deviation from the training data, without recalculating them. This ensures that both training and test data are scaled consistently.

If you scaled the entire dataset before splitting, information from the test set would leak into the training process because the mean and standard deviation would include the test data. This is called data leakage, and it can make your model’s performance appear better than it actually is.

One student in our program scaled first and split later during their graduation thesis. This was considered data leakage and was one of the reasons they failed. So always split first, then scale, and remember, this applies to any transformation where parameters are learned from the data.

3. Feature Transformations

Feature scaling is just one type of feature transformation. Some features require transformations that go beyond simple scaling. Let’s see why:

plt.hist(X['Population'])
plt.ylabel('Count')
plt.xlabel('Population')

The Population column represents the total number of people living in each California census block group. The distribution is right-skewed, meaning most block groups have populations below 5,000, but a few densely populated areas reach much higher values, up to around 35,000.

We can visually see this from the histogram, but we can quantify with some useful numpy and pandas functionality as well.

np.percentile returns the value below which a given percentage of data falls, for example, the 99th percentile is the value below which 99% of the data lie. 🔗 NumPy documentation
nlargest (in pandas) returns the top n largest values or rows from a Series or DataFrame column. 🔗 pandas documentation

# 99% of block groups have a population below roughly 5,800
np.percentile(X['Population'], 99)

# However, some block groups are much more densely populated
X['Population'].nlargest(5)

While skewed features cause multiple issues in machine learning, one of the most fundamental problems is easy to grasp and highlights a key challenge in building reliable models.

Skewed features can easily mislead your model, during random train-test split. If, by chance, those extreme values, like ~35,000 and ~28,000, end up only in the test set, your model will train on the remaining data. This means the model might learn that the maximum value for the feature is only ~16,000, simply because that was the highest value in its training set. When this trained model, especially regression models, later sees the ~35,000 or ~28,000 values, it’s encountering numbers far beyond its experience. It won’t know how to properly factor in such high figures, leading to wild and inaccurate predictions and making it unreliable.

Even if you manage the split, the core point of machine learning is to develop models for new, unseen data. A skewed feature distribution means your model is fundamentally unprepared to handle any future real-world value that is even more extreme than what was in your entire dataset.

To address this, as explained in Chapter 2 of the course textbook under Feature Scaling and Transformation, a common and effective fix is to apply a logarithmic transformation (log) to the feature.

log_feature = np.log(X['Population'])
plt.hist(log_feature)

the skew is eliminated, which is what we want.

YOUR TURN

Can you think of a case where we cannot apply np.log() to skewed data? The answer follows from the definition of the logarithm.

# YOUR ANSWER/CODE HERE

Hint: the domain of the log function is…?

YOUR TURN

We discussed that it’s crucial to apply scaling after the train/test split. Do you think the same rule applies to a log transformation? Why or why not?

# YOUR ANSWER
# Use markdown if you want

4. Missing Data

Handling missing values with pandas is part of the prerequisites. If you need a reminder, the pandas user guide covers it in depth:

https://pandas.pydata.org/docs/user_guide/missing_data.html

Therefore I will keep this section short, but I would like to emphasize two important points specific to scikit-learn.

Point 1:

Some models can handle missing features natively. For such models, it might be better not to process missing values.

# let's generate dummy data with a single feature that contains one nan value
X = np.array([[-1], [-5], [10], [7], [2], [-7], [np.nan]])
y = np.array([0, 0, 1, 1, 1, 0, 0])

# most models will complain
# please read the error message
model = SVR()
try:
    model.fit(X, y)
except Exception as e:
    print(e)

# but there are a few that could handle nans natively
from sklearn.ensemble import HistGradientBoostingClassifier
clf = HistGradientBoostingClassifier()
clf.fit(X, y)

Point 2:

If you use models that do not accept missing values (NaNs), you have three main options:

Drop missing values: Simple and keeps only complete data, but you may lose valuable information if many rows contain NaNs.
Impute missing values: Replace them with estimated numbers (like the mean or training a model to predict features). This preserves all samples but can introduce bias if the imputation method is not appropriate.
Fill with a constant (e.g., -99): Technically a form of imputation, but usually a poor choice because models may treat it as a real value. It can be acceptable for tree-based models that can handle missingness internally or if you add a missing indicator column to tell the model which values were originally missing.

Read more about the available imputers and options in scikit-learn here:

https://scikit-learn.org/stable/modules/impute.html#impute

at the end of the documentation you will see a list of estimators that handle NaN values.

Point 3:

Missing values are not the only values that models refuse. Some calculations return infinity (np.inf or -np.inf), for example a division by zero, or a molecular descriptor that cannot be computed properly for a molecule. Infinite values are not NaNs: isna() does not find them, and most scikit-learn models and the imputers raise an error when they see one. A few models, such as HistGradientBoostingClassifier, accept them, but treat them as just a very large number.

Check for infinite values with np.isinf, then decide how to handle them. A common choice is to turn them into NaNs, so that you can handle them like the other missing values:

df = df.replace([np.inf, -np.inf], np.nan)

# a feature with one infinite value
from sklearn.impute import SimpleImputer
X = np.array([[-1], [-5], [10], [7], [2], [-7], [np.inf]])
print("NaNs:", np.isnan(X).sum(), "| infinite values:", np.isinf(X).sum())

# please read the error message
try:
    SimpleImputer().fit(X)
except Exception as e:
    print(e)

# turn the infinite value into a NaN, then impute as usual
X_clean = np.where(np.isinf(X), np.nan, X)
SimpleImputer().fit_transform(X_clean)

5. Merging Data

Toy datasets usually come with X and y already prepared. In contrast, real-world data is often scattered across multiple files or sources, for example, one table in a SQL database and another in a CSV file.

Therefore, it’s essential to learn how to merge and align data from multiple sources to construct your X (features) and y (target) before modeling.

Let’s create dummy data

df1 = pd.DataFrame({
    "PatientID": [1, 2, 3],
    "Feature1": [10, 20, 30],
    "Feature2": [0.5, 0.8, 0.3]
})

df2 = pd.DataFrame({
    "PatientID": [2, 3, 4],
    "Feature3": [5, 7, 9]
})

df3 = pd.DataFrame({
    "PatientID": [1, 2, 3, 4],
    "Target": [0, 1, 0, 1]
})

We have data on 4 patients. df1 and df2 each contain different features (Feature1, Feature2, and Feature3) for overlapping but not identical sets of patients, while df3 contains the target values for all patients.

df1

df2

df3

We want to obtain a complete table with all patients (1–4), their available features, and a target column ready for machine learning. The merge method merges two DataFrames, we can do this twice to merge 3 dataframes as follows:

merged = df3.merge(df1, on="PatientID", how="left")
merged

merged = merged.merge(df2, on="PatientID", how="left")
merged

In a pandas merge,

The on parameter specifies the key column that the dataframes share, in this case, "PatientID". It tells pandas which column to match rows on when combining data.
The how parameter controls which rows are kept in the final result:
- "inner" keeps only rows with matching keys in both tables,
- "left" keeps all rows from the left table (adding NaN where no match is found),
- "right" keeps all rows from the right table,
- "outer" keeps all rows from both tables.

In our example, we use on="PatientID" to align patient records and how="left" to keep all patients from the target table, even if they are missing some feature values. Please read the documentation for more info:

https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.merge.html

The merged DataFrame can then be converted to X and y easily by:

X = merged[['Feature1', 'Feature2', 'Feature3']]
y = merged[['Target']]

6. Encoding Data

So far we have only dealt with numerical attributes, but your data may also contain text attributes.

Machine learning models generally work with numbers, not text. Therefore, we must convert text (categorical) values into numerical form, this process is called encoding.

Let’s explore three common encoding methods: 1. LabelEncoder 2. OrdinalEncoder 3. OneHotEncoder

LabelEncoder

LabelEncoder is used when you have a single column (a Series) of labels such as class names. It converts each unique category into an integer label.

Let’s create a simple example:

from sklearn.preprocessing import LabelEncoder

# dummy data for demo
y = pd.Series(["dog", "cat", "dog", "rabbit", "cat"])

encoder = LabelEncoder()
y_encoded = encoder.fit_transform(y)

y_encoded

Each unique category got a number. LabelEncoder is typically used for target variables (y), not for input features. Note that scikit-learn models can handle text labels in y directly, you don’t have to encode them.

OrdinalEncoder

OrdinalEncoder is used for feature columns when the categories have a meaningful order (e.g., low < medium < high). It encodes each category with an integer.

from sklearn.preprocessing import OrdinalEncoder

# dummy data for demo
df = pd.DataFrame({
    "Size": ["Large", "Small", "Medium", "Large", "Medium", "Small"]
})

encoder = OrdinalEncoder()
df["Size_encoded"] = encoder.fit_transform(df[["Size"]])

df

Notice that:

Large -> 0 Small -> 2 Medium -> 1

which does not make sense. We want the numbers to follow the order, we can use the categories parameter when calling OrdinalEncoder:

# dummy data for demo
df = pd.DataFrame({
    "Size": ["Large", "Small", "Medium", "Large", "Medium", "Small"]
})

encoder = OrdinalEncoder(categories=[['Small', 'Medium', 'Large']])
df["Size_encoded"] = encoder.fit_transform(df[["Size"]])

df

This encoding makes sense only when the categories are ordered.

OneHotEncoder

OneHotEncoder is used when the categories have no natural order (e.g. animals) It creates one new column per category with 0/1 values.

from sklearn.preprocessing import OneHotEncoder

# dummy data for demo
df = pd.DataFrame({
    "Animal": ["dog", "cat", "dog", "rabbit", "cat"]
})

encoder = OneHotEncoder(sparse_output=False)
encoded = encoder.fit_transform(df[["Animal"]])

# create a df to visualize
# dog -> [0, 1, 0]
# cat -> [1, 0, 0]
# rabbit -> [0, 0, 1]
encoded_df = pd.DataFrame(encoded, columns=encoder.get_feature_names_out(["Animal"]))
pd.concat([df, encoded_df], axis=1)

# any new data can be encoded with the encoder
new_df = pd.DataFrame({
    "Animal": ["rabbit", "cat"]
})

encoder.transform(new_df)

# what if there is a new animal?
new_df = pd.DataFrame({
    "Animal": ["horse", "cat"]
})

try:
    encoder.transform(new_df)
except Exception as e:
    print(e)

It will give an error because the encoder has never seen this animal before. In such cases, we might prefer an output like [0, 0, 0], since it’s not a cat, dog, or rabbit.

You can control this behavior using the handle_unknown parameter, check the documentation of OneHotEncoder to learn how.

YOUR TURN

Modify the code below (only the line with # YOUR CODE HERE) so that when a new animal appears in the data, the encoder outputs zeros for all known categories instead of raising an error.

df = pd.DataFrame({
    "Animal": ["dog", "cat", "dog", "rabbit", "cat"]
})
encoder = OneHotEncoder(sparse_output=False) # YOUR CODE HERE
encoded = encoder.fit_transform(df[["Animal"]])
encoded_df = pd.DataFrame(encoded, columns=encoder.get_feature_names_out(["Animal"]))
pd.concat([df, encoded_df], axis=1)

# RUN AS IS
# If you modified the code above correctly it should print [[0. 0. 0.]]
new_df = pd.DataFrame({
    "Animal": ["horse"]
})

try:
    print(encoder.transform(new_df))
except Exception as e:
    print(e)

Summary

Encoder	Use Case
LabelEncoder	Target variable
OrdinalEncoder	Ordered categorical feature
OneHotEncoder	Unordered categorical feature

7. Exercise

Now you will combine everything you have learned in a single exercise. For this we need some dummy data.

# GENERATES DUMMY DATA
# DO NOT MODIFY
rng = np.random.default_rng(42)

# --- IDs ---
all_ids = np.arange(1, 101)
num_ids = np.arange(1, 96)   # 1–95
cat_ids = np.arange(5, 101)  # 5–100

# --- numerical_features_df ---
# feature1: log-like with negatives
log_vals = rng.lognormal(mean=3.0, sigma=2.0, size=num_ids.size)
sign = np.where(rng.random(num_ids.size) < 0.2, -1.0, 1.0)
feature1 = log_vals * sign

# feature2: small gaussian 0–0.001 (scaled up)
feature2 = np.clip(rng.normal(loc=5e-4, scale=2e-4, size=num_ids.size), 0, 1e-3) * 1000

numerical_features_df = pd.DataFrame({
    "PatientID": num_ids,
    "feature1": feature1,
    "feature2": feature2
})

# Add some NaNs
for col in ["feature1", "feature2"]:
    nan_idx = rng.choice(numerical_features_df.index, size=3, replace=False)
    numerical_features_df.loc[nan_idx, col] = np.nan

# --- categorical_features_df (medical-style unordered) ---
blood_types = ["A", "B", "AB", "O"]
scan_types = ["MRI", "CT", "Xray", "Ultrasound"]

feature3 = rng.choice(blood_types, size=cat_ids.size, p=[0.4, 0.1, 0.1, 0.4])
feature4 = rng.choice(scan_types, size=cat_ids.size, p=[0.25, 0.25, 0.25, 0.25])

categorical_features_df = pd.DataFrame({
    "PatientID": cat_ids,
    "feature3": feature3,
    "feature4": feature4
})

# Add NaNs
for col in ["feature3", "feature4"]:
    nan_idx = rng.choice(categorical_features_df.index, size=3, replace=False)
    categorical_features_df.loc[nan_idx, col] = np.nan

# --- generate target with actual relationship ---
temp = (
    pd.DataFrame({"PatientID": all_ids})
    .merge(numerical_features_df, on="PatientID", how="left")
    .merge(categorical_features_df, on="PatientID", how="left")
)

# encode categories numerically to simulate influence
blood_map = {"A": 0, "B": 1, "AB": 2, "O": 3}
scan_map = {"MRI": 0, "CT": 1, "Xray": 2, "Ultrasound": 3}

blood_code = temp["feature3"].map(blood_map).fillna(0)
scan_code = temp["feature4"].map(scan_map).fillna(0)

# relationship + noise → "disease probability"
risk_score = (
    0.5 * temp["feature1"].fillna(0)
    + 1.5 * temp["feature2"].fillna(0)
    + 2.0 * blood_code
    + 3.0 * scan_code
    + rng.normal(0, 5, size=temp.shape[0])
)

# Convert to probability between 0 and 1 (sigmoid)
prob = 1 / (1 + np.exp(-0.1 * (risk_score - np.median(risk_score))))

# Create binary diagnosis based on probability threshold 0.5
diagnosis = (prob > 0.5).astype(int)

target_df = pd.DataFrame({
    "PatientID": all_ids,
    "diagnosis": diagnosis
})

YOUR TURN

Goal: Build a clean feature matrix X from three source tables, then train a simple model. You must follow good data prep practices: split first, learn parameters on the training set only, and avoid leakage.

Data you are given:

numerical_features_df with columns PatientID, feature1, feature2
categorical_features_df with columns PatientID, feature3, feature4
target_df with columns PatientID, diagnosis

After merging, the columns must be exactly:

PatientID, feature1, feature2, feature3, feature4, diagnosis

What to do (steps):

Merge Merge the three tables on PatientID so the columns are in the exact order shown above.
Explore missingness Count NaNs per column to understand what needs imputing or encoding.
Create X and y, then split Set X = ['feature1', 'feature2', 'feature3', 'feature4'] and y = diagnosis. Do a train/test split with test_size=0.2 and a fixed random_state for reproducibility.
Log-transform feature1 safely Apply a log transform that works for negative values and zero. Hint: transform the magnitude with np.log1p(np.abs(x)) (this is log(1 + |x|), which is 0 for x = 0) and restore the sign with np.sign(x). Do this separately on X_train and X_test.
Standardize numerical features Standardize feature1 and feature2 using StandardScaler. Fit on the training set and only transform the test set.
Handle missing values for numeric features For each of feature1 and feature2:

Add a missingness indicator column named <col>_missing that is 1 if the original value is missing, else 0.
Impute missing values with the median learned from the training set, then apply to the test set.

Encode feature3 (blood type) with a single missing flag Create two binary columns from feature3: A_antigen and B_antigen with the rules:

A → A_antigen=1, B_antigen=0
B → A_antigen=0, B_antigen=1
AB → A_antigen=1, B_antigen=1
O → A_antigen=0, B_antigen=0 Add one missingness flag named feature3_missing where 1 indicates feature3 was NaN. For modeling, fill NaNs in A_antigen and B_antigen with 0. Drop the original feature3 column after encoding.

One-hot encode feature4 Treat feature4 as an unordered category.

First fill missing values with the string "Unknown" in both train and test.
Use OneHotEncoder(handle_unknown="ignore", sparse_output=False) on train, then transform test.
The resulting columns must match the expected names below.

Train a simple model Fit a classifier (any is fine, e.g., SVM) on the prepared X_train and evaluate on X_test.

After all processing, the columns in X_train and X_test must be exactly:

['feature1', 'feature2', 'feature1_missing', 'feature2_missing',
 'A_antigen', 'B_antigen', 'feature3_missing', 'feature4_CT',
 'feature4_MRI', 'feature4_Ultrasound', 'feature4_Unknown',
 'feature4_Xray']

Hints and pitfalls to avoid

Do not take a plain log of negative numbers or zero; use np.log1p on the magnitude and keep the sign.
Fit the scaler and imputer on training data only.
Use a single missing flag for feature3 (do not create separate missing flags for A and B antigens).
Include an "Unknown" category for feature4 so all rows are encodable.
After encoding feature3, drop the original feature3 column; for feature4, replace it with the one-hot columns.

numerical_features_df.head()

categorical_features_df.head()

target_df.head()

Merge Merge the three tables on PatientID so the columns are in the exact order shown above.

# YOUR CODE HERE

Explore missingness Count NaNs per column to understand what needs imputing or encoding.

# YOUR CODE HERE

Create X and y, then split Set X = ['feature1', 'feature2', 'feature3', 'feature4'] and y = diagnosis. Do a train/test split with test_size=0.2 and a fixed random_state for reproducibility.

# YOUR CODE HERE

Log-transform feature1 safely Apply a log transform that works for negative values and zero. Hint: transform the magnitude with np.log1p(np.abs(x)) (this is log(1 + |x|), which is 0 for x = 0) and restore the sign with np.sign(x). Do this separately on X_train and X_test.

# YOUR CODE HERE

Standardize numerical features Standardize feature1 and feature2 using StandardScaler. Fit on the training set and only transform the test set.

# YOUR CODE HERE

Handle missing values for numeric features For each of feature1 and feature2:

Add a missingness indicator column named <col>_missing that is 1 if the original value is missing, else 0.
Impute missing values with the median learned from the training set, then apply to the test set.

# YOUR CODE HERE

Encode feature3 (blood type) with a single missing flag Create two binary columns from feature3: A_antigen and B_antigen with the rules:

A → A_antigen=1, B_antigen=0
B → A_antigen=0, B_antigen=1
AB → A_antigen=1, B_antigen=1
O → A_antigen=0, B_antigen=0 Add one missingness flag named feature3_missing where 1 indicates feature3 was NaN. For modeling, fill NaNs in A_antigen and B_antigen with 0. Drop the original feature3 column after encoding.

# YOUR CODE HERE

One-hot encode feature4 Treat feature4 as an unordered category.

First fill missing values with the string "Unknown" in both train and test.
Use OneHotEncoder(handle_unknown="ignore", sparse_output=False) on train, then transform test.
The resulting columns must match the expected names below.

# YOUR CODE HERE

Train a simple model Fit SVM classifier on the prepared X_train and evaluate on X_test.

After all processing, the columns in X_train and X_test must be exactly:

['feature1', 'feature2', 'feature1_missing', 'feature2_missing',
 'A_antigen', 'B_antigen', 'feature3_missing', 'feature4_CT',
 'feature4_MRI', 'feature4_Ultrasound', 'feature4_Unknown',
 'feature4_Xray']

# YOUR CODE HERE
# I got 0.75 accuracy with SVC, default params, compare to your score.

1. Motivation

2. Feature Scaling

fit_transform vs transform

3. Feature Transformations

4. Missing Data

5. Merging Data

6. Encoding Data

LabelEncoder

OrdinalEncoder

OneHotEncoder

7. Exercise

`fit_transform` vs `transform`