Data Preprocessing for Regression

# CHAPTER 9

Data Preprocessing for Regression #

1. Introduction #

2. Learning Objectives #

• Handle missing values using SimpleImputer.

• Identify and mitigate Outliers.

• Understand the mathematical need for Feature Scaling.

• Implement Standardization (StandardScaler).

• Implement Normalization (MinMaxScaler).

3. Handling Missing Values #

import numpy as np
from sklearn.impute import SimpleImputer

# Dataset with a missing value (NaN)
X = np.array([[10], [20], [np.nan], [40], [50]])

# Create an imputer that replaces NaN with the average (mean) of the column
imputer = SimpleImputer(strategy='mean')

# Apply it to the data
X_clean = imputer.fit_transform(X)

print(X_clean)
# The NaN is replaced by 30 (the average of 10, 20, 40, 50)

4. The Outlier Problem #

5. Why Do We Need Feature Scaling? #

• Bedrooms: Ranges from 1 to 5.

• SquareFeet: Ranges from 1,000 to 5,000.

In regression, the algorithm multiplies weights by these inputs. Because SquareFeet has such massive numbers, the math will naturally assume Square_Feet is 1,000 times more important than Bedrooms simply because the numbers are bigger! This is a disaster. Feature Scaling squashes all columns down to the exact same numerical scale (e.g., between 0 and 1) so the algorithm treats them fairly.

6. Standardization (Z-Score Scaling) #

from sklearn.preprocessing import StandardScaler
import numpy as np

# Raw data: massive differences in scale
X = np.array([[1, 10000], 
              [2, 20000], 
              [3, 30000]])

scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

print(X_scaled)
# Output:
# [[-1.22  -1.22]
#  [ 0.     0.  ]
#  [ 1.22   1.22]]
# Now both columns are on the exact same scale!

7. Normalization (Min-Max Scaling) #

from sklearn.preprocessing import MinMaxScaler

scaler = MinMaxScaler()
X_normalized = scaler.fit_transform(X)

print(X_normalized)
# Output:
# [[0.  0. ]
#  [0.5 0.5]
#  [1.  1. ]]

8. Standardization vs. Normalization #

• Use Standardization (StandardScaler): 90% of the time in Linear Regression, Logistic Regression, and Deep Learning. It handles outliers slightly better than Normalization.

• Use Normalization (MinMaxScaler): When you specifically need the data bounded between 0 and 1 (e.g., Image pixel data for Neural Networks).

9. Common Mistakes #

• Data Leakage: A critical mistake is scaling the *entire* dataset before splitting it into Train and Test sets. If you do this, information from the Test set "leaks" into the scaler's calculations (Mean/Max). You must fit_transform the Training data, and only transform the Test data.

• Scaling the Target Variable (y): Usually, you only scale the Input Features (X). Scaling the target variable (y) makes the final predictions unreadable (e.g., predicting a salary as 0.85 instead of $85,000).

10. Best Practices #

• Build Pipelines: Instead of manually calling imputer.fit(), then scaler.fit(), then model.fit(), scikit-learn allows you to chain them together using from sklearn.pipeline import Pipeline. This prevents Data Leakage and makes deployment incredibly easy.

11. Exercises #

1. 1. If an age column has a minimum value of 20 and a maximum value of 60, what will the age 40 become after applying MinMaxScaler?

1. 2. Why does a massive outlier drastically change the slope of a Simple Linear Regression line?

12. MCQ Quiz with Answers #

13. Interview Questions #

• Q: Explain "Data Leakage" in the context of applying a StandardScaler. How do you prevent it?

• Q: In Linear Regression, explain why an extreme outlier in the training data negatively impacts the "Line of Best Fit."

14. FAQs #

15. Summary #

16. Next Chapter Recommendation #