Advanced Data Science Techniques

# CHAPTER 29

Advanced Data Science Techniques #

1. Chapter Introduction #

2. Feature Engineering #

Feature Engineering is the art of creating new, highly predictive columns (Features) from existing data. An algorithm can only learn from what you give it.

Example 1: Date Engineering If you have a Timestamp column, a Regression model can't understand it. But if you extract the Month or DayOfWeek, the model might suddenly realize that sales always spike on Fridays in December.

import pandas as pd

df = pd.DataFrame({'Date': pd.to_datetime(['2023-12-01', '2023-07-15'])})

# Feature Engineering: Extracting discrete numbers from a date
df['Month'] = df['Date'].dt.month
df['DayOfWeek'] = df['Date'].dt.dayofweek # 0=Monday, 6=Sunday
print(df)

Example 2: Mathematical Interactions If you have Width and Height, creating a new column Area = Width * Height might provide a much stronger signal to the algorithm than the two individual columns alone.

3. Scikit-Learn Pipelines #

In Chapter 22, we manually imputed NaNs, then manually encoded text, then manually scaled data. If you write this in 20 lines of code, you have to rewrite it all when new data arrives tomorrow.

A Pipeline chains all these preprocessing steps and the model into a single, unified object.

from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression

# Define the automated sequence of steps
# 1. Impute (fill) missing numbers with the mean
# 2. Scale the data
# 3. Train the model
pipeline = Pipeline([
    ('imputer', SimpleImputer(strategy='mean')),
    ('scaler', StandardScaler()),
    ('model', LogisticRegression())
])

# You only have to call .fit() ONCE! The pipeline handles everything.
# pipeline.fit(X_train, y_train)

# You only call .predict() ONCE! It automatically scales the test data for you.
# predictions = pipeline.predict(X_test)

*Pipelines completely eliminate the risk of Data Leakage.*

4. Hyperparameter Tuning (GridSearchCV) #

When you initialize a DecisionTreeClassifier(max_depth=3), how do you know 3 is the best depth? Maybe it's 5? Maybe 10?

These settings are called Hyperparameters. Instead of guessing, we use GridSearchCV to force the computer to train 100 different versions of the model with different settings and automatically pick the best one.

from sklearn.model_selection import GridSearchCV
from sklearn.tree import DecisionTreeClassifier

model = DecisionTreeClassifier()

# Define a "Grid" of settings you want to test
param_grid = {
    'max_depth': [3, 5, 10, None],
    'min_samples_split': [2, 5, 10]
}

# GridSearchCV will train 4 * 3 = 12 different models!
grid_search = GridSearchCV(model, param_grid, cv=5, scoring='accuracy')

# Run the massive test
# grid_search.fit(X_train, y_train)

# Ask it what the winning combination was
# print("Best parameters:", grid_search.best_params_)

5. Advanced EDA: Profiling Libraries #

Writing 20 Seaborn plots takes time. Professional Data Scientists often use automated profiling libraries like ydata-profiling to generate a massive HTML report of the entire dataset in one line of code.

# Installation required: !pip install ydata-profiling
# import pandas as pd
# from ydata_profiling import ProfileReport

# df = pd.read_csv('data.csv')
# profile = ProfileReport(df, title="Automated EDA Report")
# profile.to_file("report.html")

*(This opens a gorgeous, interactive web page showing correlations, missing values, and histograms for every single column!)*

6. Common Mistakes #

• Over-Engineering Features: Creating 500 new columns by combining every single variable together. This causes "The Curse of Dimensionality," making the model slower, harder to interpret, and prone to overfitting.

• Tuning without Cross-Validation: If you use GridSearchCV without cross-validation (cv=5), it might pick settings that just got lucky on that specific Train-Test split.

7. MCQs #

8. Interview Questions #

• Q: Explain what Feature Engineering is. Give an example of how you might engineer a new feature from an "Address" column to predict House Prices.

• Q: What is the purpose of Hyperparameter Tuning? Explain how GridSearchCV accomplishes this.

9. Summary #

10. Next Chapter Recommendation #