Skip to main content
Regression Models
CHAPTER 07 Intermediate

Multiple Linear Regression

Updated: May 16, 2026
6 min read

# CHAPTER 7

Multiple Linear Regression

1. Introduction

Simple Linear Regression (using one feature) is great for learning, but terrible for reality. If you predict a house's price using *only* its Square Footage, you will be shockingly inaccurate because you ignored the number of Bedrooms, the Age of the house, and the Distance to the city center. Multiple Linear Regression solves this by allowing the algorithm to analyze dozens, hundreds, or even thousands of features simultaneously to make a highly accurate prediction.

2. Learning Objectives

By the end of this chapter, you will be able to:
  • Understand the math behind Multiple Linear Regression.
  • Train a model using multiple input features.
  • Interpret multiple Coefficients to determine Feature Importance.
  • Build a House Price prediction model.

3. The Math: Expanding the Equation

In Simple Linear Regression, the formula was a simple 2D line: $y = (m \times x) + b$. In Multiple Linear Regression, we simply add more variables! If we have 3 features (Size, Bedrooms, Age), the model creates a 4D hyper-plane: $$y = (m1 \times X1) + (m2 \times X2) + (m3 \times X3) + b$$
  • $m1$ (Coefficient 1): The weight assigned to Size.
  • $m2$ (Coefficient 2): The weight assigned to Bedrooms.
  • $m_3$ (Coefficient 3): The weight assigned to Age.
The algorithm calculates the perfect combination of these weights to minimize the prediction error.

4. Mini Project: House Price Prediction

Let's build a model predicting house prices using 3 features: Size (sqft), Bedrooms, and Age (years).
python
123456789101112131415161718192021222324252627 
import numpy as np
from sklearn.linear_model import LinearRegression

# 1. Provide the Data
# X is now a 2D matrix with 3 columns: [Size, Bedrooms, Age]
X_train = np.array([
    [1500, 3, 10], 
    [2000, 4, 5],  
    [2500, 4, 2],  
    [1200, 2, 20], 
    [3000, 5, 1]   
])

# y = Price in $
y_train = np.array([300000, 450000, 550000, 200000, 650000])

# 2. Initialize and Train the Model
model = LinearRegression()
model.fit(X_train, y_train)

# 3. Make a Prediction!
# Predict price for a 2200 sqft, 3 bed, 8-year-old house
X_test = np.array([[2200, 3, 8]])
prediction = model.predict(X_test)

print(f"Predicted House Price: ${prediction[0]:.2f}")
# Output: Predicted House Price: $464303.80

5. Feature Importance (Interpreting Coefficients)

The most powerful aspect of Linear Regression is that it is a "White Box" model. We can look inside and see exactly *why* it made its decision by looking at the generated Coefficients ($m$).
python
1234567891011 
# The model generated 3 coefficients, one for each column
print(f"Base Price (Intercept): ${model.intercept_:.2f}")
print(f"Coefficient for Size: ${model.coef_[0]:.2f}")
print(f"Coefficient for Bedrooms: ${model.coef_[1]:.2f}")
print(f"Coefficient for Age: ${model.coef_[2]:.2f}")

# Example Output:
# Base Price (Intercept): $45000.00
# Coefficient for Size: $150.00
# Coefficient for Bedrooms: $25000.00
# Coefficient for Age: $-5000.00

Interpretation: The algorithm mathematically proved that:

  • Every extra square foot adds $150 to the value.
  • Every extra bedroom adds $25,000 to the value.
  • Every year the house gets older, it *loses* $5,000 in value (Negative Correlation!).

6. The Problem of Scale

If you look at the coefficients above, Bedrooms is $25,000 and Size is only $150. Does this mean Bedrooms are the most important feature? No! Bedrooms only go up by 1 or 2. Size goes up by 1000s. Because the raw data is on entirely different scales, the raw coefficients can be misleading. To truly compare feature importance, we must "Standardize" the data first (which we cover deeply in Chapter 9).

7. Why Visualizing Multiple Regression is Hard

You can easily visualize Simple Linear Regression on a 2D piece of paper (X and Y axis). If you have two features (Size, Bedrooms), the graph becomes a 3D cube, and the line becomes a flat "Plane". If you have three features, it becomes a 4D hyper-cube, which human brains cannot visualize. Therefore, we rely heavily on mathematical error metrics (like R-Squared) rather than visual graphs to judge accuracy in multiple regression.

8. Common Mistakes

  • The Dummy Variable Trap: If you include categorical data (e.g., "City: NY, LA, SF"), you must convert them to 1s and 0s (One-Hot Encoding). If you don't drop one of the encoded columns, it creates perfect Multicollinearity, crashing the mathematical equation.
  • Throwing Garbage Data In: Adding 100 random, irrelevant columns (like the homeowner's favorite color) to a multiple regression model will not make it smarter. It will confuse the algorithm and cause Overfitting.

9. Best Practices

  • Feature Selection: More features do not equal a better model. Only include features that logically have a correlation to the target.

10. Exercises

  1. 1. In the equation $y = (m1 \times X1) + (m2 \times X2) + b$, what does $X2$ represent in our housing example?
  1. 2. Why would the coefficient ($m$) for the "Age" of a car be a negative number when predicting the car's price?

11. MCQ Quiz with Answers

Question 1

What is the primary difference between Simple Linear Regression and Multiple Linear Regression?

Question 2

If model.coef[2] (representing Age of a house) equals -4000, what does this mathematically signify to the business?

12. Interview Questions

  • Q: Explain why comparing the raw, unscaled coefficients in a Multiple Linear Regression model is not a reliable way to determine feature importance.
  • Q: How does a Multiple Linear Regression model differ geometrically from a Simple Linear Regression model?

13. FAQs

Q: How many features is too many? A: There is no hard limit (some models use thousands), but there is a rule of thumb: If you have more features (columns) than you have actual data rows, Linear Regression will mathematically fail and overfit completely. You must use Regularization (Chapter 12) if this happens!

14. Summary

By expanding our mathematical formula to include multiple $X$ variables, Multiple Linear Regression allows us to model complex, real-world scenarios. It not only provides accurate predictions by analyzing multiple factors simultaneously, but its transparency allows analysts to report exactly *how much* each feature contributes to the final outcome.

15. Next Chapter Recommendation

Linear Regression is powerful, but it is mathematically rigid. It relies on strict statistical rules to function correctly. If your dataset breaks these rules, the model will output confidently wrong answers. In Chapter 8: Regression Assumptions Explained, we will uncover the secret rules you must check before trusting your model.

Finish this Chapter

Save your progress on your learning path and prepare for coding interview challenges.

Previous Chapter Simple Linear Regression
Next Chapter Regression Assumptions Explained

Discussion

Join the discussion

Log in or create a free account to participate.

Sort: ·

Explore More

📖 Related Tutorials 4

🧪 Related Labs 1