NumPy Broadcasting and Vectorization

# CHAPTER 12

NumPy Broadcasting and Vectorization #

1. Chapter Introduction #

2. What is Vectorization? #

Vectorization is the process of executing operations on entire arrays at once, rather than iterating through them item by item.

When you vectorize, you push the loop down into the C language level, which avoids Python's slow type-checking overhead.

import numpy as np

# Two arrays
a = np.array([1, 2, 3, 4])
b = np.array([10, 20, 30, 40])

# Element-wise addition (Vectorized)
# Adds index 0 to index 0, index 1 to index 1, etc.
c = a + b 
print(c) # [11 22 33 44]

# Element-wise multiplication
d = a * b
print(d) # [10 40 90 160]

*Rule of thumb in Data Science: If you are writing a for loop to do math on a NumPy array or Pandas DataFrame, you are doing it wrong.*

3. What is Broadcasting? #

In linear algebra, you can only add matrices if they have the exact same dimensions. NumPy relaxes this rule using Broadcasting.

Broadcasting describes how NumPy treats arrays with different shapes during arithmetic operations. It "broadcasts" (stretches) the smaller array across the larger array so they have compatible shapes.

Example 1: Array + Scalar (Single Number)

arr = np.array([10, 20, 30])
scalar = 5

# NumPy "broadcasts" the 5 into [5, 5, 5] behind the scenes
result = arr + scalar
print(result) # [15 25 35]

Example 2: 2D Matrix + 1D Array

matrix = np.array([
    [10, 20, 30],
    [40, 50, 60]
])

row_to_add = np.array([1, 2, 3])

# NumPy stretches 'row_to_add' down to match the 2 rows of the matrix
result = matrix + row_to_add
print(result)
# [[11 22 33]
#  [41 52 63]]

4. The Rules of Broadcasting #

NumPy compares shapes from right to left. Two dimensions are compatible if:

1. 1. They are equal, OR

1. 2. One of them is 1.

Example of Compatibility:

• Array A shape: (4, 3)

• Array B shape: (3,)

• *Compatible!* B stretches to (4, 3).

Example of Incompatibility:

• Array A shape: (4, 3)

• Array B shape: (4,)

• *Incompatible!* It tries to match the 3 and the 4. This will throw a ValueError: operands could not be broadcast together.

5. Universal Functions (ufuncs) #

NumPy provides fast, vectorized math functions called ufuncs that operate element-wise.

arr = np.array([1, 4, 9, 16])

# Square root of every element
print(np.sqrt(arr)) # [1. 2. 3. 4.]

# Exponential (e^x)
print(np.exp(arr))

# Logarithm
print(np.log(arr))

6. Performance Optimization Proof #

Let's prove the speed difference between a Python loop and NumPy vectorization.

import time

# Create 10 million numbers
size = 10_000_000
list_a = list(range(size))
list_b = list(range(size))
arr_a = np.arange(size)
arr_b = np.arange(size)

# 1. Slow Python Loop
start = time.time()
list_c = [list_a[i] + list_b[i] for i in range(size)]
print(f"Python Loop Time: {time.time() - start:.3f} seconds")

# 2. Fast NumPy Vectorization
start = time.time()
arr_c = arr_a + arr_b
print(f"NumPy Vectorization Time: {time.time() - start:.3f} seconds")

*NumPy will execute 10 million additions almost instantly, while Python will take noticeably longer.*

7. Common Mistakes #

• Broadcasting mismatch: Trying to add an array of shape (3,) to a column of shape (3, 1). You must use .reshape() to ensure the arrays align properly before broadcasting.

• Using math.sqrt() instead of np.sqrt(): The standard Python math module expects a single scalar number. If you pass an array to it, it crashes. Always use np functions on arrays.

8. MCQs #

Q10. If Array A is (3, 1) and Array B is (1, 3), can they be broadcast together? a) Yes, they stretch to (3, 3) b) No, they are incompatible — Answer: a

9. Interview Questions #

• Q: Explain the concept of Broadcasting in NumPy. Give an example of how it is useful.

• Q: A junior developer writes a for loop to subtract the mean from every column in a matrix. How would you refactor this code to be "Pythonic" and faster using NumPy?

10. Summary #

11. Next Chapter Recommendation #