Data Analysis with Pandas in Jupyter

# CHAPTER 10

Data Analysis with Pandas in Jupyter #

1. Chapter Introduction #

2. Installing and Importing Pandas #

If you installed Anaconda, Pandas is already included. If not, run !pip install pandas in a cell.

Cell 1:

import pandas as pd
# 'pd' is the universal industry standard alias for pandas

3. The Pandas DataFrame #

A DataFrame is a 2-dimensional labeled data structure. Think of it exactly like an Excel spreadsheet or a SQL table.

Cell 2:

# Creating a DataFrame from a dictionary
data = {
    'Name': ['Alice', 'Bob', 'Charlie', 'David'],
    'Age': [25, 30, 35, 40],
    'Department': ['Sales', 'IT', 'HR', 'IT'],
    'Salary': [55000, 75000, 60000, 80000]
}

df = pd.DataFrame(data)

# In Jupyter, simply typing 'df' at the end of the cell 
# renders a beautiful, styled HTML table!
df

4. Reading Data from a CSV #

In the real world, you don't type data out by hand. You load it from a CSV (Comma Separated Values) file.

Cell 3:

# Load a CSV file into a DataFrame
# (Assuming 'employees.csv' is in your folder)
# df = pd.read_csv('employees.csv')

# Look at the first 3 rows
df.head(3)

Jupyter Pro-Tip: If your dataset has 10,000 rows, do NOT just type df and hit Enter. Jupyter will try to render the massive table and might freeze your browser. Always use df.head() to preview the first 5 rows, or df.tail() to see the last 5.

5. Data Inspection #

Once data is loaded, you must inspect it to understand its shape and data types.

Cell 4:

# Check how many rows and columns (Rows, Columns)
print("Shape:", df.shape)

# Get a summary of data types and non-null values
df.info()

# Get statistical summaries of numeric columns (mean, min, max, etc.)
df.describe()

6. Filtering and Querying Data #

Pandas allows you to slice and dice your data easily.

Cell 5:

# Select a single column (returns a Pandas Series)
ages = df['Age']

# Filter rows based on a condition (e.g., IT department only)
it_staff = df[df['Department'] == 'IT']

# Display the filtered data
it_staff

7. Basic Data Cleaning #

Data is rarely perfect. Pandas provides tools to handle missing values (NaN).

Cell 6:

import numpy as np

# Let's introduce a missing value for demonstration
df.loc[1, 'Salary'] = np.nan
print("Data with missing salary:")
display(df) # display() forces Jupyter to render the HTML table inside print-like logic

# Option A: Fill missing values with a default number
df_filled = df.fillna(0)

# Option B: Drop any row that contains a missing value
df_dropped = df.dropna()

8. Mini Project: Student Analytics Notebook #

Create a new notebook and run this workflow.

Cell 1:

import pandas as pd
# Load student grades
grades = pd.DataFrame({
    'Student': ['Tom', 'Jane', 'Bill', 'Sara'],
    'Math': [88, 92, 79, 95],
    'Science': [85, 90, 82, 98]
})

Cell 2:

# Calculate Average Grade per student
grades['Average'] = (grades['Math'] + grades['Science']) / 2
grades

Cell 3:

# Find Top Performers (Average > 90)
top_students = grades[grades['Average'] > 90]
print("Students eligible for honors:")
display(top_students)

9. Common Mistakes #

• Printing DataFrames: Beginners often write print(df). This outputs ugly, raw text aligned with spaces. Instead, use display(df) or simply leave df on the last line of the cell. Jupyter will render it as a styled HTML table.

• Forgetting the alias: Always import pandas as pd. If you just import pandas, you have to type pandas.DataFrame every time instead of pd.DataFrame, which gets tedious.

10. MCQs #

11. Interview Questions #

• Q: Explain the difference between a Pandas Series and a Pandas DataFrame.

• Q: You load a CSV into Jupyter and it has 5 million rows. What commands do you run first to understand the data without crashing the notebook?

12. Summary #

13. Next Chapter Recommendation #