AI Specialization Program

 WEEK 2  Lesson 2 

Title: Pandas for Data Loading & Exploration

Theme: Inspect Before You Model

Core Principle: Never train on data you haven’t explored.

Lesson Objective

By the end of this lesson, students will:

✔ Load CSV, JSON, and Excel files

✔ Inspect dataset structure

✔ Select and filter data

✔ Compute basic statistics

✔ Detect missing values

✔ Develop exploratory discipline

This lesson builds data confidence — a critical AI engineering skill.

1. What Is Pandas?

Pandas is Python’s most powerful data analysis library.

Pandas allows you to:

• Load structured data

• Clean datasets

• Filter rows & columns

• Compute statistics

• Prepare data for ML models.

See illustrations below.

Pandas works primarily with:

• DataFrame (table)

• Series (single column)

2 Loading Data Files

🔹 Load CSV

Python

import pandas as pd

df = pd.read_csv("data.csv")

🔹 Load Excel

Python

df = pd.read_excel("data.xlsx")

🔹 Load JSON

Python

df = pd.read_json("data.json")

Engineering Insight

Always inspect immediately after loading.

Never assume structure.

3. Inspecting Your Dataset

These are the first commands every AI engineer runs.

🔹 View First Rows

Python

df.head()

Shows the first 5 rows.

🔹 Dataset Info

Python

df.info()

Shows:

• Column names

• Data types

• Missing values

• Memory usage

🔹 Summary Statistics

Python

df.describe()

Provides:

• Mean

• Standard deviation

• Min/Max

• Percentiles

This reveals scale and outliers.

4. Selecting Columns

🔹 Select One Column

Python

df["age"]

🔹 Select Multiple Columns

Python

df[["age", "income"]]

5. Filtering Rows

Filtering is how you inspect patterns.

🔹 Example: Age > 30

Python

df[df["age"] > 30]

🔹 Example: Specific Category

Python

df[df["region"] == "East Africa"]

This helps uncover bias or imbalance.

6. Detecting Missing Values

Missing data breaks models.

🔹 Check Missing Values

Python

df.isnull().sum()

This shows how many missing values per column.

🔹 Percentage Missing

Python

df.isnull().mean() * 100

Now you know severity.

7. Basic Statistics Engineers Always Check

• Distribution range

• Skewness (large differences in mean vs median)

• Class imbalance

• Outliers

• Unique values

Example:

Python

df["category"].value_counts()

This reveals imbalance.

African Dataset Context

Real-world African datasets often have:

• Missing infrastructure records

• Inconsistent spellings

• Multiple languages

• Categorical noise

• Sparse documentation

Exploration is not optional.

It is survival.

 Common Beginner Mistakes

❌ Training without checking missing values

❌ Ignoring data types (numbers stored as text)

❌ Not checking label balance

❌ Trusting dataset blindly

❌ Skipping exploratory analysis

Professional engineers explore first.

Mini Practical Exercise

Students must:

1. Load a public dataset.

2. Run:

• head()

• info()

• describe()

• value_counts() on label

3. Identify:

• Any missing values

• Any imbalance

• Any suspicious data types

4. Write short summary (5–8 lines).

Deliverable: Exploratory report.

Lesson 2 Outcome

By the end of this lesson, students:

✔ Load structured data confidently

✔ Inspect dataset structure

✔ Detect missing values

✔ Identify imbalance

✔ Understand data types

✔ Develop exploratory mindset

Engineering Principle of the Week

“If you haven’t explored your dataset, you don’t understand your problem.”

Next:

 Week 2 — Lesson 3: Data Cleaning & Transformations (Pipelines Mindset)

Powered by Soft AI Africa | Training the Next Generation of AI Leaders in Africa.