AI Specialization Program

🔷 WEEK  3  Lesson 2

Title:

Regression Models

Learning Objectives

By the end of this lesson, learners will:

Understand what regression models are used for

Build intuition for linear regression

Understand the idea of a cost function

Understand the concept of gradient descent

Learn common regression evaluation metrics (MAE, MSE, RMSE)

Implement a simple regression model in Python

This lesson introduces predictive modeling for numerical outcomes.

1. What is Regression?

Regression is a type of supervised learning used to predict continuous numerical values.

Examples of regression problems:

• Predict house prices

• Forecast crop yield

• Estimate energy consumption

• Predict sales revenue

• Predict rainfall levels

Example dataset:

House Size    Rooms     Price

   1200              3       200000

   1500              4       250000

Goal:

Text

Use input features → predict a numeric value

Regression models learn relationships between variables.

2. Linear Regression Intuition

Linear regression assumes a linear relationship between variables.

The model tries to fit a straight line through the data.

Mathematical form:

Text

y = mx + b

Where:

y = predicted value

x = input feature

m = slope (relationship strength)

b = intercept

Example:

Predict house price based on house size.

The model learns the best line that fits the data.

The goal is to minimize prediction error.

3. The Cost Function (Measuring Error)

To improve predictions, the model needs a way to measure error.

This is done using a cost function.

The cost function calculates how far predictions are from actual values.

Example concept:

Text

Error = Predicted Value − Actual Value

The model adjusts its parameters to reduce this error.

One common cost function is Mean Squared Error (MSE).

4. Gradient Descent (Optimization Concept)

Gradient Descent is the method used to minimize the cost function.

Instead of guessing the best parameters randomly, the algorithm:

a. Starts with random values

b. Calculates error

c. Adjusts parameters slightly

d. Repeats the process many times

This gradually moves the model toward lower error.

Conceptual process:

Text

Initial Guess → Calculate Error → Adjust Parameters → Repeat

Over time, the model finds the best fitting line.

Here, students do not need heavy mathematics yet — only the intuition.

5. Regression Evaluation Metrics

Once a model is trained, we must evaluate its performance.

Several metrics are used for regression.

Mean Absolute Error (MAE)

Measures the average absolute difference between predictions and actual values.

Text

MAE = average(|prediction − actual|)

Easy to interpret.

Mean Squared Error (MSE)

Squares errors before averaging.

Text

MSE = average((prediction − actual)²)

Penalizes large errors more strongly.

Root Mean Squared Error (RMSE)

Square root of MSE.

Text

RMSE = √MSE

Often easier to interpret because it returns values in the original unit.

6. Implementing Regression in Python

Using scikit-learn, one of the most common ML libraries.

Example code:

Python:

from sklearn.linear_model import LinearRegression

from sklearn.model_selection import train_test_split

from sklearn.metrics import mean_squared_error

# Example data

X = df[['house_size']]

y = df['price']

# Split data

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

# Train model

model = LinearRegression()

model.fit(X_train, y_train)

# Make predictions

predictions = model.predict(X_test)

# Evaluate model

mse = mean_squared_error(y_test, predictions)

print(mse)

This simple workflow is the starting point for many ML projects.

7.  Engineering Mindset for Regression

Regression models are powerful but require careful thinking.

Engineers must consider:

• Data quality

• Feature relevance

• Outliers

• Proper evaluation

• Avoiding data leakage

Even simple models can perform extremely well when data is properly prepared.

Mini Practical Exercise

Students should:

a. Use a simple dataset containing numeric features.

b. Train a linear regression model.

c. Split data into training and test sets.

d. Make predictions.

e. Calculate MSE or RMSE.

Write a short explanation:

What does the error metric tell you about the model performance?

Week 3 – Lesson 2 Outcome

Students now:

✔ Understand regression problems

✔ Understand linear regression intuition

✔ Understand cost functions

✔ Understand gradient descent concept

✔ Know key regression evaluation metrics

✔ Can implement a regression model in Python

Students have now built their first predictive Machine Learning model.

Next:

Week 3 Lesson 3 — Classification Models

Where students learn how machines predict categories instead of numbers.

Powered by Soft AI Africa | Training the Next Generation of AI Leaders in Africa.