Mathemaitcal Foundation

Linear regression is a fundamental statistical method used to model the relationship between a dependent varible and one or more independent variables. In simple linear regression, we examine how a single predictor variable influences an outcome variable through a linear relationship.

The mathematical formula for simple linear regression is:

y = a×x + b

where:

• y represents the dependent variable (target)
• x represents the independent variable (feature)
• a is the slope (coefficient)
• b is the y-intercept

The goal of regression is to find the optimal values for a and b that minimize the sum of squared differences between the predicted values and the actual observed values.

Python Implemantation

Installing Required Libraries

The scikit-learn library provides a comprehensive set of machine learning algorithms. Before installation, ensure you have the following dependencies:

umpip install numpynpip install scipynpip install matplotlibnpip install scikit-learn

scikit-learn is built on top of NumPy, SciPy, and matplotlib, so these packages must be installed first.

Visualizing Data Points

Let's create a scatter plot to visualize our dataset:

import matplotlib.pyplot as plt

# Experience in years (independent variable)
# Must be formatted as a 2D array for sklearn compatibility
work_experience = [[2], [3], [5], [7], [8], [10]]

# Salary in thousands (dependent variable)
salary = [15, 20, 35, 42, 48, 62]

# Create scatter plot
plt.scatter(work_experience, salary, color='blue', marker='o')
plt.xlabel('Years of Experience')
plt.ylabel('Salary (in thousands)')
plt.title('Experience vs Salary Distribution')
plt.show()

Building the Regression Model

from sklearn.linear_model import LinearRegression

# Initialize the linear regression model
predictor = LinearRegression()

# Fit the model with our data
predictor.fit(work_experience, salary)

Making Predictions

# Predict salary for 4 years of experience
new_experience = [[4]]
predicted_salary = predictor.predict(new_experience)
print(f"Predicted salary for 4 years: {predicted_salary[0]:.2f}")

# Multiple predictions
new_experiences = [[4], [6], [9]]
multiple_predictions = predictor.predict(new_experiences)
print(f"Predicted salaries: {multiple_predictions}")

Output:

Predicted salary for 4 years: 27.67
Predicted salaries: [27.67 38.52 54.31]

Extracting Model Parameters

# Get the slope coefficient
slope = predictor.coef_[0]

# Get the y-intercept
intercept = predictor.intercept_

print(f"Slope (coefficient): {slope:.4f}")
print(f"Intercept: {intercept:.4f}")

Output:

Slope (coefficient): 3.8636
Intercept: 12.2121

Full Regression Equation

The final regression equation is:

salary = 3.8636 × work_experience + 12.2121

This means that for each additional year of work experience, the salary increases by approximately 3,86 thousand units, and the base salary (with zero experience) is approximately 12.21 thousand units.

Complete Visualization

import numpy as np

# Generate prediction line data
exp_range = np.linspace(0, 12, 100).reshape(-1, 1)
predicted_line = predictor.predict(exp_range)

# Plot original data points
plt.scatter(work_experience, salary, color='blue', label='Actual Data')

# Plot regression line
plt.plot(exp_range, predicted_line, color='red', linewidth=2, label='Regression Line')

plt.xlabel('Years of Experience')
plt.ylabel('Salary (in thousands)')
plt.title('Simple Linear Regression: Experience vs Salary')
plt.legend()
plt.grid(True, alpha=0.3)
plt.show()

This visualization shows how the regression line best fits the scattered data points, demonstrating the underlying linear relationship between work experience and salary.