This project implements a comprehensive house price prediction pipeline using the Ames Housing dataset—a widely adopted benchmark in regression modeling and feature engineering education. The workflow spans exploratory data analysis (EDA), statistical visualization, missing value handling, feature selection, model training, and evaluation.

Data Loading and Initial Setup

The analysis begins by importing essential libraries and configuring visualization defaults:

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.ensemble import RandomForestRegressor
from sklearn.linear_model import Ridge, Lasso
from sklearn.metrics import mean_squared_error, r2_score
from sklearn.preprocessing import StandardScaler, LabelEncoder

plt.style.use('seaborn-v0_8')
sns.set_palette("husl")
plt.rcParams.update({'figure.figsize': (10, 6), 'font.size': 11})

Next, the training and test datasets are loaded:

train_df = pd.read_csv("ames_train.csv")
test_df = pd.read_csv("ames_test.csv")

# Display basic info
print(f"Training set shape: {train_df.shape}")
print(f"Test set shape: {test_df.shape}")
print("\nFirst five rows of training data:")
train_df.head()

Target Variable Analysis: SalePrice

The target variable SalePrice is examined for distribution characteristics:

price_stats = train_df['SalePrice'].describe()
print(price_stats)

plt.figure(figsize=(10, 4))
plt.subplot(1, 2, 1)
sns.histplot(train_df['SalePrice'], kde=True, bins=50, color='steelblue')
plt.title('Distribution of SalePrice')

plt.subplot(1, 2, 2)
sns.boxplot(y=train_df['SalePrice'], color='lightcoral')
plt.title('Boxplot of SalePrice')
plt.tight_layout()
plt.show()

skewness = train_df['SalePrice'].skew()
kurtosis = train_df['SalePrice'].kurtosis()
print(f"\nSkewness: {skewness:.3f}, Kurtosis: {kurtosis:.3f}")

The distribution exhibits positive skew—common in real estate pricing—suggesting log-transformation may improve model performance.

Categorical Feature Exploration

Key categorical predictors are visualized via grouped boxplots to assess their relationship with sale price:

cat_features = ['Utilities', 'Heating', 'Central_Air', 'Garage_Type', 'Neighborhood', 'Overall_Qual']

fig, axes = plt.subplots(3, 2, figsize=(16, 18))
axes = axes.flatten()

for i, feature in enumerate(cat_features):
    sns.boxplot(data=train_df, x=feature, y='SalePrice', ax=axes[i])
    axes[i].set_title(f'SalePrice vs {feature}', fontsize=12)
    axes[i].tick_params(axis='x', rotation=45)

plt.tight_layout()
plt.show()

Notably, Overall_Qual shows a strong monotonic trend with price, while Neighborhood reveals substantial inter-district variation—highlighting georgaphic influence on valuation.

Numerical Feature Relationships

Scatter plots with regression lines evaluate linear associations between numeric features and SalePrice:

num_features = ['Gr_Liv_Area', 'Total_Bsmt_SF', 'Lot_Area', 'TotRms_AbvGrd']

fig, axes = plt.subplots(2, 2, figsize=(14, 10))
axes = axes.flatten()

for i, col in enumerate(num_features):
    sns.regplot(
        data=train_df,
        x=col,
        y='SalePrice',
        scatter_kws={'alpha': 0.3, 's': 10},
        line_kws={'color': 'darkorange'},
        ax=axes[i]
    )
    axes[i].set_title(f'SalePrice vs {col}')

plt.tight_layout()
plt.show()

Gr_Liv_Area demonstrates high correlation and linearity; outliers beyond ~4,500 sq ft are capped to reduce leverage effects:

def cap_outliers(df, column, upper_bound):
    df[column] = np.clip(df[column], None, upper_bound)
    return df

train_df = cap_outliers(train_df, 'Gr_Liv_Area', 4500)
test_df = cap_outliers(test_df, 'Gr_Liv_Area', 4500)

Correlation Heatmap and Feature Selection

A correlation matrix highlights multicollinearity and guides feature prioritization:

numeric_cols = train_df.select_dtypes(include=[np.number]).columns.tolist()
corr_matrix = train_df[numeric_cols].corr().round(2)

plt.figure(figsize=(12, 10))
mask = np.triu(np.ones_like(corr_matrix, dtype=bool))
sns.heatmap(
    corr_matrix,
    mask=mask,
    center=0,
    cmap='RdBu_r',
    square=True,
    linewidths=0.5,
    cbar_kws={"shrink": .8}
)
plt.title('Feature Correlation Matrix (Numerical Variables)')
plt.show()

# Select top correlated predictors excluding SalePrice
top_corrs = corr_matrix['SalePrice'].abs().sort_values(ascending=False)[1:7].index.tolist()
print("Top 6 numerical features by absolute correlation with SalePrice:")
print(top_corrs)

Selected features include Overall_Qual, Gr_Liv_Area, Total_Bsmt_SF, Year_Built, Garage_Cars, and Mas_Vnr_Area.

Preprocessing and Model Training

Missing values in continuous columns are imputed using median (more robust than mean for skewed distributions):

impute_cols = ['Total_Bsmt_SF', 'Mas_Vnr_Area', 'Garage_Cars']
for col in impute_cols:
    train_df[col].fillna(train_df[col].median(), inplace=True)
    test_df[col].fillna(train_df[col].median(), inplace=True)  # Use training stats for consistency

X = train_df[top_corrs]
y = np.log1p(train_df['SalePrice'])  # Log-transform target

X_train, X_val, y_train, y_val = train_test_split(
    X, y, test_size=0.2, random_state=42
)

# Scale features for regularization models
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_val_scaled = scaler.transform(X_val)

# Train multiple models
models = {
    'Random Forest': RandomForestRegressor(n_estimators=300, random_state=42),
    'Ridge Regression': Ridge(alpha=10),
    'Lasso Regression': Lasso(alpha=0.001)
}

results = {}
for name, model in models.items():
    if 'Regression' in name:
        model.fit(X_train_scaled, y_train)
        preds = model.predict(X_val_scaled)
    else:
        model.fit(X_train, y_train)
        preds = model.predict(X_val)
    
    rmse = np.sqrt(mean_squared_error(y_val, preds))
    r2 = r2_score(y_val, preds)
    results[name] = {'RMSE': rmse, 'R²': r2}
    print(f"{name}: RMSE = {rmse:.4f}, R² = {r2:.4f}")

Prediction and Submission

The best-performing model (e.g., Random Forest) generates predictions on the test set:

best_model = models['Random Forest']
best_model.fit(X, y)

X_test = test_df[top_corrs]
test_predictions = best_model.predict(X_test)

# Inverse transform log predictions
final_predictions = np.expm1(test_predictions)

submission = pd.DataFrame({
    'Id': test_df['Id'],
    'SalePrice': final_predictions
})
submission.to_csv('submission_house_prices.csv', index=False)
print("Submission file saved.")