Dataset Overview

This tutorial utilizes the Kaggle Titanic training dataset. In this dataset, the second column "Survived" represents the target variable, while all other columns serve as features. The dataset contains 891 rows, 11 features, and 1 target variable. Notably, the "Age" feature has data for only 714 rows, the "Cabin" feature for 204 rows, and the "Embarked" feature for 889 rows, with the remaining entries containing missing values.

Removing Rows or Columns

The DataFrame.drop() method allows deletion of specific rows or columns:

Syntax: DataFrame.drop(labels=None, axis=0, index=None, columns=None, inplace=False)

Parameters:

• labels: Names of rows or columns to remove (specified as a list)
• axis: Default is 0 (rows). Set to 1 for columns
• index: Directly specify rows to remove
• columns: Directly specify columns to remove
• inplace: If False (default), returns a new DataFrame. If True, modifies the original DataFrame

Example - Removing columns:

modified_data = original_data.drop(labels=["Cabin", "Name"], axis=1, inplace=False)

Example - Removing rows:

modified_data = original_data.drop(labels=[0,1,2,3,4], axis=0, inplace=False)

Dropping Rows or Columns with Missing Values

The dropna() method handles missing values:

Syntax: DataFrame.dropna(axis=0, how='any', thresh=None, subset=None, inplace=False)

Parameters:

• axis: 0 or 'index' for rows, 1 or 'columns' for columns
• how: 'any' removes row/column with any missing value, 'all' removes only if all values are missing
• thresh: Minimum number of non-null values required to keep a row/column
• subset: Subset of rows/columns to consider when dropping
• inplace: If True, modifies the original DataFrame

Example - Removing rows with missing values:

clean_data = original_data.dropna(axis=0, inplace=False)

Example - Removing columns with missing values:

clean_data = original_data.dropna(axis=1, inplace=False)

Imputing Missing Values

Scikit-learn's SimpleImputer class fills missing values:

Syntax: SimpleImputer(*, missing_values=nan, strategy='mean', fill_value=None, verbose=0, copy=True, add_indicator=False)

Parameters:

• missing_values: The placeholder for missing values (default: np.nan)
• strategy: Imputation strategy ('mean', 'median', 'most_frequent', 'constant')
• fill_value: Value to use when strategy='constant'
• add_indicator: If True, adds a binary indicator for missing values

Example - Using median to fill missing "Age" values:

age_values = original_data.loc[:, "Age"].values.reshape(-1, 1)

from sklearn.impute import SimpleImputer
imputer = SimpleImputer(strategy="median")
imputed_values = imputer.fit_transform(age_values)

original_data.loc[:, "Age"] = imputed_values

Data Type Conversion

To convert the "Embarked" column to integer type:

unique_values = original_data["Embarked"].unique().tolist()
original_data["Embarked"] = original_data["Embarked"].apply(lambda x: unique_values.index(x))

Splitting Data into Training and Testing Sets

Scikit-learn's train_test_split divides the dataset:

Syntax: train_test_split(*arrays, test_size=None, train_size=None, random_state=None, shuffle=True, stratify=None)

Example:

# Separate features and target
features = original_data.loc[:, original_data.columns != "Survived"]
target = original_data.loc[:, original_data.columns == "Survived"]

# Split into training and testing sets
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(features, target, test_size=0.3)

Reordering Indices

After splitting datasets, indices may become disordered. To correct this:

for dataset in [X_train, X_test, y_train, y_test]:
    dataset.index = range(dataset.shape[0])

Cross-Validation

Cross-validation evaluates model performance:

Syntax: cross_val_score(estimator, X, y, scoring=None, cv=None, n_jobs=None, verbose=0)

Example:

from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import cross_val_score

model = DecisionTreeClassifier(random_state=30)
model.fit(X_train, y_train)

cv_scores = cross_val_score(model, X_train, y_train, cv=10)
mean_score = cv_scores.mean()

Grid Search

Grid search optimizes hyperparameters:

Example:

from sklearn.model_selection import GridSearchCV

# Define parameter grid
param_grid = {
    "criterion": ["gini", "entropy"],
    "splitter": ["best", "random"],
    "max_depth": list(range(1, 11))
}

base_model = DecisionTreeClassifier(random_state=30)
grid_search = GridSearchCV(base_model, param_grid, cv=10)
grid_search.fit(X_train, y_train)

# Best parameters and score
best_params = grid_search.best_params_
best_score = grid_search.best_score_

Learning Curves

Learning curves visualize model performance across different training set sizes:

import matplotlib.pyplot as plt
import sklearn.tree as tree

train_scores = []
for depth in range(1, 11):
    model = tree.DecisionTreeClassifier(
        max_depth=depth,
        criterion="entropy",
        random_state=24,
        splitter="random"
    )
    model.fit(X_train, y_train)
    accuracy = model.score(X_test, y_test)
    train_scores.append(accuracy)

plt.plot(range(1, 11), train_scores, color="red", label="Model Accuracy")
plt.xlabel("Tree Depth")
plt.ylabel("Accuracy")
plt.legend()
plt.title("Learning Curve for Decision Tree Classifier")
plt.show()