Machine Learning Pipeline Overview

Machine learning involves the extraction of patterns from data to build predictive models. A standard workflow consists of several critical stages: data collection, data preparation, model selection, training, evaluation, and prediction.

During data preparation, it is crucial to shuffle the dataset to prevent the model from learning the order of samples. Visualization should be used to check for class imbalances. If the dataset is biased toward one class, the model may predict that class more frequently. The data must be split into a training set and a testing set—typically using a ratio such as 4:1 or 7:3—to ensure the model is evaluated on unseen data. This separation prevents the model from simply memorizing the training samples (overfitting) and provides a realistic measure of its perofrmance in production environments.

Data Types and Sources

Data generally comes from enterprise logs, government databases, or scientific experiments. It is primarily categorized into two types:

• Discrete Data: Integer values that represent counts (e.g., number of users, number of clicks). These are typically used in classification problems.
• Continuous Data: Values that can take any number within a range (e.g., height, weight, temperature). These are usually associated with regression problems.

Feature engineering is the process of transforming this raw data into a format that better represents the underlying problem, thereby improving model accuracy.

Scikit-Learn Installation

The primary library used for these operations is Scikit-Learn. It can be installed via pip:

pip install scikit-learn

Feature Extraction

Dictionary Feature Extraction

The DictVectorizer class converts lists of feature dictionaries into a NumPy array or sparse matrix. This is useful for processing categorical data where features are key-value pairs.

from sklearn.feature_extraction import DictVectorizer

def process_dict_data():
    # Sample data: Location and Temperature
    raw_data = [
        {'location': 'Beijing', 'temp': 30},
        {'location': 'Shanghai', 'temp': 25},
        {'location': 'Guangzhou', 'temp': 35}
    ]
    
    # Initialize vectorizer
    vectorizer = DictVectorizer(sparse=False)
    
    # Transform data
    features = vectorizer.fit_transform(raw_data)
    
    print("Feature Names:", vectorizer.get_feature_names_out())
    print("Transformed Data:\n", features)
    
    # Inverse transformation to verify
    print("Original Data:", vectorizer.inverse_transform(features))

if __name__ == "__main__":
    process_dict_data()

Text Feature Extraction

CountVectorizer tokenizes text and builds a vocabulary of known words, converting the text into a matrix of token counts.

from sklearn.feature_extraction.text import CountVectorizer

def process_text_counts():
    corpus = [
        "Data science involves statistics and coding",
        "Machine learning is a subset of data science"
    ]
    
    # Initialize vectorizer
    count_vec = CountVectorizer()
    
    # Fit and transform
    X = count_vec.fit_transform(corpus)
    
    print("Vocabulary:", count_vec.get_feature_names_out())
    print("Token Counts:\n", X.toarray())

if __name__ == "__main__":
    process_text_counts()

TF-IDF Text Processing

Term Frequency-Inverse Document Frequency (TF-IDF) evaluates how relevant a word is to a document in a collection. It increases the weight of words that are unique to a specific document while reducing the weight of common words.

from sklearn.feature_extraction.text import TfidfVectorizer

def process_tfidf():
    docs = [
        "The algorithm processes the data",
        "Data processing is important for the algorithm"
    ]
    
    # Initialize with stop words
    tfidf_vec = TfidfVectorizer(stop_words=['the', 'is'])
    
    # Transform
    tfidf_matrix = tfidf_vec.fit_transform(docs)
    
    print("Features:", tfidf_vec.get_feature_names_out())
    print("TF-IDF Matrix:\n", tfidf_matrix.toarray())

if __name__ == "__main__":
    process_tfidf()

Chinese Text Segmentation

Processing Chinese text requires segmentation because words are not separated by spaces. The jieba library is commonly used for this purpose.

import jieba
from sklearn.feature_extraction.text import TfidfVectorizer

def segment_chinese_text():
    text_a = "自然语言处理是人工智能的重要领域"
    text_b = "深度学习在图像识别中表现出色"
    
    # Segment words
    words_a = " ".join(jieba.cut(text_a))
    words_b = " ".join(jieba.cut(text_b))
    
    corpus = [words_a, words_b]
    
    # Apply TF-IDF
    vectorizer = TfidfVectorizer()
    result = vectorizer.fit_transform(corpus)
    
    print("Keywords:", vectorizer.get_feature_names_out())
    print("Result:\n", result.toarray())

if __name__ == "__main__":
    segment_chinese_text()

Feature Preprocessing

Normalization (Min-Max Scaling)

Normalization scales features to a fixed range, usually [0, 1]. This is useful when features have different units or scales. The formula is:

X' = (X - X_min) / (X_max - X_min)

However, this method is sensitive to outliers.

from sklearn.preprocessing import MinMaxScaler

def normalize_data():
    # Sample dataset with different scales
    data = [[100, 0.001],
            [200, 0.005],
            [300, 0.002]]
    
    # Scale to range [0, 1]
    scaler = MinMaxScaler(feature_range=(0, 1))
    normalized = scaler.fit_transform(data)
    
    print("Normalized Data:\n", normalized)

if __name__ == "__main__":
    normalize_data()

Standardization (Z-Score)

Standardization transforms data to have a mean of 0 and a standard deviation of 1. It handles outliers better than normalization. The formula is:

X' = (X - μ) / σ

from sklearn.preprocessing import StandardScaler

def standardize_data():
    data = [[1.0, -10.0, 3.0],
            [2.0, 0.0, 0.0],
            [0.0, 10.0, 1.0]]
    
    scaler = StandardScaler()
    std_data = scaler.fit_transform(data)
    
    print("Means:", scaler.mean_)
    print("Standardized Data:\n", std_data)

if __name__ == "__main__":
    standardize_data()

Missing Value Imputation

Real-world data often contains missing values (NaN). SimpleImputer replaces these missing values using a strategy such as mean, median, or most frequent value.

import numpy as np
from sklearn.impute import SimpleImputer

def handle_missing_values():
    # Matrix with NaN values
    X = [[1, 2], 
         [np.nan, 3], 
         [7, np.nan]]
    
    imputer = SimpleImputer(strategy='mean')
    cleaned_data = imputer.fit_transform(X)
    
    print("Imputed Data:\n", cleaned_data)

if __name__ == "__main__":
    handle_missing_values()

Feature Selection and Dimensionality Reduction

Variance Threshold

Removing features with low variance (i.e., features that are mostly constant) helps reduce noise and computational cost.

from sklearn.feature_selection import VarianceThreshold

def filter_low_variance():
    # The first column has zero variance
    X = [[0, 1, 0.5],
         [0, 2, 1.5],
         [0, 3, 2.5]]
    
    selector = VarianceThreshold(threshold=0.0)
    X_reduced = selector.fit_transform(X)
    
    print("Reduced Features:\n", X_reduced)

if __name__ == "__main__":
    filter_low_variance()

Principal Component Analysis (PCA)

PCA reduces the dimensionality of data by projecting it onto orthogonal axes (principal components) that maximize variance. This technique is useful for visualizing high-dimensional data or speeding up algorithms.

from sklearn.decomposition import PCA

def apply_pca():
    # 4-dimensional data
    X = [[1, 2, 3, 4],
         [5, 6, 7, 8],
         [9, 10, 11, 12],
         [13, 14, 15, 16]]
    
    # Reduce to 2 dimensions
    pca = PCA(n_components=2)
    X_pca = pca.fit_transform(X)
    
    print("PCA Transformed Data:\n", X_pca)

if __name__ == "__main__":
    apply_pca()