Feature selection is a critical stage in machine learning workflows that helps in reducing model complexity and improving predictive accuracy. Ranking features by their relative importance allows practitioners to implement selection strategies such as Top-N (selecting the highest-ranked N features) or Top-percent (selecting features that fall with in a specific percentile of importance).

In R, the caret package provides a robust interface for calculating varible importance across various model types. The following example demonstrates how to evaluate and visualize feature importance using a Learning Vector Quantization (LVQ) model and k-fold cross-validation.

# Load necessary libraries
library(mlbench)
library(caret)

# Load the Sonar dataset (Classification of sonar signals)
data(Sonar)

# Configure the training process with 10-fold cross-validation
validation_scheme <- trainControl(method = "cv", number = 10)

# Train a Learning Vector Quantization (LVQ) model
# Pre-processing includes scaling to ensure features are on the same magnitude
lvq_fit <- train(Class ~ ., 
                 data = Sonar, 
                 method = "lvq", 
                 preProcess = "scale", 
                 trControl = validation_scheme)

# Compute variable importance scores
# Setting scale = FALSE preserves the raw importance values
importance_metrics <- varImp(lvq_fit, scale = FALSE)

# Display the summary of feature rankings
print(importance_metrics)

# Generate a visual representation of variable importance
plot(importance_metrics)

The varImp function evaluates how much each predictor contributes to the model's decision-making process. For LVQ models, importance is typically calculated based on the class-specific distances. By analyzing the resulting plot, you can identify which sensors in the Sonar dataset provide the most discriminative power between "M" (metal) and "R" (rock) classes, facilitating more efficient feature pruning.