Classification Tree with the Iris Dataset

Load and prepare the Fisher iris data, then split into training and test partitions after standardizing the feature matrix.

load fisheriris;
features = meas;           % 4 predictors
speciesLabel = species;     % response

% Standardize
[featStd, muVals, sigVals] = zscore(features);

% Hold-out partition
holdPart = cvpartition(speciesLabel, 'HoldOut', 0.3);
trainFeat = featStd(training(holdPart) ,:);
trainResp = speciesLabel(training(holdPart));
testFeat  = featStd(test(holdPart) ,:);
testResp  = speciesLabel(test(holdPart));

Train a CART classification tree, limiting tree depth and leaf size.

classTree = fitctree(trainFeat, trainResp, ...
    'PredictorNames', {'SepalLength','SepalWidth','PetalLength','PetalWidth'}, ...
    'MaxNumSplits', 10, ...
    'MinLeafSize', 5);

Visualize the tree structure and inspect it interactively.

figure;
view(classTree, 'Mode', 'graph');
title('Classification Tree Structure');

Evaluate on the hold-out set and show a confusion matrix.

predictedClasses = predict(classTree, testFeat);
acc = sum(strcmp(predictedClasses, testResp)) / numel(testResp);
fprintf('Hold-out accuracy: %.2f%%\n', acc * 100);

figure;
cmPlot = confusionchart(testResp, predictedClasses);
cmPlot.Title = 'Confusion Matrix';
cmPlot.XLabel = 'Predicted';
cmPlot.YLabel = 'Actual';

Manual Regression Tree

Synthesize a nonlinear dataset with two features and an interaction term. The target is a mix of polynomial and sinusoidal components plus Gaussian noise.

rng(42);
n = 500;
f1 = 2 + 8 * rand(n,1);        % [2, 10]
f2 = 1 + 6 * rand(n,1);        % [1, 7]
output = 20 + 5*f1 + 3*f2 + 2*f1.^2 + 1.5*sin(2*f2) + 3*randn(n,1);

figure;
scatter3(f1, f2, output, 40, output, 'filled');
xlabel('Feature A'); ylabel('Feature B'); zlabel('Response');
title('Synthetic Data');

A recursive tree builder splits nodes by minimizing MSE.

function nd = growTree(X, y, curDepth, maxDepth, minParent, minLeaf)
    nd.isLeaf = false;
    nd.predVal = mean(y);
    nd.nSamp   = length(y);

    if curDepth >= maxDepth || length(y) < minParent || var(y) < 1e-6
        nd.isLeaf = true;
        return;
    end

    [bestCol, bestCut] = pickSplit(X, y);

    leftMask  = X(:, bestCol) <= bestCut;
    rightMask = ~leftMask;

    nd.splitCol  = bestCol;
    nd.splitVal  = bestCut;
    nd.left      = growTree(X(leftMask ,:), y(leftMask),  curDepth+1, maxDepth, minParent, minLeaf);
    nd.right     = growTree(X(rightMask,:), y(rightMask), curDepth+1, maxDepth, minParent, minLeaf);
end

function [bestCol, bestVal] = pickSplit(X, y)
    nFeat = size(X, 2);
    bestErr = inf;

    for c = 1:nFeat
        candidates = unique(X(:,c));
        for t = candidates'
            leftIdx  = X(:,c) <= t;
            rightIdx = ~leftIdx;
            if sum(leftIdx) == 0 || sum(rightIdx) == 0
                continue;
            end
            mse = (sum((y(leftIdx) - mean(y(leftIdx))).^2) + ...
                   sum((y(rightIdx) - mean(y(rightIdx))).^2)) / length(y);
            if mse < bestErr
                bestErr = mse;
                bestCol = c;
                bestVal = t;
            end
        end
    end
end

Train the tree and evaluate RMSE on a hold-out partition.

allFeat = [f1, f2];
holdOut = cvpartition(size(allFeat,1), 'HoldOut', 0.3);
Xtr = allFeat(training(holdOut), :);
yTr = output(training(holdOut));
Xte = allFeat(test(holdOut), :);
yTe = output(test(holdOut));

regTree = growTree(Xtr, yTr, 0, 5, 10, 5);

% Simple recursive prediction function
predVals = nan(size(Xte,1),1);
for i = 1:size(Xte,1)
    predVals(i) = treePredict(regTree, Xte(i,:));
end

rmse = sqrt(mean((yTe - predVals).^2));
fprintf('Test RMSE: %.3f\n', rmse);

function val = treePredict(nd, row)
    if nd.isLeaf
        val = nd.predVal;
    else
        if row(nd.splitCol) <= nd.splitVal
            val = treePredict(nd.left, row);
        else
            val = treePredict(nd.right, row);
        end
    end
end

Visualize predicted vs. actual responses and a decision surface.

figure;
hold on;
plot(yTe, 'b', 'LineWidth', 1.5);
plot(predVals, 'r--', 'LineWidth', 1.5);
legend('Actual', 'Predicted');
title('Regression Results');
xlabel('Sample index'); ylabel('Response');

% Surface plot for two features
[gA, gB] = meshgrid(linspace(2,10,50), linspace(1,7,50));
Zsurf = zeros(size(gA));
for r = 1:size(gA,1)
    for c = 1:size(gA,2)
        Zsurf(r,c) = treePredict(regTree, [gA(r,c), gB(r,c)]);
    end
end
figure;
surf(gA, gB, Zsurf, 'EdgeColor', 'none');
hold on;
scatter3(f1, f2, output, 40, output, 'filled');
title('Decision Surface and Samples');
xlabel('Feature A'); ylabel('Feature B'); zlabel('Prediction / Response');

Common adjustments for different scenarios include replacing the dataset for medical diagnosis or industrial fault detection, modifying the tree builder to accept sequential features, and introducing domain-specific feature engineering such as debt-to-income ratios for credit risk modeling.

Troubleshoooting quick reference