Object detection models frequantly generate multiple bounding boxes for a single object. To isolate the most accurate localization, a post-processing step is required to filter redundant proposals. Non-Maximum Suppression (NMS) is the standard technique employed to select the optimal box while suppressing overlapping candidates.

Core Algorithm Mechanics

The suppression process operates on a set of candidate boxes paired with confidence scores. The procedure iterates through the following logic:

1. Identify the bounding box with the highest confidence score.
2. Move this box to the final selection list.
3. Calculate the Intersection over Union (IoU) between this selected box and all remaining candidates.
4. Discard any remaining boxes that exceed a predefined IoU threshold.
5. Repeat until no candidates remain.

Standard NMS Implementation

The following implementation utilizes NumPy for efficient vectorized calculations. Variable names and flow have been structured for clarity.

import numpy as np

def standard_nms(predictions, overlap_limit=0.5):
    """
    predictions: Array of shape (N, 5) where columns are [x1, y1, x2, y2, score]
    overlap_limit: IoU threshold for suppression
    """
    if len(predictions) == 0:
        return []
    
    # Extract coordinates and confidence values
    left = predictions[:, 0]
    top = predictions[:, 1]
    right = predictions[:, 2]
    bottom = predictions[:, 3]
    confidence = predictions[:, 4]
    
    # Calculate area for each box
    widths = right - left + 1
    heights = bottom - top + 1
    areas = widths * heights
    
    # Sort indices by confidence in descending order
    sorted_indices = np.argsort(confidence)[::-1]
    
    selected_indices = []
    
    while sorted_indices.size > 0:
        # Pick the box with the highest score
        current_idx = sorted_indices[0]
        selected_indices.append(current_idx)
        
        # Calculate IoU with remaining boxes
        remaining_indices = sorted_indices[1:]
        
        if len(remaining_indices) == 0:
            break
            
        # Find intersection coordinates
        inter_left = np.maximum(left[current_idx], left[remaining_indices])
        inter_top = np.maximum(top[current_idx], top[remaining_indices])
        inter_right = np.minimum(right[current_idx], right[remaining_indices])
        inter_bottom = np.minimum(bottom[current_idx], bottom[remaining_indices])
        
        # Compute intersection area
        inter_w = np.maximum(0.0, inter_right - inter_left + 1)
        inter_h = np.maximum(0.0, inter_bottom - inter_top + 1)
        intersection = inter_w * inter_h
        
        # Compute IoU
        union = areas[current_idx] + areas[remaining_indices] - intersection
        iou_values = intersection / union
        
        # Keep boxes that do not exceed the overlap limit
        valid_indices = np.where(iou_values <= overlap_limit)[0]
        sorted_indices = sorted_indices[valid_indices + 1]
        
    return selected_indices

Limitations and Soft NMS

Standard NMS applies a hard threshold. If two valid objects are close together, the one with the slightly lower score might be incorrectly suppresed if the overlap exceeds the threshold. Soft NMS addresses this by decaying the confidence scores of overlapping boxes rather than removing them immediately.

The score decay can be linear or based on a Gaussian function. Boxes with high overlap see their scores reduced significantly, while those with moderate overlap retain enough score to survive subsequent thresholding.

def soft_nms_strategy(boxes, scores, sigma=0.5, iou_cutoff=0.3, score_thresh=0.001):
    """
    boxes: (N, 4) coordinates
    scores: (N,) confidence values
    """
    N = len(boxes)
    if N == 0:
        return []
        
    kept_indices = []
    
    # Work on copies to avoid modifying original data directly during iteration
    current_boxes = boxes.copy()
    current_scores = scores.copy()
    
    for i in range(N):
        # Find max score box from remaining pool
        max_idx = np.argmax(current_scores[i:]) + i
        if current_scores[max_idx] < score_thresh:
            break
            
        # Swap current position with max score position
        current_boxes[[i, max_idx]] = current_boxes[[max_idx, i]]
        current_scores[[i, max_idx]] = current_scores[[max_idx, i]]
        
        kept_indices.append(i)
        
        # Calculate IoU with remaining boxes
        for j in range(i + 1, N):
            if current_scores[j] < score_thresh:
                continue
                
            # Compute IoU logic (simplified for brevity)
            x1 = max(current_boxes[i][0], current_boxes[j][0])
            y1 = max(current_boxes[i][1], current_boxes[j][1])
            x2 = min(current_boxes[i][2], current_boxes[j][2])
            y2 = min(current_boxes[i][3], current_boxes[j][3])
            
            inter = max(0, x2 - x1) * max(0, y2 - y1)
            area_i = (current_boxes[i][2] - current_boxes[i][0]) * (current_boxes[i][3] - current_boxes[i][1])
            area_j = (current_boxes[j][2] - current_boxes[j][0]) * (current_boxes[j][3] - current_boxes[j][1])
            union = area_i + area_j - inter
            
            if union > 0:
                iou = inter / union
                weight = 1.0
                
                if iou > iou_cutoff:
                    if sigma > 0:
                        # Gaussian decay
                        weight = np.exp(-(iou * iou) / sigma)
                    else:
                        # Linear decay
                        weight = 1 - iou
                        
                current_scores[j] *= weight
                
    # Filter based on final scores
    final_selection = [idx for idx, score in enumerate(current_scores) if score >= score_thresh]
    return final_selection

Advanced IoU Metrics

Beyond standard IoU, several variations improve suppression accuracy, particularly when boxes do not overlap significantly or when aspect ratios differ.

Generalized IoU (GIoU) GIoU introduces a penalty term based on the smallest enclosing box covering both the prediction and the ground truth. This helps when boxes are disjoint, as standard IoU becomes zero. However, GIoU may require more iterations to converge during training.

Distance IoU (DIoU) DIoU adds a penalty term minimizing the distance between the center points of the two boxes. This accounts for spatial separation regardless of overlap area. Replacing standard IoU with DIoU in detection heads has shown measurable improvements in mean Average Precision (mAP).

Complete IoU (CIoU) CIoU extends DIoU by including an aspect ratio consistency term. It considers overlap area, center distance, and shape similarity. The loss function combines these factors to ensure predicted boxes align closely with ground truth in all geometric dimensions.

L_{ciou} = 1 - IoU + \frac{\rho^2(b, b^{gt})}{c^2} + \alpha v

Where $v$ measures aspect ratio similarity. These metrics refine the suppression logic by providing a more nuanced understanding of box similarity than simple area overlap.