Multi-Object Tracking (MOT) extends object detection by assigning a persistent ID to detected entities across video frames. While detection models like YOLO identify objects in individual frames, MOT ensures continuity, recognizing that an object in frame t is the same as in frame t+1. Algorithms such as SORT and DeepSORT have historically dominated this space, but ByteTrack offers a refined aproach to handling detection confidence scores.

Tracking Methodologies

Common approaches to object tracking include:

1. Feature-based Tracking: Utilizing intrinsic properties like color histograms, shape contours, or texture patterns.
2. Template Matching: Correlating a predefined template of the target against the video stream.
3. Correlation Tracking: Measuring similarity between the target region and candidate regions in subsequent frames.
4. Deep Learning-based Tracking: Leveraging neural networks trained on massive datasets to handle detection and Re-ID (Re-Identification) simultaneously.

The ByteTrack Logic

Traditional MOT algorithms often discard detection bounding boxes with low confidence scores (e.g., below 0.5). ByteTrack argues that this creates irreversible errors. Low-confidence boxes often represent occluded objects or objects in motion blur. Filtering them out leads to fragmented trajectories and missed detections.

However, simply keeping all low-confidence boxes introduces false positives (background noise). ByteTrack solves this through a two-step data association process:

1. High-Score Matching: First, it matches high-confidence detections with existing tracklets using IoU (Intersection over Union) or motion similarity.
2. Low-Score Recovery: Next, it takes the remaining unmatched tracklets and attempts to match them with the previously discarded low-confidence detections. If a low-confidence box matches a tracklet's predicted location (via Kalman Filtering), it is recovered. Background noise (false positives) usually fails this second match and is discarded.

Data Association Mechanics

The core of MOT lies in calculating similarity and applying matching strategies.

• Similarity Metrics:

  • Motion & Location: SORT uses Kalman Filters to predict the next location of a tracklet and calculates IoU. This works well for short-term occlusions.
  • Appearence: For long-term occlusions, visual appearance (Re-ID features) is crucial. DeepSORT uses a separate network to extract these features, calculating cosine similarity.

• Matching Strategy:

  • Algorithms typically use the Hungarian Algorithm to optimize the assignment of detections to tracklets.
  • DeepSORT uses a cascaded matching strategy, prioritizing recent tracklets over lost ones. ByteTrack simplifies this by focusing on the confidence threshold split.

BYTE Algorithm Workflow

The alogrithm processes a video frame by frame:

1. Detect objects using a model (e.g., YOLO) and obtain confidence scores.
2. Split detections into high_conf (above threshold) and low_conf (below threshold).
3. Predict new locations for existing tracklets using a Kalman Filter.
4. First Association: Match high_conf detections with tracklets.
5. Second Association: Match low_conf detections with the remaining unmatched tracklets.
6. Initialize new tracklets for remaining unmatched high_conf detections.

Performance

ByteTrack generally achieves higher MOTA (Multi-Object Tracking Accuracy) scores compared to SORT and DeepSORT because it minimizes identity switches and fragmentation without the computational overhead of constant Re-ID feature extraction for every box.

Implementation with YOLOv8 and Supervision

The following script demonstrates tracking vehicles in a traffic video. It filters for specific classes (cars and trucks), applies ByteTrack, and counts vehicles crossing a defined line.

import argparse
import numpy as np
from tqdm import tqdm
from ultralytics import YOLO
import supervision as sv

def run_traffic_analysis(
    model_path: str,
    input_video: str,
    output_video: str,
    conf_level: float = 0.25,
    iou_val: float = 0.5
):
    # Initialize YOLO model
    yolo_model = YOLO(model_path)
    category_names = yolo_model.names

    # Define line coordinates for counting (start and end points)
    line_start = sv.Point(50, 550)
    line_end = sv.Point(1920, 550)

    # Initialize Tracker and Annotators
    byte_tracker = sv.ByteTrack()
    box_drawer = sv.BoundingBoxAnnotator()
    text_drawer = sv.LabelAnnotator()
    
    # Line counter setup
    counter = sv.LineZone(start=line_start, end=line_end)
    counter_display = sv.LineZoneAnnotator(thickness=2, text_thickness=2, text_scale=0.6)

    # Video processing setup
    frame_provider = sv.get_video_frames_generator(source_path=input_video)
    video_meta = sv.VideoInfo.from_video_path(video_path=input_video)

    with sv.VideoSink(target_path=output_video, video_info=video_meta) as sink:
        for frame in tqdm(frame_provider, total=video_meta.total_frames):
            # Inference
            result = yolo_model(frame, verbose=False, conf=conf_level, iou=iou_val)[0]
            detections = sv.Detections.from_ultralytics(result)

            # Filter for specific classes: Car (2) and Truck (7) in COCO dataset
            vehicle_mask = np.isin(detections.class_id, [2, 7])
            detections = detections[vehicle_mask]

            # Update tracker
            detections = byte_tracker.update_with_detections(detections)

            # Create labels: ID, Class, Confidence
            labels = [
                f"ID:{tid} {category_names[cid]} {conf:.2f}"
                for tid, cid, conf in zip(
                    detections.tracker_id, 
                    detections.class_id, 
                    detections.confidence
                )
            ]

            # Annotate frame
            annotated_img = box_drawer.annotate(scene=frame.copy(), detections=detections)
            annotated_img = text_drawer.annotate(scene=annotated_img, detections=detections, labels=labels)
            
            # Update and draw line counter
            counter.trigger(detections=detections)
            final_frame = counter_display.annotate(frame=annotated_img, line_counter=counter)
            
            sink.write_frame(frame=final_frame)

if __name__ == "__main__":
    parser = argparse.ArgumentParser(description="Traffic Analysis using YOLOv8 and ByteTrack")
    parser.add_argument("--weights", required=True, help="Path to YOLOv8 weights file", type=str)
    parser.add_argument("--input", required=True, help="Path to source video", type=str)
    parser.add_argument("--output", required=True, help="Path to save processed video", type=str)
    parser.add_argument("--conf", default=0.25, help="Detection confidence threshold", type=float)
    parser.add_argument("--iou", default=0.5, help="IOU threshold for NMS", type=float)
    
    args = parser.parse_args()
    run_traffic_analysis(
        model_path=args.weights,
        input_video=args.input,
        output_video=args.output,
        conf_level=args.conf,
        iou_val=args.iou
    )

Run the script via CLI:

python traffic_monitor.py --weights yolov8s.pt --input highway.mp4 --output output.mp4 --conf 0.1

Practical Applications

• Traffic Management: Monitoring vehicle flow, detecting wrong-way drivers, and analyzing intersection density.
• Industrial Automation: Counting and tracking items on a conveyor belt.
• Retail Analytics: Analyzing customer movement patterns, dwell time near specific products, and interaction rates.
• Security: Identifying suspicious lingering behavior in restricted areas.