Implementing YOLOv9 Object Detection with TensorRT in C++

Deploying YOLOv9 with TensorRT for object detection requires a systematic approach covering environment preparation, model transformation, code implementation, and inference execution.

Begin by verifying that your development environment includes NVIDIA TensorRT. This SDK specializes in high-performance inference, optimizing models from frameworks such as TensorFlow and PyTorch to maximize execution speed on NVIDIA GPUs.

The subsequent step involves converting the YOLOv9 model into TensorRT format. This utilizes TensorRT APIs to interpret the original YOLOv9 model and convert it into a compatible format. During this conversion, TensorRT implements numerous optimizations including layer fusion and precision calibration to boost inference performance.

Once the model is converted, you can develop C++ code to load and execute the TensorRT model. Employ TensorRT's C++ interfaces to load the model and establish an execution context. Subsequently, provide input data—whether from real-time camera feeds or static images—to the model for processing.

During inference, TensorRT performs computational operations that yield object detection results. These outputs typically contain object categories, confidence levels, and bounding box coordinates. These can be visualized on the original image or processed further, such as transmitting to another system for additional analysis.

Implementation Procedure

1. Obtain the YOLOv9 repository and the yolov9-c.pt model file.
2. Convert to ONNX format:
   • Initially, modify utils/general.py following the YOLOv9 official documentation.
   • Export the model using:

     python export.py --weights yolov9-c.pt --include onnx

3. Transform to TensorRT engine:

   trtexec.exe --onnx=yolov9-c.onnx --explicitBatch --saveEngine=yolov9-c.engine --fp16

4. Configure CMakeLists.txt with appropriate paths for OpenCV and TensorRT:

   # Locate and include OpenCV
   set(OpenCV_DIR "your path to OpenCV installation")
   find_package(OpenCV REQUIRED)
   include_directories(${OpenCV_INCLUDE_DIRS})
   
   # Specify TensorRT path if environment variables are not configured
   set(TENSORRT_DIR "your path to TensorRT installation")

5. Build the project:

   mkdir build
   cd build
   cmake ..
   cmake --build . --config Release

6. Usage instructions:
   • Process an image:

     yolov9-tensorrt.exe yolov9-c.engine test.jpg

   • Process an entire directory:

     yolov9-tensorrt.exe yolov9-c.engine data_folder

   • Analyze a video file:

     yolov9-tensorrt.exe yolov9-c.engine video_input.mp4

Development Environment

• TensorRT 8.4.2.4
• CUDA 11.7 with cuDNN 8.8.0
• Windows 10 operating system