Directory

1. Advantages of Darknet
2. Darknet Structure
3. Installation
4. Training
5. Detection

Advantages of Darknet

Darknet is a deep learning framework written entirely in C, offering several unique advantages over other frameworks:

1. Easy Installation: Simply select the desired options (CUDA, cuDNN, OpenCV, etc.) in the Makefile and run make. The installation completes within minutes.
2. No Dependencies: The entire framework is written in C and can operate without any external libraries. Even the OpenCV author has written replacement functions.
3. Clear Structure: The source code is easy to read and modify. Core framework files reside in the src folder, while high-level detection and classification functions are in the examples folder.
4. Python Interface: Despite being written in C, Darknet provides a Python interface for directly calling trained .weights models using Python functions.
5. Portability: Deployment to local machines is straightforward. It can utilize both CPU and GPU, making it particularly suitable for on-premises detection and recognition tasks.

Darknet Structure

Directory Layout

|
|-- \cfg
|   |-- coco.data
|   |-- darknet.cfg
|   |-- ...
|   |-- yolov3.cfg
|
|-- \data
|   |-- coco.names
|   |-- ...
|
|-- \examples
|   |-- classifier.c
|   |-- detector.c
|   |-- ...
|
|-- \include
|   |-- darknet.h
|
|-- \python
|   |-- darknet.py
|   |-- ...
|
|-- \scripts
|   |-- get_coco_dataset.sh
|   |-- ...
|
|-- \src
|   |-- convolutional_layer.c
|   |-- convolutional_layer.h
|   |-- convolutional_kernels.cu
|   |-- ...
|
|-- darknet53.conv.74
|-- LICENSE
|-- Makefile
|-- ReadMe.md

Folder Descriptions

1. cfg: Contains model architecture configuration files that define the entire network structure.
2. data: Stores label files and sample images.
3. src: Contains low-level framework definitions, including all layer implementations.
4. examples: Contains higher-level functions such as detection and classification, which directly call the low-level functions.
5. include: Stores header files.
6. python: Contains the Python interface for calling models (mainly in darknet.py).
7. scripts: Contains scripts for tasks like downloading the COCO dataset or converting VOC format data to the training required format.
8. Other files: Include the LICENSE and Makefile.

Installation

1. Open the Makefile and set the desired options to 1. For GPU and cuDNN support:

   • GPU=1
   • CUDNN=1

2. Open a terminal, navigate to the Darknet root directory, and run:

   make
   

3. After compilation, the root directory will contain additional files and folders:

   • obj/: Object files from compilation.
   • darknet: Executable binary.
   • libdarknet.a: Static library.
   • libdarknet.so: Dynamic library.

   The executable darknet can be used locally. For porting to other platforms, use the dynamic library libdarknet.so. Note that the dynamic library only includes the basic functions from src, not the high-level functions from examples, so detection functions must be defined manually.

Training

1. Data Preparation

Convert ground truth data to Darknet's required format. For VOC format XML files, use the following Python script:

import xml.etree.ElementTree as ET
import pickle
import os
from os import listdir, getcwd
from os.path import join

# Define all classes to be detected
classes = ["car"]

def convert(size, box):
    dw = 1. / size[0]
    dh = 1. / size[1]
    x = (box[0] + box[1]) / 2.0
    y = (box[2] + box[3]) / 2.0
    w = box[1] - box[0]
    h = box[3] - box[2]
    x = x * dw
    w = w * dw
    y = y * dh
    h = h * dh
    return (x, y, w, h)

def convert_annotation(image_id):
    in_file = open(xml_path)          # Path to the XML file corresponding to the image
    out_file = open(txt_save_path, 'w')  # Full path for the output TXT file
    tree = ET.parse(in_file)
    root = tree.getroot()
    size = root.find('size')
    w = int(size.find('width').text)
    h = int(size.find('height').text)

    for obj in root.iter('object'):
        cls = obj.find('name').text
        if cls not in classes:
            continue
        cls_id = classes.index(cls)
        xmlbox = obj.find('bndbox')
        b = (float(xmlbox.find('xmin').text), float(xmlbox.find('xmax').text),
             float(xmlbox.find('ymin').text), float(xmlbox.find('ymax').text))
        bb = convert((w, h), b)
        out_file.write(str(cls_id) + " " + " ".join([str(a) for a in bb]) + '\n')
    out_file.close()

Set xml_path and txt_save_path appropriately. The script converts bounding box coordinates (xmin, xmax, ymin, ymax) to normalized center coordinates and dimensions.

2. Prepare the .data File

The .data file must include the following fields:

• classes: Number of detection classes.
• train: Path to a list of training images (e.g., train=data/trainlist.txt).
• names: Path to the class names file (e.g., names=data/plate/car.names).
• backup: Directory for saving model checkpoints and final weights.

Example:

classes= 1
train = /home/user/yolov3/data/car/train.list
names = data/plate/car.names
backup = /home/user/yolov3/data/car/models

To generate train.list, use:

find /path/to/images -name \*.jpg > trainlist.txt

3. Prepare the .cfg File

If using YOLOv3, modify cfg/yolov3.cfg:

• Batch and subdivisions: Comment out the testing section and enable the training section:

  # Testing
  #batch=1
  #subdivisions=1
  # Training
  batch=64
  subdivisions=16
  

  batch is the total batch size. subdivisions allows processing smaller mini-batches (e.g., 64/16=4 images per iteration) while accumulating gradients for the full batch.

• Classes: Change classes in each [yolo] layer (there are three) to your number of classes (e.g., classes=1).

• Filters: Modify the filters value in the convolutional layer immediately before each [yolo] layer. The formula is (5 + classes) * 3. For 1 class, it becomes 18.

  Example modification:

  [convolutional]
  size=1
  stride=1
  pad=1
  filters=255        # changed to 18
  activation=linear
  
  [yolo]
  mask = 0,1,2
  anchors = 10,13, 16,30, 33,23, 30,61, 62,45, 59,119, 116,90, 156,198, 373,326
  classes=80          # changed to 1
  num=9
  jitter=.3
  ignore_thresh = .5
  truth_thresh = 1
  random=1
  

• random: Enables multi-scale training as mentioned in the paper. Set to 1 if memory permits, otherwise set to 0 to disable input size variation.

4. Prepare Pre-trained Weights (Optional)

Download pre-trained weights from the official website (e.g., darknet53.conv.74) to initialize the model and accelerate convergence. This step is optional.

5. Start Training

• With pre-trained weights:

  ./darknet detector train data/detect.data data/yolov3.cfg data/yolov3.weights
  

• Without pre-trained weights:

  ./darknet detector train data/detect.data data/yolov3.cfg
  

Note: For classification tasks, use:

./darknet classifier train cfg/cifar.data cfg/cifar_small.cfg [weights]

Detection

Run detection using:

./darknet detector test data/detect.data data/yolov3.cfg data/yolov3.weights

The executable darknet first calls the main function in examples/darknet.c. The parameter detector calls run_detector:

else if (0 == strcmp(argv[1], "detector")){ run_detector(argc, argv); }

run_detector (in examples/detector.c) then dispatches to the appropriate function based on the command:

• test: test_detector
• train: train_detector
• valid: validate_detector
• recall: validate_detector_recall
• demo: Real-time camera detection

The .data file provides class information, training list, class names, and backup path. The .cfg file defines the network architecture, and the .weights file contains the trained model parameters.