Setting Up the Cross-Compilation Environment

1. Clone the ax-pipeline repository along with its submodules:

git clone --recursive https://github.com/AXERA-TECH/ax-pipeline.git

2. Download the SDK:

cd ax-pipeline
./download_ax_bsp.sh ax650
cd ax650n_bsp_sdk
wget https://github.com/ZHEQIUSHUI/assets/releases/download/ax650/drm.zip
mkdir third-party
unzip drm.zip -d third-party
cd ..

3. Download OpenCV:

mkdir 3rdparty
cd 3rdparty
wget https://github.com/ZHEQIUSHUI/assets/releases/download/ax650/libopencv-4.5.5-aarch64.zip
unzip libopencv-4.5.5-aarch64.zip
cd ..

4. Set up the ARM cross-compiler:

wget https://developer.arm.com/-/media/Files/downloads/gnu-a/9.2-2019.12/binrel/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu.tar.xz
tar -xvf gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu.tar.xz
export PATH=$PATH:$PWD/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu/bin/

5. Build the project:

cd ax-pipeline
mkdir build && cd build
cmake -DAXERA_TARGET_CHIP=AX650 \
      -DBSP_MSP_DIR=$PWD/../ax650n_bsp_sdk/msp/out \
      -DOpenCV_DIR=$PWD/../3rdparty/libopencv-4.5.5-aarch64/lib/cmake/opencv4 \
      -DSIPY_BUILD=OFF \
      -DCMAKE_BUILD_TYPE=Release \
      -DCMAKE_TOOLCHAIN_FILE=../toolchains/aarch64-none-linux-gnu.toolchain.cmake \
      -DCMAKE_INSTALL_PREFIX=install ..
make -j12
make install

Deploying to the Development Board

After building, the output is located under install/bin/:

bin
├── config
│   ├── custom_model.json
│   ├── dinov2.json
│   ├── dinov2_depth.json
│   ├── glpdepth.json
│   ├── ppyoloe.json
│   ├── scrfd.json
│   ├── scrfd_recognition.json
│   ├── yolo_nas.json
│   ├── yolov5_seg.json
│   ├── yolov5s.json
│   ├── yolov5s_face.json
│   ├── yolov5s_face_recognition.json
│   ├── yolov6.json
│   ├── yolov7.json
│   ├── yolov7_face.json
│   ├── yolov8.json
│   ├── yolov8_pose.json
│   └── yolox.json
├── sample_demux_ivps_npu_hdmi_vo
├── sample_demux_ivps_npu_rtsp
├── sample_demux_ivps_npu_rtsp_hdmi_vo
├── sample_multi_demux_ivps_npu_hdmi_vo
├── sample_multi_demux_ivps_npu_multi_rtsp
├── sample_multi_demux_ivps_npu_multi_rtsp_hdmi_vo
├── sample_vin_ivps_npu_hdmi_vo
└── sample_vin_ivps_npu_venc_rtsp

Configuring the Model

Modify the yolov5s.json configuration file to match your model. Below is an example with custom classes:

{
    "MODEL_TYPE": "MT_DET_YOLOV5",
    "MODEL_PATH": "/root/Desktop/install/bin/config/models/yolov5s_hat.axmodel",
    "TRACK_ENABLE": true,
    "STRIDES": [8, 16, 32],
    "ANCHORS": [
        10.0, 13.0, 16.0, 30.0, 33.0, 23.0,
        30.0, 61.0, 62.0, 45.0, 59.0, 119.0,
        116.0, 90.0, 156.0, 198.0, 373.0, 326.0
    ],
    "CLASS_NAMES": ["hat", "person"],
    "CLASS_NUM": 2,
    "NMS_THRESHOLD": 0.45,
    "PROB_THRESHOLD": 0.4
}

Udpate the label names, CLASS_NUM, and MODEL_PATH accordingly. Anchors may remain unchanged if using models converted from the official PyTorch YOLOv5.

Running Inference on HDMI

To display inference results on HDMI, first stop the fb_vo process:

ps aux | grep fb_vo
kill -9 <pid>

Connect the HDMI cable to port HDMI0 (the one farther from the Ethernet port). To revert to the origianl display configuration, execute /root/runVoHook.sh.

The following command reads a MP4 file, runs NPU inference, and outputs the result to HDMI:

./sample_demux_ivps_npu_hdmi_vo -p config/yolov5s.json -f test.mp4

Once running, the annotated video will appear on the HDMI display.