The Hybrid Convolution and Attention Network (HCANet) introduced a Convolution and Attention Fusion Module (CAFM) that jointly models local features via convolutions and global context via self‑attention. This design is particular beneficial for detecting small objects, where fine spatial details and long‑range dependencies both matter. In this article we adapt the CAFM for YOLOv8, presenting a lightweight standalone implementation and explaining how to embed it into the detector's backbone or neck.

Core CAFM Block

Below is a re‑engineered version of the attention‑convolution fusion unit. It splits the input into a global self‑attention pathway and a local depthwise convolution pathway, then merges their outputs. The implementation uses only standard 2D convolutions, avoiding the 3D convolutions of the original HSI‑oriented design.

import torch
import torch.nn as nn
import torch.nn.functional as F

class LayerNorm2d(nn.Module):
    def __init__(self, dim, eps=1e-6):
        super().__init__()
        self.weight = nn.Parameter(torch.ones(dim))
        self.bias = nn.Parameter(torch.zeros(dim))
        self.eps = eps

    def forward(self, x):
        mu = x.mean(1, keepdim=True)
        sigma = (x - mu).pow(2).mean(1, keepdim=True)
        x = (x - mu) / torch.sqrt(sigma + self.eps)
        return x * self.weight[:, None, None] + self.bias[:, None, None]

class ConvAttnFusion(nn.Module):
    def __init__(self, dim, num_heads=8, bias=False):
        super().__init__()
        self.num_heads = num_heads
        self.temperature = nn.Parameter(torch.ones(num_heads, 1, 1))

        # QKV projection + depthwise convolution for global pathway
        self.qkv = nn.Conv2d(dim, dim * 3, 1, bias=bias)
        self.dwconv = nn.Conv2d(dim * 3, dim * 3, 3, padding=1, groups=dim * 3, bias=bias)
        self.proj = nn.Conv2d(dim, dim, 1, bias=bias)

        # local branch: depthwise -> gelu -> pointwise
        self.local_conv = nn.Sequential(
            nn.Conv2d(dim, dim, 3, padding=1, groups=dim, bias=bias),
            nn.GELU(),
            nn.Conv2d(dim, dim, 1, bias=bias)
        )

    def forward(self, x):
        B, C, H, W = x.shape

        # ---- global self-attention ----
        qkv = self.dwconv(self.qkv(x))
        q, k, v = torch.chunk(qkv, 3, dim=1)
        q = q.reshape(B, self.num_heads, C // self.num_heads, H * W)
        k = k.reshape(B, self.num_heads, C // self.num_heads, H * W)
        v = v.reshape(B, self.num_heads, C // self.num_heads, H * W)

        q = F.normalize(q, dim=2)
        k = F.normalize(k, dim=2)
        attn = (q @ k.transpose(-2, -1)) * self.temperature
        attn = attn.softmax(dim=-1)
        global_out = (attn @ v).reshape(B, C, H, W)
        global_out = self.proj(global_out)

        # ---- local convolution ----
        local_out = self.local_conv(x)

        return global_out + local_out

Embedding into YOLOv8

1. Create a custom module file

Place the above code in a file named hca_cafm.py inside the ultralytics/nn/modules/ directory. You can optionally wrap the fusion block with a pre‑normalisasion scheme:

class CAFMBlock(nn.Module):
    def __init__(self, dim, num_heads=8, bias=False):
        super().__init__()
        self.norm1 = LayerNorm2d(dim)
        self.fusion = ConvAttnFusion(dim, num_heads, bias)
        self.norm2 = LayerNorm2d(dim)
        self.ffn = nn.Sequential(
            nn.Conv2d(dim, dim*4, 1),
            nn.GELU(),
            nn.Conv2d(dim*4, dim, 1)
        )

    def forward(self, x):
        x = x + self.fusion(self.norm1(x))
        x = x + self.ffn(self.norm2(x))
        return x

2. Register the module

In ultralytics/nn/tasks.py, import CAFMBlock (or ConvAttnFusion) and add it to the module dicsionary that maps YAML strings to classes. For example:

from ultralytics.nn.modules.hca_cafm import CAFMBlock

# inside parse_model, ensure the class is recognized:
if m in {CAFMBlock, ...}:
    ...

3. Adjust the YAML configuration

Replace a C2f layer in the backbone or neck with the CAFM block. A typical insertion might look like:

# in the backbone section
- [-1, 1, CAFMBlock, [256, 8]]   # CAFM with 256 channels and 8 heads

Make sure the input channel dimensions match the preceding layer. Because CAFMBlock does not change the tensor shape, it can be placed wherever feature refinement is desired.

4. Train the model

Launch training as usual:

python train.py --model yolov8n-cafm.yaml --data your_dataset.yaml --epochs 100

The fused local and global processing helps the detector to better capture small instances, often leading to higher mAP@0.5:0.95, especially on datasets with numerous small objects.