Processing Volumetric Input Tensors

Traditional convolutional operations are frequently introduced using single-plane, two-dimensional arrays. Real-world sensor data, however, typically arrives as volumetric tensors containing multiple parallel planes, such as color imagery with red, green, and blue components. To process such structures, deep learning frameworks treat the depth axis as an explicit dimension, representing inputs as tensors with the shape C_in × H × W. This structural shift necessitates a modification to how sliding-window filters interact with the underlying data.

Depth-Aligned Filtering Operations

When an input tensor contains multiple planes, the convolution filter must possess an identical depth to ensure complete spatial and channel-wise alignment. For an input with c_i planes, the filter must also span c_i planes. Given a spatial footprint of k_h × k_w, the complete filter becomes a three-dimensional weight block of shape c_i × k_h × k_w. The mathematical procedure involves computing a discrete cross-correlation independently for each matching pair of input and filter slices. The resulting two-dimensional maps are subsequently aggregated through element-wise addition, producing a single output feature map that synthesizes information across all input modalities at every spatial coordinate.

The following implementation demonstrates this depth-wise aggregation strategy using tensor slicing and explicit accumulation:

import torch

def aggregate_channel_correlations(input_volume, filter_weights):
    out_height = input_volume.shape[1] - filter_weights.shape[1] + 1
    out_width = input_volume.shape[2] - filter_weights.shape[2] + 1
    result_plane = torch.zeros((out_height, out_width))
    
    for depth_index in range(input_volume.size(0)):
        plane_input = input_volume[depth_index]
        plane_kernel = filter_weights[depth_index]
        k_h, k_w = plane_kernel.shape
        
        for i in range(out_height):
            for j in range(out_width):
                receptive_field = plane_input[i:i+k_h, j:j+k_w]
                result_plane[i, j] += torch.sum(receptive_field * plane_kernel)
                
    return result_plane

Validation with aligned sample tensors confirms the aggregation logic:

sample_data = torch.tensor([[[0., 1., 2.], [3., 4., 5.], [6., 7., 8.]],
                            [[1., 2., 3.], [4., 5., 6.], [7., 8., 9.]]])
sample_filter = torch.tensor([[[0., 1.], [2., 3.]], [[1., 2.], [3., 4.]]])

aggregate_channel_correlations(sample_data, sample_filter)

tensor([[ 56.,  72.],
        [104., 120.]])

Multi-Map Feature Extraction

While single-output filters are mathematically straightforward, modern architectures rely heavily on extracting diverse representations simultaneously. To produce c_o distinct feature maps from an input with c_i channels, the layer requires a collection of independent filters. Each output plane is generated by applying a unique c_i × k_h × k_w kernel across the input volume. Consequent, the complete weight tensor expands to four dimensions, structured as c_o × c_i × k_h × k_w. During the forward pass, each independent kernel performs the channel-summed cross-corrrelation described previously, and the individual results are stacked along a new depth axis.

This multi-output expansion can be programmed by iterating over the filter bank and applying the aggregation function per kernel:

def compute_multiple_feature_maps(source_tensor, weight_bank):
    feature_maps = []
    for independent_kernel in weight_bank:
        feature_maps.append(aggregate_channel_correlations(source_tensor, independent_kernel))
    return torch.stack(feature_maps)

Extending the previous weight block to produce three parallel outputs requires stacking shifted copies along the output axis:

extended_bank = torch.stack([sample_filter, sample_filter + 1.0, sample_filter + 2.0])
print(extended_bank.shape)

torch.Size([3, 2, 2, 2])

Executing the operation yields a volumetric output where each depth slice corresponds to a distinct linear projection:

compute_multiple_feature_maps(sample_data, extended_bank)

tensor([[[ 56.,  72.],
         [104., 120.]],
        [[ 76., 100.],
         [148., 172.]],
        [[ 96., 128.],
         [192., 224.]]])

Pointwise Channel Transformations

Convolutional filters with a spatial extent of 1 × 1 appear counterintuitive at first, as they do not aggregate neighboring spatial information. Their primary utility lies in manipulating the depth dimension without altering height or width resolution. A 1 × 1 operation treats every spatial location (x, y) independently, performing a linear projection across the channel axis. Mathematically, this is equivalent to applying a fully connected network at every pixel coordinate, transforming c_i input values into c_o output values using a shared weight matrix. This mechanism enables efficient channel reduction, expansion, or non-linear feature mixing while preserving spatial topology.

Because the kernel dimensions collapse to 1 × 1, the operation simplifies to a batched matrix multiplication after appropriate tensor reshaping:

def execute_pointwise_projection(input_volume, unit_kernels):
    channels_in, spatial_h, spatial_w = input_volume.shape
    channels_out = unit_kernels.shape[0]
    
    flat_spatial = input_volume.view(channels_in, spatial_h * spatial_w)
    projection_matrix = unit_kernels.view(channels_out, channels_in)
    
    linear_output = torch.mm(projection_matrix, flat_spatial)
    return linear_output.view(channels_out, spatial_h, spatial_w)

Verifying that this matrix approach yields numerically identical results to the generalized sliding-window implementation:

random_tensor = torch.randn(3, 3, 3)
random_bank = torch.randn(2, 3, 1, 1)

fc_result = execute_pointwise_projection(random_tensor, random_bank)
conv_result = compute_multiple_feature_maps(random_tensor, random_bank.squeeze(-1).squeeze(-1))

assert torch.allclose(fc_result, conv_result, atol=1e-6)