Understanding SIMD Architectures

x86 architecture, originally introduced by Intel in 1978 with their 16-bit microprocessor, refers to a family of instruction set architectures. Modern compilers like Intel ICC and GCC provide intrinsic functions for SSE/AVX instruction sets through headers like immintrin.h.

AVX Instruction Set Fundamentals

AVX (Advanced Vector Extensions) intrinsic functions follow a specific naming convention:

• Prefix: _mm for SSE (128-bit), _mm256 for AVX (256-bit)
• Operation: add, mul, load, store, etc.
• Data Type: ps (packed single-precision), pd (packed double-precision), epi64 (64-bit integers)

Example: _mm256_add_ps performs 256-bit vector addition on single-precision floating-point numbers.

To check your processor's AVX support:

gcc -march=native -Q --help=target | grep march
# or
cat /proc/cpuinfo | grep avx

Vector Addition with AVX

Here's an optimized implementation of vector addition using AVX instructions:

#include <immintrin.h>
#include <iostream>
#include <chrono>

void demonstrate_vector_addition() {
    const int VECTOR_SIZE = 8;
    int32_t operand_a[VECTOR_SIZE] = {10, 20, 30, 40, 50, 60, 70, 80};
    int32_t operand_b[VECTOR_SIZE] = {1, 2, 3, 4, 5, 6, 7, 8};
    int32_t result[VECTOR_SIZE];
    
    __m256i vec_a = _mm256_loadu_si256((__m256i*)operand_a);
    __m256i vec_b = _mm256_loadu_si256((__m256i*)operand_b);
    __m256i vec_result = _mm256_add_epi32(vec_a, vec_b);
    _mm256_storeu_si256((__m256i*)result, vec_result);
    
    for (int i = 0; i < VECTOR_SIZE; i++) {
        std::cout << result[i] << " ";
    }
    std::cout << std::endl;
}

Optimized Matrix Multiplication with AVX

Matrix multiplication can be significantly accelerated using AVX instructions:

#include <immintrin.h>
#include <chrono>
#include <random>

struct Matrix {
    double* elements;
    size_t row_count;
    size_t col_count;
};

Matrix* create_matrix(size_t rows, size_t cols) {
    Matrix* mat = new Matrix();
    mat->row_count = rows;
    mat->col_count = cols;
    mat->elements = new double[rows * cols];
    return mat;
}

void initialize_random_matrix(Matrix* mat) {
    std::random_device rd;
    std::mt19937 gen(rd());
    std::uniform_real_distribution<> dis(0.0, 1.0);
    
    for (size_t i = 0; i < mat->row_count * mat->col_count; i++) {
        mat->elements[i] = dis(gen);
    }
}

Matrix* multiply_matrices_avx(Matrix* a, Matrix* b) {
    Matrix* c = create_matrix(a->row_count, b->col_count);
    
    auto start = std::chrono::high_resolution_clock::now();
    
    for (size_t i = 0; i < a->row_count; i++) {
        for (size_t j = 0; j < b->col_count; j++) {
            __m256d accumulator = _mm256_setzero_pd();
            
            size_t k = 0;
            for (; k <= a->col_count - 4; k += 4) {
                __m256d row_vec = _mm256_loadu_pd(&a->elements[i * a->col_count + k]);
                __m256d col_vec = _mm256_set_pd(
                    b->elements[(k+3) * b->col_count + j],
                    b->elements[(k+2) * b->col_count + j],
                    b->elements[(k+1) * b->col_count + j],
                    b->elements[k * b->col_count + j]
                );
                __m256d product = _mm256_mul_pd(row_vec, col_vec);
                accumulator = _mm256_add_pd(accumulator, product);
            }
            
            double temp[4];
            _mm256_storeu_pd(temp, accumulator);
            double sum = temp[0] + temp[1] + temp[2] + temp[3];
            
            for (; k < a->col_count; k++) {
                sum += a->elements[i * a->col_count + k] * 
                       b->elements[k * b->col_count + j];
            }
            
            c->elements[i * c->col_count + j] = sum;
        }
    }
    
    auto end = std::chrono::high_resolution_clock::now();
    auto duration = std::chrono::duration_cast<std::chrono::microseconds>(end - start);
    std::cout << "AVX optimization time: " << duration.count() << " μs" << std::endl;
    
    return c;
}

Compile with: g++ matrix_mult.cpp -o matrix_mult -mavx -mavx2 -O3

ARM NEON SIMD Programming

NEON technology is ARM's 128-bit SIMD architecture extension for Cortex-A series processors. It's designed for parallel processing and supports 128-bit vector operations.

NEON Data Types and Registers

NEON provides several vector data types:

• float32x4_t: 4 × 32-bit floating-point numbers
• int16x8_t: 8 × 16-bit integers
• int8x16_t: 16 × 8-bit integers

ARMv7 NEON architecture has 16 × 128-bit registers (q0-q15) which can be split into 32 × 64-bit registers (d0-d31).

Common NEON Intrinsics

Key NEON intrinsic functoins include:

• vdupq_n_f32(value): Duplicate scalar value across vector
• vld1q_f32(ptr): Load 4 floats from memory
• vst1q_f32(ptr, val): Store 4 floats to memory
• vaddq_f32(a, b): Vector addition
• vmulq_f32(a, b): Vector multiplication
• vmlaq_f32(a, b, c): Multiply-accumulate (a + b*c)
• vmaxvq_f32(a): Horizontal maximum across vector

Array Summation with NEON

#include <arm_neon.h>
#include <iostream>

float compute_vector_sum(float* data, int length) {
    if (!data || length <= 0) return 0.0f;
    
    int vector_length = length >> 2;
    int remaining = length & 3;
    
    float32x4_t sum_vector = vdupq_n_f32(0.0f);
    
    for (int i = 0; i < vector_length; i++) {
        float32x4_t data_vector = vld1q_f32(data + (i * 4));
        sum_vector = vaddq_f32(sum_vector, data_vector);
    }
    
    float sum = vgetq_lane_f32(sum_vector, 0) + 
                vgetq_lane_f32(sum_vector, 1) + 
                vgetq_lane_f32(sum_vector, 2) + 
                vgetq_lane_f32(sum_vector, 3);
    
    for (int i = vector_length * 4; i < length; i++) {
        sum += data[i];
    }
    
    return sum;
}

Dot Product Implementation

void compute_matrix_vector_product(float matrix[][1000], 
                                  float vector[], 
                                  float result[], 
                                  int dimension) {
    for (int i = 0; i < dimension; i++) {
        float32x4_t accumulator = vdupq_n_f32(0.0f);
        
        for (int j = 0; j < dimension; j += 4) {
            float32x4_t matrix_row = vld1q_f32(matrix[i] + j);
            float32x4_t vector_elements = vld1q_f32(vector + j);
            accumulator = vmlaq_f32(accumulator, matrix_row, vector_elements);
        }
        
        result[i] = vgetq_lane_f32(accumulator, 0) + 
                    vgetq_lane_f32(accumulator, 1) + 
                    vgetq_lane_f32(accumulator, 2) + 
                    vgetq_lane_f32(accumulator, 3);
    }
}

Vector Addition with NEON

void perform_vector_addition(float* destination, 
                             float* source_a, 
                             float* source_b, 
                             int element_count) {
    int i = 0;
    for (; i <= element_count - 4; i += 4) {
        float32x4_t vec_a = vld1q_f32(source_a + i);
        float32x4_t vec_b = vld1q_f32(source_b + i);
        float32x4_t vec_result = vaddq_f32(vec_a, vec_b);
        vst1q_f32(destination + i, vec_result);
    }
    
    for (; i < element_count; i++) {
        destination[i] = source_a[i] + source_b[i];
    }
}

Automatic vs Manual Vectorization

Enabling Automatic Vectorization

For GCC, enable automatic vectorization with:

-ftree-vectorize -mfpu=neon -mcpu=cortex-a7

Note: -O3 optimization level automatically enables -ftree-vectorize.

Manual Vectorization Benefits

Manual vectorization using intrinsics provides: - Better control over memory access pattersn - Ability to optimize for specific data layouts - Fine-tuned performance for critical sections - Predictable behavior across different compilers

Getting Started with PyTorch

PyTorch is a powerful machine learning framework that provides:

• Dynamic computation graphs
• GPU acceleration via CUDA
• Automatic differentiation
• Rich ecosystem of tools and libraries

Basic tensor operations example:

import torch

# Create tensors
x = torch.randn(3, 4)
y = torch.randn(4, 5)

# Matrix multiplication
result = torch.mm(x, y)

# CUDA support
if torch.cuda.is_available():
    device = torch.device("cuda")
    x_gpu = x.to(device)
    y_gpu = y.to(device)
    result_gpu = torch.mm(x_gpu, y_gpu)

For comprehensive documentation, refer to the official PyTorch guides and API references.