Freaks City

This article explores the computational essence of SIMD (Single Instruction Multiple Data) and SIMT (Single Instruction Multiple Thread) in the context of AI chips, detailing their relationship and how NVIDIA CUDA implements these concepts at a low level. The Core of SIMD Computation SIMD applies the same operation to multiple data elements sim ...

Understanding SIMD Architectures x86 architecture, originally introduced by Intel in 1978 with their 16-bit microprocessor, refers to a family of instruction set architectures. Modern compilers like Intel ICC and GCC provide intrinsic functions for SSE/AVX instruction sets through headers like immintrin.h. AVX Instruction Set Fundamentals AVX ( ...

PyTorch is a tensor library optimized for deep learning that leverages both GPU and CPU capabilities Chinese documentation: https://pytorch.org/resources Gradient and Derivative Calculation # gradient_calculation.py import torch import numpy as np input_val = torch.tensor(3.) weight = torch.tensor(4., requires_grad=True) bias = torch.tensor(5 ...