Freaks City

This article explores the computational essence of SIMD (Single Instruction Multiple Data) and SIMT (Single Instruction Multiple Thread) in the context of AI chips, detailing their relationship and how NVIDIA CUDA implements these concepts at a low level. The Core of SIMD Computation SIMD applies the same operation to multiple data elements sim ...

Understanding CUDA Programming When leveraging GPU processing to solve computational problems, we must first transfer data from CPU memory to GPU memory via high-speed interconnects like PCIe or NVLink. The second step involves executing the actual computation, typically through CUDA kernel functions. Finally, after computation completes, resul ...

In this notebook, we will: Parallelize a simple algorithm Learn about the performance of different parallel strategies Implement the parallelization in Julia Problem Description Asssumptions All matrices, including A, B, and C, are initially stored on the master process. The final result will overwrite the matrix C. Steps To implement paral ...

In the era of big data and artificial intelligence, the demand for distributed computing frameworks is increasingly urgent. Python Ray library was created to meet this demand. It is a high-performance, easy-to-use distributed application framework designed specifically for solving parallel and distributed computing problems in Python. This arti ...

KernelsCUDA C++ extends the standard C++ language by allowing the definition of kernels. A kernel is a function that, when called, is executed N times in parallel by N different CUDA threads, as opposed to a regular C++ function which executes only once. Kernels are defined using the __global__ declaration specifier and are invoked using a spec ...