Freaks City

Attention Mechanisms Multi-Head Attention The attention mechanism computes: The scaling factor \(\sqrt{d_k}\) prevents large inner product values that could cause gradient instability. Assuming Q and K elements have mean 0 and variance \(\sigma^2\), the variance of \(QK^T\) grows with \(d_k\). Scaling by \(\sqrt{d_k}\) maintains stable varianc ...

PagedAttention Memory Layout and Block Mapping PagedAttention replaces traditional contiguous key-value cache allocations with a virtual-to-physical block mapping scheme. This approach mirrors operating system memory paging, allowing non-contiguous GPU memory segments to serve sequential generation tasks without fragmentation overhead. Each req ...