GPU Memory Hierarchy

High-bandwidth memory vs on-chip SRAM and registers, memory bandwidth as the bottleneck for ML kernels, and arithmetic intensity.

Demerzel Video

High-bandwidth memory vs on-chip SRAM and registers, memory bandwidth as the bottleneck for ML kernels, and arithmetic intensity.