CPU vs GPU vs NPU: The Real Maths Behind AI Chips

Every AI answer is one tiny sum, a multiply and an add, done billions of times. We work it out for real, then follow it onto a CPU, a GPU and an NPU to see why each one does it so differently. ✖️➕ 0:00 One tiny sum 0:22 The multiply-add 0:52 A layer is a matrix 1:20 A whole model 1:48 The CPU 2:31 The GPU 3:20 Training: why GPUs won 4:11 Inference is a different job 4:54 The real bottleneck: moving numbers 5:43 The inference chip 6:44 Three ways to do one sum Sources: Kaplan et al. 2020 (scaling laws) · Meta 2024 (Llama 3) · Jouppi et al. 2017 (inference chip) · Horowitz 2014 and Han et al. 2015 (energy) · Dettmers et al. 2022 (8-bit) · NVIDIA and Intel documentation. Full list in the episode notes. #TechExplained #AI #GPU #NPU #CPU #MachineLearning #Semiconductors