Speculative Decoding - 2-3 X faster LLMs same answer

Big AI models are smart but slow. Small ones are fast but make more mistakes. Speculative decoding gets you the big model’s exact answer at close to small-model speed, and it’s already running inside Google Search, DeepSeek and Cursor. In this video: 0:00 Intro 0:07 Smart or fast? The wall every AI team hits 1:06 Why big models are slow 1:41 The delivery rider analogy (the memory wall) 2:43 The trick borrowed from CPUs 3:04 Who came up with it (Google 2022, DeepMind 2023) 3:23 How it works: draft model + target model 4:13 Rejection sampling: why the answer stays the same 4:31 What the draft model gets right 4:57 The code-edit twist (prompt lookup / speculative edits) 5:23 Where it runs in production 6:56 What it means for agents 7:34 What it means for evals 7:55 The catch 8:49 Recap Sources: Leviathan, Kalman & Matias, Fast Inference from Transformers via Speculative Decoding (Google Research, 2022): https://arxiv.org/abs/2211.17192 Chen et al., Accelerating LLM Decoding with Speculative Sampling (DeepMind, 2023): https://arxiv.org/abs/2302.01318 Google Research, Looking back at speculative decoding: https://research.google/blog/looking-back-... Cursor, Instant Apply: https://cursor.com/blog/instant-apply Cerebras, Inference 3x faster: https://www.cerebras.ai/blog/cerebras-infe... Apple, ReDrafter on NVIDIA TensorRT-LLM: https://machinelearning.apple.com/research... DeepSeek, DSpark: https://arxiv.org/abs/2607.05147 vLLM, Speculative decoding performance: https://vllm.ai/blog/spec-decode