Accelerating LLM Inference: Speculative Decoding and Diffusion LLMs | AI Scale Talks EP.2

The second episode of AI Scale Talks goes inside LLM inference, where serving cost and latency are actually decided. Junbum Lee, researcher at Lablup and the creator of KcBERT and KoAlpaca, covers why prefill and decode run into different limits, how quantization and compressed attention shrink what has to be loaded per token, and what speculative decoding and diffusion LLMs change about generation. AI Scale Talks runs four Wednesdays under the theme From Cell to Factory. Each episode scales up one level, from a single inference engine to full AI infrastructure operations. EP.2 stays at the token level: one forward pass, and the milliseconds it spends waiting on memory. ⏰ Chapters 00:00 Welcome and series overview 02:31 Housekeeping 03:19 Meet the speaker 04:18 What this session covers 05:06 What inference actually costs 07:45 Prefill and decode 10:51 KV cache 13:30 TTFT and TPOT 17:04 Prefill is compute-bound 19:59 Decode is memory-bound 21:50 Quantization 23:40 Why 4-bit is faster 26:00 MoE and DeepSeek V4 Flash 29:21 TurboQuant for the KV cache 31:39 Speculative decoding: the idea 35:39 Medusa and DeepSeek MTP 38:23 Eagle-3 39:54 DFlash and DSpark 44:55 Diffusion LLMs 46:13 A2D: autoregressive to diffusion 47:23 WeDLM 48:40 Training DFlash and a dLLM 50:33 DFlash for agentic coding 53:26 A2D on Qwen3-30B-A3B 56:05 What's next 56:46 Q&A