DeepSeek Just Split the Transformer in Two

What if the next AI breakthrough isn't a bigger model, but a model that stops spending the same effort on reading as it does on writing? DeepSeek V4.1-Flash uses a causal encoder–decoder: a 20-layer encoder reads your prompt with about 8B active parameters, and a 20-layer decoder writes the answer with about 16B. In this video we break down why that matters, starting with the 2017 transformer and decoder-only chatbots. We then cover prefill vs. decode, mixture of experts (6 of 384 routed experts per token), the KV cache and multi-head latent attention, and why a 1M-token context window isn't the same as understanding 1M tokens. Then we zoom out: what this means for AI coding agents that make hundreds of model calls per task, why cheap tokens aren't the same as cheap tasks, the hardware and kernel engineering that makes new architectures practical, and the Jevons paradox (why more efficient AI might mean more AI, not less compute). Chapters 0:00 Hook 2:03 The Transformer 3:42 Decoder-only 4:53 Prefill vs. decode 6:02 Asymmetric computation 7:12 Mixture of experts 9:18 Experts + asymmetry 10:48 Memory and the KV cache 13:18 A common misunderstanding: long context 14:15 AI coding agents 16:03 The big question 17:26 The economics 19:44 The limits of scaling 21:50 One final question: the efficiency paradox 23:14 The verdict Sources shown on screen DeepSeek V4.1-Flash release note: https://api-docs.deepseek.com/news/ne... DeepSeek V4.1-Flash model card: https://huggingface.co/deepseek-ai/De... Attention Is All You Need: https://arxiv.org/abs/1706.03762 Bahdanau et al., Neural Machine Translation by Jointly Learning to Align and Translate: https://arxiv.org/abs/1409.0473 Sutskever et al., Sequence to Sequence Learning: https://arxiv.org/abs/1409.3215 The Illustrated Transformer (Jay Alammar): https://jalammar.github.io/illustrate... DistServe, Disaggregating Prefill and Decoding: https://arxiv.org/abs/2401.09670 Splitwise, Phase Splitting for LLM Inference: https://arxiv.org/abs/2311.18677 Sparsely-Gated Mixture-of-Experts: https://arxiv.org/abs/1701.06538 Switch Transformers: https://arxiv.org/abs/2101.03961 Mixture of Experts Explained (Hugging Face): https://huggingface.co/blog/moe DeepSeek-V2 (MLA): https://arxiv.org/abs/2405.04434 DeepSeek-V3 Technical Report: https://arxiv.org/abs/2412.19437 DeepSeek-R1: https://arxiv.org/abs/2501.12948 DeepEP: https://github.com/deepseek-ai/DeepEP FlashMLA: https://github.com/deepseek-ai/FlashMLA DeepGEMM: https://github.com/deepseek-ai/DeepGEMM PagedAttention / vLLM: https://arxiv.org/abs/2309.06180 · https://github.com/vllm-project/vllm FlashAttention: https://arxiv.org/abs/2205.14135 Gemini 1.5 (long context): https://arxiv.org/abs/2403.05530 Lost in the Middle: https://arxiv.org/abs/2307.03172 RULER: https://arxiv.org/abs/2404.06654 Needle In A Haystack: https://github.com/gkamradt/LLMTest_N... SWE-bench: https://arxiv.org/abs/2310.06770 · https://www.swebench.com SWE-agent: https://arxiv.org/abs/2405.15793 Vision Transformer (ViT): https://arxiv.org/abs/2010.11929 T5: https://arxiv.org/abs/1910.10683 Whisper: https://arxiv.org/abs/2212.04356 Scaling Laws for Neural Language Models: https://arxiv.org/abs/2001.08361 Scaling Test-Time Compute: https://arxiv.org/abs/2408.03314 NVIDIA H100: https://www.nvidia.com/en-us/data-cen... Latent Space coverage: https://www.latent.space/p/ainews-dee... Machine Learns (erogol): https://erogol.substack.com/p/deepsee... Wikipedia: Transformer, Recurrent neural network, LSTM, GPT, Mixture of experts, Jevons paradox Let me know in the comments: will decoder-only transformers keep dominating, or are specialised architectures about to take the lead?