How much VRAM does Qwen 3.8 27B really need? Every GB, 8GB to 32GB..

Qwen 3.8 27B is a 55 GB local LLM that people run on 8 GB GPUs: here's how much VRAM it really needs, gigabyte by gigabyte, for 8, 12, 16, 24 and 32 GB cards and Macs. The five things that eat your VRAM (the file, the KV cache, the extras, scratch space, your screen), why its KV cache is a quarter of Qwen 3 32B's (only 16 of 64 layers keep one), the three different "Q4_K_M" files (16.46, 16.81 and 18.97 GB), Ollama's 4K default context, and the one formula that explains every card's tokens per second in llama.cpp. Watch next: Strata runs Qwen 3.8 Flash Next on a 12GB GPU.. here's the catch    • Strata runs Qwen 3.8 Flash Next on a 12GB ...   Chapters 0:00 Qwen 3.8 27B: a 55 GB model on an 8 GB card? 0:30 Every Qwen 3.8 27B file, from 6 GB to 55 GB 1:33 Chapter 1: Qwen 3.8 27B at every quant, and what each file keeps 4:50 Chapter 2: the Qwen 3.8 27B KV cache, and why it's 4x smaller 8:21 Chapter 3: the 3 hidden VRAM costs: vision, scratch space, your desktop 11:01 Chapter 4: why Qwen 3.8 27B runs at the speed it does 15:20 Chapter 5: Qwen 3.8 27B on 8GB, 12GB, 16GB, 24GB, 32GB and Macs 18:09 So, your move: Qwen 3.8 27B on every GPU Sources Qwen/Qwen3.8-27B model card (Hugging Face): https://huggingface.co/Qwen/Qwen3.8-27B Qwen/Qwen3.8-27B config.json: https://huggingface.co/Qwen/Qwen3.8-2... Hugging Face API: Qwen/Qwen3.8-27B: https://huggingface.co/api/models/Qwe... unsloth/Qwen3.8-27B-GGUF files: https://huggingface.co/unsloth/Qwen3.... ggml-org/Qwen3.8-27B-GGUF files: https://huggingface.co/ggml-org/Qwen3... lmstudio-community/Qwen3.8-27B-GGUF files: https://huggingface.co/lmstudio-commu... Unsloth docs: Qwen3.8: https://unsloth.ai/docs/de/modelle/qw... Qwen/Qwen3-32B config.json: https://huggingface.co/Qwen/Qwen3-32B... Quesma: Qwen3.8 27B quantizations benchmarked (Piotr Migdał): https://quesma.com/blog/qwen38-27b-qu... Ollama docs: context length: https://docs.ollama.com/context-length Hugging Face forum: How many tokens will an old 3090 produce?: https://discuss.huggingface.co/t/how-... thc1006: Qwen3.8-27B speculative decoding on an RTX 3090: https://hackmd.io/@thc1006/qwen3-8-27... The Autodidacts: How to fit Qwen3.8-27B into 16GB VRAM: https://www.autodidacts.io/how-to-fit... Anass Kartit: Qwen3.8-27B on a 24GB MacBook: https://www.kartit.net/blog/qwen38-27... localbench: KV cache quantization, KL divergence results: https://localbench.substack.com/p/kv-... Red Stapler: I ran Qwen3.8 27B on single 8GB Card!:    • I ran Qwen3.8 27B on single 8GB Card! (Bet...   Wikipedia: GeForce RTX 30, 40 and 50 series: https://en.wikipedia.org/wiki/GeForce... Apple newsroom: Apple introduces M4 Pro and M4 Max: https://www.apple.com/newsroom/2024/1... Gated Delta Networks: Improving Mamba2 with Delta Rule: https://arxiv.org/abs/2412.06464 Subscribe:    / @kernelbrief   #Qwen3827B #LocalLLM #LocalAI