How to Run a 177B AI Model on an 8GB GPU

A 177B model is about 85 GB. An 8 GB graphics card should not be able to run it. One tester got it running at around 35 tokens per second using 6.6 GB of VRAM. This video walks through how. Only a small slice of the model is active for each token, so the work gets split three ways: the card, system RAM and the SSD. Then one simple sum tells you roughly what speed your own PC will get, and where the real tests land below it. Every number on screen carries a tag: vendor figures, measured results from named testers, or our own arithmetic. Chapters 0:00 The impossible number 0:48 The six-second token 1:36 500 experts sit still 2:45 The three-way split 3:51 The 2.7 kilobyte job 5:13 One sum 6:00 The fine print 7:47 What a normal PC gets 10:16 DeepSeek's table 11:25 Which setup Sources Qwen model card https://huggingface.co/Qwen/Qwen3.8-F... lna-lab, flash-next-8gb (the 8 GB test) https://github.com/lna-lab/flash-next... lukaLLM VRAM benchmark https://github.com/lukaLLM/Qwen3.8-Fl... Owner report, Hugging Face (5070 Ti + 64 GB) https://huggingface.co/unsloth/Qwen3.... Owner reports, r/LocalLLaMA   / 1sw07j4     / 1idseqb   Red Stapler    • I run 177B model on SSD (It's Fast!) | Qwe...   xCreate    • Qwen 3.8 Flash via SSD: Max Quants on Low ...   DeepSeek model card https://huggingface.co/deepseek-ai/De... DeepSeek Engram paper https://cdn.jsdelivr.net/gh/deepseek-... SSD read-disturb study (ASPLOS '21) https://doi.org/10.1145/3445814.3446733 Image credits (logos removed, modified from the originals) DDR5 RAM: ITEagle Europe, Sebastiaan Broekhoven, CC BY-SA 4.0, Wikimedia Commons Threadripper CPU: PantheraLeo1359531, CC BY 4.0, Wikimedia Commons Motherboard: Jacek Halicki, CC BY-SA 4.0, Wikimedia Commons Intel Optane module: Jacek Halicki, CC BY-SA 4.0, Wikimedia Commons NVMe SSD: User5515, CC0, Wikimedia Commons Graphics cards: ASUS product images. Laptop: Apple press image. #LocalAI #LocalLLM #GPU