Best Local AI For Your GPU (4GB To 512GB)

A researched local AI coding pick and exact download for every memory size from 4GB to 512GB, Qwen 3.5 to DeepSeek V4.1 Flash, with room left for your code. Pick the right local coding model for your GPU without maxing out capacity. Start with Qwen 3.5-2B on 4GB or Qwen 3.8-27B on 24GB. Qwen's own evaluation (SWE-bench Pro via Claude Code) reports 61.7% for 3.8 versus 53.5% for the older 3.6, for the source models rather than these local files. This guide maps Ollama, llama.cpp, and GGUF quantizations across seven hardware tiers from entry-level cards through 512GB workstations and Macs. Each tier names the specific model, quantization level, download size in GB, and why that choice preserves headroom for your agent to read repos, write edits, and run terminal commands. You'll learn the difference between model weights and working memory, why Ornith 1.5-9B leads the 8-16GB tier (the maker's own Terminal-Bench table: 46.2 vs 21.3 for the base Qwen 3.5-9B), when gpt-oss-20b and gpt-oss-120b fit, how Flash Next uses SSD-streamed lookup tables on Mac unified memory, and why Laguna's current 96GB file won't fit 80GB rigs despite old guides. At 256GB, GLM 5.3 Flash (a 192.9GB 4-bit file) is an experimental candidate; at 512GB the route splits: DeepSeek V4.1 Flash in antirez's DwarfStar 4-bit package on a Mac (about 294 GiB of main weights in memory, 189 GiB of lookup tables on a fast SSD) or GLM 5.3 in Unsloth's UD-Q4_K_XL build (about 467GB of weight files) on a multi-GPU workstation. These are researched starting points based on publisher evaluations and community quantizations, not guaranteed context-window wins. Built for developers running agents locally and the GPU-curious who want actual numbers instead of hype. Chapters: 0:00 Intro 0:53 4–6GB 2:34 8–16GB 4:20 20–24GB 6:14 32–48GB 7:38 64–96GB 10:18 128–256GB 12:41 288–512GB 15:47 Conclusion Tools & resources mentioned: Ollama: https://ollama.com llama.cpp: https://github.com/ggerganov/llama.cpp Qwen 3.5-2B: https://huggingface.co/Qwen/Qwen3.5-2B Qwen 3.5-4B: https://huggingface.co/Qwen/Qwen3.5-4B Qwen 3.8-27B: https://huggingface.co/Qwen/Qwen3.8-27B qtum Qwen 3.8-27B GGUF: https://huggingface.co/qtum/Qwen3.8-2... ggml-org Qwen 3.8-27B GGUF: https://huggingface.co/ggml-org/Qwen3... Unsloth Qwen 3.5-4B GGUF: https://huggingface.co/unsloth/Qwen3.... Ornith 1.5-9B: https://huggingface.co/ornith-ai/Orni... Ornith 1.5-9B GGUF: https://huggingface.co/ornith-ai/Orni... Ornith 1.5-35B-A3B: https://huggingface.co/ornith-ai/Orni... Ornith 1.5-397B: https://huggingface.co/ornith-ai/Orni... gpt-oss-20b: https://huggingface.co/openai/gpt-oss... gpt-oss-120b: https://huggingface.co/openai/gpt-oss... Qwen 3.8-Flash-Next: https://huggingface.co/Qwen/Qwen3.8-F... AtomicChat Qwen 3.8-Flash-Next GGUF: https://huggingface.co/AtomicChat/Qwe... Qwen 3-Coder-Next: https://huggingface.co/Qwen/Qwen3-Cod... GLM-5.3-Flash: https://huggingface.co/zai-org/GLM-5.... vcruz305 GLM-5.3-Flash GGUF: https://huggingface.co/vcruz305/GLM-5... Unsloth GLM 5.3 GGUF: https://huggingface.co/unsloth/GLM-5.... DeepSeek V4.1-Flash: https://huggingface.co/deepseek-ai/De... antirez deepseek-v4.1-flash-gguf: https://huggingface.co/antirez/deepse... About The Stack The Stack helps you build with AI. Each video takes one tool, model, or workflow and shows how it works in a few focused minutes, with the real benchmarks and real costs. We go deep on Claude Code and Cursor for AI coding, AI agents and MCP servers, the open-source AI tools and GitHub repos most people miss, RAG and vector search, fine-tuning, and running local LLMs on your own machine with Ollama and LM Studio. We compare models like ChatGPT and Claude, test AI automation with Zapier, Make, and n8n, and flag the tools that actually ship. Subscribe for new breakdowns: https://www.youtube.com/@the-stack-ai... #localai #ollama #codingmodels