Unfortunately, A $20 Plan Beats A Free Local AI Setup

Run free AI locally on 16GB: which models actually fit your PC, which need GPU, and where the hard limit truly is Most open AI models download for free, but your hardware decides what actually runs. This video tests a practical stack on a 16GB PC with no discrete GPU, from Phi-4-mini's 4 billion parameters running at 20 tokens/second on CPU alone, to Qwen's 28 billion model crawling at 3 tokens/second when memory-bound, to sparse models like Qwen 3.8 and DeepSeek-V4 that only activate a slice of their weights per token. You'll learn the real math: tokens per second equals memory bandwidth divided by model size, which means a standard desktop processor hits an 8B-parameter ceiling, while a used RTX 3090 (24GB VRAM) unlocks 28B models at reading speed. The video covers speech transcription with NVIDIA's Parakeet (627M params, CPU-only), document embedding with Nomic's Embed-Text (137M params, runs on any machine), local image generation with Biniou (CPU-capable but arithmetic-bound, needs GPU for speed), and the actual cost curve: second-hand 24GB cards are the one worthwhile upgrade for most builders; 128GB systems hide their memory bandwidth specs and often cost more while running slower; anything above 300B parameters requires renting cloud GPUs, making open weights a pricing control mechanism rather than a local ownership path. For builders, developers, and anyone curious whether a homelab AI stack makes financial or practical sense. Chapters: 0:00 Four billion parameters and two point four trillion, both free 1:29 Tokens a second is bandwidth divided by model size 3:10 A twenty-eight billion model reads at walking pace 4:33 Parakeet transcribes an hour with 0.6B parameters 5:43 Drawing one picture is not a reading problem 7:00 The most-owned card on Hugging Face is a 3060 8:07 An RTX 3090 holds a 28B model whole 9:31 NVIDIA sells 128GB and never prints the speed 11:28 The 2.4 trillion model only reads 95 billion of itself 12:54 512 gigabytes on one wall socket 14:11 One eight-card server holds 1,440 gigabytes 15:02 Four point nine terabytes needs four of those servers 16:15 Six dollars sixty nine an hour, times thirty-two cards 17:24 The top is a market, not a machine 18:37 The line falls at one used 24 gigabyte card 19:50 The ladder has one step worth taking 21:32 Nobody charges for the model, only the bandwidth Tools & resources mentioned: Ollama: https://ollama.ai llama.cpp: https://github.com/ggerganov/llama.cpp Biniou: https://github.com/bes/biniou Nomic Embed Text: https://huggingface.co/nomic-ai/nomic... Hugging Face: https://huggingface.co About The Stack The Stack helps you build with AI. Each video takes one tool, model, or workflow and shows how it works in a few focused minutes, with the real benchmarks and real costs. We go deep on Claude Code and Cursor for AI coding, AI agents and MCP servers, the open-source AI tools and GitHub repos most people miss, RAG and vector search, fine-tuning, and running local LLMs on your own machine with Ollama and LM Studio. We compare models like ChatGPT and Claude, test AI automation with Zapier, Make, and n8n, and flag the tools that actually ship. Subscribe for new breakdowns: https://www.youtube.com/@the-stack-ai... #homelab #localai #ollama