EmbeddingGemma 2 vs nomic-embed-text: Homelab RAG on a CPU

nomic-embed-text is the default embedding model in a lot of homelab RAG setups. Google's EmbeddingGemma 2 (Apache 2.0, 8,192-token context, Matryoshka 768/512/256/128) is the new contender. The bigiron.cc guide ran both through the same llama.cpp server on a 6-core CPU. On the BEIR SciFact benchmark, EmbeddingGemma 2 scored 0.908 nDCG@10 against 0.784 for nomic, and even cut to 128 dimensions it still won. It was about 30% slower, used 723 MiB against 287 MiB, and the Ollama library build would not run on x86 Linux. This video walks through every measurement and what it means for your index. What's covered: • The two models: 137M vs 740M parameters, 2K vs 8K context, Matryoshka dimensions, prompts • The Ollama trap: "this model requires MLX support" on Ollama 0.40.3 for x86-64 Linux • What worked: llama-server --embedding with the official GGUF, and why -b and -ub matter • SciFact retrieval: nDCG@10 0.908 vs 0.784, recall@10 0.970 vs 0.876 • Truncation: 512 dimensions was free; EmbeddingGemma 2 at 128 (0.845) beat nomic at 768 • Speed and memory: 2.2 vs 3.1 documents per second, 723 vs 287 MiB • Long documents: 400 tokens per second at 8K, about 12 seconds per article, 1,429 MiB • Forgetting the prompts: EmbeddingGemma 2 fell to 0.874 with no error; nomic did not care here • Storage: 4 bytes per dimension, 2.9 GiB per million vectors at 768 • Switching safely: re-embed everything, build side by side, test on your own questions ⏱️ Chapters 0:00 Time to Drop nomic? 0:45 Two Models, One Engine 3:24 Retrieval Quality 5:31 Speed and Memory 7:24 The Prompts You Forget 8:54 Before You Switch 📖 The full guide (every results table, the evaluation script for your own documents, test method, troubleshooting): → EmbeddingGemma 2 vs nomic-embed-text for Homelab RAG on a CPU: https://www.bigiron.cc/guides/embeddi... 🛠️ Gear (affiliate — same price to you, supports the channel): • Mini PC for a CPU-only RAG box → https://www.amazon.com/s?k=mini+pc+32... • DDR4 RAM upgrade for vector indexes held in memory → https://www.amazon.com/s?k=32gb+ddr4+... • NVMe SSD for the document index → https://www.amazon.com/s?k=1tb+nvme+s... As an Amazon Associate we earn from qualifying purchases. Affiliate links — same price to you, supports the channel. 👉 Subscribe for self-hosted AI + homelab, with new videos every day. #RAG #Embeddings #EmbeddingGemma #LocalAI #SelfHosted #Homelab #LlamaCpp #Ollama #VectorSearch #LLM