Ternary Models Are BETTER, But Why No One Uses Them?

1.58 bit LLMs promise a big model on a small machine. Every weight is just minus one, zero or plus one, so a 27B model like Ternary Bonsai 2 fits in about 5.9 GB, and its maker says it keeps 98.2% of what the original Qwen 3.8 27B scores. So why does almost nobody run a ternary LLM yet? Complete Local AI course: join the waitlist → https://localanalysis.ai/md/curso The video starts with one person running Bonsai 2 on an 8 GB card for an audiobook pipeline, then goes back to the Microsoft paper behind BitNet b1.58, what bitnet.cpp really showed when it ran big models on a CPU, and the first real one, BitNet b1.58 2B4T, trained from scratch on four trillion tokens. You will see why you cannot simply round an existing model to three values, how quantization-aware training gets around that, and how it compares with the LLM quantization you already use, the 4-bit and 2-bit builds of the same model. Then the part the headlines skip. On a 16 GB card a ternary build finished the same tasks, but wrote about three times as many tokens and took three times as long. On an agent benchmark, the first ternary 27B scored below a 9B model that fits on the same card. Those figures come from community posts: they are individual user reports on their own hardware, illustrative, not averages. With 8 GB or less, a 1 bit LLM can be the only way to get a 27B class model fully on the card. With 16 GB, a normal local LLM quantization of Qwen 3.8 27B is usually the better trade. Tell me in the comments which card you run and what you got. 00:00 An audiobook on an 8 GB card 01:47 What I believed at first 03:06 What everyone said 03:32 What the paper actually says 04:19 Where the time goes when you generate text 05:34 A quick stop to understand the 1.58 06:47 Why you can't just round 07:36 How you train a model made of switches 09:17 BitNet, from paper to the first real model 10:14 PrismML's shortcut 11:41 Why almost nobody uses it 12:48 What people measured at home 14:07 The math nobody did 15:45 What nobody tells you before you download it 16:39 Who it's for and who it isn't 17:43 What nobody knows yet 18:04 The questions you can answer now 18:41 Back to the audiobook #bitnet #localllm #llm