Qwen Just Ran an 82GB Model on a 12GB Phone

An 82GB-class AI model is generating text directly on an Android phone with only 12GB of RAM. No cloud server. No external GPU. No hidden desktop. BigMoeOnEdge runs Qwen3.8-Flash-Next locally using CPU compute, RAM, and the phone’s own UFS storage. Qwen3.8-Flash-Next is a 125B-parameter Mixture-of-Experts model with roughly 6B parameters activated per token. It also contains a massive 51B-parameter N-gram embedding table and serves as an experimental preview of the architecture being developed for Qwen4. The checkpoint is roughly seven times larger than the phone’s available memory—yet it keeps generating. 🔔 Subscribe to Code Unpacked:    / @code-unpacked   📌 Chapters: 00:00 - An 82GB Model on a 12GB Phone 01:03 - Why Most of the Model Does Not Need Fast Memory 01:56 - Mixture-of-Experts and Expert Routing 03:11 - Page-Fault Storms vs. Deliberate Streaming 04:25 - Two Kinds of Sparsity: MoE and N-Gram Memory 05:24 - From 2.03 to 3.48 Tok/s 06:43 - Latency, Heat, Power and Context Limits 07:44 - Lossless Streaming vs. Quality Tradeoffs 08:21 - The Third Local-AI Limit: Residency 09:14 - Qwen4 and Future Model Working Sets 10:00 - RAM as a Working Set, Not a Hard Ceiling 10:51 - What This Phone Demo Changes In this video, we unpack how BigMoeOnEdge turns flash storage into an active part of the model’s memory hierarchy. Instead of forcing the entire checkpoint into RAM, the runtime keeps frequently used weights in an expert cache and reads only the expert slices selected by the router for the current token. Parallel storage reads, bounded caching, direct I/O, and compute–I/O overlap help prevent the page-fault storms normally created when a model is larger than system memory. Qwen3.8-Flash-Next is unusually well suited to this approach: • Hundreds of routed experts exist, but only a small subset participates in each token • Frequently used experts can remain cached in RAM • Less frequently used weights can wait on UFS storage • The 51B N-gram table behaves like a massive learned lookup system • Only the required N-gram entries need to be retrieved • Fast memory is reserved for the model’s current working set The original documented phone run reached approximately 2.03 tokens per second. A newer Q2_K configuration is reported at approximately 3.48 tokens per second on the same 12GB-class device. That may sound slow compared with cloud inference, but speed is not the most important result. The important part is that a model far larger than the phone’s RAM can generate usable text without relying on a server. The core streaming mechanism can also be lossless, producing the same output as the same quantized model running resident in memory. Optional settings such as using fewer experts or dropping less important cache misses can improve speed further, but those settings change the computation. There are still serious limitations. Prompt processing, storage latency, memory bandwidth, cache behavior, heat, power consumption, context memory, and inference-engine support all matter. Running this checkpoint on a phone does not turn the device into a data-center server. But it changes the upper boundary of local AI. RAM can become a working set instead of a hard ceiling. Compute handles the part of the model required now, RAM caches what is likely to be reused, and storage holds the much larger pool of parameters waiting for its turn. The same idea could extend to laptops with large SSDs, small desktops, and edge servers with limited accelerator memory but inexpensive NVMe storage. The key question is no longer: “How large a model can fit in RAM?” It is: “Which parts of the model actually need to be there right now?” What do you think: will SSD and phone storage become first-class model memory, or will storage bandwidth remain the real bottleneck? Let me know in the comments. 🔗 Sources & Further Reading: BigMoeOnEdge: https://github.com/Helldez/BigMoeOnEdge Android Benchmarks: https://github.com/Helldez/BigMoeOnEd... Android Release: https://github.com/Helldez/BigMoeOnEd... Official Qwen Announcement: https://qwen.ai/blog?id=qwen3.8-flash... Qwen3.8-Flash-Next Model: https://huggingface.co/Qwen/Qwen3.8-F... Official Architecture Report: https://arxiv.org/abs/2608.30320 llama.cpp: https://github.com/ggml-org/llama.cpp #Qwen4 #Qwen #Qwen38FlashNext #BigMoeOnEdge #LocalAI #LLM #MixtureOfExperts #Android #AIHardware #EdgeAI