Avalon Just Beat Strata by 47% on AMD (9x Faster)

Avalon, a llama.cpp fork, beat Strata by 47% on a used Radeon RX 7900 XT, the one card Strata's maintainers never ran. On October 22, Avalon's developer (freriadoc on GitHub) published a same-machine benchmark: Strata 0.1.43 at 61.9 tokens per second, Avalon at 91.0. The machine is a 2020 desktop: Ryzen 7 3800XT, dual-channel DDR4-3200, an AMD Radeon RX 7900 XT 20 GB on ROCm, and a PCIe 3.0 slot on a B450 board. Same night, greedy decoding, 512 tokens, median of three runs, Strata's own drafters switched on. Avalon's model file is about 13% bigger than the one Strata served and scores slightly better on perplexity, so the speed doesn't come from a smaller model. The 47% is two numbers stacked: about 27% from the engine (49.0 to 62.4 tokens per second without speculative decoding) and about 15% from drafting (Avalon's MTP draft head gives 1.46x, Strata's 1.26x). Both come from the same place: what a 20 GB card does with its VRAM when every expert it can't hold has to come from 34 GB/s of DDR4. At about a gigabyte of routed experts per token, that RAM alone caps you near 34 tokens per second. Everyone reads this as one local AI engine beating another. The comparison that explains it is each engine against the card it was built on. The Strata build in the table was seven releases old the day it was published and is fourteen behind today, and the AMD gains in those releases were measured on RDNA 4 (Radeon AI PRO R9700). Strata's 0.2.0 notes say its maintainers have not run their RX 7000 packages on an RX 7000. The baseline is still fair: RDNA 3 owners report 59 to 62 tokens per second from Strata on their own machines. Off that card the lead shrinks: on a rented RTX 4090 the author calls it about the same speed as Strata. WHAT THIS VIDEO COVERS Avalon vs Strata on the same AMD machine: 61.9 vs 91.0 tok/s, and the 27% engine gap plus 15% drafting gap inside the 47% Why speculative decoding is expensive on a mixture of experts model The DDR4 floor: about 1 GB of experts per token against 33 to 35 GB/s of measured bandwidth Avalon reads a bigger file (about 88 vs 78 GiB) at slightly lower perplexity, with no task benchmarks published yet Strata's AMD gains since 0.1.43 landed on RDNA 4, not on the RX 7900 XT / XTX 32 GB of RAM works for writing (91.2 to 89.0 tok/s) and costs you on new prompts (1,283 to 889 tok/s) Who should run Avalon today (RX 7900 XT / XTX, Linux, ROCm) and who should stay on Strata (NVIDIA, RDNA 4, Windows, desktop app) CHAPTERS 0:00 61.9 to 91 on a 2020 desktop 1:13 Why it reads as revenge 2:23 A fork built for one model 3:37 A €500 card in a 2020 box 5:09 125B counted, 177B stored 6:10 The RAM floor and the cache 7:48 Every byte of VRAM is cache 9:01 47% is two numbers 9:57 Drafting on a mixture of experts 11:25 Faster on a bigger file 12:32 The Strata build from that night 13:34 The card Strata never ran 14:47 61.9 is what owners get 15:58 Off the home card, it shrinks 17:21 32 gigs and the prompt bill 18:54 The owner this is wrong for 20:11 Run what your card's author runs HARDWARE AND SOFTWARE IN THIS VIDEO GPU: AMD Radeon RX 7900 XT 20 GB (RDNA 3, 800 GB/s) · CPU: AMD Ryzen 7 3800XT (Zen 2, 8 cores) · RAM: DDR4-3200 dual channel · Board: B450, PCIe 3.0 x16 Model: Qwen3.8-Flash-Next (125B parameters, about 6B active per token, plus a 51B lookup table) Engines: Avalon (llama.cpp fork, MIT), Strata 0.1.43 HIP build (current release 0.2.8) Also: Radeon AI PRO R9700 32 GB, Radeon Pro W7800 48 GB, RTX 4090, RTX 3060 Every speed here was published by an engine's developer or posted by an owner. None of it was re-run for this video. SOURCES Avalon README (every Avalon table, the Strata comparison, the RTX 4090 run, the 32 GB test): https://github.com/freriadoc/Avalon Avalon model files and quantization recipe: https://huggingface.co/freriadoc/Qwen... Avalon issue #1, two Radeon Pro W7800 48 GB: https://github.com/freriadoc/Avalon/i... Avalon launch thread, r/ROCm:   / 1xdwk8z   Strata: https://github.com/Niko1221/Strata Strata releases (0.1.43, 0.2.0-beta RX 7000 note, 0.2.7 and 0.2.8 R9700 numbers): https://github.com/Niko1221/Strata/re... Strata AMD support table and HIP backend history: https://github.com/Niko1221/Strata/bl... Strata community benchmarks (RX 7900 XTX, GRE, 4x 7900 XT): https://github.com/Niko1221/Strata/bl... Strata vs llama.cpp head-to-head, RTX 3060, Oct 6: https://github.com/Niko1221/Strata/bl... llama.cpp PR #30971, adaptive single-graph MoE offload (draft): https://github.com/ggml-org/llama.cpp... llama.cpp PR #30966, missed experts on the CPU: https://github.com/ggml-org/llama.cpp... RX 7900 XT used price, EU: https://bestvaluegpu.com/en-eu/histor... #Strata #LocalAI #AMD