Can Local AI Beat an Optimised $20 Cloud AI? I Ran 5 Agentic Tests!
Weights & Wonders
0:00 / 0:00
Can Local AI Beat an Optimised $20 Cloud AI? I Ran 5 Agentic Tests!
864 просмотра · 8 ч назад
Weights & Wonders
1,02 тыс. подписчиков
864 просмотра · 8 ч назад
Get started with Abacus AI Supercomputer here: https://supercomputer.abacus.ai/wei
Can a $20 cloud AI setup do the work better than my RTX 3090? I gave both the same five real jobs to find out.
This video is sponsored by Abacus AI.
Qwen 3.8 27B on the 3090 and Abacus AI Supercomputer both ran Hermes, with the same prompts and an hour per job: build a website and put it online, research GPU prices with three helper agents, fix a real bug in loguru, analyse a messy 1,076-row spreadsheet, and answer ten questions about a 572-page report.
The cloud passed four of the five on its first try and got through all five in about fifteen and a half minutes. The 3090 needed more than an hour and a half and second tries at two jobs, and it still left the bug only partly fixed. But it matched the cloud on the 572-page report, with every page right, and all five jobs cost it about 7 cents of electricity.
So for time-sensitive work, like fixing a bug or providing support, the cloud is the better pick. For agents that run 24/7 on work that isn't urgent, or data that can't leave your house, the 3090 costs far less.
If you already own a 3090:
1. Load Qwen's vision file, so your agent can look at screenshots.
2. Turn on multi-token prediction. It took us from about 40 to more than 60 tokens a second.
3. Let your helper agents work one at a time, so each one gets the whole card.
4. Squeeze the KV cache to 8-bit to fit twice the context in the same VRAM.
The setup: Qwen 3.8 27B (Unsloth's 4-bit UD-Q4_K_XL, 17.6 GB) · llama.cpp with vision, multi-token prediction and a 65K context · RTX 3090 24 GB · Hermes Agent. Against Abacus AI Supercomputer ($20 a month ChatLLM Pro plan, 30,000 credits) · RouteLLM picking the model for every message · Hermes Agent.
0:00 Local AI vs a $20 cloud setup
1:22 The contenders
1:44 Abacus AI Supercomputer
3:32 The rules
5:00 Test 1: Build a website
6:25 Test 2: Research with three helpers
10:19 Test 3: Fix a real bug
12:03 Test 4: A messy spreadsheet
12:56 Test 5: A 572-page report
13:40 The bill
15:26 Which one deserves your money?
Music: "Decompress", "Cloudloop" and "Systematic" by Lee Rosevere, licensed under CC BY 4.0. Source: Music For Podcasts 5 and 6, Internet Archive.
Narration uses an AI voice.
#LocalAI #Qwen #RTX3090 #AbacusAI #AIAgents