AI on a $12 USB Stick for Your Mac: 34 Seconds or 3 Minutes
Mind The Prompt
0:00 / 0:00
AI on a $12 USB Stick for Your Mac: 34 Seconds or 3 Minutes
29 просмотров · 1 ч назад
Mind The Prompt
707 подписчиков
29 просмотров · 1 ч назад
A cheap USB stick doesn't make your Mac's AI slow to talk. It makes it slow to start. The same 5 GB model is ready in about 34 seconds from a
$17.99 SanDisk Ultra Flair, or about 3 minutes from the $11.99 SanDisk Cruzer Blade, and then both run at your Mac's own speed. I went through
the kit from the viral USB AI videos, one owner's Mac log, and the measured read speeds of every stick I could find.
In this video:
→ A $12 stick: 34 seconds or 3 minutes
→ What the wait costs on your Mac
→ Why the USB AI kit caught on
→ Why a slow stick doesn't mean slow answers
→ One owner's clock: 1:34, then 74 tok/s
→ Every stick, by read speed
→ Where the sums are too kind
→ Offline, after one online start
→ FAT32: the file that won't copy
→ The $12 stick vs the $18 one
→ Check your stick
The short version: load time is the model file divided by the stick's read speed. The Cruzer Blade is USB 2.0 and reads about 25–26 MB/s
(HWPure lab, UserBenchmark average of 34,316 runs), so a 5 GB model takes 3 min 12 s to 3 min 22 s, nearer 4 on slower copies. The Ultra Flair
32 GB is USB 3.0 and reads 149 MB/s: 34 seconds. $6 more buys back about 2 min 40 s every time you start a model, and nothing once it's running.
One owner's M1 MacBook Pro loaded a 9 GB model from a ~98 MB/s Kingston stick in 1 m 34 s, then wrote 74.22 tokens/s. The catch: Portable Local
Studio's Mac setup downloads Node.js and llama.cpp the first time it runs (about 60 MB, plus the models), so do the first run at home; sticks of
32 GB or less often come as FAT32, which can't hold a file over 4.29 GB, so erase yours as ExFAT; and eject before you pull it out.
Fair warnings: the load times are modelled, not timed by me; the read speeds are other people's benchmarks, mostly on Windows; the 74 tok/s is
one owner's run with no internal-drive comparison; mapped loading can be slower than the sum, and nobody has timed it from a stick on a Mac. Use
it on a Mac you own or are allowed to use. The voice is a digital one; the research and the editing are mine.
WATCH NEXT
(generated by `tools/crosslinks.py`)
CHAPTERS
0:00 A $12 stick: 34 seconds or 3 minutes
1:02 What the wait costs on your Mac
1:50 Why the USB AI kit caught on
2:43 Why a slow stick doesn't mean slow answers
3:30 One owner's clock: 1:34, then 74 tok/s
4:20 Every stick, by read speed
5:15 Where the sums are too kind
6:20 Offline, after one online start
7:12 FAT32: the file that won't copy
8:07 The $12 stick vs the $18 one
9:11 Check your stick
DOWNLOADS
• Portable Local Studio (the kit): https://github.com/techjarves/Portabl...
• SanDisk Ultra Flair 32 GB: https://www.sandisk.com/products/usb-...
• SanDisk Cruzer Blade 32 GB: https://www.sandisk.com/products/usb-...
• Apple: file system formats in Disk Utility: https://support.apple.com/guide/disk-...
SOURCES
• One owner's Mac run (1 m 34 s load, 74.22 tok/s): https://techtips.work/blog/optimizing...
• Cruzer Blade 32 GB read speeds: https://usb.userbenchmark.com/SpeedTe...
• Ultra Flair 32 GB lab test: https://ssd-tester.com/sandisk_ultra_...
• FAT32's 4 GiB file limit (Microsoft Learn): https://learn.microsoft.com/en-us/win...
• llama.cpp mapped-load speed (issue #9244): https://github.com/ggml-org/llama.cpp...
#LocalAI #Mac #USB