Перейти к содержимому

AI on a $12 USB Stick for Your Mac: 34 Seconds or 3 Minutes

Mind The Prompt

0:00 / 0:00

AI on a $12 USB Stick for Your Mac: 34 Seconds or 3 Minutes

29 просмотров · 1 ч назад
Mind The Prompt
707 подписчиков
29 просмотров · 1 ч назад
A cheap USB stick doesn't make your Mac's AI slow to talk. It makes it slow to start. The same 5 GB model is ready in about 34 seconds from a $17.99 SanDisk Ultra Flair, or about 3 minutes from the $11.99 SanDisk Cruzer Blade, and then both run at your Mac's own speed. I went through the kit from the viral USB AI videos, one owner's Mac log, and the measured read speeds of every stick I could find. In this video: → A $12 stick: 34 seconds or 3 minutes → What the wait costs on your Mac → Why the USB AI kit caught on → Why a slow stick doesn't mean slow answers → One owner's clock: 1:34, then 74 tok/s → Every stick, by read speed → Where the sums are too kind → Offline, after one online start → FAT32: the file that won't copy → The $12 stick vs the $18 one → Check your stick The short version: load time is the model file divided by the stick's read speed. The Cruzer Blade is USB 2.0 and reads about 25–26 MB/s (HWPure lab, UserBenchmark average of 34,316 runs), so a 5 GB model takes 3 min 12 s to 3 min 22 s, nearer 4 on slower copies. The Ultra Flair 32 GB is USB 3.0 and reads 149 MB/s: 34 seconds. $6 more buys back about 2 min 40 s every time you start a model, and nothing once it's running. One owner's M1 MacBook Pro loaded a 9 GB model from a ~98 MB/s Kingston stick in 1 m 34 s, then wrote 74.22 tokens/s. The catch: Portable Local Studio's Mac setup downloads Node.js and llama.cpp the first time it runs (about 60 MB, plus the models), so do the first run at home; sticks of 32 GB or less often come as FAT32, which can't hold a file over 4.29 GB, so erase yours as ExFAT; and eject before you pull it out. Fair warnings: the load times are modelled, not timed by me; the read speeds are other people's benchmarks, mostly on Windows; the 74 tok/s is one owner's run with no internal-drive comparison; mapped loading can be slower than the sum, and nobody has timed it from a stick on a Mac. Use it on a Mac you own or are allowed to use. The voice is a digital one; the research and the editing are mine. WATCH NEXT (generated by `tools/crosslinks.py`) CHAPTERS 0:00 A $12 stick: 34 seconds or 3 minutes 1:02 What the wait costs on your Mac 1:50 Why the USB AI kit caught on 2:43 Why a slow stick doesn't mean slow answers 3:30 One owner's clock: 1:34, then 74 tok/s 4:20 Every stick, by read speed 5:15 Where the sums are too kind 6:20 Offline, after one online start 7:12 FAT32: the file that won't copy 8:07 The $12 stick vs the $18 one 9:11 Check your stick DOWNLOADS • Portable Local Studio (the kit): https://github.com/techjarves/Portabl... • SanDisk Ultra Flair 32 GB: https://www.sandisk.com/products/usb-... • SanDisk Cruzer Blade 32 GB: https://www.sandisk.com/products/usb-... • Apple: file system formats in Disk Utility: https://support.apple.com/guide/disk-... SOURCES • One owner's Mac run (1 m 34 s load, 74.22 tok/s): https://techtips.work/blog/optimizing... • Cruzer Blade 32 GB read speeds: https://usb.userbenchmark.com/SpeedTe... • Ultra Flair 32 GB lab test: https://ssd-tester.com/sandisk_ultra_... • FAT32's 4 GiB file limit (Microsoft Learn): https://learn.microsoft.com/en-us/win... • llama.cpp mapped-load speed (issue #9244): https://github.com/ggml-org/llama.cpp... #LocalAI #Mac #USB