Local AI vs. Cloud APIs: The True Cost & Speed Breakdown

Two machines answer the same question. One sits on your desk, the other in a data centre — and only one of them bills you per token. This is the arithmetic, both ways: what setup actually costs, where the network hides in your wait, what token throughput means when you read the units, and how the two cost curves behave over years rather than afternoons. CHAPTERS 0:00 Two machines 0:24 Setup: the form or the desk 1:09 Latency: where local wins 2:03 Throughput: read the units 2:38 Cost: the meter and the watt 3:41 The two curves 4:35 Which cost you can see SOURCES API pricing (May–October 2026): www.respan.ai/articles/openai-vs-anthropic-pricing-2026 www.morphllm.com/llm-api amnic.com/blogs/llm-cost-comparison Latency (edge TTFB and time to first token, measured November 2026): llmlatency.dev llmlatency.dev/time-to-first-token Local throughput and hardware: www.xda-developers.com/used-rtx-3090-still-best-for-local-ai-in-value/ www.aquanode.io/blog/rtx-5090-vs-rtx-4090 Power and running costs: aifoss.dev/blog/always-on-ai-server-electricity-cost-by-region-2026/ openclawdc.com/blog/undervolt-gpu-24-7-local-agent/ www.indiehackers.com/post/i-ran-the-self-host-vs-managed-math-on-our-5-agents-this-weekend-heres-where-the-crossover-actually-lives-a2998d182f dev.to/shadie_ai/why-i-stopped-self-hosting-ai-models-and-you-probably-should-too-18if Cost crossover and caveats: www.computer.org/csdl/proceedings-article/bigdata/2025/11400957/2eDsQMYBM6Q zenodo.org/records/21195375 CAVEATS Prices and latency move month to month; these are point measurements from public trackers, not a forecast. Throughput figures are single-request generation speed on consumer cards, not a batched cloud fleet. "Local wins on latency" means time to first token, not tokens per second, and the 3090 numbers assume a model small enough to fit on one card. Every drawing, the narration and the animation in this video were generated locally for this channel.