What Is a Token? How AI Chat Bills Are Counted | Part 1
Step Into
0:00 / 0:00
What Is a Token? How AI Chat Bills Are Counted | Part 1
364 просмотра · 4 дн. назад
Step Into
95 подписчиков
364 просмотра · 4 дн. назад
You ask a chat assistant one question, and a line lands on the bill: 30 input tokens, 309 output tokens. Not words, not letters. This first part of The token economy shows what a token is, using a real tokenizer split (GPT-2's public tokenizer, r50k_base), why the same question costs more in German than in English, why code and emoji count differently, why token counts can't be compared across models, where the count shows up in an API response, and how to count tokens before you send.
Chapters
00:00 One question, one bill line
00:30 Not words, not letters
01:16 Where the pieces come from
01:54 Same question, different price
02:45 Every model, its own dictionary
03:23 Where the count shows up
04:11 Count before you send
Prices as of 2026-10-25 (Claude Opus 5.5: $4 per million input tokens, $20 per million output tokens).
Sources
Anthropic, Using the Messages API (example response and usage block): https://platform.claude.com/docs/en/build-...
Anthropic, Pricing: https://platform.claude.com/docs/en/about-...
Anthropic, Token counting: https://platform.claude.com/docs/en/build-...
OpenAI tiktoken: https://github.com/openai/tiktoken
Radford et al. 2019, Language Models are Unsupervised Multitask Learners (GPT-2): https://cdn.openai.com/better-language-mod...
Next: Part 2, why an output token costs five times an input token. #tokens #LLM #AIpricing