Перейти к содержимому

What Is a Token? How ChatGPT and Gemini Actually Read You #AI #ChatGPT

WatTheAi

0:00 / 0:00

What Is a Token? How ChatGPT and Gemini Actually Read You #AI #ChatGPT

4 просмотра · 6 часов назад
WatTheAi
24 подписчика
4 просмотра · 6 часов назад
ChatGPT has never read a single word you typed. Neither has Gemini. They read tokens: small pieces of words, turned into numbers. In under five minutes, in plain words: what a token is, what really happens when you press send, how much text a token count is, what a "1 million token" context window means, and why tokens decide what you pay. Chapters 0:00 Why ChatGPT never reads your words 0:18 What is a token in AI? 1:12 What happens when you press send in ChatGPT 1:58 How many tokens is a page, a novel, a 1,000-page book? 2:27 What is a context window? (1 million tokens explained) 3:28 Why tokens decide what you pay 3:57 One question, start to finish 4:16 Try it yourself Key numbers 1 token ≈ 4 characters ≈ 3/4 of a word; 100 tokens ≈ 75 words (OpenAI). "What's the best pizza in New York?" = 8 tokens; "pepperoni" = pep · per · oni (OpenAI's open o200k_base tokenizer). Rough sizes (at ~300 words a page): 1 page ≈ 400 tokens · 300-page novel ≈ 120,000 · 1,000-page book ≈ 400,000. Context windows today: GPT-6 models 1.05M tokens, Gemini 1.05M, Claude Opus 5.5 / Sonnet 5 1M. Meta lists Llama 4 Scout at 10M (on paper). In the ChatGPT app, the quick model's window by plan: Free 27K, Plus 54K, Pro 128K tokens. Try it: https://platform.openai.com/tokenizer Sources OpenAI, Understanding and counting tokens: https://help.openai.com/en/articles/4... OpenAI model pages (GPT-6 Astra / Sol / Luna): https://developers.openai.com/api/doc... OpenAI, ChatGPT pricing (context window by plan): https://chatgpt.com/pricing/ Google, Gemini models: https://ai.google.dev/gemini-api/docs... Google, long context: https://ai.google.dev/gemini-api/docs... Anthropic, context windows: https://platform.claude.com/docs/en/b... Anthropic, pricing: https://platform.claude.com/docs/en/a... Meta, Llama 4 model card: https://www.llama.com/docs/model-card... #AI #ChatGPT #Gemini #LLM #AIexplained