Why One Sentence Became 11 Tokens in English and 81 in Hindi

The text appearing in an AI chat isn't just a typing animation. You're watching an answer being built, one token at a time. But what is a token? I ran an English sentence and its Hindi equivalent through five tokenizers. On GPT-2, the English version took 11 tokens. The Hindi version took 81. Same meaning, very different counts. In this lesson, we unpack why that happens, then follow the loop behind an LLM's response: predict, pick, append, repeat. You'll see how streaming works, why an AI correction doesn't erase its earlier output, how longer answers affect cost, and four ways generation can stop. This is episode 2 of the AI Engineering course. No coding experience needed. CHAPTERS 00:00 That typing effect is the model working 00:52 What a token actually is 04:47 Predict, pick, append, repeat 05:41 Why AI responses stream 07:42 Why the model can't take back its words 09:19 Why longer answers cost more and can drift 11:03 Four ways generation stops 13:29 Try it yourself: compare token counts CONTINUE LEARNING [AI Engineering course and community](https://www.skool.com/the-rabbit-hole...) [Watch episode 1: What an LLM actually is](   • The Only Video You Need to Understand LLMs  ) TRY THIS Count the tokens in a paragraph you normally write, then compare an English version using the same tokenizer. Share your language, tokenizer and both counts in the comments. NOTES The 11-versus-81 comparison is specific to this sentence and GPT-2, not a universal difference between English and Hindi. Pricing shown is dated 15 September 2026. The discussion of drift explains a possible mechanism, not a measured error rate. #AIEngineering #LLM #Tokenization