From Text to Prediction: Token IDs, Embeddings, Training and Inference in an LLM

How does a word become something a machine can process? This 8-minute visual explainer follows one word, "cat", from text to a token ID to an embedding vector. Then it shows how training tunes those numbers and how a trained model predicts the next token. What it covers: How a tokenizer turns text into token IDs, and why an ID is only a label How an ID selects a row in the embedding table, and what that row is Where the starting numbers come from, and how training changes them: prediction, loss, backpropagation and the optimizer What a dimension is, and why a vector can be drawn as a point How an initial embedding becomes a contextual representation How inference predicts the next token: "The cat is an" becomes "The cat is an animal" Chapters 00:00 Introduction 00:33 Text becomes a token ID 01:29 The embedding table 02:26 Where the numbers come from 03:10 Training from prediction errors 04:16 What a dimension is 04:36 The geometry of representations 05:47 From embedding to context 06:28 Predicting the next token 07:40 The whole path Every token ID, vector and probability in this video is invented for teaching. The 3D view is an illustrative projection, not actual model coordinates. Real models use hundreds or thousands of dimensions. Narration: a synthetic voice (Microsoft Zira, Windows text-to-speech). Music: Calm Ambient 1 (Synthwave 4k), The Cynic Project / cynicmusic.com / pixelsphere.org, CC0 (https://opengameart.org/content/calm-ambie...) #LLM #Embeddings #MachineLearning