Ep. 1 | AI Can't Read a Single Word. So How Does It Understand You? | AI Fundamentals
李自在AI
0:00 / 0:00
Ep. 1 | AI Can't Read a Single Word. So How Does It Understand You? | AI Fundamentals
10 просмотров · 2 нед. назад
李自在AI
1 подписчик
10 просмотров · 2 нед. назад
Every sentence you type to a chatbot is, to the model, not words at all. It only does math.
This episode follows one sentence on its way into a model: it gets cut into tokens, each token gets an ID, each ID becomes a long list of numbers called a vector, and that vector is a point in a high-dimensional space. Along the way: how subwords and BPE break text into building blocks, why per-token pricing costs more in some languages than others, and how cosine similarity uses direction to tell whether two words are close.
We also look at the famous "king minus man plus woman is roughly queen" example: where it came from, and why it often gets oversold.
By the end, you'll see that what a model calls "meaning" lives in the directions and distances between points, and so does bias.
Chapters:
00:00 Cold open: machines can't read
00:08 Roadmap
00:19 Cutting: tokens and subwords
00:58 How BPE merges
01:42 Vocabulary size trade-offs
02:03 Per-token pricing across languages
02:28 IDs and embeddings
03:13 High-dimensional space
03:33 Coordinates are learned
03:55 Cosine similarity
04:18 King − man + woman ≈ queen
04:37 Why that example gets oversold
05:38 Looking ahead: progress and my take
07:16 Recap
07:32 A question to leave you with, and what's next
#AIFundamentals #LLM #Tokens #Embeddings #AIExplained
The voiceover in this episode is AI-synthesized.