電腦如何理解人類語言？ 從 Embedding 說起

大家好，我是 Jim，這裡是 AI Notebook。你拿起手機打開相簿，在搜尋框裡打「海邊的日落」，幾張照片就跳了出來。可是你從來沒有幫這些照片打過任何標籤，也沒有寫過一個字。手機憑什麼知道，那幾張就是海邊的日落？今天我要帶你看懂背後那個負責懂意思的零件，它叫 embedding。你每一次搜尋，每一次收到推薦，每一次 AI 幫你查資料，它都在裡面。而這整件事的起點，是一位語言學家在一九五七年寫下的一句話。 📎 資料來源 • Efficient Estimation of Word Representations in Vector Space (word2vec) https://arxiv.org/abs/1301.3781 • Distributed Representations of Words and Phrases and their Compositionality (NIPS 2013) https://arxiv.org/abs/1310.4546 • NeurIPS 2023 Paper Awards — Test of Time https://blog.neurips.cc/2023/12/11/an... • Fair Is Better than Sensational: Man Is to Doctor as Woman Is to Doctor (CL 2020) https://aclanthology.org/2020.cl-2.7/ • Indexing by Latent Semantic Analysis (LSA, 1990) https://asistdl.onlinelibrary.wiley.c... • A Neural Probabilistic Language Model (JMLR 2003) https://www.jmlr.org/papers/v3/bengio... • GloVe: Global Vectors for Word Representation (Stanford) https://nlp.stanford.edu/projects/glove/ • Deep contextualized word representations (ELMo, NAACL 2018) https://arxiv.org/abs/1802.05365 • BERT (NAACL 2019 Best Long Paper) https://arxiv.org/abs/1810.04805 • NAACL-HLT 2019 Best Papers announcement https://naacl.org/naacl-hlt-2019/blog... • Sentence-BERT (EMNLP 2019) https://arxiv.org/abs/1908.10084 • Learning Transferable Visual Models From Natural Language Supervision (CLIP, ICML 2021) https://proceedings.mlr.press/v139/ra... • MTEB: Massive Text Embedding Benchmark (EACL 2023) https://arxiv.org/abs/2210.07316 • Matryoshka Representation Learning https://arxiv.org/abs/2205.13147 • OpenAI — New embedding models and API updates https://openai.com/index/new-embeddin... • Introducing Voyager, Spotify's new nearest-neighbor search library https://engineering.atspotify.com/202... • Pinecone Series B announcement https://www.pinecone.io/blog/series-b/ • Quote Investigator — You shall know a word by the company it keeps https://quoteinvestigator.com/2022/09... --- Jim AI Notebook | Note the Future 每日深度解析 AI 最新進展 #AI #embedding #word2vec #BERT #RAG #向量資料庫 章節： 00:00 開場：相簿搜尋為什麼找得到 00:52 意思，就是地圖上的座標 02:35 one-hot 的語意沙漠 04:15 分佈假說：看一個詞交什麼朋友 06:10 對的想法，錯的年代 06:47 一台桌機打敗機房 08:30 word to vector 的猜詞遊戲 10:00 king 減 man 加 woman 10:40 誠實面：修過圖的魔法 11:10 一個座標裝不下兩個意思 13:00 從查表變成現場算 15:10 同一張地圖也能放圖片 16:30 RAG：把文件切成卡片 18:00 便宜到能把全部畫進地圖 19:00 維度的俄羅斯娃娃