Positional embeddings in transformers EXPLAINED | Demystifying positional encodings.

What are positional embeddings and why do transformers need positional encodings? In this video, we explain why Attention is all you need has these weird sine and cosine embeddings. :) 📺 Follow-up video: Concatenate or add positional encodings? Learned positional embeddings.    • Adding vs. concatenating positional embedd...   ➡️ AI Coffee Break Merch! 🛍️ https://aicoffeebreak.creator-spring.... ► Outline: 00:00 What are positional embeddings? 03:39 Requirements for positional embeddings 04:23 Sines, cosines explained: The original solution from the “Attention is all you need” paper 📺 Transformer explained:    • The Transformer neural network architectur...   ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀ NEW (channel update): 🔥 Optionally, pay us a coffee to boost our Coffee Bean production! ☕ Patreon:   / aicoffeebreak   Ko-fi: https://ko-fi.com/aicoffeebreak ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀ Paper 📄 Vaswani, Ashish, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. "Attention is all you need." In Advances in neural information processing systems, pp. 5998-6008. 2017. https://proceedings.neurips.cc/paper/... ✍️ Arabic Subtitles by Ali Haidar Ahmad   / ali-ahmad-0706a51bb   . Music 🎵 : Discovery Hit by Kevin MacLeod is licensed under a Creative Commons Attribution 4.0 licence. https://creativecommons.org/licenses/... Source: http://incompetech.com/music/royalty-... Artist: http://incompetech.com/ --------------------------- 🔗 Links: AICoffeeBreakQuiz:    / aicoffeebreak   Twitter:   / aicoffeebreak   Reddit:   / aicoffeebreak   YouTube:    / aicoffeebreak   #AICoffeeBreak #MsCoffeeBean #MachineLearning #AI #research​