Multi-Head Attention Explained: Why Transformers Use Several Heads

Why does a Transformer run several attention heads instead of one? This beginner-friendly video explains multi-head attention. See why one head gives only one way of looking at a sentence, and how each extra head has its own query, key and value weights. Work through a second head by hand for "The cat sleeps" and see it focus on a different word. Then learn what concatenation means, how the output projection W_O mixes the heads and brings each word back to its original size, and why the output must keep the same shape as the input. Finally, see the real sizes in the original Transformer, eight heads of 64 numbers each, and read the multi-head attention formula symbol by symbol.