Self Attention with torch.nn.MultiheadAttention Module

This video explains how the torch multihead attention module works in Pytorch using a numerical example and also how Pytorch takes care of the dimension. Having a good understanding of the dimension really helps a lot in understanding the how self attention is computed through the key, query and value. torch version is 1.9.0