MiniMax-01: Scaling Foundation Models with Lightning Attention

MiniMax-01: https://arxiv.org/abs/2501.08313 Lightning attention: https://arxiv.org/abs/2405.17381 Lightning attention v2: https://arxiv.org/abs/2401.04658 Lightning attention code: https://github.com/OpenNLPLab/lightni... Huggingface page with models: https://huggingface.co/papers/2501.08313 Notes (lightning attention): https://drive.google.com/file/d/1DMPL... Notes (Minimax): https://drive.google.com/file/d/1DINk... 00:00 Intro 01:30 Linear attention 15:04 Lightning attention 29:11 Lightning attention code and some remarks 34:20 MiniMax