Building Google's Time Series AI from Scratch, TimesFM

In this video we build a working Time Series Foundation Model (TimesFM from Google) step by step: patching, RMSNorm, RoPE, causal self-attention, quantile loss, and the full training loop. What we Cover: 1. Binning and quantization for continuous data 2. Why foundation models can forecast data they have never seen (zero-shot) 3. Patching: treating time series like tokens 4. Input normalization that doesn't break on padded data 5. RMSNorm vs LayerNorm — and why the switch matters 6. Rotary Position Embeddings (RoPE) — fast and slow frequencies explained visually 8. Decode-only architecture 8. Quantile (Pinball) Loss and probabilistic forecasting *Timestamps:* 00:00 - Introduction 00:13 - Meet 00:19 - Forecast example 00:53 - Loading data 01:39 - Times series samples 02:20 - Google TimesFM 02:40 - TimesFM output 03:49 - Patching 04:26 - Preprocessing 06:08 - Residual block 07:15 - RMSNorm 09:05 - RoPE 10:40 - Self-Attention 13:15 - Decode layer 15:39 - Stacked decoder 18:01 - Quantile loss 20:40 - Data processing 23:15 - Training dataset 24:23 - Training config 25:32 - Training results 26:23 - Comparing to TimesFM 27:17 - Conclusion *Code & Resources:* 📁 Notebook available on GitHub: https://github.com/ialekseiiv/time-se... 📁 Notebook in Google Colab: https://colab.research.google.com/dri... *Papers Referenced:* Title: A DECODER-ONLY FOUNDATION MODEL FOR TIME-SERIES FORECASTING. Link: https://arxiv.org/pdf/2310.10688 Title: RoFormer: Enhanced Transformer with Rotary Position Embedding. Link: https://arxiv.org/abs/2104.09864 Title: Attention Is All You Need. Link: https://arxiv.org/abs/1706.03762 #MachineLearning #DeepLearning #Transformers #TimesFm #Google #TimeSeries #Forecasting #PyTorch #AI #TutorialFromScratch