What's new in Apache Airflow 3?
Data with Marc
0:00 / 0:00
What's new in Apache Airflow 3?
22 109 просмотров · 1 год назад
Data with Marc
41,2 тыс. подписчиков
22 109 просмотров · 1 год назад
What's new in Apache Airflow 3?
This is it! Airflow 3.0 is finally out!
In this video, you will discover:
✅ The new architecture with the API server
✅ The new user interface, powered by React and FastAPI
✅ DAG versioning
✅ Event-driven scheduled data pipelines
✅ The new Asset syntax for writing DAGs
✅ The new backfilling mechanism
and SO much more!
🏆 BECOME A PRO: https://www.udemy.com/course/the-ulti...
👍 Smash the like button to become an Airflow Super Hero!
❤️ Subscribe to my channel to become a master of Airflow
📖 Chapters
00:00 - Airflow 3.0 Has Arrived! 🚀
00:23 - The Completely Rebuilt Modern UI
00:33 - The New Home: Powerful Monitoring at a Glance
01:01 - The New DAGs View: Better Manage your DAGs and Tasks
02:17 - Overview of a DAG With Direct Access to Failed Tasks
03:23 - The New Task Tab: Easier Log Analysis & Troubleshooting
04:19 - Graph View: Visualize Your Workflow Dependencies
04:35 - Trigger your DAGs With Advanced Options
04:51 - Updates on the Pools View with Task Status Tracking
05:01 - Minor Updates on the UI
05:18 - New API Endpoints
05:47 - Visualize the Documentation of your DAGs
06:03 - Assets: The Next Step for your DAGs
07:32 - Implement your First Assets
08:03 - Finally! Some Clarity on the Task Context
08:24 - Visualizing and Managing Assets With the Asset View
09:51 - Producing Multiple Assets at Once
10:21 - Backfill Made Easy: UI & API Support!
12:03 - Fetch DAGs from Various Sources with DAG Bundles Effortlessly
13:43 - DAG Versioning, Never Lose Track Again
14:32 - Event-Driven Scheduling: Real-time Pipeline Triggers
17:49 - New Scheduling Behavior: No More Waiting!
19:25 - New Architecture: Better Security & Scalability
21:25 - Run Task Anywhere With The EdgeExecutor
22:01 - Multiple Schedule Support For Your DAGs
22:20 - Closing Thoughts & Community Thanks
Thank you again for the amazing comitters and anybody who made that possible ❤️