The Complete Hands-On Introduction to Apache Airflow 3
Last updated 6/2025
Duration: 4h 37m | .MP4 1280x720 30 fps(r) | AAC, 44100 Hz, 2ch | 1.36 GB
Genre: eLearning | Language: English
Learn to author, schedule and monitor data pipelines through practical examples using Apache Airflow
What you'll learn
- Create plugins to add functionalities to Apache Airflow.
- Using Docker with Airflow and different executors
- Master core functionalities such as DAGs, Operators, Tasks, Workflows, etc
- Understand and apply advanced concepts of Apache Airflow such as XCOMs, Branching and SubDAGs.
- The difference between Sequential, Local and Celery Executors, how do they work and how can you use them.
- Use Apache Airflow in a Big Data ecosystem with Hive, PostgreSQL, Elasticsearch etc.
- Install and configure Apache Airflow
- Think, answer and implement solutions using Airflow to real data processing problems
Requirements
- At least 8 gigabytes of memory
- Some prior programming or scripting experience. Python experience will help you a lot but since it's a very easy language to learn, it shouldn't be too difficult if you are not familiar with.
Description
Apache Airflow is an open-source platform for programmatically authoring, scheduling, and monitoring workflows. If you have many ETLs to manage, Airflow is a must-have.
In this course, you are going to learn everything you need to start using Apache Airflow 3 through theory and practical videos.
You will start with the basics such as:
What is Apache Airflow?
The core concepts of Airflow
Different architectures to run Airflow
What happens when a workflow runs
Then you will create your first data pipeline covering many Airflow features such as:
Sensors, to wait for specific conditions
Hooks, for interacting with a database
Taskflow, for writing efficient, easy-to-read DAGs
XCOMs, for sharing data
and much more.
At the end of the project, you will be equipped for creating your own workflows!
After the project, you will also discover the new Asset syntax that completely change your way of thinking about your tasks in Airflow 3.
What is an Asset
How to create dependencies between Assets
How to materialize an Asset
and more.
You will discover the different executors for running Airflow at scale. More specifically, the CeleryExecutor which is extremely popular.
How to configure Airflow for using the CeleryExecutor
How to distribute your tasks on different Workers
How to choose your Workers with Queues
and more.
You will explore advanced features to elevate your DAGs to a new level, and conclude by creating your own Airflow provider and a new decorator for executing SQL requests.
If you're working in a company with Airflow, you will love that part.
Enjoy
Who this course is for:
- People being curious about data engineering.
- People who want to learn basic and advanced concepts about Apache Airflow.
- People who like hands-on approach.
Bitte
Anmelden
oder
Registrieren
um Links zu sehen.
Please check out others courses in your favourite language and bookmark them
- - - -
DDownload
Code:
Bitte
Anmelden
oder
Registrieren
um Code Inhalt zu sehen!
Code:
Bitte
Anmelden
oder
Registrieren
um Code Inhalt zu sehen!
Code:
Bitte
Anmelden
oder
Registrieren
um Code Inhalt zu sehen!
Code:
Bitte
Anmelden
oder
Registrieren
um Code Inhalt zu sehen!
Code:
Bitte
Anmelden
oder
Registrieren
um Code Inhalt zu sehen!
Code:
Bitte
Anmelden
oder
Registrieren
um Code Inhalt zu sehen!
