Airflow Quickstart
A great way to explore Airflow features and use cases is using Astronomer's Airflow Quickstart.
This quickstart uses open source tools including Airflow, DuckDB, Streamlit, and the Astro Python SDK to implement an ELT pipeline that creates an interactive dashboard showing global and local temperature changes over time.
You can run the quickstart without installing anything locally by using GitHub Codespaces, or you can run locally using the Astro CLI.
This project is designed to show what Airflow can do and provide a sandbox for you to explore important Airflow concepts. The repository contains one pre-built pipeline that will create and populate the interactive dashboard with no user code changes, as well as several exercises that will enhance the dashboard when completed. For more instructions on using the quickstart, see the project Readme.
If you are new to Airflow and want more instructions for getting started, check out our step-by-step Get Started with Airflow tutorial.
Concepts covered
The pre-built pipeline and exercises in this quickstart show the implementation of several key Airflow features, which you can learn more about in our Learn resources:
- Datasets: This feature helps make Airflow data-aware and expands Airflow scheduling capabilities. Using datasets, you can schedule DAGs to run only when other tasks that update shared data have completed.
- Dynamic task mapping: This feature allows you to write DAGs that dynamically generate parallel tasks at runtime based on the results of previous tasks or external criteria.
- The Astro Python SDK: This open source package built on top of Airflow provides functions and classes that simplify common ELT operations such as loading files or using SQL or Pandas to transform data.