Astro Python SDK for ETL
Use the Astro Python SDK to implement ELT use cases in Airflow.
Branches
Learn about Airflow’s multiple options for building conditional logic and branching within DAGs, including the BranchPythonOperator and ShortCircuitOperator.
Context
Access the Airflow context in your tasks.
Cross-DAG dependencies
How to implement dependencies between your Airflow DAGs.
Custom hooks and operators
How to correctly import custom hooks and operators.
DAG notifications
Master the basics of Apache Airflow notifications. Learn how to set up automatic email and Slack notifications to be alerted of events in your DAGs.
DAG parameters
Learn about all important DAG-level parameters in Airflow.
DAG writing best practices
Keep up to date with the best practices for developing efficient, secure, and scalable DAGs using Airflow. Learn about DAG design and data orchestration.
Debug DAGs
Troubleshoot Airflow DAGs
Dynamic tasks
How to dynamically create tasks at runtime in your Airflow DAGs.
Jinja templates
Learn about Jinja templating in Apache Airflow and see examples of how to pass dynamic information into task instances at runtime.
Params
Create and use DAG and task-level params in Airflow.
Pass data between tasks
Learn more about the most common methods to implement data sharing between your Airflow tasks, including an in-depth explanation of XCom.
Rerun DAGs and tasks
How to configure retries, catchup, backfill, and clear task instances in Airflow.
SubDAGs
Use SubDAGs to create Airflow modular workflows.
TaskFlow API & decorators
An overview of Airflow decorators and how they can improve the DAG authoring experience.
Task groups
Follow Astronomer’s step-by-step guide to use task groups for organizing tasks within the grid view of the Airflow user interface.