Make Requests to the Airflow REST API
Overview
This guide explains how to make requests to Airflow's REST API for your Deployments.
You can use Airflow's REST API to automate various Airflow workflows in your Deployments. For example, you can externally trigger a DAG run without accessing your Deployment directly by making an HTTP request in Python or cURL to the corresponding endpoint in the Airflow REST API.
Prerequisites
To make an Airflow API request, you need:
- A Deployment API key.
- A Deployment on Astro.
- cURL.
Step 1: Retrieve an Access Token and Deployment URL
Calling the Airflow REST API for a Deployment requires:
- An Astro access token.
- A Deployment URL.
To retrieve an Astro access token, run the following API request with your Deployment API key ID and secret:
curl --location --request POST "https://auth.astronomer.io/oauth/token" \
--header "content-type: application/json" \
--data-raw '{
"client_id": "<api-key-id>",
"client_secret": "<api-key-secret>",
"audience": "astronomer-ee",
"grant_type": "client_credentials"}'
info
Note that this token is only valid for 24 hours. If you need to call the Airflow API only once, you can retrieve a single 24-hour access token at https://cloud.astronomer.io/token
in the Cloud UI.
If you have an automated CI/CD process configured, we recommend including logic to generate a fresh access token. If you add a step to your CI/CD pipeline that automatically generates an API access token, for example, you can avoid having to generate the token manually.
To retrieve your Deployment URL, open your Deployment in the Cloud UI and click Open Airflow. The Deployment URL is the URL for the Airflow UI homepage up until /home
. It includes the name of your Organization and a short Deployment ID. For example, a Deployment with an ID dhbhijp0
that is part of an Organization called mycompany
would have a Deployment URL of https://mycompany.astronomer.run/dhbhijp0
.
Step 2: Make an Airflow API Request
You can now execute requests against any endpoint that is listed in the Airflow Rest API Reference. To make a request based on Airflow documentation, make sure to:
- Replace
https://airflow.apache.org
with your Deployment URL - Use the Astro access token for authentication
Example API Requests
The following topic contains common examples of API requests that you can run against a Deployment.
List DAGs
To retrieve a list of all DAGs in a Deployment, you can run a GET
request to the dags
endpoint
cURL
curl -X GET <deployment-url>/api/v1/dags \
-H 'Cache-Control: no-cache' \
-H 'Authorization: Bearer <your-access-token>'
Python
import requests
token = "<your-access-token>"
deployment_url = "<your-deployment-url>"
response = requests.get(
url=f"{deployment_url}/api/v1/dags",
headers={"Authorization": f"Bearer {token}"}
)
print(response.json())
# Prints data about all DAGs in your Deployment
Trigger a DAG Run
You can trigger a DAG run by executing a POST
request to Airflow's dagRuns
endpoint.
This will trigger a DAG run for the DAG you specify with a logical_date
value of NOW()
, which is equivalent to clicking the Play button in the main DAGs view of the Airflow UI.
cURL
curl -X POST <your-deployment-url>/api/v1/dags/<your-dag-id>/dagRuns \
-H 'Content-Type: application/json' \
-H 'Cache-Control: no-cache' \
-H 'Authorization: Bearer <your-access-token>' \
-d '{}'
Python
import requests
token = "<your-access-token>"
deployment_url = "<your-deployment-url>"
dag_id = "<your-dag-id>"
response = requests.post(
url=f"{deployment_url}/api/v1/dags/{dag_id}/dagRuns",
headers={
"Authorization": f"Bearer {token}",
"Content-Type": "application/json"
},
data='{}'
)
print(response.json())
# Prints metadata of the DAG Run that was just triggered
Trigger a DAG Run by Date
You can also specify a logical_date
at the time in which you wish to trigger the DAG run by passing the logical_date
with the desired timestamp with the request's data
field. The timestamp string is expressed in UTC and must be specified in the format "YYYY-MM-DDTHH:MM:SSZ"
, where:
YYYY
represents the year.MM
represents the month.DD
represents the day.HH
represents the hour.MM
represents the minute.SS
represents the second.Z
stands for "Zulu" time, which represents UTC.
cURL
curl -v -X POST <your-deployment-url>/api/v1/dags/<your-dag-id>/dagRuns \
-H 'Authorization: Bearer <your-access-token>' \
-H 'Cache-Control: no-cache' \
-H 'content-type: application/json' \
-d '{"logical_date":"2021-11-16T11:34:00Z"}'
Python
Using Python:
import requests
token = "<your-access-token>"
deployment_url = "<your-deployment-url>"
dag_id = "<your-dag-id>"
response = requests.post(
url=f"{deployment_url}/api/v1/dags/{dag_id}/dagRuns",
headers={
"Authorization": f"Bearer {token}",
"Content-Type": "application/json"
},
data='{"logical_date": "2021-11-16T11:34:01Z"}'
)
print(response.json())
# Prints metadata of the DAG Run that was just triggered
Pause a DAG
You can pause a given DAG by executing a PATCH
command against the dag
endpoint.
cURL
curl -X PATCH <your-deployment-url>/api/v1/dags/<your-dag-id> \
-H 'Content-Type: application/json' \
-H 'Cache-Control: no-cache' \
-H 'Authorization: Bearer <your-access-token>' \
-d '{"is_paused": true}'
Python
import requests
token = "<your-access-token>"
deployment_url = "<your-deployment-url>"
dag_id = "<your-dag-id>"
response = requests.patch(
url=f"{deployment_url}/api/v1/dags/{dag_id}",
headers={
"Authorization": f"Bearer {token}",
"Content-Type": "application/json"
},
data='{"is_paused": true}'
)
print(response.json())
# Prints data about the DAG with id <dag-id>