Regularly upgrading your Software Deployments ensures that your Deployments continue to be supported and that your Airflow instance has the latest features and functionality.
To upgrade your Airflow Deployment to a later version of Airflow:
- Select a new Airflow version with the Software UI or CLI to start the upgrade.
- Change the FROM statement in your project's
Dockerfileto reference an Astronomer Certified (AC) or Astro Runtime image that corresponds to your current Airflow version. See Customize Your Image.
- Deploy to Astronomer.
Available Astronomer image versions
A cron job automatically pulls new Astronomer image versions from the Astronomer update service and adds them to the Software UI and CLI within 24 hours of their publication. You don't have to upgrade Astronomer to upgrade Airflow.
If you don't want to wait for new Astronomer image versions, you can manually trigger the cron job with the following Kubernetes command:
kubectl create job --namespace astronomer --from=cronjob/astronomer-houston-update-airflow-check airflow-update-check-first-run
If you get a message indicating that a job already exists, delete the job and rerun the command.
Step 1. Start the upgrade process
Starting the upgrade process doesn't interrupt or otherwise impact your Airflow Deployment. It only signals to Astronomer your intent to upgrade at a later time.
The Software UI and CLI only provide Airflow versions that are later than the version currently running in your
Dockerfile. For example, Airflow
2.1.0 is not available for an Airflow Deployment running
With the Software UI
- Go to Deployment > Settings > Basics > Airflow Version.
- Select an Airflow version.
- Click Upgrade.
With the Astro CLI
$ astro auth login <base-domain>to confirm you're authenticated.
Run the following command to view your Airflow Deployment
astro deployment list
DEPLOYMENT IDvalue and run the following command to list the available Airflow versions:
astro deployment airflow upgrade --deployment-id=<deployment-id>
Enter the Airflow version you want to upgrade to and press
Step 2: Update your Astro project
In your Astro project, open your
FROMline of your project's
Dockerfileto reference a new Astronomer image. For example, to upgrade to the latest version of Astro Runtime, you would change the
For a list of currently supported Astronomer images, see:danger
After you upgrade your Airflow version, you can't revert to an earlier version.
Optional. Test your upgrade on your local machine by running:
astro dev restart
All 4 Airflow Docker containers restart and begin running your new image.
To confirm that your migration was successful, open the Airflow UI at
localhost:8080and scroll to the bottom of any page. You should see your new Runtime version in the footer.
Step 3: Deploy to Astronomer
To push your upgrade to a Deployment on Astronomer Software, run:
Due to a schema change in the Airflow metadata database, upgrading a Software Deployment to AC 2.3.0 can take significant time. Depending on the size of your metadata database, upgrades can take 10 minutes to an hour or longer depending on the number of task instances that have been recorded in the Airflow metadata database. During this time, scheduled tasks continue to execute but new tasks are not scheduled.
To minimize the upgrade time for a Deployment, contact Astronomer support. Minimizing your upgrade time requires removing records from your metadata database.
In the Software UI, open your Deployment and click Open Airflow.
In the Airflow UI, scroll to the bottom of any page. You should see your new Runtime version in the footer.
Cancel Airflow upgrade
You can cancel an Airflow Deployment upgrade at any time if you haven't yet changed the Astronomer image in your
Dockerfile and deployed it.
In the Software UI, select Cancel next to Airflow Version.
Using the Astro CLI, run:
astro deployment airflow upgrade --cancel --deployment-id=<deployment-id>
For example, if you cancel an upgrade from Airflow 2.1.0 to Airflow 2.2.0 in the CLI, the following message appears:
astro deployment airflow upgrade --cancel --deployment-id=ckguogf6x0685ewxtebr4v04x
Airflow upgrade process has been successfully canceled. Your Deployment was not interrupted and you are still running Airflow 2.1.0.
Canceling the Airflow upgrade process does not interrupt or otherwise impact your Airflow Deployment or code that's running.
Upgrade to a hotfix version of Astronomer Certified
To upgrade to the latest hotfix version of Astronomer Certified, replace the image referenced in your
Dockerfile with a pinned version that specifies a particular hotfix. Astro Runtime does not support hotfix versions.
To upgrade to the latest Astronomer Certified 2.3.0 patch fix, for example, you would:
- Check the AC 2.3.0 Changelog.
- Identify the latest patch (e.g.
- Pin the image in your Dockerfile to that patch version. For example,
Note: If you're pushing code to an Airflow Deployment using the Astro CLI and install a new Astronomer Certified image for the first time without pinning a specific hotfix version, the latest available version is automatically pulled.
If a hotfix release becomes available after you've already built an Astronomer Certified image for the first time, subsequent code pushes do not automatically pull the latest corresponding hotfix. Follow the hotfix upgrade process to pin your image to a particular version.