Skip to main content

Don't miss the biggest Airflow event of the year: Airflow Summit 2023, Sept 19-21

Join Us →

View Deployment logs

View task and component logs for your DAGs to troubleshoot your data pipelines and better understand the behavior of your tasks and their execution environment.

Log and message types

  • Scheduler logs describe the performance of the scheduler, which is responsible for scheduling and queueing task runs. For more information on configuring the scheduler on Astro, see Scheduler resources.

  • Triggerer logs describe the performance of the triggerer, the Airflow component responsible for running triggers and signaling tasks to resume when their conditions have been met. The triggerer is used exclusively for tasks that are run with deferrable operators.

  • Worker logs are generated by Celery Workers. This type of log is not available when using the Kubernetes Exeuctor.

  • Webserver logs relate to the health and performance of the Airflow UI.

Log Levels

Logs and messages might also be associated with one of the following log levels:

  • Error: Emitted when a process fails or does not complete. For example, these logs might indicate a missing DAG file, an issue with your scheduler's connection to the Airflow database, or an irregularity with your scheduler's heartbeat.
  • Warn: Emitted when Airflow detects an issue that may or may not be of concern but does not require immediate action. This often includes deprecation notices marked as DeprecationWarning. For example, Airflow might recommend that you upgrade your Deployment if there was a change to the Airflow database or task execution logic.
  • Info: Emitted frequently by Airflow to show that a standard scheduler process, such as DAG parsing, has started. These logs are frequent and can contain useful information. If you run dynamically generated DAGs, for example, these logs will show how many DAGs were created per DAG file and how long it took the scheduler to parse each of them.

View logs in the Cloud UI

You can access scheduler, triggerer, and task logs in the Cloud UI to find the past 24 hours of logs for any Deployment on its Logs page.

  1. In the Cloud UI, select a Workspace and then a Deployment.

  2. Click the Logs tab.

The maximum number of lines returned is 10,000, with 25 results displayed per page. If there are no logs available for a given Deployment, the following message appears:

No matching events have been recorded in the past 24 hours.

Typically, this indicates that the Deployment you selected does not currently have any DAGs running.

Filter options

You can use the following options to specify the types of logs or messages that you want to view.

  • String search: Enter a string, keyword, or phrase to find in your logs. You can also search with suffix wildcards by adding a * to your search query. For example, acti* returns results that include action and acting. The string search does not include fuzzy matching, so misspelled strings or incomplete strings without a wildcard, *, return zero results.

  • Time range: Filter the logs displayed based on time.

  • Log type: Filter based on whether the log message is from a scheduler, worker, webserver, or trigger.

View Airflow task logs on Astro

Airflow task logs for both local Airflow environments and Deployments on Astro are available in the Airflow UI. Task logs can help you troubleshoot a specific task instance that failed or retried.

On Astro, Airflow task logs are stored in your cluster. On Amazon Web Services (AWS), they are stored in S3. On Google Cloud Platform (GCP), they are stored in Cloud Storage. On Azure, they are stored in Azure Blob Storage.

On clusters hosted in your own cloud, task logs are stored indefinitely. On clusters hosted in Astronomer's cloud, task logs are retained for 90 days. The task log retention policy is not currently configurable.

To access task logs from the Cloud UI:

  1. In the Cloud UI, select a Workspace.
  2. Click DAGs.
  3. Click the DAG you want to view task logs for.
  4. Click a task run in the DAG run grid. The task run's logs appear in the blue window.

To view task logs in the Airflow UI

  1. Access the Airflow UI. To access the Airflow UI for a Deployment, open the Deployment in the Cloud UI and click Open Airflow. To access the Airflow UI in a local environment, open a browser and go to http://localhost:8080.
  2. Click a DAG.
  3. Click Graph.
  4. Click a task run.
  5. Click Instance Details.
  6. Click Log.

View Airflow scheduler logs

You can access the past 24 hours of scheduler logs for any Deployment on its Logs page. Logs are color-coded according to their type. Scheduler logs can help you understand scheduler performance and indicate if a task failed due to an issue with the scheduler. For more information on configuring the scheduler on Astro, see Scheduler resources.

  1. In the Cloud UI, select a Workspace and then a Deployment.

  2. Click the Logs tab.

  3. Optional. Select one or more options in the Log Level menu and click Apply. These are the available options:

    • Error: Emitted when a process fails or does not complete. For example, these logs might indicate a missing DAG file, an issue with your scheduler's connection to the Airflow database, or an irregularity with your scheduler's heartbeat.
    • Warn: Emitted when Airflow detects an issue that may or may not be of concern but does not require immediate action. This often includes deprecation notices marked as DeprecationWarning. For example, Airflow might recommend that you upgrade your Deployment if there was a change to the Airflow database or task execution logic.
    • Info: Emitted frequently by Airflow to show that a standard scheduler process, such as DAG parsing, has started. These logs are frequent and can contain useful information. If you run dynamically generated DAGs, for example, these logs will show how many DAGs were created per DAG file and how long it took the scheduler to parse each of them.

When a Deployment generates more than 500 lines of logs in 24 hours, only the most recent 500 lines are shown. If there are no scheduler logs available for a given Deployment, the following message appears:

No matching events have been recorded in the past 24 hours.

Typically, this indicates that the Deployment you selected does not currently have any DAGs running.

Access Airflow component logs locally

To show logs for your Airflow scheduler, webserver, or triggerer locally, run the following Astro CLI command:

astro dev logs

Once you run this command, the most recent logs for these components appear in your terminal window.

By default, running astro dev logs shows logs for all Airflow components. To see logs only for a specific component, add any of the following flags to your command:

  • --scheduler
  • --webserver
  • --triggerer

To continue monitoring logs, run astro dev logs --follow. The --follow flag ensures that the latest logs continue to appear in your terminal window. For more information about this command, see CLI Command Reference.

Export task logs to Datadog

You can forward Airflow task logs from a Deployment to Datadog using a Datadog API key. Complete the following setup to view Airflow task logs from your Datadog instance.

Prerequisites

Setup

  1. Create a new Datadog API key or copy an existing API key. See API and Application Keys.

  2. Set the following environment variable on your Deployment:

    • Key 1: DATADOG_API_KEY

    • Value 1: Your Datadog API key.

    • Key 2: ASTRO_DATADOG_TASK_LOGS_ENABLED

    • Value 2: true

      Select the Secret? checkbox for DATADOG_API_KEY. This ensures that your Datadog API key is saved securely and is not available to Workspace users in plain text.

    info

    By default, the Astro Datadog integration also sends a Deployment's Airflow metrics to Datadog. To send only task logs to Datadog, additionally set the following environment variable:

    • Key: ASTRO_DATADOG_METRICS_DISABLED
    • Value: true
  3. (Optional) Set the following environment variable on your Deployment to send your metrics to a specific Datadog site:

    • Key: DATADOG_SITE
    • Value: Your Datadog site name. For example, US3.
  4. Click Save variable.

Astro also supports exporting Airflow metrics to Datadog. See Export Airflow metrics to Datadog.

Was this page helpful?

Sign up for Developer Updates

Get a summary of new Astro features once a month.

You can unsubscribe at any time.
By proceeding you agree to our Privacy Policy, our Website Terms and to receive emails from Astronomer.