GCP Hybrid cluster settings
This document applies only to Astro Hybrid. To see whether you're an Astro Hybrid user, click the Astronomer logo in the upper left corner of the Cloud UI and go to Settings > General. Your Astro product type is listed under Product Type.
To create a Deployment on Astro Hosted, see Astro resource reference.
Unless otherwise specified, new Clusters on Google Cloud Platform (GCP) are created with a set of default resources that our team has deemed appropriate for most use cases.
Read the following document for a reference of our default resources as well as supported cluster configurations.
Default cluster values
|Resource||Description||Quantity/ Default Size||Configurable|
|GKE Cluster||A GKE cluster is required to run the Astro data plane, which hosts the resources and data required to execute Airflow tasks. Workload Identity is enabled on this cluster.||1x, IP Ranges are |
|Worker node pool||A node pool that hosts all workers with the ||1x pool of e2-standard-4 nodes||Yes. See Manage worker node pools.|
|Airflow node pool||A node pool that runs all core Airflow components, including the scheduler and webserver, for all Deployments in the cluster. This node pool is fully managed by Astronomer.||1x pool of e2-standard-4 nodes|
|Astro system node pool||A node pool that runs all other system components required in Astro. The availability zone determines how many nodes are created. This node pool is fully managed by Astronomer.||1x pool of n2-standard-2 nodes|
|Cloud SQL for PostgreSQL||The Cloud SQL instance is the primary database for the Astro data plane. It hosts the metadata database for each Airflow Deployment hosted on the GKE cluster. All Cloud SQL instances are multi-AZ.||1x regional instance with 4 vCPUs, 16GB memory||Yes. See Configure your relational database.|
|VPC||Virtual private network for hosting GCP resources||1x /19||Yes. See Connect Astro to GCP data sources.|
|Subnet||A single subnet is provisioned in the VPC.||1, IP Range is |
|Service Network Peering||The Astro VPC is peered to the Google Service Networking VPC.||1, IP Range is |
|NAT Router (External)||Required for connectivity with the Astro control plane and other public services||1.|
|Workload Identity Pool||Astro uses the fixed Workload Identity Pool for your cluster. One is created if it does not exist.|
|Google Cloud Storage (GCS) Bucket||Stores Airflow task logs.||1 bucket with name |
Supported cluster regions
You can host Astro Hybrid clusters in the following GCP regions:
|Montreal, North America|
|Toronto, North America|
|Sau Paolo, South America|
|Santiago, South America|
|Iowa, North America|
|South Carolina, North America|
|Virginia, North America|
|Columbus, North America|
|Dallas, North America|
|Oregon, North America|
|Los Angeles, North America|
|Salt Lake City, North America|
|Nevada, North America|
Modifying the region of an existing Astro cluster isn't supported. If you're interested in a GCP region that isn't listed, contact Astronomer support.
Supported Cloud SQL instance types
The following Cloud SQL instance types are supported on Astro:
- Small General Purpose (2 CPU, 8 GiB MEM)
- Medium General Purpose (4 CPU, 16 GiB MEM)
- Small Memory Optimized (2 CPU, 12 GiB MEM)
- Medium Memory Optimized (4 CPU, 24 GiB MEM)
For detailed information about each instance type, see the Cloud SQL documentation. If you're interested in an Cloud SQL instance type that is not on this list, contact Astronomer support.
Supported worker node pool instance types
Each worker node in a pool runs a single worker Pod. A worker Pod's actual available size is equivalent to the total capacity of the instance type minus Astro’s system overhead.
The following table lists all available instance types for worker node pools, as well as the Pod size that is supported for each instance type. As the system requirements of Astro change, these values can increase or decrease.
|Node Instance Type||CPU||Memory|
|e2-standard-4||3 CPUs||13 GiB MEM|
|e2-standard-8||7 CPUs||29 GiB MEM|
|e2-standard-16||15 CPUs||61 GiB MEM|
|e2-highmem-4||3 CPUs||29 GiB MEM|
|e2-highmem-8||7 CPUs||61 GiB MEM|
|e2-highmem-16||15 CPUs||125 GiB MEM|
|e2-highcpu-4||3 CPUs||1 GiB MEM|
|e2-highcpu-8||7 CPUs||5 GiB MEM|
|e2-highcpu-16||15 CPUs||13 GiB MEM|
|n2-standard-4||3 CPUs||13 GiB MEM|
|n2-standard-8||7 CPUs||29 GiB MEM|
|n2-standard-16||15 CPUs||61 GiB MEM|
|n2-highmem-4||3 CPUs||29 GiB MEM|
|n2-highmem-8||7 CPUs||61 GiB MEM|
|n2-highmem-16||15 CPUs||125 GiB MEM|
|n2-highcpu-4||3 CPUs||1 GiB MEM|
|n2-highcpu-8||7 CPUs||5 GiB MEM|
|n2-highcpu-16||15 CPUs||13 GiB MEM|
|c2-standard-4||3 CPUs||13 GiB MEM|
|c2-standard-8||7 CPUs||29 GiB MEM|
If your Organization is interested in using an instance type that supports a larger worker size, contact Astronomer support. For more information about configuring worker size on Astro, see Configure a Deployment.