On September 15, 2026 , all Cloud Composer 1 versions and versions 2.0.x of Cloud Composer 2 will reach their planned end of life . You will not be able to use environments with these versions. We recommend planning migration to Cloud Composer 3 . Cloud Composer 2 versions 2.1.x and later are still supported and are not impacted by this change.

Scale environments

Cloud Composer 3 | Cloud Composer 2 | Cloud Composer 1

This page describes how to scale Cloud Composer environments.

For information about how environment scaling works, see Environment scaling .

Scale vertically and horizontally

In Cloud Composer 1, you don't define specific CPU and memory resources for Cloud Composer and Airflow components such as workers and schedulers. Instead, you specify the number and type of machines for nodes in your environment's cluster.

Options for horizontal scaling:

Adjust the number of nodes
Adjust the number of schedulers

Options for vertical scaling:

Adjust the machine type of the Cloud SQL instance
Adjust the web server machine type

Adjust scheduler parameters

Your environment can run more than one Airflow scheduler at the same time. Use multiple schedulers to distribute load between several scheduler instances for better performance and reliability.

If your environment uses Airflow 2, you can specify a number of schedulers up to the number of nodes in your environment.

When scaling schedulers, use the following considerations:

Increasing the number of schedulers doesn't always improve Airflow performance.

For example, this might happen when the extra scheduler isn't utilized, and consumes resources of your environment without contributing to the overall performance. The actual scheduler performance depends on the number of Airflow workers, the number of DAGs and tasks that run in your environment, and the configuration of both Airflow and the environment.
We recommend starting with two schedulers and then monitoring the performance of your environment.

For more information about configuring multiple schedulers, see Airflow documentation .

Console

In the Google Cloud console, go to the Environmentspage.

Go to Environments
In the list of environments, click the name of your environment. The Environment detailspage opens.
Go to the Environment configurationtab.
In the Resources > Workloads configurationitem, click Edit.
In the Resources > Number of schedulersitem, click Edit.
In the Scheduler configurationpane, in the Number of schedulersfield, specify the number of schedulers for your environment.
Click Save.

gcloud

The following Airflow scheduler parameters are available:

--scheduler-count : the number of schedulers in your environment.

Run the following Google Cloud CLI command:

 gcloud  
composer  
environments  
update  
 ENVIRONMENT_NAME 
  
 \ 
  
--location  
 LOCATION 
  
 \ 
  
--scheduler-count  
 SCHEDULER_COUNT

Replace the following:

ENVIRONMENT_NAME : the name of the environment.
LOCATION : the region where the environment is located.

Example:

 gcloud  
composer  
environments  
update  
example-environment  
 \ 
  
--location  
us-central1  
 \ 
  
--scheduler-count  
 2

API

Construct an environments.patch API request.
In this request:
1. In the updateMask parameter, specify the config.workloadsConfig.schedulerCount mask.
2. In the request body, specify the number of schedulers for your environment.

  "config" 
 : 
  
 { 
  
 "workloadsConfig" 
 : 
  
 { 
  
 "scheduler" 
 : 
  
 { 
  
 "count" 
 : 
  
  SCHEDULER_COUNT 
 
  
 } 
  
 } 
 }

Replace the following:

ENVIRONMENT_NAME : the name of the environment.
LOCATION : the region where the environment is located.
SCHEDULER_COUNT : the number of schedulers.

Example:

  // PATCH https://composer.googleapis.com/v1/projects/example-project/ 
 // locations/us-central1/environments/example-environment?updateMask= 
 // config.workloadsConfig.scheduler 
 "config" 
 : 
  
 { 
  
 "workloadsConfig" 
 : 
  
 { 
  
 "scheduler" 
 : 
  
 { 
  
 "count" 
 : 
  
 2 
  
 } 
  
 } 
 }

Terraform

The following fields in the workloads_config.scheduler block control the Airflow scheduler parameters. Each scheduler uses the specified amount of resources.

scheduler.count : the number of schedulers in your environment.

  resource 
  
 "google_composer_environment" 
  
 "example" 
  
 { 
  
 provider 
  
 = 
  
 google-beta 
  
 name 
  
 = 
  
 " ENVIRONMENT_NAME 
" 
  
 region 
  
 = 
  
 " LOCATION 
" 
  
 config 
  
 { 
  
 workloads_config 
  
 { 
  
 scheduler 
  
 { 
  
 count 
  
 = 
  
  SCHEDULER_COUNT 
 
  
 } 
  
 } 
  
 } 
 }

Replace the following:

ENVIRONMENT_NAME : the name of the environment.
LOCATION : the region where the environment is located.
SCHEDULER_COUNT : the number of schedulers.

Example:

  resource 
  
 "google_composer_environment" 
  
 "example" 
  
 { 
  
 provider 
  
 = 
  
 google-beta 
  
 name 
  
 = 
  
 "example-environment" 
  
 region 
  
 = 
  
 "us-central1" 
  
 config 
  
 { 
  
 workloads_config 
  
 { 
  
 scheduler 
  
 { 
  
  
 count 
  
 = 
  
 2 
  
 } 
  
 } 
  
 } 
 }

Adjust the number of nodes

You can change the number of nodes in your environment.

This number corresponds to the number of Airflow workers in your environment. In addition to running Airflow workers, your environment nodes also run Airflow schedulers and other environment components.

Console

In the Google Cloud console, go to the Environmentspage.

Go to Environments
In the list of environments, click the name of your environment. The Environment detailspage opens.
Go to the Environment configurationtab.
In the Worker nodes > Node countitem, click Edit.
In the Worker nodes configurationpane, in the Node countfield, specify the number of nodes in your environment.
Click Save.

gcloud

The --node-count argument controls the number of nodes in your environment:

 gcloud  
composer  
environments  
update  
 ENVIRONMENT_NAME 
  
 \ 
  
--location  
 LOCATION 
  
 \ 
  
--zone  
 NODE_ZONE 
  
 \ 
  
--node-count  
 NODE_COUNT

Replace the following:

ENVIRONMENT_NAME : the name of the environment.
LOCATION : the region where the environment is located.
NODE_COUNT : the number of nodes. The minimum number of nodes is 3 .
NODE_ZONE : the Compute Engine zone for your environment VMs.

Example:

 gcloud  
composer  
environments  
update  
example-environment  
 \ 
  
--location  
us-central1  
 \ 
  
--zone  
us-central1-a  
 \ 
  
--node-count  
 6

API

Create an environments.patch API request.
In this request:
1. In the updateMask parameter, specify the config.nodeCount mask.
2. In the request body, specify the number of nodes for your environment.

   
 "config" 
 : 
  
 { 
  
 "nodeCount" 
 : 
  
  NODE_COUNT 
 
  
 }

Replace the following:

NODE_COUNT : the number of nodes. The minimum number of nodes is 3 .

Example:

  // PATCH https://composer.googleapis.com/v1/projects/example-project/ 
 // locations/us-central1/environments/example-environment?updateMask= 
 // config.nodeCount 
 "config" 
 : 
  
 { 
  
 "nodeCount" 
 : 
  
 6 
 }

Terraform

The node_count field in the node_config block specifies the number of nodes in your environment.

  resource 
  
 "google_composer_environment" 
  
 "example" 
  
 { 
  
 config 
  
 { 
  
 node_config 
  
 { 
  
 node_count 
  
 = 
  
  NODE_COUNT 
 
  
 } 
 }

Replace the following:

NODE_COUNT : the number of nodes.

Example:

  resource 
  
 "google_composer_environment" 
  
 "example" 
  
 { 
  
 name 
  
 = 
  
 "example-environment" 
  
 region 
  
 = 
  
 "us-central1" 
  
 config 
  
 { 
  
 node_config 
  
 { 
  
 node_count 
  
 = 
  
 4 
  
 } 
 }

Adjust the machine type of the Cloud SQL instance

You can change the machine type of the Cloud SQL instance that stores the Airflow database of your environment.

Console

In the Google Cloud console, go to the Environmentspage.

Go to Environments
In the list of environments, click the name of your environment. The Environment detailspage opens.
Go to the Environment configurationtab.
In the Resources > Cloud SQL machine typeitem, click Edit.
In the Cloud SQL configurationpane, in the Cloud SQL machine typedrop-down list, select the machine type for the Cloud SQL instance of your environment.
Click Save.

gcloud

The --cloud-sql-machine-type arguments controls the machine type of the Cloud SQL instance in your environment.

Run the following Google Cloud CLI command:

 gcloud  
composer  
environments  
update  
 ENVIRONMENT_NAME 
  
 \ 
  
--location  
 LOCATION 
  
 \ 
  
--cloud-sql-machine-type  
 SQL_MACHINE_TYPE

Replace the following:

ENVIRONMENT_NAME : the name of the environment.
LOCATION : the region where the environment is located.
SQL_MACHINE_TYPE : the machine type for the Cloud SQL instance .

Example:

 gcloud  
composer  
environments  
update  
example-environment  
 \ 
  
--location  
us-central1  
 \ 
  
--cloud-sql-machine-type  
db-n1-standard-2

API

Create an environments.patch API request.
In this request:
1. In the updateMask parameter, specify the config.databaseConfig.machineType mask.
2. In the request body, specify the machine type for the Cloud SQL instance.

  { 
  
 "config" 
 : 
  
 { 
  
 "databaseConfig" 
 : 
  
 { 
  
 "machineType" 
 : 
  
 " SQL_MACHINE_TYPE 
" 
  
 } 
  
 } 
 }

Replace the following:

SQL_MACHINE_TYPE : the machine type for the Cloud SQL instance .

Example:

  // PATCH https://composer.googleapis.com/v1/projects/example-project/ 
 // locations/us-central1/environments/example-environment?updateMask= 
 // config.databaseConfig.machineType 
 { 
  
 "config" 
 : 
  
 { 
  
 "databaseConfig" 
 : 
  
 { 
  
 "machineType" 
 : 
  
 "db-n1-standard-2" 
  
 } 
  
 } 
 }

Terraform

The machine_type field in the database_config block specifies the machine type for the Cloud SQL instance.

  resource 
  
 "google_composer_environment" 
  
 "example" 
  
 { 
  
 config 
  
 { 
  
 database_config 
  
 { 
  
 machine_type 
  
 = 
  
 " SQL_MACHINE_TYPE 
" 
  
 } 
  
 } 
 }

Replace the following:

SQL_MACHINE_TYPE : the machine type for the Cloud SQL instance .

Example:

  resource 
  
 "google_composer_environment" 
  
 "example" 
  
 { 
  
 name 
  
 = 
  
 "example-environment" 
  
 region 
  
 = 
  
 "us-central1" 
  
 config 
  
 { 
  
 database_config 
  
 { 
  
 machine_type 
  
 = 
  
 "db-n1-standard-2" 
  
 } 
 }

Adjust the web server machine type

You can change the machine type for the Airflow web server of your environment.

Console

In the Google Cloud console, go to the Environmentspage.

Go to Environments
In the list of environments, click the name of your environment. The Environment detailspage opens.
Go to the Environment configurationtab.
In the Resources > Web server machine typeitem, click Edit.
In the Web server configurationpane, in the Web server machine typedrop-down list, select the machine type for the Airflow web server.
Click Save.

gcloud

The --web-server-machine-type arguments controls the machine type of the Airflow web server instance in your environment.

Run the following Google Cloud CLI command:

 gcloud  
composer  
environments  
update  
 ENVIRONMENT_NAME 
  
 \ 
  
--location  
 LOCATION 
  
 \ 
  
--web-server-machine-type  
 WS_MACHINE_TYPE

Replace the following:

ENVIRONMENT_NAME : the name of the environment.
LOCATION : the region where the environment is located.
WS_MACHINE_TYPE : the machine type for the Airflow web server instance .

Example:

 gcloud  
composer  
environments  
update  
example-environment  
 \ 
  
--location  
us-central1  
 \ 
  
--web-server-machine-type  
composer-n1-webserver-2

API

Create an environments.patch API request.
In this request:
1. In the updateMask parameter, specify the config.webServerConfig.machineType mask.
2. In the request body, specify the machine type for the web server.

  { 
  
 "config" 
 : 
  
 { 
  
 "webServerConfig" 
 : 
  
 { 
  
 "machineType" 
 : 
  
 " WS_MACHINE_TYPE 
" 
  
 } 
  
 } 
 }

Replace the following:

WS_MACHINE_TYPE : the machine type for the Airflow web server instance .

Example:

  // PATCH https://composer.googleapis.com/v1/projects/example-project/ 
 // locations/us-central1/environments/example-environment?updateMask= 
 // config.webServerConfig.machineType 
 { 
  
 "config" 
 : 
  
 { 
  
 "webServerConfig" 
 : 
  
 { 
  
 "machineType" 
 : 
  
 "composer-n1-webserver-2" 
  
 } 
  
 } 
 }

Terraform

The machine_type field in the web_server_config block specifies the machine type for the Airflow web server instance.

  resource 
  
 "google_composer_environment" 
  
 "example" 
  
 { 
  
 config 
  
 { 
  
 web_server_config 
  
 { 
  
 machine_type 
  
 = 
  
 " WS_MACHINE_TYPE 
" 
  
 } 
  
 } 
 }

Replace the following:

WS_MACHINE_TYPE : the machine type for the Airflow web server instance .

Example:

  resource 
  
 "google_composer_environment" 
  
 "example" 
  
 { 
  
 name 
  
 = 
  
 "example-environment" 
  
 region 
  
 = 
  
 "us-central1" 
  
 config 
  
 { 
  
 web_server_config 
  
 { 
  
 machine_type 
  
 = 
  
 "composer-n1-webserver-2" 
  
 } 
 }

Scale environments Stay organized with collections Save and categorize content based on your preferences.

Scale vertically and horizontally

Adjust scheduler parameters

Console

gcloud

API

Terraform

Adjust the number of nodes

Console

gcloud

API

Terraform

Adjust the machine type of the Cloud SQL instance

Console

gcloud

API

Terraform

Adjust the web server machine type

Console

gcloud

API

Terraform

What's next

Scale environments