Upgrade environments

Cloud Composer 3  |  Cloud Composer 2 |  Cloud Composer 1

This page describes how to upgrade your environment to a new Cloud Composer or Airflow version.

About upgrade operations

You can upgrade the Cloud Composer version, the Airflow version, or both at the same time.

When you change the version of Airflow or Cloud Composer used by your environment:

• Cloud Composer re-deploys your environment's components using new versions of Cloud Composer images .

• Cloud Composer applies Airflow configuration changes, such as custom PyPI packages or Airflow configuration option overrides, if your environment had them before the upgrade.

• Cloud Composer updates the Airflow airflow_db connection to point to the new Cloud SQL database.

Upgrading doesn't change how you connect to the resources in your environment, such as the URL of your environment's bucket, or Airflow web server.

Limitations of upgrade operations

Upgrade operations have the following limitations:

• Cloud Composer releases are gradually rolled out to all regions supported by Cloud Composer over several days. The latest version from a release in progress might not be available in your region yet.

• You can't downgrade to an earlier Cloud Composer or Airflow version.

• In Google Cloud console, you can upgrade only to the latest supported versions of Cloud Composer 2.

• In Google Cloud CLI, API, or Terraform, you can upgrade an environment to the latest supported versions, three previous versions of Cloud Composer 2, the last patch version of the previous minor version, and to versions with an extended upgrade timeline. It's not possible to upgrade to other versions of Cloud Composer 2, even if they are still supported and can be used to create a new environment.

  Example of upgrade choices for Google Cloud CLI, API, and Terraform:

  • Latest versions: composer-2.11.5-airflow-*
  • Three previous Cloud Composer 2 versions: composer-2.11.4-airflow-* , composer-2.11.3-airflow-* , composer-2.11.2-airflow-* .
  • Last patch of the previous Cloud Composer 2 minor version: composer-2.10.2-airflow-*
  • Cloud Composer 2 versions with an extended upgrade timeline: composer-2.10.2-airflow-* , composer-2.10.1-ariflow-* , composer-2.9.7-airflow-* , and other versions with an extended upgrade timeline.

• The version that you upgrade to must support your environment's current Python version.

• The Compute Engine API quota limits can prevent upgrades. In Google Cloud console, navigate to the IAM and Admin > Quotas and System Limitspage and check if the Compute Engine APIquota for CPU is not exceeded. If the quota threshold is being approached, request Quota extension before proceeding with upgrade operation.

• You can't upgrade your environment if the Airflow database contains more than 16 GB of data. During an upgrade, a warning is shown if the Airflow database size is more than 16 GB. In this case, perform the database maintenance to reduce the database size.

• If you use the XCom mechanism to transfer files, make sure that you use it according to Airflow's guidelines . Transferring big files or a large number of files using XCom impacts Airflow database's performance and can lead to failures when loading snapshots or upgrading your environment. Consider using alternatives such as Cloud Storage to transfer large volumes of data.

Before you begin

• We recommend to create a new snapshot of the environment to be able to re-create the environment in case this is needed.

• Your account must have a role that can trigger environment upgrade operations. In addition, the service account of the environment must have a role that has enough permissions to perform upgrade operations. For more information, see Access control .

• You might notice Compute Engine metadata entries for the GKE clusters that belong to environments in your project. During GKE cluster creation and upgrades, Google Kubernetes Engine automatically adds project-level metadata entries ( google_compute_project_metadata ) to track secondary IP address range usage. Don't modify or remove these entries. Google Kubernetes Engine manages them automatically.

Check that your environment is up to date

Cloud Composer displays warnings when your environment's image approaches its end of full support date. You can use these warnings to always keep your environment supported .

Cloud Composer keeps track of the Cloud Composer image version that your environment is based on. When it approaches the end of support date , you can see a warning in the list of environments and on the Environment detailspage.

To check if your environment is up to date:

View suggested upgrades

Cloud Composer provides a list of Cloud Composer versions that you can upgrade your environment to.

To view Cloud Composer versions suggested for an upgrade:

 gcloud  
composer  
environments  
list-upgrades  
 \ 
  
 ENVIRONMENT_NAME 
  
 \ 
  
--location  
 LOCATION 
 

 gcloud  
composer  
environments  
list-upgrades  
example-environment  
 \ 
  
--location  
us-central1 

  // GET https://composer.googleapis.com/v1/projects/example-project/ 
 // locations/us-central1/imageVersions 
 

Check for PyPI package conflicts before upgrading

You can check if PyPI packages installed in your environment have any conflicts with preinstalled packages in the new Cloud Composer image.

A successful check means that there are no conflicts in PyPI package dependencies between the current and the specified version. However, an upgrade operation might still not be successful because of other reasons.

 gcloud  
composer  
environments  
check-upgrade  
 \ 
  
 ENVIRONMENT_NAME 
  
 \ 
  
--location  
 LOCATION 
  
 \ 
  
--image-version  
 VERSION 
 

 gcloud  
composer  
environments  
check-upgrade  
example-environment  
 \ 
  
--location  
us-central1  
 \ 
  
--image-version  
composer-2.14.1-airflow-2.10.5 

 Waiting for [projects/example-project/locations/us-central1/environments/
example-environment] to be checked for PyPI package conflicts when upgrading
to composer-2.14.1-airflow-2.10.5. Operation [projects/example-project/locations/
us-central1/operations/04d0e8b2-...]...done.
...

Response:
'@type': type.googleapis.com/
google.cloud.orchestration.airflow.service.v1.CheckUpgradeResponse
buildLogUri: https://console.cloud.google.com/cloud-build/builds/79738aa7-... containsPypiModulesConflict: CONFLICTpypiConflictBuildLogExtract: |-
The Cloud Build image build failed: Build failed; check build logs for
details. Full log can be found at https://console.cloud.google.com/
cloud-build/builds/79738aa7-... Error details: tensorboard 2.2.2 has requirement
setuptools>=41.0.0, but you have setuptools 40.3.0. 

  { 
  
 "imageVersion" 
 : 
  
 " VERSION 
" 
 } 
 

Upgrade your environment

To upgrade your environment to a later version of Cloud Composer or Airflow:

 gcloud  
composer  
environments  
update  
 \ 
  
 ENVIRONMENT_NAME 
  
 \ 
  
--location  
 LOCATION 
  
 \ 
  
--image-version  
 VERSION 
 

 gcloud  
composer  
environments  
update  
example-environment  
 \ 
  
--location  
us-central1  
 \ 
  
--image-version  
composer-2.14.1-airflow-2.10.5 

  // PATCH https://composer.googleapis.com/v1/projects/example-project/ 
 // locations/us-central1/environments/example-environment?updateMask= 
 // config.softwareConfig.imageVersion 
  
 { 
  
 "config" 
 : 
  
 { 
  
 "softwareConfig" 
 : 
  
 { 
  
 "imageVersion" 
 : 
  
 "composer-2.14.1-airflow-2.10.5" 
  
 } 
  
 } 
  
 } 
 

   
 resource 
  
 "google_composer_environment" 
  
 "example" 
  
 { 
  
 provider 
  
 = 
  
 google-beta 
  
 name 
  
 = 
  
 " ENVIRONMENT_NAME 
" 
  
 region 
  
 = 
  
 " LOCATION 
" 
  
 config 
  
 { 
  
 software_config 
  
 { 
  
 image_version 
  
 = 
  
 " VERSION 
" 
  
 } 
  
 } 
 } 
 

  resource 
  
 "google_composer_environment" 
  
 "example" 
  
 { 
  
 provider 
  
 = 
  
 google-beta 
  
 name 
  
 = 
  
 "example-environment" 
  
 region 
  
 = 
  
 "us-central1" 
  
 config 
  
 { 
  
 software_config 
  
 { 
  
 image_version 
  
 = 
  
 "composer-2.14.1-airflow-2.10.5" 
  
 } 
  
 } 
 } 
 

What's next

• List of versions
• Cloud Composer versioning
• Update environments