Managed Service for Apache Airflow versioning overview

Composer 3

Google supports an Airflow build for a period of time after it was released:

• 0-12 months from the release date: Managed Airflow (Gen 3) environments running this Airflow build are supported.

• 12+ months from the release date: Managed Airflow (Gen 3) environments running these versions are unsupported.

  After an Airflow build is no longer supported, environments running this Airflow build are also unsupported and entirely user-managed.

Environments work with supported and unsupported Airflow builds in the following way:

• You can create environments with all Airflow builds that are supported.

• You can still use an environment with an Airflow build that is no longer supported. The environment still works, you can keep using it, and you can upgrade it to a supported Airflow build .

• In some cases, an upgrade to a later Airflow version or build is the only available solution for a security vulnerability.

Composer 2

Google supports Managed Airflow (Gen 2) versions for a period of time after the version's release:

• 0-12 months from the release date: Managed Airflow (Gen 2) environments running these versions are supported.

• 12+ months from the release date: Managed Airflow (Gen 2) environments running these versions are unsupported.

You can use Managed Airflow versions after the date they are no longer supported. For example, if your environment is based on a Managed Airflow version that is past the support period, the environment still works and you can keep using it. In this case, we recommend you to upgrade your environment to a supported version .

Composer 1

New Managed Airflow (Legacy Gen 1) versions were released until the end of March 2023. All released Managed Airflow (Legacy Gen 1) versions follow the same version deprecation policy as Managed Airflow (Gen 2) and are supported according to it.

Composer 3

In Managed Airflow (Gen 3), new Airflow versions and builds become available as the time goes.

Your environment can function with all previous Airflow versions and builds, and receives updates to its infrastructure components automatically. Each version and build is supported as described in Version deprecation and support , and you can choose to upgrade your Airflow version to a later version or build.

Composer 2

For a list of Airflow versions supported in each Managed Airflow version, see the list of Managed Airflow versions .

• In each release, Managed Airflow (Gen 2) supports two minor versions of Airflow 2.

  For example, Managed Airflow 2.4.0 supports Airflow 2.4.* and Airflow 2.5.*.

• For each minor version of Airflow 2, Managed Airflow supports one patch version.

  For example, for Airflow 2.4.*, Managed Airflow supports Airflow 2.4.3.

• When a new patch version of Airflow 2 becomes available in Managed Airflow, the corresponding release has two patch versions of Airflow 2 available for the same minor version of Airflow. The previous patch version is then removed in one of the next releases.

  For example, Managed Airflow 2.3.4 supports both Airflow 2.5.1 and Airflow 2.5.3.

• Managed Airflow can skip some Airflow patch versionsbetween releases of Managed Airflow. In addition, if there are functional, quality, or performance issues with a specific patch version of Airflow, Managed Airflow can also skip this version.

  For example, Airflow 2.1.3 is skipped and Airflow 2.1.4 is available.

• A Managed Airflow release running a stable Airflow version can include Airflow updates that are backported from a later Airflow version.

Composer 1

The latest version of Airflow supported in Managed Airflow (Legacy Gen 1) is Airflow 2.4.3.

Only Managed Airflow (Legacy Gen 1) supports Airflow 1.10.15, which is the latest released version of Airflow 1.

Composer 3

Managed Airflow (Gen 3) versioning schema captures Airflow versions and builds:

composer-3-airflow-x.y.z-build.t

Where:

• composer-3 is the major version of Managed Airflow.

• airflow-x.y.z.build.t is the Airflow version and build. Every next release of the same Airflow version in Managed Airflow (Gen 3) has a higher build number. The build number is defined by Managed Airflow and not by the Airflow open source project.

Composer 2

Managed Airflow (Gen 2) image versioning schema captures Managed Airflow and Airflow versions:

composer-2.b.c-airflow-x.y.z

Where:

• composer-2.b.c denotes the minor and patch version of Managed Airflow-managed customizations in Managed Airflow (Gen 2). As a general rule, minor versions of Managed Airflow correspond to significant changes in functionality compared to the previous minor version, such as when the available set of Airflow versions is changed. Patch versions are updated in subsequent releases.

• airflow-x.y.z is the Airflow release, it denotes the major, minor, and patch version of Airflow that's used in a specific Managed Airflow image.

Composer 1

Managed Airflow (Legacy Gen 1) image versioning schema captures Managed Airflow and Airflow versions:

composer-1.b.c-airflow-x.y.z

Where:

• composer-1.b.c is the version of Managed Airflow-managed customizations
• airflow-x.y.z is the Airflow release

Composer 3

• --image-version argument specifies the Airflow version and build in the composer-3-airflow-x.y.z-build.t format. You can use version aliases , which are resolved by Google Cloud CLI into the full format.

• --airflow-version specifies the Airflow version in the following formats:

  • The x.y.z format is for the latest build of the specified version of Airflow. This format is equal to the composer-3-airflow-x.y.z version alias.

  • The x.y format is for the latest build of Airflow available within x.y major and minor versions. This format is equal to the composer-3-airflow-x.y version alias.

  • x.y.z-build.t is the specified version and build of Airflow. This format is equal to composer-3-airflow-x.y.z-build.t .

Composer 2

• --image-version argument specifies Managed Airflow (Gen 2) and Airflow versions:

  • composer-a.b.c-airflow-x.y.z format creates an environment with the specified versions of Managed Airflow and Airflow.
  • Version aliases first resolve to the composer-a.b.c-airflow-x.y.z format and then create an environment with the specified versions of Managed Airflow (Gen 2) and Airflow.

• --airflow-version specifies the Airflow version in the following formats:

  • The x.y.z format is for the latest version of Managed Airflow (Gen 2) and the specified version of Airflow. This format is equal to the composer-2-airflow-x.y.z version alias.

  • The x.y format is for the latest version of Managed Airflow (Gen 2) and the latest version of Airflow available within x.y major and minor versions. This format is equal to the composer-2-airflow-x.y version alias.

Composer 1

• --image-version argument specifies Managed Airflow (Legacy Gen 1) and Airflow versions:

  • composer-a.b.c-airflow-x.y.z format creates an environment with the specified versions of Managed Airflow and Airflow.
  • Version aliases first resolve to the composer-a.b.c-airflow-x.y.z format and then create an environment with the specified versions of Managed Airflow (Legacy Gen 1) and Airflow.

• --airflow-version specifies the Airflow version in the following formats:

  • The x.y.z format is for the latest version of Managed Airflow (Legacy Gen 1) and the specified version of Airflow. This format is equal to the composer-1-airflow-x.y.z version alias.

  • The x.y format is for the latest version of Managed Airflow (Legacy Gen 1) and the latest version of Airflow available within x.y major and minor versions. This format is equal to the composer-1-airflow-x.y version alias.