Upgrade a cluster

This document explains how to upgrade the Kubernetes version of a Google Distributed Cloud (GDC) air-gapped Kubernetes cluster. You must plan routine upgrades for your clusters to ensure they incorporate the latest Kubernetes features and fixes.

This document is for audiences such as IT administrators, security engineers, and network administrators within the platform administrator group, who are responsible for managing Kubernetes resources within their organization. For more information, see Audiences for GDC air-gapped documentation .

Before you begin

• Download and install the gdcloud CLI.

• Install the kubectl CLI. For more information, see Install components .

• Generate a kubeconfig file for the management API server in your targeted zone. For more information, see Zonal Kubernetes cluster resources .

• To get the permissions that you need to list available upgrade versions, ask your Organization IAM Admin to grant you the User Cluster Admin ( user-cluster-admin ) role.

  If you manage a standard cluster and are unable to obtain the User Cluster Admin role from your platform administrator group, you must request they provide you with the list of available Kubernetes upgrade versions.

• To get the permissions that you need to configure a cluster upgrade, ask your Organization IAM Admin to grant you the Standard Cluster Admin ( standard-cluster-admin ) role.

Upgrade a Kubernetes cluster

Use the UserClusterUpgradeRequest API to upgrade an existing Kubernetes cluster.

To upgrade a Kubernetes cluster using the API, follow these steps:

1. List the available Kubernetes upgrade versions by printing the kubernetesVersion value of the UserClusterMetadata objects:

    kubectl  
   --kubeconfig  
    MANAGEMENT_API_SERVER 
     
    \ 
     
   get  
   userclustermetadata  
   -o = 
   custom-columns = 
    'TARGET_VERSION:.spec.kubernetesVersion' 
    
   

   Replace MANAGEMENT_API_SERVER with the path to the kubeconfig file of the management API server.

   The command returns a list of available Kubernetes versions:

    TARGET_VERSION
   1.30.1000-gke.85 
   

2. Note the Kubernetes version from the previous output that you want to upgrade your cluster to. The targetVersion you specify in the upgrade request in the next step must be one of these available versions.

3. Request the cluster upgrade by creating a UserClusterUpgradeRequest resource:

      
    kubectl --kubeconfig MANAGEMENT_API_SERVER 
   apply -f - <<EOF 
     
    apiVersion 
    : 
     
    cluster.gdc.goog/v1 
     
    kind 
    : 
     
    UserClusterUpgradeRequest 
     
    metadata 
    : 
     
    name 
    : 
     
     CLUSTER_NAME 
    
     
    namespace 
    : 
     
     CLUSTER_NAMESPACE 
    
     
    spec 
    : 
     
    clusterRef 
    : 
     
    name 
    : 
     
     CLUSTER_NAME 
    
     
    targetVersion 
    : 
     
     TARGET_VERSION 
    
     
    EOF 
    
   

   Replace the following:

   • MANAGEMENT_API_SERVER : the path to the kubeconfig file of the management API server.
   • CLUSTER_NAME : the name of the Kubernetes cluster to upgrade, such as user-vm-2 .
   • CLUSTER_NAMESPACE : the namespace of the Kubernetes cluster. For shared clusters, use the platform namespace. For standard clusters, use the project namespace of the cluster.
   • TARGET_VERSION : the target version for the upgrade identified in the previous step, such as 1.30.1000-gke.85 .

   The average upgrade time for a Kubernetes cluster takes approximately 20 minutes.

4. Verify the cluster upgrade by monitoring the UserClusterUpgradeRequest resource:

    kubectl  
   --kubeconfig  
    MANAGEMENT_API_SERVER 
     
    \ 
     
   describe  
   userclusterupgraderequest  
    CLUSTER_NAME 
     
    \ 
     
   -n  
    CLUSTER_NAMESPACE 
    
   

   Replace the following:

   • MANAGEMENT_API_SERVER : the path to the kubeconfig file of the management API server.
   • CLUSTER_NAME : the name of the Kubernetes cluster to upgrade, such as user-vm-2 .
   • CLUSTER_NAMESPACE : the namespace of the Kubernetes cluster. For shared clusters, use the platform namespace. For standard clusters, use the project namespace of the cluster.

   Inspect the Spec section of the output. It contains Current Version and Target Version . The upgrade is still in progress if the Current Version and the Target Version differ.

    Spec:
     ...
     Current Version: 1.29.500-gke.60
     Target Version: 1.30.1000-gke.85
   Status:
     Conditions:
     - Message: Upgrade is in progress
       Reason: UpgradeInProgress
       Status: "False"
       Type: Succeeded 
   

   The upgrade is complete and successful when the Succeeded condition has a status of True , and the Current Version matches the Target Version .

   If the Succeeded condition has a status of False , the upgrade failed. If you encounter errors related to your Kubernetes cluster upgrade, contact the infrastructure operator group for further assistance.

Verify cluster subcomponents after upgrade

After your Kubernetes cluster is successfully upgraded, we recommend confirming that all subcomponents of the cluster are healthy.

Check for subcomponent failures in your cluster:

1. Verify that there are no subcomponents that show a ReconciliationError message:

    kubectl  
   --kubeconfig  
    MANAGEMENT_API_SERVER 
     
    \ 
     
   get  
   subcomponent  
   -n  
    CLUSTER_NAME 
     
   -o  
   json  
    | 
     
   jq  
   -r  
    \ 
     
    '.items[] |  select(.status.conditions[]?.reason == "ReconciliationError") | 
    select(.status.featureDisabled != true) |  "Sub-Component: \(.metadata.name) 
    - \(.status.conditions[]?.message)"' 
    
   

   If the output returns any subcomponents, reach out to your infrastructure operator group for further assistance.

2. Verify that there are no subcomponents that show a Reconciling message:

    kubectl  
   --kubeconfig  
    MANAGEMENT_API_SERVER 
     
    \ 
     
   get  
   subcomponent  
   -n  
    CLUSTER_NAME 
     
   -o  
   json  
    | 
     
   jq  
   -r  
    \ 
     
    '.items[] |  select(.status.conditions[]?.reason == "Reconciling") | 
    select(.status.featureDisabled != true) | select( "\(.status)" | 
    contains("PreinstallPending") | not) | "Sub-Component: \(.metadata.name) 
    - \(.status.conditions[]?.message)"' 
    
   

   For Kubernetes clusters with three control plane nodes and three worker nodes, the upgrade time takes approximately 20 minutes. If you still see subcomponents in a state of Reconciling after 20 minutes, contact your infrastructure operator group for further assistance.

What's next

• Design workload separation
• Maintain Kubernetes clusters
• Manage node pools