Scale container resource requests and limits

Console

To view suggested resource requests in the Google Cloud console, you must have an existing workload deployed that is at least 24 hours old. Some suggestions might not be available or relevant for certain workloads, such as those created within the last 24 hours, standalone Pods, and apps written in Java.

1. Go to the Workloadspage in the Google Cloud console.

   Go to Workloads

2. In the workloads list, click the name of the workload you want to scale.

3. Click list Actions > Scale > Edit resource requests.

   The Analyze resource utilization data section shows historic usage data that the Vertical Pod Autoscaler controller analyzed to create the suggested resource requests in the Adjust resource requests and limits section.

Cloud Monitoring

To view suggested resource requests in Cloud Monitoring, you must have an existing workload deployed.

1. Go to the Metrics Explorerpage in the Google Cloud console.

   Go to Metrics Explorer

2. Click Configuration.

3. Expand the Select a Metricmenu.

4. In the Resourcemenu, select Kubernetes Scale.

5. In the Metric categorymenu, select Autoscaler.

6. In the Metricmenu, select Recommended per replicate request bytesand Recommended per replica request core.

7. Click Apply.

gcloud CLI

To view suggested resource requests, you must create a VerticalPodAutoscaler object and a Deployment.

1. For Standard clusters, enable vertical Pod autoscaling for your cluster. For Autopilot clusters, vertical Pod autoscaling is enabled by default.

    gcloud  
   container  
   clusters  
   update  
    CLUSTER_NAME 
     
   --enable-vertical-pod-autoscaling 
   

   Replace CLUSTER_NAME with the name of your cluster.

2. Save the following manifest as my-rec-deployment.yaml :

     apiVersion 
    : 
     
    apps/v1 
    kind 
    : 
     
    Deployment 
    metadata 
    : 
     
    name 
    : 
     
    my-rec-deployment 
    spec 
    : 
     
    replicas 
    : 
     
    2 
     
    selector 
    : 
     
    matchLabels 
    : 
     
    app 
    : 
     
    my-rec-deployment 
     
    template 
    : 
     
    metadata 
    : 
     
    labels 
    : 
     
    app 
    : 
     
    my-rec-deployment 
     
    spec 
    : 
     
    containers 
    : 
     
    - 
     
    name 
    : 
     
    my-rec-container 
     
    image 
    : 
     
    nginx 
    
   

   This manifest describes a Deployment that does not have CPU or memory requests. The containers.name value of my-rec-deployment specifies that all Pods in the Deployment belong to the VerticalPodAutoscaler .

3. Apply the manifest to the cluster:

    kubectl  
   create  
   -f  
   my-rec-deployment.yaml 
   

4. Save the following manifest as my-rec-vpa.yaml :

     apiVersion 
    : 
     
    autoscaling.k8s.io/v1 
    kind 
    : 
     
    VerticalPodAutoscaler 
    metadata 
    : 
     
    name 
    : 
     
    my-rec-vpa 
    spec 
    : 
     
    targetRef 
    : 
     
    apiVersion 
    : 
     
    "apps/v1" 
     
    kind 
    : 
     
    Deployment 
     
    name 
    : 
     
    my-rec-deployment 
     
    updatePolicy 
    : 
     
    updateMode 
    : 
     
    "Off" 
    
   

   This manifest describes a VerticalPodAutoscaler . The updateMode value of Off means that when Pods are created, the Vertical Pod Autoscaler controller analyzes the CPU and memory needs of the containers and records those recommendations in the status field of the resource. The Vertical Pod Autoscaler controller does not automatically update the resource requests for running containers.

5. Apply the manifest to the cluster:

    kubectl  
   create  
   -f  
   my-rec-vpa.yaml 
   

6. After some time, view the VerticalPodAutoscaler :

    kubectl  
   get  
   vpa  
   my-rec-vpa  
   --output  
   yaml 
   

   The output is similar to the following:

    ...
     recommendation:
       containerRecommendations:
       - containerName: my-rec-container
         lowerBound:
           cpu: 25m
           memory: 262144k
         target:
           cpu: 25m
           memory: 262144k
         upperBound:
           cpu: 7931m
           memory: 8291500k
   ... 
   

   This output shows recommendations for CPU and memory requests.

Console

1. Go to the Workloadspage in the Google Cloud console.

   Go to Workloads

2. In the workloads list, click the name of the workload you want to scale.

3. Click list Actions > Scale > Edit resource requests.

   1. The Adjust resource requests and limitssection shows the current CPU and memory requests for each container as well as suggested CPU and memory requests.

4. Click Apply Latest Suggestionsto view suggested requests for each container.

5. Click Save Changes.

6. Click Confirm.

kubectl

Vertically scale your workload with minimal disruption

Preview

This feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms . Pre-GA features are available "as is" and might have limited support. For more information, see the launch stage descriptions .

Starting in Kubernetes version 1.33 , you can use the kubectl patch command to vertically scale your workload by updating the resources that are assigned to a container, without recreating the Pod. For more information, including limitations, see the Kubernetes documentation for resizing CPU and memory resources .

To use the kubectl patch command, specify the updated resource request under the --patch flag. For example, to scale my-app to 800 mCPUs, run the following command:

 kubectl  
patch  
pod  
my-app  
--subresource  
resize  
--patch  
 \ 
  
 '{"spec":{"containers":[{"name":"pause", "resources":{"requests":{"cpu":"800m"}, "limits":{"cpu":"800m"}}}]}}' 
 

Vertically scale your workload

To set resource requests for a Pod, set the requests.cpu and memory.cpu values in your Deployment manifest. In this example, you manually modify the Deployment created in Analyze resource requests with suggested resource requests.

1. Save the following example manifest as my-adjusted-deployment.yaml :

     apiVersion 
    : 
     
    apps/v1 
    kind 
    : 
     
    Deployment 
    metadata 
    : 
     
    name 
    : 
     
    my-rec-deployment 
    spec 
    : 
     
    replicas 
    : 
     
    2 
     
    selector 
    : 
     
    matchLabels 
    : 
     
    app 
    : 
     
    my-rec-deployment 
     
    template 
    : 
     
    metadata 
    : 
     
    labels 
    : 
     
    app 
    : 
     
    my-rec-deployment 
     
    spec 
    : 
     
    containers 
    : 
     
    - 
     
    name 
    : 
     
    my-rec-container 
     
    image 
    : 
     
    nginx 
     
    resources 
    : 
     
    requests 
    : 
     
    cpu 
    : 
     
    25m 
     
    memory 
    : 
     
    256Mi 
    
   

   This manifest describes a Deployment that has two Pods. Each Pod has one container that requests 25 milliCPU and 256 MiB of memory.

2. Apply the manifest to the cluster:

    kubectl  
   apply  
   -f  
   my-adjusted-deployment.yaml 
   

Console

To set resource requests automatically, you must have a cluster with the vertical Pod autoscaling feature enabled. Autopilot clusters have the vertical Pod autoscaling feature enabled by default.

Enable Vertical Pod Autoscaling

1. Go to the Google Kubernetes Enginepage in Google Cloud console.

   Go to Google Kubernetes Engine

2. In the cluster list, click the name of the cluster you want to modify.

3. In the Automationsection, click edit Editfor the Vertical Pod Autoscalingoption.

4. Select the Enable Vertical Pod Autoscalingcheckbox.

5. Click Save changes.

Configure Vertical Pod Autoscaling

1. Go to the Workloadspage in Google Cloud console.

   Go to Workloads

2. In the workloads list, click the name of the Deployment you want to configure vertical Pod autoscaling for.

3. Click list Actions > Autoscale > Vertical pod autoscaling.

4. Choose an autoscaling mode:

   • Auto mode: Vertical Pod autoscaling updates CPU and memory requests during the life of a Pod.
   • Initial mode: Vertical Pod autoscaling assigns resource requests only at Pod creation and never changes them later.

5. (Optional) Set container policies. This option lets you ensure that the recommendation is never set above or below a specified resource request.

   1. Click expand_more Add Policy.
   2. Select Autofor Edit container mode.
   3. In Controlled resources, select which resources you want to autoscale the container on.
   4. Click Add Ruleto set one or more minimum or maximum ranges for the container's resource requests:
      • Min. allowed Memory: the minimum amount of memory that the container should always have, in MiB.
      • Min. allowed CPU: the minimum amount of CPU that the container should always have, in mCPU.
      • Max allowed Memory: the maximum amount of memory that the container should always have, in MiB.
      • Max allowed CPU: the maximum amount of CPU that the container should always have, in mCPU.

6. Click Done.

7. Click Save.

gcloud

To set resource requests automatically, you must use a cluster that has the vertical Pod autoscaling feature enabled. Autopilot clusters have the feature enabled by default.

1. For Standard clusters, enable vertical Pod autoscaling for your cluster:

    gcloud  
   container  
   clusters  
   update  
    CLUSTER_NAME 
     
   --enable-vertical-pod-autoscaling 
   

   Replace CLUSTER_NAME with the name of your cluster.

2. Save the following manifest as my-auto-deployment.yaml :

     apiVersion 
    : 
     
    apps/v1 
    kind 
    : 
     
    Deployment 
    metadata 
    : 
     
    name 
    : 
     
    my-auto-deployment 
    spec 
    : 
     
    replicas 
    : 
     
    2 
     
    selector 
    : 
     
    matchLabels 
    : 
     
    app 
    : 
     
    my-auto-deployment 
     
    template 
    : 
     
    metadata 
    : 
     
    labels 
    : 
     
    app 
    : 
     
    my-auto-deployment 
     
    spec 
    : 
     
    containers 
    : 
     
    - 
     
    name 
    : 
     
    my-container 
     
    image 
    : 
     
    registry.k8s.io/ubuntu-slim:0.14 
     
    resources 
    : 
     
    requests 
    : 
     
    cpu 
    : 
     
    100m 
     
    memory 
    : 
     
    50Mi 
     
    command 
    : 
     
    [ 
    "/bin/sh" 
    ] 
     
    args 
    : 
     
    [ 
    "-c" 
    , 
     
    "while 
     
    true; 
     
    do 
     
    timeout 
     
    0.5s 
     
    yes 
     
   > /dev/null; 
     
    sleep 
     
    0.5s; 
     
    done" 
    ] 
    
   

   This manifest describes a Deployment that has two Pods. Each Pod has one container that requests 100 milliCPU and 50 MiB of memory.

3. Apply the manifest to the cluster:

    kubectl  
   create  
   -f  
   my-auto-deployment.yaml 
   

4. List the running Pods:

    kubectl  
   get  
   pods 
   

   The output shows the names of the Pods in my-deployment :

    NAME                            READY     STATUS             RESTARTS   AGE
   my-auto-deployment-cbcdd49fb-d6bf9   1/1       Running            0          8s
   my-auto-deployment-cbcdd49fb-th288   1/1       Running            0          8s 
   

5. Save the following manifest as my-vpa.yaml :

     apiVersion 
    : 
     
    autoscaling.k8s.io/v1 
    kind 
    : 
     
    VerticalPodAutoscaler 
    metadata 
    : 
     
    name 
    : 
     
    my-vpa 
    spec 
    : 
     
    targetRef 
    : 
     
    apiVersion 
    : 
     
    "apps/v1" 
     
    kind 
    : 
     
    Deployment 
     
    name 
    : 
     
    my-auto-deployment 
     
    updatePolicy 
    : 
     
    updateMode 
    : 
     
    "Auto" 
    
   

   This manifest describes a VerticalPodAutoscaler with the following properties:

   • targetRef.name : specifies that any Pod that is controlled by a Deployment named my-deployment belongs to this VerticalPodAutoscaler .
   • updateMode: Auto : specifies that the Vertical Pod Autoscaler controller can delete a Pod, adjust the CPU and memory requests, and then start a new Pod.

   You can also configure vertical Pod autoscaling to assign resource requests only at Pod creation time, using updateMode: "Initial" .

6. Apply the manifest to the cluster:

    kubectl  
   create  
   -f  
   my-vpa.yaml 
   

7. Wait a few minutes, and view the running Pods again:

    kubectl  
   get  
   pods 
   

   The output shows that the Pod names have changed:

    NAME                                 READY     STATUS             RESTARTS   AGE
   my-auto-deployment-89dc45f48-5bzqp   1/1       Running            0          8s
   my-auto-deployment-89dc45f48-scm66   1/1       Running            0          8s 
   

   If the Pod names have not changed, wait a bit longer, and then view the running Pods again.

Console

To opt out specific containers from vertical Pod autoscaling, you must have a cluster with the vertical Pod autoscaling feature enabled. Autopilot clusters have the vertical Pod autoscaling feature enabled by default.

Enable Vertical Pod Autoscaling

1. Go to the Google Kubernetes Enginepage in Google Cloud console.

   Go to Google Kubernetes Engine

2. In the cluster list, click the name of the cluster you want to modify.

3. In the Automationsection, click edit Editfor the Vertical Pod Autoscalingoption.

4. Select the Enable Vertical Pod Autoscalingcheckbox.

5. Click Save changes.

Configure Vertical Pod Autoscaling

1. Go to the Workloadspage in Google Cloud console.

   Go to Workloads

2. In the workloads list, click the name of the Deployment you want to configure vertical Pod autoscaling for.

3. Click list Actions > Autoscale > Vertical pod autoscaling.

4. Choose an autoscaling mode:

   • Auto mode: Vertical Pod autoscaling updates CPU and memory requests during the life of a Pod.
   • Initial mode: Vertical Pod autoscaling assigns resource requests only at Pod creation and never changes them later.

5. Click expand_more Add Policy.

6. Select the container you want to opt out.

7. For Edit container mode, select Off.

8. Click Done.

9. Click Save.

gcloud

To opt out specific containers from vertical Pod autoscaling, perform the following steps:

1. Save the following manifest as my-opt-vpa.yaml :

     apiVersion 
    : 
     
    autoscaling.k8s.io/v1 
    kind 
    : 
     
    VerticalPodAutoscaler 
    metadata 
    : 
     
    name 
    : 
     
    my-opt-vpa 
    spec 
    : 
     
    targetRef 
    : 
     
    apiVersion 
    : 
     
    "apps/v1" 
     
    kind 
    : 
     
    Deployment 
     
    name 
    : 
     
    my-opt-deployment 
     
    updatePolicy 
    : 
     
    updateMode 
    : 
     
    "Auto" 
     
    resourcePolicy 
    : 
     
    containerPolicies 
    : 
     
    - 
     
    containerName 
    : 
     
    my-opt-sidecar 
     
    mode 
    : 
     
    "Off" 
    
   

   This manifest describes a VerticalPodAutoscaler . The mode: "Off" value turns off recommendations for the container my-opt-sidecar .

2. Apply the manifest to the cluster:

    kubectl  
   apply  
   -f  
   my-opt-vpa.yaml 
   

3. Save the following manifest as my-opt-deployment.yaml :

     apiVersion 
    : 
     
    apps/v1 
    kind 
    : 
     
    Deployment 
    metadata 
    : 
     
    name 
    : 
     
    my-opt-deployment 
    spec 
    : 
     
    replicas 
    : 
     
    1 
     
    selector 
    : 
     
    matchLabels 
    : 
     
    app 
    : 
     
    my-opt-deployment 
     
    template 
    : 
     
    metadata 
    : 
     
    labels 
    : 
     
    app 
    : 
     
    my-opt-deployment 
     
    spec 
    : 
     
    containers 
    : 
     
    - 
     
    name 
    : 
     
    my-opt-container 
     
    image 
    : 
     
    nginx 
     
    - 
     
    name 
    : 
     
    my-opt-sidecar 
     
    image 
    : 
     
    busybox 
     
    command 
    : 
     
    [ 
    "sh" 
    , 
    "-c" 
    , 
    "while 
     
    true; 
     
    do 
     
    echo 
     
    Doing 
     
    sidecar 
     
    stuff!; 
     
    sleep 
     
    60; 
     
    done" 
    ] 
    
   

4. Apply the manifest to the cluster:

    kubectl  
   apply  
   -f  
   my-opt-deployment.yaml 
   

5. After some time, view the Vertical Pod Autoscaler:

    kubectl  
   get  
   vpa  
   my-opt-vpa  
   --output  
   yaml 
   

   The output shows recommendations for CPU and memory requests:

    ...
     recommendation:
       containerRecommendations:
       - containerName: my-opt-container
   ... 
   

   In this output, there are only recommendations for one container. There are no recommendations for my-opt-sidecar .

   The Vertical Pod Autoscaler never updates resources on opted out containers. If you wait a few minutes, the Pod recreates but only one container has updated resource requests.