When you enable autoscaling, the autoscaler makes scaling decisions based on the options that you specify. Each scaling decision is logged by Cloud Logging . Read these logs in the Logs Explorer to better understand the autoscaler's scaling decisions.
Using the Logs Explorer, you can see events related to:
- Autoscaler recommendation for resizing a managed instance group (MIG). If the autoscaler calculates a new recommended size for a MIG, the change in the recommended size is described in the autoscaler logs. See Resize log entry .
- Changes in autoscaler status. For example, if the autoscaler was working as expected but then encountered an issue, the change in status would be described in the autoscaler logs. See Status change log entry .
Before you begin
- Read the Autoscaling documentation.
- Have an active autoscaler for which you want to view logs.
- Review how to use the Logs Explorer .
- If you haven't already, set up authentication
.
Authentication verifies your identity for access to Google Cloud services and APIs. To run
code or samples from a local development environment, you can authenticate to
Compute Engine by selecting one of the following options:
Select the tab for how you plan to use the samples on this page:
Console
When you use the Google Cloud console to access Google Cloud services and APIs, you don't need to set up authentication.
gcloud
-
Install the Google Cloud CLI. After installation, initialize the Google Cloud CLI by running the following command:
gcloud init
If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity .
- Set a default region and zone .
REST
To use the REST API samples on this page in a local development environment, you use the credentials you provide to the gcloud CLI.
Install the Google Cloud CLI. After installation, initialize the Google Cloud CLI by running the following command:
gcloud init
If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity .
For more information, see Authenticate for using REST in the Google Cloud authentication documentation.
-
View autoscaler logs
To view autoscaler logs, use the Google Cloud console, the gcloud CLI, or REST.
Console
If you've turned on autoscaling, then you can see autoscaling logs in the Google Cloud console .
- Go to the Logs Explorerpage.
- Pull down the Resourcedrop-down menu and select Autoscaler.
You can also navigate to the Logs Explorerpage from your MIG as follows:
-
In the Google Cloud console, go to the Instance groupspage.
-
Click the name of the MIG for which you want to view the logs.
-
Click Monitoringtab.
-
In the Monitoringtab, the Logspanel is at the bottom of the page. Open the panel by clicking the icon at the end of this panel.
-
To open the Logs explorerpage, click the View in logs explorericon in this panel. The log explorer opens with a default query to fetch all autoscaler logs of the MIG.
gcloud
To look up all logs related to autoscaling, use the logging read
command
.
For example:
gcloud logging read "resource.type=autoscaler" --limit 10 \ --format json
To view the autoscaler logs of a specific MIG, specify the instance_group_manager_name
as follows:
gcloud logging read "resource.type=autoscaler AND \ resource.labels.instance_group_manager_name=example-igm " \ --limit 10 --format json
To view the log entries related to autoscaler resize actions
, specify the compute.autoscalers.resize
method
name:
gcloud logging read "resource.type=autoscaler AND \ resource.labels.instance_group_manager_name=example-igm AND \ protoPayload.methodName=compute.autoscalers.resize" \ --limit 10 --format json
To view the log entries related to autoscaler status changes
, specify the compute.autoscalers.changeStatus
method name:
gcloud logging read "resource.type=autoscaler AND \ resource.labels.instance_group_manager_name=example-igm AND \ protoPayload.methodName=compute.autoscalers.changeStatus" \ --limit 10 --format json
REST
Make a request to the Logging V2 API
.
The request body must contain the filter
parameter to use when searching
for logs and the project
you want to view logs for. For example, to make
a request to get a list of resize actions of a specific MIG:
POST https://logging.googleapis.com/v2/entries:list { "filter": "resource.type=autoscaler AND resource.labels.instance_group_manager_name=example-igm AND protoPayload.methodName=compute.autoscalers.resize", "pageSize": 10, "resourceNames": [ "projects/example-project" ] }
To make a request to get a list of status changes of a specific MIG:
POST https://logging.googleapis.com/v2/entries:list { "filter": "resource.type=autoscaler AND resource.labels.instance_group_manager_name=example-igm AND protoPayload.methodName=compute.autoscalers.changeStatus", "pageSize": 10, "resourceNames": [ "projects/example-project" ] }
Resize log entry
When the autoscaler calculates a recommended size for your MIG, Compute Engine creates a log entry capturing the new and old recommended sizes. For zonal MIGs, the log entry also captures the reason for the change in recommended size.
The following is an example of a log entry for a zonal MIG. In this example,
based on predictive autoscaling the autoscaler calculated 3
VMs to achieve the
CPU utilization target of 35%
. But, the maximum number of instances
configured in the autoscaling policy is only 2
. So, the autoscaler limits the
recommended size to 2
.
{ insertId: "1l68z7sg4jw7kzo" logName: "projects/example-project/logs/cloudaudit.googleapis.com%2Fsystem_event" protoPayload: { @type: "type.googleapis.com/google.cloud.audit.AuditLog" metadata: { @type: "type.googleapis.com/autoscaler.AutoscalerSizeChangeExplanation" autoscalingMode: "ON" autoscalingReason : { scalingLimit: { limitPolicy: { maxNumReplicas: 2 } limitSize: 2 name: "MAX_INSTANCES" } scalingSignal: { calculatedSize: 3 calculationDetails: { servingSize: 1 signalTarget: 0.35 signalValue: 1.03533 } name: "PREDICTED_CPU_UTILIZATION" signalPolicy: { coolDownPeriodSec: 15 cpuUtilization: { predictiveMethod: "OPTIMIZE_AVAILABILITY" utilizationTarget: 0.35 } } } summary: "The autoscaler's recommended size changed from 1 to 2 because the autoscaler predicted that in 15 seconds the average CPU utilization across 1 serving instance will be 103.533%, which is above the utilization target of 35%. The calculated size was 3 to achieve the target CPU utilization but the size was limited by the maximum number of instances set in the autoscaling policy." } newSize: 2 oldSize: 1 } methodName: "compute.autoscalers.resize" resourceName: "projects/example-project/zones/us-east1-d/autoscalers/example-autoscaler" serviceName: "compute.googleapis.com" } receiveTimestamp: "2022-06-13T22:34:43.045973046Z" resource: { labels: { autoscaler_id: "1234567890123456789" autoscaler_name: "example-autoscaler" instance_group_manager_id: "1357908642148074125" instance_group_manager_name: "example-igm" location: "us-east1-d" project_id: "example-project" } type: "autoscaler" } severity: "INFO" timestamp: "2022-06-13T22:34:42.810216614Z" }
The following table describes the fields from the log entry:
autoscalingMode
newSize
in the log. Depending on
the autoscaling mode, the autoscaler resizes the MIG as follows: -
ON
: The autoscaler resizes the MIG tonewSize
. -
OFF
: The autoscaler does not resize the MIG. -
ONLY_SCALE_OUT
: The autoscaler resizes the MIG only ifnewSize
is greater than the target size of the MIG.
autoscalingReason
autoscalingReason.scalingLimit
calculatedSize
, which is the size that autoscaler calculated
based on the autoscaling signal. The autoscaling features that can limit
the calculatedSize
are maximum or minimum number of
instances, the scale-in controls
,
or the stabilization period
.
If no limit applies to calculatedSize
, then the scalingLimit
field does not appear in the logs.autoscalingReason.scalingLimit.limitPolicy
calculatedSize
. The limitPolicy
field is not set when stabilization limits the calculatedSize
because stabilization is a
default feature of autoscaling.autoscalingReason.scalingLimit.limitSize
calculatedSize
.autoscalingReason.scalingLimit.name
calculatedSize
.autoscalingReason.scalingSignal
autoscalingReason.scalingSignal.calculatedSize
scalingLimit
applies, then the autoscaler limits calculatedSize
to scalingLimit.limitSize
.autoscalingReason.scalingSignal.calculationDetails
calculatedSize
. -
servingSize
: The number of VMs in the MIG excluding the VMs that are still in their initialization period . -
signalTarget
: The target value that the autoscaler maintains for CPU utilization (real-time), predicted CPU utilization, load balancing utilization, or a metric configured with utilization target. -
singleInstanceAssignment
: If scaling is based on a metric value that represents the total amount of work to be done, thensingleInstanceAssignment
is the quantity of work that each VM can handle. -
signalValue
: The value of the autoscaling signal. IfsignalTarget
is set, then the autoscaler compares thesignalValue
with thesignalTarget
to calculate how many VMs are needed, which is represented bycalculatedSize
.
autoscalingReason.scalingSignal.name
calculatedSize
in the log.autoscalingReason.scalingSignal.signalPolicy
calculatedSize
.autoscalingReason.summary
newSize
autoscalingMode
, the autoscaler resizes the MIG to the newSize
as follows: -
ON
: The autoscaler resizes the MIG tonewSize
. -
OFF
: The autoscaler does not resize the MIG. -
ONLY_SCALE_OUT
: The autoscaler resizes the MIG only ifnewSize
is greater than the target size of the MIG.
oldSize
For information about the other fields in the log entry, see Format of audit log entries .
Status change log entry
When the status of the autoscaler changes, Compute Engine creates a log entry capturing the old and new status messages. You can use these status messages to find out why your autoscaler is not behaving as you expect or to debug past issues you noticed. For some of the common status messages, see Commonly returned status messages .
The following fields in the protoPayload
describe the old and new statuses:
-
metadata.newStatus.details
: The new status of the autoscaler. -
metadata.oldStatus.details
: The previous status of the autoscaler.
The following is an example of a log entry that describes a status change. In this example, you might notice the new status message indicating problems with your load balancing configuration.
{ insertId: "ivho6kg4icqfio" logName: "projects/example-project/logs/cloudaudit.googleapis.com%2Fsystem_event" protoPayload: { metadata: { @type: "type.googleapis.com/autoscaler.AutoscalerStatusChange" newStatus : { details: "The autoscaler is configured to scale based on a load balancing signal but the instance group has not received any utilization data from the load balancer or the utilization is constantly 0. Check that the load balancing configuration is working" } oldStatus : { details: "OK" }, methodName: "compute.autoscalers.changeStatus" resourceName: "projects/example-project/zones/us-east1-b/autoscalers/example-autoscaler" serviceName: "compute.googleapis.com" } receiveTimestamp: "2022-06-08T03:54:12.332329320Z" resource: { labels: { autoscaler_id: "1234567890123456789" autoscaler_name: "example-autoscaler" instance_group_manager_id: "1357908642148074125" instance_group_manager_name: "example-igm" location: "us-east1-b" project_id: "example-project" } type: "autoscaler" } severity: "INFO" timestamp: "2022-06-08T03:54:10.675416812Z" } }
For information about the other fields in the log entry, see Format of audit log entries .
What's next
- Update your autoscaler using information that you received from the logs.
- Learn how autoscaler makes decisions .
- Review autoscaling options .