Collect and view cAdvisor/Kubelet metrics

This page describes how to configure a Google Kubernetes Engine (GKE) cluster to send a curated set of cAdvisor/Kubelet metrics to Cloud Monitoring using Google Cloud Managed Service for Prometheus. This page also describes how these metrics are formatted when they are written to Monitoring, and how to query metrics.

The curated set of metrics is a subset of the set of cAdvisor/Kubelet metrics built into every Kubernetes deployment by default and includes metrics related to reducing ingestion volume and associated costs.

Before you begin

Before you start, make sure that you have performed the following tasks:

• Enable the Google Kubernetes Engine API.
Enable Google Kubernetes Engine API
• If you want to use the Google Cloud CLI for this task, install and then initialize the gcloud CLI. If you previously installed the gcloud CLI, get the latest version by running gcloud components update .

Requirements

To collect cAdvisor/Kubelet metrics, your GKE cluster must meet the following requirements:

• The cluster must be running GKE version 1.29.3-gke.1093000 or later. cAdvisor/Kubelet packages are enabled by default when you create a cluster running version 1.29.3-gke.1093000 or later.
• The cluster must have system metrics enabled.
• The cluster has Google Cloud Managed Service for Prometheus managed collection enabled. Google Cloud Managed Service for Prometheus managed collection is enabled by default for new clusters.

Configuring collection of cAdvisor/Kubelet metrics

You can enable cAdvisor/Kubelet metrics in an existing GKE cluster using the Google Cloud console, the gcloud CLI, or Terraform.

 gcloud  
container  
clusters  
update  
 CLUSTER_NAME 
  
 \ 
  
--location = 
 COMPUTE_LOCATION 
  
 \ 
  
--enable-managed-prometheus  
 \ 
  
--monitoring = 
SYSTEM,CADVISOR,KUBELET 

Metric format

All Kubernetes cAdvisor/Kubelet metrics written to Cloud Monitoring use the resource type prometheus_target . Each metric name is prefixed with prometheus.googleapis.com/ and has a suffix indicating the Prometheus metric type, such as /gauge , /histogram , or /counter . Otherwise, each metric name is identical to the metric name exposed by open source Kubernetes.

Exporting from Cloud Monitoring

The cAdvisor/Kubelet metrics can be exported from Cloud Monitoring by using the Cloud Monitoring API . Because all cAdvisor/Kubelet metrics are ingested by using Google Cloud Managed Service for Prometheus , cAdvisor/Kubelet metrics can be queried by using Prometheus Query Language (PromQL) . They can also be queried by using by using Monitoring Query Language (MQL) .

Querying metrics

When you query cAdvisor/Kubelet metrics, the name you use depends on whether you are using PromQL or Cloud Monitoring-based features like MQL or the Metrics Explorer menu-driven interface .

The following tables of cAdvisor/Kubelet metrics show two versions of each metric name:

• PromQL metric name : When using PromQL in Cloud Monitoring pages of the Google Cloud console or in PromQL fields of the Cloud Monitoring API , use the PromQL metric name.
• Cloud Monitoring metric name When using other Cloud Monitoring features, use the Cloud Monitoring metric name in the tables below. This name must be prefixed with prometheus.googleapis.com/ , which has been omitted from the entries in the table.

cAdvisor metrics

The Cloud Monitoring metric names in this table must be prefixed with prometheus.googleapis.com/ . That prefix has been omitted from the entries in the table.

Kubelet metrics

The Cloud Monitoring metric names in this table must be prefixed with prometheus.googleapis.com/ . That prefix has been omitted from the entries in the table.