View Vertex AI Inference DCGM metrics

This page covers how to explore NVIDIA Data Center GPU Manager (DCGM) metrics associated with your Vertex AI Inference endpoints.

What is DCGM

NVIDIA Data Center GPU Manager (DCGM) is a set of tools from NVIDIA that let you manage and monitor NVIDIA GPUs. Vertex AI Inference automatically exports Vertex AI DCGM metrics to Cloud Monitoring if your endpoints utilize supported GPUs . Those metrics provide a comprehensive view of GPU utilization, performance, and health.

Prerequisites

Before you start, make sure your project has enabled Cloud Monitoring. See Enable the Monitoring API for more information.

Use DCGM metrics

To view DCGM metrics on Metrics Explorer, do the following:

1. Go to the Metrics Explorerpage in the Google Cloud console.

   Go to Metrics Explorer

2. Under Select a metric, select Prometheus Target.

3. Under Active metric categories, select Vertex.

4. Under Active metrics, select the desired metric.

5. Click Apply.

You can also query metrics using Grafana , or Prometheus API or UI .

Quota

DCGM metrics consume the Time series ingestion requests per minutequota of the Cloud Monitoring API. Before enabling the metrics packages, check your recent peak usage of that quota. If you are already approaching that quota limit, you can request a quota-limit increase .

Vertex AI DCGM metrics

The Cloud Monitoring metric names in this table must be prefixed with prometheus.googleapis.com/ . That prefix has been omitted from the entries in the table.

Along with labels on the prometheus_target monitored resource, all collected DCGM metrics on Vertex AI have the following labels attached to them:

GPU labels:

• gpu_model : the GPU device model, such as NVIDIA L4 .
• gpu_uuid : the GPU device UUID.
• gpu_i_id : the NVIDIA Multi-Instance GPU (MIG) instance ID.

Vertex AI labels:

• deployed_model_id : the ID of a deployed model which serves inference requests.
• model_display_name : the display name of a deployed model.
• replica_id : the unique ID corresponding to the deployed model replica (pod name).
• endpoint_id : the ID of a model endpoint.
• endpoint_display_name : the display name of a model endpoint.
• product : the name of the feature under Vertex AI. This is always Online Inference .

Supported GPUs

All NVIDIA GPUs are supported, except the following, due to resource constraints:

1. NVIDIA P100
2. NVIDIA V100
3. NVIDIA P4
4. NVIDIA T4

What's next

• Learn more about the Metrics Explorer .