Monitor instances and operations

Cloud Monitoring automatically collects and stores information about your Managed Lustre instance.

This document provides a detailed overview of the metrics available for monitoring Managed Lustre on Google Cloud. These metrics help you understand the performance, capacity, and health of your Managed Lustre file systems, so you can identify bottlenecks, troubleshoot issues, and optimize resource utilization.

You can use these metrics in Cloud Monitoring to create custom dashboards, set up alerts, and gain deeper insights into your Managed Lustre instance's behavior.

Cloud Monitoring is automatically enabled for Managed Lustre. There's no charge for the collection of data or to view metrics in the Google Cloud console. API calls may incur charges; see Cloud Monitoring pricing for pricing details.

Required IAM roles

The following roles are required:

• Monitoring Viewer( roles/monitoring.viewer ), or equivalent permissions, to view metrics in Cloud Monitoring.
• Monitoring Editor( roles/monitoring.editor ), or equivalent permissions, to configure alerts.

Learn how to grant an IAM role .

View metrics

Cloud Monitoring metrics are available from two locations in the Google Cloud console:

• The Managed Lustre instance details page displays available metrics. In addition to the metrics listed on this page, it computes the bandwidth of bytes copied and the rate of objects copied.

• The Cloud Monitoring page provides multiple chart options and customizations.

View metrics on the instance details page

To view a specific instance's metrics:

1. Go to the Instancespage in the Google Cloud console.

   Go to Instances

2. Click the instance for which to view metrics. The Instance detailspage appears.

3. Click the Monitoringtab. The default dashboard is displayed.

View metrics in Cloud Monitoring

To view Managed Lustre metrics in Cloud Monitoring, do the following:

1. Go to the Metrics Explorerpage in the Google Cloud console.

   Go to Monitoring: Metrics Explorer

2. Follow the instructions in Create charts with Metrics Explorer to select and display your metrics.

Set up alerts

You can configure alerting policies in Cloud Monitoring to notify you when your Managed Lustre file system meets specific conditions, such as exceeding storage capacity or throughput limits.

Prerequisites

To create alerting policies, you must have the Monitoring Editor( roles/monitoring.editor ) IAM role on the project.

Create an alerting policy

To set up an alert, define a condition using a metric or a PromQL query and configure notification channels.

1. In the Google Cloud console, go to the Alertingpage in the Google Cloud console.

   Go to Monitoring: Alerting

2. Click + Create policy.

3. Select Builderand select your metric, or choose Code editorto enter a query with PromQL . In the metric picker, Managed Lustre metrics fall under the Lustre instanceand Lustre locationresources.

4. Configure your trigger logic and define your notification channels and notification settings.

5. Click Create policy.

For more information about creating triggers and other options, see:

• Create metric-threshold alerting policies
• Create metric-absence alerting policies
• Create forecasted metric-value alerting policies

Example: Create a storage capacity alert

The following example demonstrates how to create an alert that triggers when your Managed Lustre instance exceeds 80% of its provisioned capacity.

1. In the Google Cloud console, go to the Alertingpage in the Google Cloud console.

   Go to Monitoring: Alerting

2. Click + Create policy.

3. Select Code editor.

4. In the Query Editor, paste the following PromQL query:

    (
     sum by (instance_id, location) (lustre_googleapis_com:instance_capacity_bytes)
     -
     sum by (instance_id, location) (lustre_googleapis_com:instance_available_bytes)
   )
   /
   sum by (instance_id, location) (lustre_googleapis_com:instance_capacity_bytes)
   > 0.8 
   

   This query calculates the usage ratio across all instances: (Total - Available) / Total . The value 0.8 represents the total bytes reaching 80% usage. To alert at 90%, change this value to 0.9 .

5. Click Run Queryto verify the syntax and view a chart of the current usage ratio.

6. Click Nextand configure the trigger to Any time series violates.

7. Click Next. In the Documentationsection, add recommended actions for resolving the capacity issue. For example:

     ## Action Required: Lustre Capacity Warning 
   The Managed Lustre instance is exceeding 80% capacity usage. **Metric:** 
   Usage Ratio > 0.8 **Severity:** 
   Warning **Recommended Actions:** 
    1. 
   Check the instance details in the Google Cloud console. 2. 
   Verify if this is expected data growth or a runaway process. 3. 
   If valid, consider expanding the storage capacity of the instance or deleting old data to free up space. 4. 
   Failure to address this may result in "No Space Left on Device" errors for client applications. 
   

Create an alerting policy with gcloud

You can create alerting policies using the Google Cloud CLI. Note that you must edit the alert in the Google Cloud console later to enable specific notification channels.

The following example creates an 80% capacity alert using gcloud :

 gcloud  
monitoring  
policies  
create  
 \ 
  
--policy-from-file = 
/dev/stdin  
<<EOF { 
  
 "displayName" 
:  
 "Lustre High Capacity Usage (>80%)" 
,  
 "severity" 
:  
 "WARNING" 
,  
 "combiner" 
:  
 "OR" 
,  
 "conditions" 
:  
 [ 
  
 { 
  
 "displayName" 
:  
 "Capacity Usage Ratio > 0.8" 
,  
 "conditionPrometheusQueryLanguage" 
:  
 { 
  
 "query" 
:  
 "(sum by (instance_id, location) (lustre_googleapis_com:instance_capacity_bytes) - sum by (instance_id, location) (lustre_googleapis_com:instance_available_bytes)) / sum by (instance_id, location) (lustre_googleapis_com:instance_capacity_bytes) > 0.8" 
,  
 "duration" 
:  
 "300s" 
,  
 "evaluationInterval" 
:  
 "60s" 
,  
 "alertRule" 
:  
 "AlwaysOn" 
  
 } 
  
 } 
  
 ] 
,  
 "documentation" 
:  
 { 
  
 "content" 
:  
 "Action Required: The Managed Lustre instance is exceeding 80% capacity usage. Please verify if storage expansion is required." 
,  
 "mimeType" 
:  
 "text/markdown" 
  
 } 
 } 
EOF 

Metric details

Managed Lustre metrics are attached to the following monitored resource types:

• lustre.googleapis.com/Instance
• lustre.googleapis.com/Job
• lustre.googleapis.com/QuotaEntity

Data is sampled every 60 seconds. After sampling, data may not be visible for up to 180 seconds.

Storage capacity metrics

Metrics related to the storage space available and provisioned on your Lustre file system.

For metric labels, the value of target uses the format <fsname>-<TYPE><HEXA> where <HEXA> is the zero-based index of the target in hexadecimal. For example, if your file system name is filesys , the 43rd OST is filesys-OST002a , and the 4th MDT is filesys-MDT0003 .

Storage capacity metrics are attached to the lustre.googleapis.com/Instance resource.

Inode (object) metrics

Metrics related to the number of inodes (objects) available and the maximum capacity.

Inode metrics are attached to the lustre.googleapis.com/Instance resource.

I/O performance metrics

Metrics providing insight into data transfer rates and operation latency.

I/O performance metrics are attached to the lustre.googleapis.com/Instance resource.

Client connection metrics

Metrics specifically for understanding client connectivity.

Client connection metrics are attached to the lustre.googleapis.com/Instance resource.

File system quota metrics

File system quota metrics allow you to monitor storage and inode consumption for specific users, groups, and projects. Use these metrics to track current usage against the soft and hard limits configured on your file system.

File system quota metrics are associated with the lustre.googleapis.com/QuotaEntity monitored resource.

Jobstats metrics

Metrics providing read, write, and metadata statistics per JobID, as configured on the client.

To collect these metrics, use lctl to configure the jobid_var parameter on your Lustre clients. For more information, see Lustre Jobstats .

To configure the client to report a specific identifier (for example, procname_uid ), use the lctl set_param jobid_var command:

 lctl  
set_param  
 jobid_var 
 = 
procname_uid 

Jobstats metrics are attached to the lustre.googleapis.com/Job resource.