Cloud Monitoring provides visibility into the performance, uptime, and overall health of cloud-powered applications. Google Cloud Observability collects and ingests metrics, events, and metadata from Dataproc Metastore services to generate insights using dashboards and charts.
-
See Monitoring Pricing to understand your costs.
-
See Monitoring Quotas and limits for information on metric data retention.
Access service metrics in Monitoring
Dataproc Metastore service resource metrics are automatically enabled on Dataproc Metastore services. Use Monitoring to view these metrics.
You can access Monitoring from the Google Cloud console or by using the Monitoring API .
Console
-
In the Google Cloud console, go to the Cloud Monitoring > Metrics explorerpage:
-
From the "Find resource type and metric" drop-down list, select the resource
Dataproc Metastore Service. -
Click again in the input box, and then select a metric from the drop-down list. Hovering over the metric name displays information about the metric.
You can select filters, group by metric labels, perform aggregations, and select chart viewing options.
REST
You can use the Monitoring timeSeries.list
API to capture and list metrics defined by a filter
expression.
Use the Try this APItemplate on the API page to send
an API request and display the response.
Dataproc Metastore service metrics in Monitoring
Metrics from Dataproc Metastore. Launch stages of these metrics: BETA GA
The "metric type" strings in this table must be prefixed
with metastore.googleapis.com/
. That prefix has been
omitted from the entries in the table.
When querying a label, use the metric.labels.
prefix; for
example, metric.labels. LABEL
=" VALUE
"
.
Display name
Monitored resources
Labels
method_name
:
The request method name.status
:
End status of the request, one of [success, client_error, server_error].version
:
The Hive Metastore version of the service.tier
:
Service tier or instance size of the service, such as ENTERPRISE.version
:
The Hive Metastore version of the service.status
:
The status of the request, such as `BAD_REQUEST`.method_name
:
The request method name.tier
:
Service tier or instance size of the service.version
:
The Hive Metastore version of the service.tier
:
Service tier or instance size of the service.version
:
The Hive Metastore version of the service.status
:
The status of the request, such as `BAD_REQUEST`.method_name
:
The request method name.tier
:
Service tier or instance size of the service.version
:
The Hive Metastore version of the service.tier
:
Service tier or instance size of the service.version
:
The Hive Metastore version of the service.method_name
:
The request method name.tier
:
Service tier or instance size of the service, such as ENTERPRISE.version
:
The Hive Metastore version of the service.release_tag
:
Dataproc Metastore release tag, populated by the service backend and may change over service lifetime.version
:
The Hive Metastore version of the service.release_tag
:
Dataproc Metastore release tag, populated by the service backend and may change over service lifetime.service_region
:
The constituent region that Dataproc Metastore service is hosted in.scaling_factor
:
Scaling factor of the service, one of [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0].method_name
:
The request method name.version
:
The Hive Metastore version of the service.service_region
:
The constituent region that Dataproc Metastore service is hosted in.scaling_factor
:
Scaling factor of the service, one of [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0].method_name
:
The request method name.version
:
The Hive Metastore version of the service.service_region
:
The constituent region that Dataproc Metastore service is hosted in.scaling_factor
:
Scaling factor of the service, one of [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0].database_type
:
The type of database, one of [SQL, Spanner].code
:
The gRPC status code.database_type
:
The type of database, one of [SQL, Spanner].code
:
The gRPC status code.method_name
:
The request method name.version
:
The Hive Metastore version of the service.state
:
State of the backup, one of [SUCCEEDED, FAILED, IN_PROGRESS].database_type
:
The type of database, one of [SQL, Spanner].version
:
The Hive Metastore version of the service.state
:
State of the backup, one of [SUCCEEDED, FAILED, IN_PROGRESS].database_type
:
The type of database, one of [SQL, Spanner].version
:
The Hive Metastore version of the service.state
:
State of the backup, one of [SUCCEEDED, FAILED, IN_PROGRESS].database_type
:
The type of database, one of [SQL, Spanner].version
:
The Hive Metastore version of the service.Table generated at 2024-06-27 18:22:22 UTC.
Build a custom Monitoring dashboard
You can build a custom Monitoring dashboard that displays charts of selected Dataproc Metastore service metrics.
-
Select + CREATE DASHBOARDfrom the Monitoring Dashboards Overview page.
-
Provide a name for the dashboard.
-
In the upper-right menu, click Add Chartto open the Add Chart window.
-
For the resource type, select
Dataproc Metastore Service. -
Select one or more metrics and metric and chart properties, and then click Saveto save the chart.
-
Add additional charts to your dashboard if you'd like.
-
To save the dashboard, click Save. Its title appears in the Monitoring Dashboards Overviewpage.
You can view, update, and delete dashboard charts from the dashboard display page.
Use Monitoring alerts
You can create a Monitoring alert that notifies you when a Dataproc Metastore service or job metric crosses a specified threshold.
Create an alert
-
In the Google Cloud console, open Monitoring Alerting .
-
To open the Create new alerting policyform, click + CREATE POLICY
-
Define an alert by adding alert conditions, policy triggers, notification channels, and documentation.
-
Select ADD CONDITIONto open the alert condition form with the Metric tab selected.
-
Fill in the fields to define an alert condition, and then click ADD.
-
After adding the alert condition, complete the alert policy by setting notification channels, policy triggers, documentation, and the alert policy name.
View alerts
When an alert is triggered by a metric threshold condition, Monitoring creates an incident and a corresponding event. You can view incidents from the Monitoring Alerting page in the Google Cloud console. If you defined a notification mechanism in the alert policy, such as an email or SMS notification, Monitoring also sends a notification of the incident.
