Cloud Logging

You can view, search, filter, and archive Dataproc Metastore job logs in Cloud Logging .

Access job logs in Logging

When you run Dataproc Metastore, a Hive metastore process output is streamed to the Google Cloud console.

You can access Logging using the Logging console , the gcloud logging command, or the Logging API .

Console

  1. In the Google Cloud console, go to the Cloud Logging > Logs(Logs Explorer) page:

    Go to the Logs Explorer page

  2. Select an existing Dataproc Metastore Google Cloud project.

  3. Under the Resourcesmenu, select Dataproc Metastore Service .

  4. Under the All logsmenu, select the log type that you want to see.

gcloud

The Google Cloud CLI has a group of commands, gcloud logging , that provide a command-line interface to the Cloud Logging API. To read your log entries, run the following command:

gcloud logging read \
    'resource.type="metastore.googleapis.com/Service"
    resource.labels.location= LOCATION 
resource.labels.service_id= SERVICE_ID 
'

REST

Follow the Logging API instructions to list log entries ( entries.list ) by using the API Explorer.

Dataproc Metastore service logs in Logging

Dataproc Metastore exports request and system logs to Cloud Logging.

Log Type Log Name logName query
Request logs
metastore.googleapis.com/requests logName=(projects/$PROJECT_ID/logs/metastore.googleapis.com%2Frequests)
System logs
metastore.googleapis.com/system logName=(projects/$PROJECT_ID/logs/metastore.googleapis.com%2Fsystem)

Whats next

Create a Mobile Website
View Site in Mobile | Classic
Share by: