Version 1.10. This version is no longer supported. For information about how to upgrade to version 1.11, seeUpgrading Anthos on bare metalin the 1.11 documentation. For more information about supported and unsupported versions, see theVersion historypage in the latest documentation.
This page shows how to configure a cluster for Google Distributed Cloud so that
custom logs and metrics from user applications are sent to Cloud Logging and
Cloud Monitoring.
Enabling Logging and Monitoring for user applications
ADMIN_KUBECONFIG: the path to the admin cluster
kubeconfig file.
Annotate workloads
To enable the collection of custom metrics from an application, add theprometheus.io/scrape: "true"annotation to the application's Service or
Deployment manifest.
Run an example application
In this section, you create an application that writes custom logs and exposes
a custom metric.
Save the following Service and Deployment manifests to a file namedmy-app.yaml. Notice that the Service has the annotationprometheus.io/scrape: "true":
UnderCLUSTER_NAME, select the name of your user cluster.
UnderNAMESPACE_NAME, selectdefault.
ClickAddand then clickRun Query.
UnderQuery results, you can see log entries from themonitoring-exampleDeployment. For example:
{"textPayload":"2020/11/14 01:24:24 Starting to listen on :9090\n","insertId":"1oa4vhg3qfxidt","resource":{"type":"k8s_container","labels":{"pod_name":"monitoring-example-7685d96496-xqfsf","cluster_name":...,"namespace_name":"default","project_id":...,"location":"us-west1","container_name":"prometheus-example-exporter"}},"timestamp":"2020-11-14T01:24:24.358600252Z","labels":{"k8s-pod/pod-template-hash":"7685d96496","k8s-pod/app":"monitoring-example"},"logName":"projects/.../logs/stdout","receiveTimestamp":"2020-11-14T01:24:39.562864735Z"}
gcloud
Run this command:
gcloudloggingread'resource.labels.project_id="PROJECT_ID" AND \resource.type="k8s_container" AND resource.labels.namespace_name="default"'
ReplacePROJECT_IDwith the ID
of your project.
In the output, you can see log entries from themonitoring-exampleDeployment. For example:
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-09-04 UTC."],[[["\u003cp\u003eThis page provides instructions on configuring a Google Distributed Cloud cluster to send custom logs and metrics from user applications to Cloud Logging and Cloud Monitoring.\u003c/p\u003e\n"],["\u003cp\u003eTo enable logging and monitoring for applications, the \u003ccode\u003eenableApplication\u003c/code\u003e field in the cluster configuration file must be set to \u003ccode\u003etrue\u003c/code\u003e, and the changes are applied using \u003ccode\u003ebmctl update\u003c/code\u003e.\u003c/p\u003e\n"],["\u003cp\u003eCustom metrics collection requires adding the \u003ccode\u003eprometheus.io/scrape: "true"\u003c/code\u003e annotation to the application's Service or Deployment manifest.\u003c/p\u003e\n"],["\u003cp\u003eApplication logs can be viewed in the Google Cloud console's Logs explorer by selecting \u003ccode\u003eKubernetes Container\u003c/code\u003e as the resource type and filtering by the cluster name and namespace, or by using \u003ccode\u003egcloud logging read\u003c/code\u003e.\u003c/p\u003e\n"],["\u003cp\u003eApplication custom metrics, like the example \u003ccode\u003eexample_monitoring_up\u003c/code\u003e, can be viewed in the Google Cloud console's Metrics explorer by selecting \u003ccode\u003eKubernetes Pod\u003c/code\u003e as the resource type and the metric name.\u003c/p\u003e\n"]]],[],null,["# Enable application logging and monitoring\n\n\u003cbr /\u003e\n\nThis page shows how to configure a cluster for Google Distributed Cloud so that\ncustom logs and metrics from user applications are sent to Cloud Logging and\nCloud Monitoring.\n\nEnabling Logging and Monitoring for user applications\n-----------------------------------------------------\n\nTo enable Logging and Monitoring for your\napplications, use the\n[`spec.clusterOperations.enableApplication`](/anthos/clusters/docs/bare-metal/1.10/reference/cluster-config-ref#clusteroperations-enableapplication)\nfield in the cluster configuration file.\n\n1. Update the cluster configuration file to set `enableApplication` to `true`:\n\n apiVersion: v1\n kind: Namespace\n metadata:\n name: cluster-user-basic\n ---\n apiVersion: baremetal.cluster.gke.io/v1\n kind: Cluster\n metadata:\n name: user-basic\n namespace: cluster-user-basic\n spec:\n type: user\n ...\n clusterOperations:\n projectID: project-fleet\n location: us-central1enableApplication: true\n ...\n\n2. Use `bmctl update` to apply your changes:\n\n bmctl update cluster -c \u003cvar translate=\"no\"\u003eCLUSTER_NAME\u003c/var\u003e --admin-kubeconfig=\u003cvar translate=\"no\"\u003eADMIN_KUBECONFIG\u003c/var\u003e\n\n Replace the following:\n - \u003cvar translate=\"no\"\u003eCLUSTER_NAME\u003c/var\u003e: the name of the cluster to upgrade.\n - \u003cvar translate=\"no\"\u003eADMIN_KUBECONFIG\u003c/var\u003e: the path to the admin cluster kubeconfig file.\n\nAnnotate workloads\n------------------\n\nTo enable the collection of custom metrics from an application, add the\n`prometheus.io/scrape: \"true\"` annotation to the application's Service or\nDeployment manifest.\n\nRun an example application\n--------------------------\n\nIn this section, you create an application that writes custom logs and exposes\na custom metric.\n\n1. Save the following Service and Deployment manifests to a file named\n `my-app.yaml`. Notice that the Service has the annotation\n `prometheus.io/scrape: \"true\"`:\n\n kind: Service\n apiVersion: v1\n metadata:\n name: \"monitoring-example\"\n namespace: \"default\"\n annotations:\n prometheus.io/scrape: \"true\"\n spec:\n selector:\n app: \"monitoring-example\"\n ports:\n - name: http\n port: 9090\n ---\n apiVersion: apps/v1\n kind: Deployment\n metadata:\n name: \"monitoring-example\"\n namespace: \"default\"\n labels:\n app: \"monitoring-example\"\n spec:\n replicas: 1\n selector:\n matchLabels:\n app: \"monitoring-example\"\n template:\n metadata:\n labels:\n app: \"monitoring-example\"\n spec:\n containers:\n - image: gcr.io/google-samples/prometheus-dummy-exporter:latest\n name: prometheus-example-exporter\n imagePullPolicy: Always\n command:\n - /bin/sh\n - -c\n - ./prometheus-dummy-exporter --metric-name=example_monitoring_up --metric-value=1 --port=9090\n resources:\n requests:\n cpu: 100m\n\n2. Create the Deployment and the Service:\n\n kubectl --kubeconfig \u003cvar scope=\"CLUSTER_KUBECONFIG\" translate=\"no\"\u003eCLUSTER_KUBECONFIG\u003c/var\u003e apply -f my-app.yaml\n\nView application logs\n---------------------\n\n### Console\n\n1. Go to the Logs explorer in the Google Cloud console.\n\n [Go to the Logs explorer](https://console.cloud.google.com/logs/query)\n2. Click **Resource** . Under **ALL_RESOURCE_TYPES** , select `Kubernetes\n Container`.\n\n3. Under **CLUSTER_NAME**, select the name of your user cluster.\n\n4. Under **NAMESPACE_NAME** , select `default`.\n\n5. Click **Add** and then click **Run Query**.\n\n6. Under **Query results** , you can see log entries from the\n `monitoring-example` Deployment. For example:\n\n {\n \"textPayload\": \"2020/11/14 01:24:24 Starting to listen on :9090\\n\",\n \"insertId\": \"1oa4vhg3qfxidt\",\n \"resource\": {\n \"type\": \"k8s_container\",\n \"labels\": {\n \"pod_name\": \"monitoring-example-7685d96496-xqfsf\",\n \"cluster_name\": ...,\n \"namespace_name\": \"default\",\n \"project_id\": ...,\n \"location\": \"us-west1\",\n \"container_name\": \"prometheus-example-exporter\"\n }\n },\n \"timestamp\": \"2020-11-14T01:24:24.358600252Z\",\n \"labels\": {\n \"k8s-pod/pod-template-hash\": \"7685d96496\",\n \"k8s-pod/app\": \"monitoring-example\"\n },\n \"logName\": \"projects/.../logs/stdout\",\n \"receiveTimestamp\": \"2020-11-14T01:24:39.562864735Z\"\n }\n\n### gcloud\n\n1. Run this command:\n\n gcloud logging read 'resource.labels.project_id=\"\u003cvar label=\"project ID\" translate=\"no\"\u003ePROJECT_ID\u003c/var\u003e\" AND \\\n resource.type=\"k8s_container\" AND resource.labels.namespace_name=\"default\"'\n\n Replace \u003cvar label=\"project ID\" translate=\"no\"\u003ePROJECT_ID\u003c/var\u003e with the ID\n of your project.\n2. In the output, you can see log entries from the `monitoring-example`\n Deployment. For example:\n\n insertId: 1oa4vhg3qfxidt\n labels:\n k8s-pod/app: monitoring-example\n k8s- pod/pod-template-hash: 7685d96496\n logName: projects/.../logs/stdout\n receiveTimestamp: '2020-11-14T01:24:39.562864735Z'\n resource:\n labels:\n cluster_name: ...\n container_name: prometheus-example-exporter\n location: us-west1\n namespace_name: default\n pod_name: monitoring-example-7685d96496-xqfsf\n project_id: ...\n type: k8s_container\n textPayload: |\n 2020/11/14 01:24:24 Starting to listen on :9090\n timestamp: '2020-11-14T01:24:24.358600252Z'\n\nView application metrics in the Google Cloud console\n----------------------------------------------------\n\nYour example application exposes a custom metric named `example_monitoring_up`.\nYou can view the values of that metric in the Google Cloud console.\n\n1. Go to the Metrics explorer in the Google Cloud console.\n\n [Go to the Metrics explorer](https://console.cloud.google.com/monitoring/metrics-explorer)\n2. For **Resource type** , select `Kubernetes Pod`.\n\n3. For **metric** , select `external/prometheus/example_monitoring_up`.\n\n4. In the chart, you can see that `example_monitoring_up` has a repeated value\n of 1."]]
