Version 1.9. This version is no longer supported. For information about how to upgrade to version 1.10, seeUpgrading Anthos on bare metalin the 1.10 documentation. For more information about supported and unsupported versions, see theVersion historypage in the latest documentation.
This page shows how to configure a cluster for Google Distributed Cloud so that
custom logs and metrics from user applications are sent to Cloud Logging and
Cloud Monitoring.
Enabling Logging and Monitoring for user applications
ADMIN_KUBECONFIG: the path to the admin cluster
kubeconfig file.
Annotate workloads
To enable the collection of custom metrics from an application, add theprometheus.io/scrape: "true"annotation to the application's Service or
Deployment manifest.
Run an example application
In this section, you create an application that writes custom logs and exposes
a custom metric.
Save the following Service and Deployment manifests to a file namedmy-app.yaml. Notice that the Service has the annotationprometheus.io/scrape: "true":
UnderCLUSTER_NAME, select the name of your user cluster.
UnderNAMESPACE_NAME, selectdefault.
ClickAddand then clickRun Query.
UnderQuery results, you can see log entries from themonitoring-exampleDeployment. For example:
{"textPayload":"2020/11/14 01:24:24 Starting to listen on :9090\n","insertId":"1oa4vhg3qfxidt","resource":{"type":"k8s_container","labels":{"pod_name":"monitoring-example-7685d96496-xqfsf","cluster_name":...,"namespace_name":"default","project_id":...,"location":"us-west1","container_name":"prometheus-example-exporter"}},"timestamp":"2020-11-14T01:24:24.358600252Z","labels":{"k8s-pod/pod-template-hash":"7685d96496","k8s-pod/app":"monitoring-example"},"logName":"projects/.../logs/stdout","receiveTimestamp":"2020-11-14T01:24:39.562864735Z"}
gcloud
Run this command:
gcloudloggingread'resource.labels.project_id="PROJECT_ID" AND \resource.type="k8s_container" AND resource.labels.namespace_name="default"'
ReplacePROJECT_IDwith the ID
of your project.
In the output, you can see log entries from themonitoring-exampleDeployment. For example:
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-09-04 UTC."],[[["\u003cp\u003eConfigure Google Distributed Cloud to send custom logs and metrics from user applications to Cloud Logging and Cloud Monitoring by setting the \u003ccode\u003eenableApplication\u003c/code\u003e field to \u003ccode\u003etrue\u003c/code\u003e in the cluster configuration file.\u003c/p\u003e\n"],["\u003cp\u003eEnable custom metric collection from an application by adding the \u003ccode\u003eprometheus.io/scrape: "true"\u003c/code\u003e annotation to the application's Service or Deployment manifest.\u003c/p\u003e\n"],["\u003cp\u003eDeploy an example application that writes custom logs and exposes a custom metric by creating a Service and Deployment manifest and then using \u003ccode\u003ekubectl apply\u003c/code\u003e.\u003c/p\u003e\n"],["\u003cp\u003eView application logs in the Google Cloud console's Logs explorer by filtering for \u003ccode\u003eKubernetes Container\u003c/code\u003e resource type, selecting your cluster, and the \u003ccode\u003edefault\u003c/code\u003e namespace, or via the \u003ccode\u003egcloud logging read\u003c/code\u003e command.\u003c/p\u003e\n"],["\u003cp\u003eAccess custom application metrics, such as \u003ccode\u003eexample_monitoring_up\u003c/code\u003e, through the Metrics explorer in the Google Cloud console by selecting \u003ccode\u003eKubernetes Pod\u003c/code\u003e as the resource type and choosing the specific custom metric.\u003c/p\u003e\n"]]],[],null,["# Enable application logging and monitoring\n\n\u003cbr /\u003e\n\nThis page shows how to configure a cluster for Google Distributed Cloud so that\ncustom logs and metrics from user applications are sent to Cloud Logging and\nCloud Monitoring.\n\nEnabling Logging and Monitoring for user applications\n-----------------------------------------------------\n\nTo enable Logging and Monitoring for your\napplications, use the\n[`spec.clusterOperations.enableApplication`](/anthos/clusters/docs/bare-metal/1.9/reference/cluster-config-ref#clusteroperations-enableapplication)\nfield in the cluster configuration file.\n\n1. Update the cluster configuration file to set `enableApplication` to `true`:\n\n apiVersion: v1\n kind: Namespace\n metadata:\n name: cluster-user-basic\n ---\n apiVersion: baremetal.cluster.gke.io/v1\n kind: Cluster\n metadata:\n name: user-basic\n namespace: cluster-user-basic\n spec:\n type: user\n ...\n clusterOperations:\n projectID: project-fleet\n location: us-central1enableApplication: true\n ...\n\n2. Use `bmctl update` to apply your changes:\n\n bmctl update cluster -c \u003cvar translate=\"no\"\u003eCLUSTER_NAME\u003c/var\u003e --admin-kubeconfig=\u003cvar translate=\"no\"\u003eADMIN_KUBECONFIG\u003c/var\u003e\n\n Replace the following:\n - \u003cvar translate=\"no\"\u003eCLUSTER_NAME\u003c/var\u003e: the name of the cluster to upgrade.\n - \u003cvar translate=\"no\"\u003eADMIN_KUBECONFIG\u003c/var\u003e: the path to the admin cluster kubeconfig file.\n\nAnnotate workloads\n------------------\n\nTo enable the collection of custom metrics from an application, add the\n`prometheus.io/scrape: \"true\"` annotation to the application's Service or\nDeployment manifest.\n\nRun an example application\n--------------------------\n\nIn this section, you create an application that writes custom logs and exposes\na custom metric.\n\n1. Save the following Service and Deployment manifests to a file named\n `my-app.yaml`. Notice that the Service has the annotation\n `prometheus.io/scrape: \"true\"`:\n\n kind: Service\n apiVersion: v1\n metadata:\n name: \"monitoring-example\"\n namespace: \"default\"\n annotations:\n prometheus.io/scrape: \"true\"\n spec:\n selector:\n app: \"monitoring-example\"\n ports:\n - name: http\n port: 9090\n ---\n apiVersion: apps/v1\n kind: Deployment\n metadata:\n name: \"monitoring-example\"\n namespace: \"default\"\n labels:\n app: \"monitoring-example\"\n spec:\n replicas: 1\n selector:\n matchLabels:\n app: \"monitoring-example\"\n template:\n metadata:\n labels:\n app: \"monitoring-example\"\n spec:\n containers:\n - image: gcr.io/google-samples/prometheus-dummy-exporter:latest\n name: prometheus-example-exporter\n imagePullPolicy: Always\n command:\n - /bin/sh\n - -c\n - ./prometheus-dummy-exporter --metric-name=example_monitoring_up --metric-value=1 --port=9090\n resources:\n requests:\n cpu: 100m\n\n2. Create the Deployment and the Service:\n\n kubectl --kubeconfig \u003cvar scope=\"CLUSTER_KUBECONFIG\" translate=\"no\"\u003eCLUSTER_KUBECONFIG\u003c/var\u003e apply -f my-app.yaml\n\nView application logs\n---------------------\n\n### Console\n\n1. Go to the Logs explorer in the Google Cloud console.\n\n [Go to the Logs explorer](https://console.cloud.google.com/logs/query)\n2. Click **Resource** . Under **ALL_RESOURCE_TYPES** , select `Kubernetes\n Container`.\n\n3. Under **CLUSTER_NAME**, select the name of your user cluster.\n\n4. Under **NAMESPACE_NAME** , select `default`.\n\n5. Click **Add** and then click **Run Query**.\n\n6. Under **Query results** , you can see log entries from the\n `monitoring-example` Deployment. For example:\n\n {\n \"textPayload\": \"2020/11/14 01:24:24 Starting to listen on :9090\\n\",\n \"insertId\": \"1oa4vhg3qfxidt\",\n \"resource\": {\n \"type\": \"k8s_container\",\n \"labels\": {\n \"pod_name\": \"monitoring-example-7685d96496-xqfsf\",\n \"cluster_name\": ...,\n \"namespace_name\": \"default\",\n \"project_id\": ...,\n \"location\": \"us-west1\",\n \"container_name\": \"prometheus-example-exporter\"\n }\n },\n \"timestamp\": \"2020-11-14T01:24:24.358600252Z\",\n \"labels\": {\n \"k8s-pod/pod-template-hash\": \"7685d96496\",\n \"k8s-pod/app\": \"monitoring-example\"\n },\n \"logName\": \"projects/.../logs/stdout\",\n \"receiveTimestamp\": \"2020-11-14T01:24:39.562864735Z\"\n }\n\n### gcloud\n\n1. Run this command:\n\n gcloud logging read 'resource.labels.project_id=\"\u003cvar label=\"project ID\" translate=\"no\"\u003ePROJECT_ID\u003c/var\u003e\" AND \\\n resource.type=\"k8s_container\" AND resource.labels.namespace_name=\"default\"'\n\n Replace \u003cvar label=\"project ID\" translate=\"no\"\u003ePROJECT_ID\u003c/var\u003e with the ID\n of your project.\n2. In the output, you can see log entries from the `monitoring-example`\n Deployment. For example:\n\n insertId: 1oa4vhg3qfxidt\n labels:\n k8s-pod/app: monitoring-example\n k8s- pod/pod-template-hash: 7685d96496\n logName: projects/.../logs/stdout\n receiveTimestamp: '2020-11-14T01:24:39.562864735Z'\n resource:\n labels:\n cluster_name: ...\n container_name: prometheus-example-exporter\n location: us-west1\n namespace_name: default\n pod_name: monitoring-example-7685d96496-xqfsf\n project_id: ...\n type: k8s_container\n textPayload: |\n 2020/11/14 01:24:24 Starting to listen on :9090\n timestamp: '2020-11-14T01:24:24.358600252Z'\n\nView application metrics in the Google Cloud console\n----------------------------------------------------\n\nYour example application exposes a custom metric named `example_monitoring_up`.\nYou can view the values of that metric in the Google Cloud console.\n\n1. Go to the Metrics explorer in the Google Cloud console.\n\n [Go to the Metrics explorer](https://console.cloud.google.com/monitoring/metrics-explorer)\n2. For **Resource type** , select `Kubernetes Pod`.\n\n3. For **metric** , select `external/prometheus/example_monitoring_up`.\n\n4. In the chart, you can see that `example_monitoring_up` has a repeated value\n of 1."]]
