OTLP attributes to resource type mapping

This document describes the rules Google Cloud Observability uses to determine the Logging monitored resource type for log data that is ingested by the Telemetry API. The monitored resource describes the source of log data, and in a LogEntry , it is represented by using labels. Google Cloud Observability determines the values of those labels by using the OTLP resource attributes.

General structure of OTLP-formatted log data

When log data is sent to Google Cloud by using the Telemetry API, this data must be in a format that is consistent with OTLP. The general structure for this data is as shown:

 "resourceLogs": [
    {
      "resource": {
        "attributes": [...]
      },
      "scopeLogs": [
        {
          "logRecords": [...]
        }
      ]
    }
]

Notice that OpenTelemetry batches individual logs, each of which is represented by a logRecord structure, with information about the source of those logs, which is represented by the resource structure.

When Google Cloud Observability receives a resourceLogs object, it constructs one LogEntry for each logRecord . Unlike OTLP which batches source information with a collection of individual logs, each LogEntry structure contains information about the source of the log and the log itself.

To learn more about the structure of OTLP-formatted log data, see the OpenTelemetry logs.proto .

Set OTLP resource attributes by using the Google Cloud resource detector

To set OTLP resource attributes in your OTLP log data, use a Google Cloud resource detector. These detectors attach to telemetry the OTLP resource attributes that describe the source of the telemetry:

For collector-based export, configure resource detection as part of the collector's processors. For example, your collector might specify two resource detectors:
```
 resourcedetection:
    detectors: ["env", "gcp"] 
```
The following documents provide examples of collectors that export metric or trace data:
- Deploy and use the collector : Describes how to send metric data to the Telemetry API.
- Deploy Google-Built OpenTelemetry Collector on Google Kubernetes Engine : Describes how to collect log, metric, and trace data when running on Google Kubernetes Engine. The collector sends trace data to the Telemetry API and metric data to a Prometheus exporter.
Note: We recommend that you use a collector-based export when you use Google Kubernetes Engine. This is the only configuration that lets you configure the resource attributes that Google Cloud Observability requires to associate telemetry with an App Hub application.
If you are using direct export, then use the appropriate Google Cloud resource detector for your language:

How resource labels are set

Each log entry contains a set of labels that identifies the source of the log entry, which is a also known as a Logging monitored resource . The following illustrates the labels for Google Kubernetes Engine.

resource: {
    labels: {
        cluster_name: "my-cluster"
        container_name: "server"
        location: "us-central1"
        namespace_name: "default"
        pod_name: "my-pod"
        project_id: "my-project"
    }
    type: "k8s_container"
}

As shown, a monitored resource has a type field and one or more labels . At a minimum, the labels for a monitored resource include either project_id or resource_container .

The following table describes the rules that Google Cloud Observability uses to map OTLP resource attributes to a monitored resource. The table is ordered. That is, if the resource attributes include cloud.platform and the value is gcp_compute_engine , then the monitored resource type is set to gce_instance . To determine the value of a monitored resource's label, Google Cloud Observability uses prioritized list of resource attributes. The value of a label is an empty string when none of the resource attributes are available.

OTLP attribute and value attached to a log record	Cloud Logging monitored resource type	Monitored resource label values as determined from OTLP resource attributes, by priority
`cloud.platform` has value `gcp_compute_engine`	`gce_instance`	`project_id` : See Infer project ID . `instance_id` : instance_id, host.id `zone` : zone, cloud.availability_zone
`cloud.platform` has value `gcp_app_engine`	`gae_app`	`project_id` : See Infer project ID . `module_id` : gae_module_id, faas.name `version_id` : gae_version_id, faas.version `zone` : zone, cloud.availability_zone
`cloud.platform` has value `aws_ec2`	`aws_ec2_instance`	`project_id` : See Infer project ID . `instance_id` : instance_id, host.id `aws_account` : aws_account, cloud.account.id `region` : region, cloud.availability_zone, cloud.region
`cloud.platform` has value `gcp_bare_metal_solution`	`baremetal instance`	`resource_container` : See Infer resource_container ID . `region` : region, cloud.availability_zone, cloud.region `instance_id` : instance_id, host.id
`service.name` and `service.instance.id` are not empty	`generic_task`	`project_id` : See Infer project ID . `location` : location, cloud.availability_zone, cloud.region `namespace` : namespace, service.namespace `job_name` : job, service.name, faas.name `task_id` : task_id, service.instance.id, faas.instance
`faas.name` and `faas.instance` are not empty	`generic_task`	`project_id` : See Infer project ID . `location` : location, cloud.availability_zone, cloud.region `namespace` : namespace, service.namespace `job_name` : job, service.name, faas.name `task_id` : task_id, service.instance.id, faas.instance
`k8s.cluster.name` and `k8s.container.name` are not empty	`k8s_container`	`project_id` : See Infer project ID . `location` : location, cloud.availability_zone, cloud.region `cluster_name` : cluster_name, k8s.cluster.name `namespace_name` : namespace_name, k8s.namespace_name `pod_name` : pod_name, k8s.pod.name `container_name` : container_name, k8s.container.name
`k8s.cluster.name` and `k8s.pod.name` are not empty	`k8s_pod`	`project_id` : See Infer project ID . `location` : location, cloud.availability_zone, cloud.region `cluster_name` : cluster_name, k8s.cluster.name `namespace_name` : namespace_name, k8s.namespace_name `pod_name` : pod_name, k8s.pod.name
`k8s.cluster.name` and `k8s.node.name` are not empty	`k8s_node`	`project_id` : See Infer project ID . `location` : location, cloud.availability_zone, cloud.region `cluster_name` : cluster_name, k8s.cluster.name `node_name` : node_name, k8s.node.name
`k8s.cluster.name` is not empty	`k8s_cluster`	`project_id` : See Infer project ID . `location` : location, cloud.availability_zone, cloud.region `cluster_name` : cluster_name, k8s.cluster.name
No other label matches	`generic_node`	`project_id` : See Infer project ID . `location` : location, cloud.availability_zone, cloud.region `namespace` : namespace, service.namespace `node_id` : node_id, host.id, host_name

Infer project or resource container ID label value

All Logging monitored resource types have either a project_id label or a resource_container label. The system searches the following attributes, in order, to determine the value of those labels:

project_id
gcp.project.id
gcp.project_id
gcp.dest_project_id
gcp.resource.container
resource_container
cloud.account.id
cloud.account_id