Migrate to the OTLP exporter

This document describes how to modify a OpenTelemetry Collector logging configuration that uses an existing exporter — in this case, the googlecloud logging exporter — to use the OTLP exporter and the Telemetry (OTLP) API , telemetry.googleapis.com .

Enable the Telemetry API

The OTLP exporter writes to the Telemetry API, so that API must be enabled in your project. Enable the Telemetry API by running the following command:

gcloud services enable telemetry.googleapis.com

Locate the exporter to be replaced

In your configuration file, locate the exporters entry for logging. Your configuration might look like the following:

  exporters 
 : 
  
 googlecloud/logging 
 : 
 

Replace googlecloud/logging exporter and add the googleauthclient extension:

  exporters 
 : 
  
 otlp_grpc/otlp_logs 
 : 
  
 auth 
 : 
  
 authenticator 
 : 
  
 googleclientauth 
  
 balancer_name 
 : 
  
 pick_first 
  
 endpoint 
 : 
  
 telemetry.googleapis.com:443 
 extensions 
 : 
  
 googleclientauth 
 : 
  
 {} 
 

Add processors

In your configuration file, locate the processors entry. Your configuration might look like the following:

  processors 
 : 
  
 resourcedetection/_global_0 
 : 
  
 [ 
 ...details omitted... 
 ] 
  
 transform/mylog__source_0 
 : 
  
 [ 
 ...details omitted... 
 ] 
 

Copy and paste the configuration for each of the following processors into your configuration file:

1. A resource processor that captures information about your Google Cloud project.

      
    resource/gcp_project_id 
    : 
     
    attributes 
    : 
     
    - 
     
    action 
    : 
     
    insert 
     
    value 
    : 
     
     PROJECT_ID 
    
     
    key 
    : 
     
    gcp.project_id 
    
   

2. A transform processor that preserves information about the instrumentation source.

      
    transform/otlp_grpc/preserve_instrumentation_source_version 
    : 
     
    error_mode 
    : 
     
    ignore 
     
    log_statements 
    : 
     
    - 
     
    context 
    : 
     
    log 
     
    statements 
    : 
     
    - 
     
    set(attributes["instrumentation_source"], instrumentation_scope.name) where instrumentation_scope.name != "" 
     
    - 
     
    set(attributes["instrumentation_version"], instrumentation_scope.version) where instrumentation_scope.version != "" 
    
   

3. A transform processor that preserves information about the resource attributes.

      
    transform/otlp_grpc/preserve_service_resource_attributes 
    : 
     
    error_mode 
    : 
     
    ignore 
     
    log_statements 
    : 
     
    - 
     
    context 
    : 
     
    log 
     
    statements 
    : 
     
    - 
     
    set(attributes["service.name"], resource.attributes["service.name"]) where resource.attributes["service.name"] != nil 
     
    - 
     
    set(attributes["service.namespace"], resource.attributes["service.namespace"]) where resource.attributes["service.namespace"] != nil 
     
    - 
     
    set(attributes["service.instance.id"], resource.attributes["service.instance.id"]) where resource.attributes["service.instance.id"] != nil 
    
   

The modified configuration looks like the following:

  processors 
 : 
  
 resourcedetection/_global_0 
 : 
  
 [ 
 ...details omitted... 
 ] 
  
 transform/mylog__source_0 
 : 
  
 [ 
 ...details omitted... 
 ] 
  
 resource/gcp_project_id 
 : 
  
 [ 
 ...details omitted... 
 ] 
  
 transform/otlp_grpc/preserve_instrumentation_source_version 
 : 
  
 [ 
 ...details omitted... 
 ] 
  
 transform/otlp_grpc/preserve_service_resource_attributes 
 : 
  
 [ 
 ...details omitted... 
 ] 
 

Modify the logging service and pipeline

In your configuration, locate the service used for logging, which includes a pipeline that uses the googlecloud/logging exporter. Your configuration might look like the following:

  service 
 : 
  
 pipelines 
 : 
  
 logs/logs_my__pipeline_mylog__source 
 : 
  
 exporters 
 : 
  
 - 
  
 googlecloud/logging 
  
 processors 
 : 
  
 - 
  
 transform/mylog__source_0 
  
 - 
  
 resourcedetection/_global_0 
  
 receivers 
 : 
  
 - 
  
 tcplog 
 

Make the following changes to this configuration:

1. Add the googleclientauth extension to the service entry.
2. Change the pipeline's exporter to otlp_grpc/otlp_logs .
3. Add the new processors to the pipeline's processors list:
   • resource/gcp_project_id
   • transform/otlp_grpc/preserve_instrumentation_source_version
   • transform/otlp_grpc/preserve_service_resource_attributes

The modified configuration looks like the following:

  service 
 : 
  
 extensions 
 : 
  
 - 
  
 googleclientauth 
  
 pipelines 
 : 
  
 logs/logs_my__pipeline_mylog__source 
 : 
  
 exporters 
 : 
  
 - 
  
 otlp_grpc/otlp_logs 
  
 processors 
 : 
  
 - 
  
 transform/mylog__source_0 
  
 - 
  
 resourcedetection/_global_0 
  
 - 
  
 resource/gcp_project_id 
  
 - 
  
 transform/otlp_grpc/preserve_instrumentation_source_version 
  
 - 
  
 transform/otlp_grpc/preserve_service_resource_attributes 
  
 receivers 
 : 
  
 - 
  
 tcplog 
 

Migrate dashboards and alerting policies, if necessary

If you have charts or alerting policies that track usage of the Cloud Logging API, then update them to monitor usage of the Telemetry API instead. The OTLP exporter sends logs to your Google Cloud project by using the Telemetry API instead of the Cloud Logging API.