Console
    Contact Us Start free

    Migrate Pub/Sub triggers from Events for Cloud Run for Anthos to Eventarc

    You can migrate event triggers so that a Google Kubernetes Engine (GKE) service (including Knative serving services) can receive Pub/Sub events using Eventarc. This guide assumes that you are migrating existing event triggers and that your GKE service is running in a GKE cluster. A migration needs to be run for each cluster.

    This migration path avoids the loss of events while minimizing their duplication.

    The migration involves the following:

    1. To allow Eventarc to manage resources in the GKE cluster, enable GKE destinations.
    2. If necessary, enable Workload Identity for the GKE cluster running your target GKE service.
    3. Identify existing event triggers.
    4. Create an Eventarc trigger that points to the same GKE service as the existing trigger.
    5. Confirm that the Eventarc trigger is delivering events as expected.
    6. Delete the original event trigger and clean up after migration.

    Before you begin

    This guide assumes that you already have:

    Set up the command-line environment

    To set up the gcloud and kubectl command-line tools:

    1. In the Google Cloud console, activate Cloud Shell.

      Activate Cloud Shell

      At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

    2. Sign in using your account:

      gcloud  
auth  
login

    3. Set the default settings for the gcloud CLI:

      gcloud  
config  
 set 
  
project  
 PROJECT_ID 
gcloud  
config  
 set 
  
run/cluster  
 CLUSTER_NAME 
gcloud  
config  
 set 
  
run/cluster_location  
 CLUSTER_LOCATION 
gcloud  
config  
 set 
  
run/platform  
gke
gcloud  
config  
 set 
  
eventarc/location  
 LOCATION

      Replace the following:

      • PROJECT_ID : your project ID
      • CLUSTER_NAME : the name of your cluster
      • CLUSTER_LOCATION : any zone where GKE is supported; for example: us-central1-a .
      • LOCATION : a location for the Eventarc trigger; for example: us-central1 .

    4. Enable the following APIs for the project:

      gcloud  
services  
 enable 
  
cloudapis.googleapis.com
gcloud  
services  
 enable 
  
cloudbuild.googleapis.com
gcloud  
services  
 enable 
  
cloudresourcemanager.googleapis.com
gcloud  
services  
 enable 
  
container.googleapis.com
gcloud  
services  
 enable 
  
containerregistry.googleapis.com
gcloud  
services  
 enable 
  
eventarc.googleapis.com

    5. Install the kubectl command-line tool:

      gcloud  
components  
install  
kubectl

    6. Update installed gcloud CLI components:

      gcloud  
components  
update

    Enable GKE destinations

    To allow Eventarc to manage resources in the GKE cluster, enable GKE destinations, and bind the Eventarc service account with the required roles.

    1. Enable GKE destinations for Eventarc:

      gcloud  
eventarc  
gke-destinations  
init

    2. At the prompt to bind the required roles, enter y .

      The following roles are bound:

      • roles/compute.viewer
      • roles/container.developer
      • roles/iam.serviceAccountAdmin

    Enable Workload Identity Federation for GKE on a cluster

    If Workload Identity Federation for GKE is already enabled on your cluster, you can skip this step.

    Workload Identity Federation for GKE is the recommended way to access Google Cloud services from applications running within GKE due to its improved security properties and manageability. It is also needed to forward GKE events using Eventarc.

    To enable Workload Identity Federation for GKE on an existing cluster, refer to Using Workload Identity .

    Identify existing event triggers

    Before migrating any existing event triggers, you must retrieve the trigger details.

    1. List the existing event triggers for the GKE cluster:

      gcloud beta events triggers list --namespace EVENTS_NAMESPACE

      Replace EVENTS_NAMESPACE with the namespace of your events broker.

      The output is similar to the following:

      TRIGGER             EVENT TYPE                                     TARGET
✔  trigger-id          google.cloud.pubsub.topic.v1.messagePublished  cloud-run-service-name

    2. You will need the topic ID to create the Eventarc trigger. Retrieve the topic ID for the existing event trigger:

      gcloud beta events triggers describe TRIGGER_ID 
\
  --platform gke --namespace EVENTS_NAMESPACE 
\
  --format="flattened(serialized_source.spec.topic,serialized_trigger.spec.filter.attributes.type,serialized_trigger.spec.subscriber.ref.name,serialized_trigger.spec.subscriber.ref.namespace)"

      Replace TRIGGER_ID with the ID of the existing event trigger or a fully qualified identifier.

      The output is similar to the following:

      topic:     topic-id
type:      google.cloud.pubsub.topic.v1.messagePublished
name:      cloud-run-service-name
namespace: events

    Create an Eventarc trigger to replace the existing trigger

    Before creating the Eventarc trigger, set up a user-managed service account and grant it specific roles so that Eventarc can manage events for GKE destinations.

    1. Create a Google Cloud service account (GSA):

       TRIGGER_GSA 
 = 
 SERVICE_ACCOUNT_NAME 
gcloud  
iam  
service-accounts  
create  
 $TRIGGER_GSA

      Replace SERVICE_ACCOUNT_NAME with a name that is between 6 and 30 characters. It can contain lowercase alphanumeric characters and dashes.

    2. Grant the pubsub.subscriber and monitoring.metricWriter roles to the service account:

       PROJECT_ID 
 = 
 $( 
gcloud  
config  
get-value  
project ) 
gcloud  
projects  
add-iam-policy-binding  
 $PROJECT_ID 
  
 \ 
  
--member  
 "serviceAccount: 
 $TRIGGER_GSA 
 @ 
 $PROJECT_ID 
 .iam.gserviceaccount.com" 
  
 \ 
  
--role  
 "roles/pubsub.subscriber" 
gcloud  
projects  
add-iam-policy-binding  
 $PROJECT_ID 
  
 \ 
  
--member  
 "serviceAccount: 
 $TRIGGER_GSA 
 @ 
 $PROJECT_ID 
 .iam.gserviceaccount.com" 
  
 \ 
  
--role  
 "roles/monitoring.metricWriter"

    3. Create a new Eventarc trigger based on the existing event trigger's configuration. All the arguments, including the destination service, cluster, and topic ID, should match that of the existing event trigger.

      gcloud  
eventarc  
triggers  
create  
 EVENTARC_TRIGGER_NAME 
  
 \ 
  
--event-filters = 
 "type=google.cloud.pubsub.topic.v1.messagePublished" 
  
 \ 
  
--location  
 LOCATION 
  
 \ 
  
--destination-gke-service = 
 DESTINATION_SERVICE 
  
 \ 
  
--destination-gke-cluster = 
 CLUSTER_NAME 
  
 \ 
  
--destination-gke-location = 
 CLUSTER_LOCATION 
  
 \ 
  
--destination-gke-namespace = 
 EVENTS_NAMESPACE 
  
 \ 
  
--destination-gke-path = 
/  
 \ 
  
--service-account = 
 $TRIGGER_GSA 
@ $PROJECT_ID 
.iam.gserviceaccount.com  
 \ 
  
--transport-topic = 
projects/ PROJECT_ID 
/topics/ TOPIC_ID

      Replace EVENTARC_TRIGGER_NAME with a name for the new Eventarc trigger and TOPIC_ID with the Pub/Sub topic ID that you retrieved previously.

      This creates an Eventarc trigger for the Pub/Sub topic ID.

    4. Confirm that the trigger was successfully created:

      gcloud  
eventarc  
triggers  
list

      The output is similar to the following:

      NAME                  TYPE                                           DESTINATION_RUN_SERVICE  DESTINATION_RUN_PATH  ACTIVE
eventarc-trigger-name google.cloud.pubsub.topic.v1.messagePublished                                                 Yes

    Test the migration and verifying the output

    To test your migration, trigger a test event by publishing to a Pub/Sub topic, and then verify that there is at least one event delivered for each of the existing event triggers and Eventarc triggers, respectively.

    For example, the output after triggering a test event would be duplicated, and would be similar to the following:

      
Hello,  
World!  
ID:  
 2896291125947199 
  
Hello,  
World!  
ID:  
 2896291125947199

    You can also use the following scenario for guidance in which the event receiver can display the event data:

    1. Publish a message to the Pub/Sub topic:

      gcloud pubsub topics publish TOPIC_ID 
\
  --message "Hello, World!"

    2. Verify the event and its successful delivery. For example, look at the logs of the receiving GKE service:

      kubectl logs \
  --selector serving.knative.dev/service= DESTINATION_SERVICE 
\
  -c user-container \
  -n EVENTS_NAMESPACE 
\
  --tail=100

      The output from the two triggers is similar to the following, with differences only in the Context Attributes and Extensions . The actual message data, with the exception of the subscription data, should be identical so that no event receiver changes are required after migrating to Eventarc.

      Eventarc event:

      ☁️  cloudevents.Event
Validation: valid
Context Attributes,
  specversion: 1.0
  type: google.cloud.pubsub.topic.v1.messagePublished
  source: //pubsub.googleapis.com/projects/project-id/topics/topic-name
  id: 2759155988927083
  time: 2021-07-22T17:47:19.964Z
  datacontenttype: application/json
Data,
  {
    "subscription": "projects/project-id/subscriptions/eventarc-us-central1-eventarc-trigger-name-sub-567", "message": {
      "data": "V29ybGQ=",
      "messageId": "2759155988927083",
      "publishTime": "2021-07-22T17:47:19.964Z"
    }}

      Knative serving event:

      ☁️  cloudevents.Event
Validation: valid
Context Attributes,
  specversion: 1.0
  type: google.cloud.pubsub.topic.v1.messagePublished
  source: //pubsub.googleapis.com/projects/project-id/topics/topic-name
  id: 2759155988927083
  time: 2021-07-22T17:47:19.964Z
  dataschema: https://raw.githubusercontent.com/googleapis/google-cloudevents/master/proto/google/events/cloud/pubsub/v1/data.proto
  datacontenttype: application/json
Extensions,
  knativearrivaltime: 2021-07-22T17:47:20.723302362Z
  knsourcetrigger: link0.09767362059083662
  traceparent: 00-f3cc6d754d361a0e49e83e5973fa3565-b4a20ef4ecffe96b-00
Data,
  {
    "subscription": "cre-src_events_source-for-trigger-name_5ffa9638-8cab-4010-900a-2edb275d9eaf", "message": {
      "messageId": "2759155988927083",
      "data": "V29ybGQ=",
      "publishTime": "2021-07-22T17:47:19.964Z"
    }}

    Clean up after the migration

    After testing and verifying the migration of the trigger to Eventarc, you can delete the original event trigger.

    1. Delete the original event trigger:

      gcloud  
beta  
events  
triggers  
delete  
 TRIGGER_NAME 
  
 \ 
  
--platform  
gke  
 \ 
  
--namespace  
 EVENTS_NAMESPACE 
  
 \ 
  
--quiet

    You have migrated an existing Knative serving trigger to Eventarc.

    What's next
    Create a Mobile Website
    View Site in Mobile | Classic
    Share by: