Receive Pub/Sub events at a private HTTP endpoint in a private GKE cluster

Create a proxy-only subnet

Unless you create an organizational policy that prohibits it, new projects start with a default network (an auto mode VPC network) that has one subnetwork (subnet) in each region. Each VPC network consists of one or more IP address ranges called subnets . Subnets are regional resources, and have IP address ranges associated with them.

1. Use the gcloud compute networks subnets create command to create a proxy-only subnet in the default network.

    gcloud  
   compute  
   networks  
   subnets  
   create  
   proxy-only-subnet  
    \ 
     
   --purpose = 
   REGIONAL_MANAGED_PROXY  
    \ 
     
   --role = 
   ACTIVE  
    \ 
     
   --region = 
   us-central1  
    \ 
     
   --network = 
   default  
    \ 
     
   --range = 
    10 
   .10.10.0/24 
   

   Note that a subnet with purpose=REGIONAL_MANAGED_PROXY is reserved for Envoy-based load balancers and that the range must provide 64 or more IP addresses.

2. Create a firewall rule that matches the proxy-only subnet's range and that allows traffic on TCP port 8080.

    gcloud  
   compute  
   firewall-rules  
   create  
   allow-proxy-connection  
    \ 
     
   --allow  
   tcp:8080  
    \ 
     
   --source-ranges  
    10 
   .10.10.0/24  
    \ 
     
   --network = 
   default 
   

Create a private GKE cluster

Use the gcloud container clusters create-auto command to create a private GKE cluster in Autopilot mode that has private nodes, and that has no client access to the public endpoint.

The following example creates a private GKE cluster named private-cluster and also creates a subnet named my-subnet :

 gcloud  
container  
clusters  
create-auto  
private-cluster  
 \ 
  
--create-subnetwork  
 name 
 = 
my-subnet  
 \ 
  
--enable-master-authorized-networks  
 \ 
  
--enable-private-nodes  
 \ 
  
--enable-private-endpoint  
 \ 
  
--region = 
us-central1 

Note the following:

• --enable-master-authorized-networks specifies that access to the public endpoint is restricted to IP address ranges that you authorize.
• --enable-private-nodes indicates that the cluster's nodes don't have external IP addresses.
• --enable-private-endpoint indicates that the cluster is managed using the internal IP address of the control plane API endpoint.

It might take several minutes for the creation of the cluster to complete. Once the cluster is created, the output should indicate that the status of the cluster is RUNNING .

Create a VM instance in a specified subnet

A Compute Engine VM instance is a virtual machine that is hosted on Google's infrastructure. The terms Compute Engine instance , VM instance , and VM are synonymous and are used interchangeably. VM instances include GKE clusters, App Engine flexible environment instances, and other Google Cloud products built on Compute Engine VMs.

Use the gcloud compute instances create command to create a Compute Engine VM instance in the subnet you created previously. Attach a service account and set the VM's access scope to cloud-platform .

 gcloud  
compute  
instances  
create  
my-vm  
 \ 
  
--service-account = 
 PROJECT_NUMBER 
-compute@developer.gserviceaccount.com  
 \ 
  
--scopes = 
https://www.googleapis.com/auth/cloud-platform  
 \ 
  
--zone = 
us-central1-a  
 \ 
  
--subnet = 
my-subnet 

For more information, see Create and start a VM instance .

Deploy an event receiver on the VM

Using a prebuilt image, us-docker.pkg.dev/cloudrun/container/hello , deploy a service on your VM that listens on port 80, and that receives and logs events.

1. Establish an SSH connection to your VM instance by running the following command:

    gcloud  
   compute  
   ssh  
   my-vm  
   --project = 
    PROJECT_ID 
     
   --zone = 
   us-central1-a 
   

   After a connection to the SSH server is established, run the remaining commands on your VM instance.

2. If necessary, install kubectl and any required plugins .

3. From your VM instance, use the get-credentials command to enable kubectl to work with the cluster you created.

    gcloud  
   container  
   clusters  
   get-credentials  
   private-cluster  
    \ 
     
   --region = 
   us-central1  
    \ 
     
   --internal-ip 
   

4. Use a Kubernetes command, kubectl create deployment , to deploy an application to the cluster.

    kubectl  
   create  
   deployment  
   hello-app  
    \ 
     
   --image = 
   us-docker.pkg.dev/cloudrun/container/hello 
   

   This creates a Deployment named hello-app . The Deployment's Pod runs the hello container image.

5. After deploying the application, you can expose your application to traffic by creating a Kubernetes Service . Run the following kubectl expose command:

    kubectl  
   expose  
   deployment  
   hello-app  
    \ 
     
   --type  
   ClusterIP  
    \ 
     
   --port  
    80 
     
    \ 
     
   --target-port  
    8080 
    
   

   You should see service/hello-app exposed in the output.

   You can ignore any messages similar to the following:

     E0418 
     
    14 
    : 
    15 
    : 
    33.970933 
     
    1129 
     
    memcache 
    . 
    go 
    : 
    287 
    ] 
     
    couldn 
    ' 
    t 
     
    get 
     
    resource 
     
    list 
     
    for 
     
    metrics 
    . 
    k8s 
    . 
    io 
    / 
    v1beta1 
    : 
     
    the 
     
    server 
     
    is 
     
    currently 
     
    unable 
     
    to 
     
    handle 
     
    the 
     
    request 
    
   

Configure Kubernetes traffic routing

A Gateway resource represents a data plane that routes traffic in Kubernetes. A Gateway can represent many different kinds of load balancing and routing depending on the GatewayClass it is derived from. For more information, see Deploying Gateways . An HTTPRoute manifest is deployed to create Routes and send traffic to application backends.

1. Deploy a Gateway in your cluster.

    kubectl  
   apply  
   -f  
   -  
   <<EOF
   kind:  
   Gateway
   apiVersion:  
   gateway.networking.k8s.io/v1beta1
   metadata:  
   name:  
   internal-http
   spec:  
   gatewayClassName:  
   gke-l7-rilb  
   listeners:  
   -  
   name:  
   http  
   protocol:  
   HTTP  
   port:  
    80 
   EOF 
   

   Note the following:

   • gatewayClassName: gke-l7-rilb specifies the GatewayClass that this Gateway is derived from. gke-l7-rilb corresponds to the internal Application Load Balancer.
   • port: 80 specifies that the Gateway exposes only port 80 for listening for HTTP traffic.

2. Validate that the Gateway has deployed correctly. It might take a few minutes for it to deploy all of its resources.

    kubectl  
   describe  
   gateways.gateway.networking.k8s.io  
   internal-http 
   

   The output is similar to the following:

    Name:         internal-http
   Namespace:    default
   ...
   API Version:  gateway.networking.k8s.io/v1beta1
   Kind:         Gateway
   ...
   Spec:
     Gateway Class Name:  gke-l7-rilb
     Listeners:
       Allowed Routes:
         Namespaces:
           From:  Same
       Name:      http
       Port:      80
       Protocol:  HTTP
   Status:
     Addresses:
       Type:   IPAddress
       Value:  10.36.172.5
   ...
   Events:
     Type    Reason  Age                From                   Message
     ----    ------  ----               ----                   -------
     Normal  ADD     80s                sc-gateway-controller  default/internal-http
     Normal  UPDATE  20s (x3 over 80s)  sc-gateway-controller  default/internal-http
     Normal  SYNC    20s                sc-gateway-controller  SYNC on default/internal-http was a success 
   

3. Deploy an HTTPRoute manifest to route HTTP traffic to the hello-app service at port 80.

    kubectl  
   apply  
   -f  
   -  
   <<EOF
   kind:  
   HTTPRoute
   apiVersion:  
   gateway.networking.k8s.io/v1beta1
   metadata:  
   name:  
   hello-app-route
   spec:  
   parentRefs:  
   -  
   kind:  
   Gateway  
   name:  
   internal-http  
   rules:  
   -  
   backendRefs:  
   -  
   name:  
   hello-app  
   port:  
    80 
   EOF 
   

Create a network attachment

A network attachment is a resource that lets a producer VPC network initiate connections to a consumer VPC network through a Private Service Connect interface .

To publish events, Eventarc uses the network attachment to establish a connection to the internal HTTP endpoint hosted in a VPC network.

You can create a network attachment that automatically accepts connections from any Private Service Connect interface that refers to the network attachment. Create the network attachment in the same network and region containing the HTTP destination service.

gcloud  
compute  
network-attachments  
create  
my-network-attachment  
 \ 
  
--region = 
us-central1  
 \ 
  
--subnets = 
my-subnet \ 
  
--connection-preference = 
ACCEPT_AUTOMATIC

For more information, see About network attachments .

Create an Eventarc trigger

Create an Eventarc trigger that creates a new Pub/Sub topic and routes events to the event receiver deployed on the VM when a message is published to the Pub/Sub topic.

1. Retrieve the Gateway address.

     GATEWAY_ADDRESS 
    = 
    $( 
   kubectl  
   get  
   gateways.gateway.networking.k8s.io  
   internal-http  
   -o = 
    jsonpath 
    = 
    "{.status.addresses[0].value}" 
    ) 
    
   

2. Create a trigger.

    gcloud  
   eventarc  
   triggers  
   create  
   my-trigger  
    \ 
     
   --location = 
   us-central1  
    \ 
     
   --destination-http-endpoint-uri = 
    "http:// 
    $GATEWAY_ADDRESS 
    :80/" 
     
    \ 
     
   --network-attachment = 
    "projects/ PROJECT_ID 
   /regions/us-central1/networkAttachments/my-network-attachment" 
     
    \ 
     
   --event-filters = 
    "type=google.cloud.pubsub.topic.v1.messagePublished" 
     
    \ 
     
   --service-account = 
    PROJECT_NUMBER 
   -compute@developer.gserviceaccount.com 
   

   Replace PROJECT_NUMBER with your Google Cloud project number. You can find your project number on the Welcome page of the Google Cloud console or by running the following command:

    gcloud  
   projects  
   describe  
    PROJECT_ID 
     
   --format = 
    'value(projectNumber)' 
    
   

For more information about configuring your trigger, see Route events to an internal HTTP endpoint in a VPC network .

Generate and view a Pub/Sub topic event

You can generate an event by publishing a message to a Pub/Sub topic.

1. Find and set the Pub/Sub topic as an environment variable.

     export 
     
    MY_TOPIC 
    = 
    $( 
   gcloud  
   eventarc  
   triggers  
   describe  
   my-trigger  
    \ 
     
   --location = 
   us-central1  
    \ 
     
   --format = 
    'value(transport.pubsub.topic)' 
    ) 
    
   

2. Publish a message to the Pub/Sub topic to generate an event.

    gcloud  
   pubsub  
   topics  
   publish  
    $MY_TOPIC 
     
   --message  
    "Hello World" 
    
   

   The Eventarc trigger routes the event to the internal HTTP endpoint in the private GKE cluster.

3. Check the application Pod logs and verify the event delivery.

     POD_NAME 
    = 
    $( 
   kubectl  
   get  
   pod  
   --selector  
    app 
    = 
   hello-app  
   --output = 
   name ) 
   kubectl  
   logs  
    $POD_NAME 
    
   

   The body of the event should be similar to the following:

     2024 
    / 
    04 
    / 
    18 
     
    20 
    : 
    31 
    : 
    43 
     
    Hello 
     
    from 
     
    Cloud 
     
    Run 
    ! 
     
    The 
     
    cont 
    ainer 
     
    started 
     
    successfully 
     
    and 
     
    is 
     
    list 
    ening 
     
    for 
     
    HTTP 
     
    requests 
     
    on 
     
    $ 
    PORT 
    { 
    "severity" 
    : 
    "INFO" 
    , 
    "eventType" 
    : 
    "google.cloud.pubsub.topic.v1.messagePublished" 
    , 
    "message" 
    : 
    "Received event of type google.cloud.pubsub.topic.v1.messagePublished. 
    Event 
     
    data 
    : 
     
    Hello 
     
    World 
    "," 
    event 
    ":{" 
    specversion 
    ":" 
    1.0 
    "," 
    id 
    ":" 
    10935738681111260 
    "," 
    source 
    ":" 
    // 
    pubsub 
    . 
    go 
    ogleapis 
    . 
    com 
    / 
    projects 
    / 
    my 
    - 
    project 
    / 
    to 
    pics 
    / 
    eventarc 
    - 
    us 
    - 
    central1 
    - 
    my 
    - 
    trigger 
    - 
    224 
    "," 
    type 
    ":" 
    go 
    ogle 
    . 
    cloud 
    . 
    pubsub 
    . 
    to 
    pic 
    . 
    v1 
    . 
    messagePublished 
    "," 
    data 
    cont 
    enttype 
    ":" 
    application 
    / 
    json 
    "," 
    time 
    ":" 
    2024 
    - 
    04 
    - 
    18 
    T20 
    : 
    40 
    : 
    03 
    Z 
    "," 
    data 
    ": 
    { 
    "message" 
    : 
    { 
    "data" 
    : 
    "SGVsbG8gV29ybGQ=" 
    , 
    "messageId" 
    : 
    "10935738681111260" 
    , 
    "publishTime" 
    : 
    "2024-04-18T20:40:03Z" 
    }}}} 
    
   

You have successfully deployed an event receiver service to an internal HTTP endpoint in a private GKE cluster, created an Eventarc trigger, generated an event from Pub/Sub, and confirmed that the event was routed as expected by the trigger to the target endpoint.