Console
    Contact Us Start free

    Cloud Load Balancing callouts overview

    Service Extensions lets you instruct supported Application Load Balancers to send a callout from the load balancing data path to user-managed callout services or Google services.

    Callouts data flow

    A load balancer communicates with a callout by using Envoy's ext proc gRPC API . This API lets the extension service respond to events in the lifecycle of an HTTP request by examining and modifying the headers or the body of the request.

    An abbreviated version of the API is as follows.

     // 
  
 The 
  
 gRPC 
  
 API 
  
 to 
  
 be 
  
 implemented 
  
 by 
  
 the 
  
 external 
  
 processing 
  
 server 
 service 
  
 ExternalProcessor 
  
 { 
  
 rpc 
  
 Process 
 ( 
 stream 
  
 ProcessingRequest 
 ) 
  
 returns 
  
 ( 
 stream 
  
 ProcessingResponse 
 ) 
  
 { 
  
 } 
 } 
 // 
  
 Envoy 
  
 sets 
  
 one 
  
 of 
  
 these 
  
 fields 
  
 depending 
  
 on 
  
 the 
  
 processing 
  
 stage 
 . 
 message 
  
 ProcessingRequest 
  
 { 
  
 oneof 
  
 request 
  
 { 
  
 HttpHeaders 
  
 request_headers 
  
 = 
  
 2 
 ; 
  
 HttpHeaders 
  
 response_headers 
  
 = 
  
 3 
 ; 
  
 HttpBody 
  
 request_body 
  
 = 
  
 4 
 ; 
  
 HttpBody 
  
 response_body 
  
 = 
  
 5 
 ; 
  
 } 
 } 
 // 
  
 Depending 
  
 on 
  
 the 
  
 processing 
  
 mode 
  
 configuration 
 , 
  
 the 
  
 server 
  
 might 
  
 operate 
  
 in 
 // 
  
 one 
  
 of 
  
 two 
  
 ways 
 . 
  
 One 
  
 way 
  
 is 
  
 to 
  
 send 
  
 a 
  
 ProcessingResponse 
  
 response 
  
 for 
  
 each 
 // 
  
 message 
  
 received 
 . 
  
 The 
  
 other 
  
 is 
  
 to 
  
 buffer 
  
 and 
  
 process 
  
 many 
  
 body 
  
 chunks 
  
 before 
 // 
  
 splitting 
  
 the 
  
 processed 
  
 body 
  
 into 
  
 smaller 
  
 chunks 
 , 
  
 with 
  
 each 
  
 response 
  
 chunk 
 // 
  
 sent 
  
 in 
  
 a 
  
 separate 
  
 ProcessingResponse 
  
 message 
 . 
 message 
  
 ProcessingResponse 
  
 { 
  
 // 
  
 The 
  
 server 
  
 must 
  
 set 
  
 one 
  
 of 
  
 these 
  
 fields 
  
 corresponding 
  
 to 
  
 the 
  
 field 
  
 set 
  
 in 
  
 // 
  
 the 
  
 ProcessingRequest 
  
 message 
 . 
  
 Alternatively 
 , 
  
 the 
  
 server 
  
 can 
  
 set 
  
 the 
  
 // 
  
 immediate_response 
  
 field 
  
 to 
  
 make 
  
 the 
  
 load 
  
 balancer 
  
 terminate 
  
 request 
  
 // 
  
 processing 
  
 and 
  
 send 
  
 the 
  
 specified 
  
 response 
  
 back 
  
 to 
  
 the 
  
 client 
 . 
  
 oneof 
  
 response 
  
 { 
  
 HeadersResponse 
  
 request_headers 
  
 = 
  
 1 
 ; 
  
 HeadersResponse 
  
 response_headers 
  
 = 
  
 2 
 ; 
  
 BodyResponse 
  
 request_body 
  
 = 
  
 3 
 ; 
  
 BodyResponse 
  
 response_body 
  
 = 
  
 4 
 ; 
  
 ImmediateResponse 
  
 immediate_response 
  
 = 
  
 7 
 ; 
  
 } 
 }

    Figure 3 shows how you can deploy the callout backend service with a gRPC server on a user-managed compute resource such as virtual machine (VM) instances or Google Kubernetes Engine (GKE) and represent it to the load balancer as a regular backend service.

    Application Load Balancers use callouts to include custom logic from callout backend services.
    Figure 3. Application Load Balancers send Service Extensions callouts to callout backend services (click to enlarge).

    For example, on receiving the headers for an HTTP request, the load balancer sends the ProcessingRequest message to the extension service with the request_headers field set to the HTTP headers from the client. The extension service must respond with a suitable ProcessingResponse message with any configured changes to the headers or body.

    For REQUEST_HEADER and RESPONSE_HEADER events, the extension service can manipulate the HTTP headers in the request or response. The service can add, modify, or delete headers by setting the request_headers or response_headers field in the ProcessingResponse message appropriately. Use the raw_value field for headers.

    You can deploy the ext_proc gRPC service on VM instances or on GKE and configure an instance group or network endpoint group (NEG) to represent the endpoints of this service.

    Traffic extensions allow changing the headers and the body of both requests and responses. The extension server can override the processing mode dynamically and allow it to enable or disable the extension for subsequent phases of request processing.

    The other extensions have the following restrictions:

    • They allow changing only the request headers. So, the extension service must not set anything other than request_headers in the ProcessingResponse message.

    • They can't override the processing mode of the ext_proc stream. Load balancers call them only for request headers.

    Load balancers don't re-evaluate route rules after calling a traffic extension.

    Supported backends for user-managed callout backend services

    You can host user-managed callout extensions on a backend service that uses one of the following types of backends that run the ext_proc gRPC service:

    Recommended optimizations for callouts

    Integrating an extension into the load balancing processing path incurs additional latency for requests and responses. Each type of data that the extension service processes—including request headers, request body, response headers, and response body—adds latency.

    Consider the following optimizations to minimize the latency:

    • Configure the extension to process only the data that you need. For example, to modify only request headers, set the supported_events field in the extension to REQUEST_HEADERS .
    • Deploy callouts in the same zones as the regular destination backend service for the load balancer. When using a cross-region internal Application Load Balancer, place the extension service backends in the same region as the load balancer's proxy-only subnets.
    • When using a global external Application Load Balancer, place the callout service backends in the geographic regions where the regular load balancer's destination VMs, GKE workloads, and Cloud Run functions are located.

    Limitations

    This section lists some limitations with callouts.

    Limitations with header manipulation

    You cannot change some headers. The following are the limitations with header manipulation:

    • Header manipulation is not supported for the following headers:

      • X-user-IP
      • CDN-Loop
      • Headers starting with X-Forwarded , X-Google , X-GFE , or X-Amz-
      • connection
      • keep-alive
      • transfer-encoding , te
      • upgrade
      • proxy-connection , proxy-authenticate , proxy-authorization
      • trailers

      For LbTrafficExtension , header manipulation is also not supported for these: :method , :authority , :scheme , or host headers.

    • When the ext_proc server specifies header values in HeaderMutation , the load balancer ignores the value field. Use the raw_value field instead.

    Limitations with HTTP/1.1 clients and backends

    The following are the limitations with HTTP/1.1 clients and backends:

    • When you configure either REQUEST_BODY or RESPONSE_BODY for an extension, if the load balancer receives a matching request, it removes the Content-Length header from the response and switches to chunked body encoding.

    • While streaming a message body to the ext_proc server, at the end, the load balancer might send a tailing ProcessingRequest message with an empty body with end_stream set to true to indicate that the stream has ended.

    Other limitations

    The following is a limitation with ProcessingResponse messages:

    • The maximum size of one ProcessingResponse message is 128KB. If a message received is over this limit, the stream is closed with a RESOURCE_EXHAUSTED error.

    • The callout backend service cannot use Cloud Armor, IAP, or Cloud CDN policies.

    • The callout backend service must use HTTP/2 as the protocol.

    • The callout backend service used by route extensions cannot override the processing mode of ext_proc stream.

    What's next
    Create a Mobile Website
    View Site in Mobile | Classic
    Share by: