Create a mixed-protocol external load balancer

Autopilot Standard

This document shows you how to expose an application running in a Google Kubernetes Engine (GKE) cluster to the internet by using a mixed-protocol, external LoadBalancer Service for both TCP and UDP traffic.

To learn more about external passthrough Network Load Balancers, see Backend service-based external passthrough Network Load Balancer .

Overview

You can expose applications that use both TCP and UDP protocols by using two separate GKE LoadBalancer Services with a manually coordinated, shared IP address. However, this approach is inefficient because it requires managing multiple Services for a single application and might lead to issues such as configuration errors or exhausted IP address quotas.

Mixed-protocol LoadBalancer Services let you use a single Service to manage traffic for both TCP and UDP. Using a single Service simplifies your configuration by letting you use a single IPv4 address and a consolidated set of forwarding rules for both protocols. This feature is supported for external passthrough Network Load Balancer.

Before you begin

Before you start, make sure that you have performed the following tasks:

Enable the Google Kubernetes Engine API.

Enable Google Kubernetes Engine API

If you want to use the Google Cloud CLI for this task, install and then initialize the gcloud CLI. If you previously installed the gcloud CLI, get the latest version by running the gcloud components update command. Earlier gcloud CLI versions might not support running the commands in this document.
Note: For existing gcloud CLI installations, make sure to set the compute/region property . If you use primarily zonal clusters, set the compute/zone instead. By setting a default location, you can avoid errors in the gcloud CLI like the following: One of [--zone, --region] must be supplied: Please specify location . You might need to specify the location in certain commands if the location of your cluster differs from the default that you set.

Ensure that you have an existing Autopilot or Standard cluster. To create a new cluster, see Create an Autopilot cluster .

Requirements

To create an external LoadBalancer Service that uses mixed protocols, your cluster must meet the following requirements:

Mixed-protocol load balancing is available only on newly-created clusters on version 1.34.1-gke.2190000 or later.
You must have the HttpLoadBalancing addon enabled on your cluster.
For new external LoadBalancer Services, to implement the load balancer, set the spec.loadBalancerClass field to networking.gke.io/l4-regional-external in the Service manifest. For existing Services, your manifest already has the cloud.google.com/l4-rbs: "enabled" annotation, and you can leave the annotation as is.

Limitations

Mixed-protocol load balancers support only IPv4 addresses.
You can't use mixed protocols in a Service manifest with the following finalizers:
- gke.networking.io/l4-ilb-v1
- gke.networking.io/l4-netlb-v1
If your manifest has these finalizers, you must delete and re-create the Service according to the preceding requirements.

Pricing

Google Cloud bills you per forwarding rule, for any external IP addresses, and for data sent. The following table describes the number of forwarding rules and external IP addresses used for specified configurations. For more information, see VPC network pricing .

Type

Transport Layer

Internet Layer

Number of Forwarding Rules

Number of External IP Addresses

External

Single (Either TCP or UDP)

IPv4

IPv6

IPv4 and IPv6(DualStack)

Mixed (Both TCP and UDP)

IPv4

Deploy a workload

This section shows how to deploy a sample workload that listens on both TCP and UDP ports. Note that the Deployment configuration is the same whether you are using a mixed-protocol LoadBalancer Service or two separate single-protocol LoadBalancer Services.

The following manifest is for a sample application that listens on port 8080 for both TCP and UDP traffic. Save the following manifest as mixed-app-deployment.yaml :

  apiVersion 
 : 
  
 apps/v1 
 kind 
 : 
  
 Deployment 
 metadata 
 : 
  
 name 
 : 
  
 mixed-app-deployment 
 spec 
 : 
  
 replicas 
 : 
  
 3 
  
 selector 
 : 
  
 matchLabels 
 : 
  
 app 
 : 
  
 mixed-app 
  
 template 
 : 
  
 metadata 
 : 
  
 labels 
 : 
  
 app 
 : 
  
 mixed-app 
  
 spec 
 : 
  
 containers 
 : 
  
 - 
  
 image 
 : 
  
 gcr.io/kubernetes-e2e-test-images/agnhost:2.6 
  
 name 
 : 
  
 agnhost 
  
 args 
 : 
  
 [ 
 "serve-hostname" 
 , 
  
 "--port=8080" 
 , 
  
 "--tcp=true" 
 , 
  
 "--udp=true" 
 , 
  
 "--http=false" 
 ] 
  
 ports 
 : 
  
 - 
  
 name 
 : 
  
 tcp8080 
  
 protocol 
 : 
  
 TCP 
  
 containerPort 
 : 
  
 8080 
  
 - 
  
 name 
 : 
  
 udp8080 
  
 protocol 
 : 
  
 UDP 
  
 containerPort 
 : 
  
 8080

Apply the manifest to your cluster:

 kubectl  
apply  
-f  
mixed-app-deployment.yaml

Create a mixed-protocol load balancer

Create a Service of type LoadBalancer that exposes the deployment to both TCP and UDP traffic.

Save the following manifest as mixed-protocol-lb.yaml :

  apiVersion 
 : 
  
 v1 
 kind 
 : 
  
 Service 
 metadata 
 : 
  
 name 
 : 
  
 mixed-protocol-lb 
 spec 
 : 
  
 loadBalancerClass 
 : 
  
 "networking.gke.io/l4-regional-external" 
  
 type 
 : 
  
 LoadBalancer 
  
 selector 
 : 
  
 app 
 : 
  
 mixed-app 
  
 ports 
 : 
  
 - 
  
 name 
 : 
  
 tcp-port 
  
 protocol 
 : 
  
 TCP 
  
 port 
 : 
  
 8080 
  
 - 
  
 name 
 : 
  
 udp-port 
  
 protocol 
 : 
  
 UDP 
  
 port 
 : 
  
 8080

The preceding Service has two ports, one for TCP and one for UDP, both on port 8080.

Apply the manifest to your cluster:

 kubectl  
apply  
--server-side  
-f  
mixed-protocol-lb.yaml

Verify the mixed-protocol load balancer

After you create the Service, verify that GKE created the load balancer successfully.

Inspect the Service:
```
 kubectl  
describe  
service  
mixed-protocol-lb 
```
The output shows the external IP address of the load balancer and the forwarding rules for both TCP and UDP. Verify the following details in the output:
- The status.loadBalancer.ingress.ip field is populated.
- Verify that the following annotations for your external load balancer are present:
  - service.kubernetes.io/tcp-forwarding-rule
  - service.kubernetes.io/udp-forwarding-rule
- The Events section contains no error messages.

Update the mixed-protocol load balancer

You can update the ports on a mixed-protocol load balancer by editing the Service manifest. To edit the Service, run the following command:

 kubectl  
edit  
service  
 SERVICE_NAME

Replace SERVICE_NAME with the name of your Service.

Update ports

To update the ports on a mixed-protocol load balancer, modify the ports section of the Service manifest. You can add, remove, or modify ports.

The following example adds a UDP port for streaming and a TCP port for game-server metadata:

  apiVersion 
 : 
  
 v1 
 kind 
 : 
  
 Service 
 metadata 
 : 
  
 name 
 : 
  
 mixed-protocol-lb 
 spec 
 : 
  
 loadBalancerClass 
 : 
  
 "networking.gke.io/l4-regional-external" 
  
 type 
 : 
  
 LoadBalancer 
  
 selector 
 : 
  
 app 
 : 
  
 mixed-app 
  
 ports 
 : 
  
 - 
  
 name 
 : 
  
 tcp-port 
  
 protocol 
 : 
  
 TCP 
  
 port 
 : 
  
 8080 
  
  - 
  
 name 
 : 
  
 streaming 
  
 protocol 
 : 
  
 UDP 
  
 port 
 : 
  
 10100 
  
 - 
  
 name 
 : 
  
 gameserver-metadata 
  
 protocol 
 : 
  
 TCP 
  
 port 
 : 
  
 10400 
  
 - 
  
 name 
 : 
  
 https 
  
 protocol 
 : 
  
 TCP 
  
 port 
 : 
  
 443

Update a single-protocol load balancer to mixed-protocol

To change a single-protocol load balancer to a mixed-protocol load balancer, edit the Service to include ports for both the TCP and UDP protocols.

The following example adds a UDP port for DNS to an existing TCP-only load balancer:

  apiVersion 
 : 
  
 v1 
 kind 
 : 
  
 Service 
 metadata 
 : 
  
 name 
 : 
  
 already-existing-single-protocol-lb 
 spec 
 : 
  
 loadBalancerClass 
 : 
  
 "networking.gke.io/l4-regional-external" 
  
 type 
 : 
  
 LoadBalancer 
  
 selector 
 : 
  
 app 
 : 
  
 mixed-app 
  
 ports 
 : 
  
 - 
  
 name 
 : 
  
 http 
  
 protocol 
 : 
  
 TCP 
  
 port 
 : 
  
 80 
  
 - 
  
 name 
 : 
  
 https 
  
 protocol 
 : 
  
 TCP 
  
 port 
 : 
  
 443 
  
  - 
  
 name 
 : 
  
 dns 
  
 protocol 
 : 
  
 UDP 
  
 port 
 : 
  
 53

Update a mixed-protocol load balancer to single-protocol

To change a mixed-protocol load balancer to a single-protocol load balancer, remove all ports for one of the protocols.

The following example removes the UDP port for DNS, which converts the load balancer to TCP-only:

  apiVersion 
 : 
  
 v1 
 kind 
 : 
  
 Service 
 metadata 
 : 
  
 name 
 : 
  
 already-existing-mixed-protocol-lb 
 spec 
 : 
  
 loadBalancerClass 
 : 
  
 "networking.gke.io/l4-regional-external" 
  
 type 
 : 
  
 LoadBalancer 
  
 selector 
 : 
  
 app 
 : 
  
 mixed-app 
  
 ports 
 : 
  
 - 
  
 name 
 : 
  
 http 
  
 protocol 
 : 
  
 TCP 
  
 port 
 : 
  
 80 
  
 - 
  
 name 
 : 
  
 https 
  
 protocol 
 : 
  
 TCP 
  
 port 
 : 
  
 443

Delete the mixed-protocol LoadBalancer

To delete the mixed-protocol-lb external LoadBalancer Service, run the following command:

 kubectl  
delete  
service  
mixed-protocol-lb

GKE automatically removes all load balancer resources created for the Service.

Troubleshooting

This section describes how to resolve common issues with mixed-protocol LoadBalancer Services.

Check for error events

The first step in troubleshooting is to check the events associated with your Service.

Get the details of your Service:

 kubectl  
describe  
service  
mixed-protocol-lb

Review the Events section at the end of the output for any error messages.

Error: Mixed Protocol is not supported for LoadBalancer

If you created the Service with the cloud.google.com/l4-rbs: "enabled" annotation, you might see a warning event from the original service controller after you create the mixed-protocol load balancer: mixed-protocol is not supported for LoadBalancer .

You can safely ignore this message because the new controller, which supports mixed-protocols, correctly provisions the load balancer.

Port definition is missing after an update

Symptom:

When you update a Service that uses the same port for both TCP and UDP (for example, port 8080), one of the port definitions is missing from the updated Service.

Cause:

This is a known issue in Kubernetes. When you update a Service with multiple protocols on the same port, the client-side patch calculation can incorrectly merge the port list, which causes one of the port definitions to be removed. This issue affects clients that use client-side patching, such as kubectl apply and the Go client with merge patches.

Solution:

The workaround for this issue depends on your client.

For kubectl: Use the --server-side flag with kubectl apply :
```
 kubectl  
apply  
--server-side  
-f  
 YOUR_SERVICE_MANIFEST 
.yaml 
```
Replace YOUR_SERVICE_MANIFEST with the name of your Service manifest.
For go-client: Don't use merge patches. Instead, use an update call to replace the Service. This requires an HTTP PUT request with the entire Service object specification.

What's next

Learn more about exposing applications using Services .
Read about LoadBalancer Services .