Set up a regional external proxy Network Load Balancer with VM instance group backends

A regional external proxy Network Load Balancer is a proxy-based regional Layer 4 load balancer that lets you run and scale your TCP service traffic in a single region behind an external regional IP address. These load balancers distribute external TCP traffic from the internet to backends in the same region.

Before you begin, read the External proxy Network Load Balancer overview .

This guide contains instructions to set up a regional external proxy Network Load Balancer with a managed instance group (MIG) backend. For this example, you configure the deployment shown in the following diagram.

Note: Regional external proxy Network Load Balancers support both the Premium and Standard Network Service Tiers. This procedure demonstrates the setup with Standard Tier.

For this example, we'll use the load balancer to distribute TCP traffic across backend VMs in two zonal managed instance groups in Region A. For purposes of the example, the service is a set of Apache servers configured to respond on port 110 . Many browsers don't allow port 110 , so the testing section uses curl .

A regional external proxy Network Load Balancer is a regional load balancer. All load balancer components must be in the same region as the load balancer.

SNI-based routing : This page also shows you an alternative deployment architecture that you can use to configure SNI-based routing. For SNI-based routing, you use TLS routes to define how traffic is distributed. For details, see Create a load balancer with TLS routes .

Permissions

To follow this guide, you must be able to create instances and modify a network in a project. You must be either a project Owner or Editor , or you must have all of the following Compute Engine IAM roles .

For more information, see the following guides:

• Roles and permissions
• IAM Conditions

Optional: Use BYOIP addresses

With bring your own IP (BYOIP), you can import your own public addresses to Google Cloud to use the addresses with Google Cloud resources. For example, if you import your own IPv4 addresses, you can assign one to the forwarding rule when you configure your load balancer. When you follow the instructions in this document to configure the load balancer , provide the BYOIP address as the IP address .

For more information about using BYOIP, see Bring your own IP addresses .

Configure the network and subnets

You need a VPC network with two subnets, one for the load balancer's backends and the other for the load balancer's proxies. This load balancer is regional. Traffic within the VPC network is routed to the load balancer if the traffic's source is in a subnet in the same region as the load balancer.

This example uses the following VPC network, region, and subnets:

• Network:a custom-mode VPC network named lb-network

• Subnet for backends:a subnet named backend-subnet in Region A that uses 10.1.2.0/24 for its primary IP address range

• Subnet for proxies:a subnet named proxy-only-subnet in Region B that uses 10.129.0.0/23 for its primary IP address range

Create the network and subnets

Create the proxy-only subnet

A proxy-only subnet provides a set of IP addresses that Google uses to run Envoy proxies on your behalf. The proxies terminate connections from the client and create new connections to the backends.

This proxy-only subnet is used by all Envoy-based load balancers in Region A of the lb-network VPC network.

gcloud compute networks subnets create proxy-only-subnet \
    --purpose=REGIONAL_MANAGED_PROXY \
    --role=ACTIVE \
    --region= REGION_A 
\
    --network=lb-network \
    --range=10.129.0.0/23

Create firewall rules

In this example, you create the following firewall rules:

• fw-allow-ssh .An ingress rule, applicable to the instances being load balanced, that allows incoming SSH connectivity on TCP port 22 from any address. You can choose a more restrictive source IP range for this rule; for example, you can specify only the IP ranges of the system from which you initiate SSH sessions. This example uses the target tag allow-ssh .

• fw-allow-health-check .An ingress rule, applicable to the instances being load balanced, that allows all TCP traffic from the Google Cloud health checking systems (in 130.211.0.0/22 and 35.191.0.0/16 ). This example uses the target tag allow-health-check .

• fw-allow-proxy-only-subnet .An ingress rule that allows connections from the proxy-only subnet to reach the backends.

Without these firewall rules, the default deny ingress rule blocks incoming traffic to the backend instances.

The target tags define the backend instances. Without the target tags, the firewall rules apply to all of your backend instances in the VPC network. When you create the backend VMs, make sure to include the specified target tags, as shown in Create a managed instance group .

Reserve the load balancer's IP address

Reserve a static IP address for the load balancer.

Create a managed instance group

This section shows you how to create two managed instance group (MIG) backends in Region A for the load balancer. The MIG provides VM instances running the backend Apache servers for this example. Typically, a regional external proxy Network Load Balancer isn't used for HTTP traffic, but Apache software is commonly used for testing.

Configure the load balancer

Test the load balancer

Now that you have configured your load balancer, you can test sending traffic to the load balancer's IP address.

1. Get the load balancer's IP address.

   To get the IPv4 address, run the following command:

   gcloud compute addresses describe ADDRESS_NAME 
   

2. Send traffic to your load balancer by running the following command. Replace LB_IP_ADDRESS with your load balancer's IPv4 address.

   curl -m1 LB_IP_ADDRESS 
   :9090

Additional configuration options

This section expands on the configuration example to provide alternative and additional configuration options. All of the tasks are optional. You can perform them in any order.

Create a load balancer with TLS routes

This section shows you how to create a load balancer that can use SNI-based routing. SNI-based routing lets your proxy Network Load Balancers route traffic to specific backend services based on the Server Name Indication (SNI) hostname provided during the TLS handshake.

To create this load balancer we use the same networks, subnets, and firewall rules created previously on this page. You configure the deployment shown in the following diagram:

Create a managed instance group backend

This section shows you how to create a managed instance group (MIG) backends for the load balancer. The MIG provides VM instances running the backend servers for this example.

The Google Cloud CLI instructions in this guide assume that you are using Cloud Shell or another environment with bash installed.

1. Create an instance template with an "echo" HTTPS service that is exposed on port 443.

   gcloud compute instance-templates create INSTANCE_TEMPLATE_NAME 
   \
   --region= REGION_A 
   \
   --network= NETWORK 
   \
   --subnet= SUBNET_A 
   \
   --stack-type=IPv4_ONLY \
   --tags=allow-ssh,allow-health-check,allow-proxy-only-subnet \
   --image-family=debian-12 \
   --image-project=debian-cloud \
   --metadata=startup-script='#! /bin/bash
   sudo sed -i "s/^#DNS=.*/DNS=8.8.8.8 8.8.4.4/" /etc/systemd/resolved.conf
   sudo systemctl restart systemd-resolved
   sudo rm -rf /var/lib/apt/lists/*
   sudo apt-get -y clean
   sudo apt-get -y update
   sudo apt-get -y install ca-certificates curl gnupg software-properties-common
   sudo curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -
   sudo add-apt-repository -y "deb [arch=amd64] https://download.docker.com/linux/debian $(lsb_release -cs) stable"
   sudo apt-get -y update
   sudo apt-get -y install docker-ce
   sudo which docker
   echo "{ \"registry-mirrors\": [\"https://mirror.gcr.io\"] }" | sudo tee -a /etc/docker/daemon.json
   sudo service docker restart
   sudo docker run -e HTTPS_PORT=9999 -p 443:9999 --rm -dt mendhak/http-https-echo:22'

   Replace the following:

   • INSTANCE_TEMPLATE_NAME : a name for the instance template.
   • REGION_A : the region for the instance template.
   • NETWORK : the name of the network.
   • SUBNET_A : the name of the subnetwork.

2. Create a managed instance group based on the instance template:

   gcloud compute instance-groups managed create INSTANCE_GROUP_NAME 
   \
     --zone= ZONE_A 
   \
     --size=2 \
     --template= INSTANCE_TEMPLATE_NAME 
   

   Replace ZONE_A with the zone for the instance group.

3. Set the name of the serving port for the managed instance group:

   gcloud compute instance-groups managed set-named-ports INSTANCE_GROUP_NAME 
   \
     --named-ports= PORT_NAME 
   : PORT_NUMBER 
   \
     --zone= ZONE_A 
   

   Replace the following:

   • PORT_NAME : a name for the serving port—for example, tcp443 .
   • PORT_NUMBER : a port number for the serving port—for example, 443 .

Configure the firewall

Configure a firewall rule to allow traffic from the load balancer and from the health check probes to the backend instances.

gcloud compute firewall-rules create FIREWALL_RULE_NAME 
\
    --network= NETWORK 
\
    --action=allow \
    --direction=ingress \
    --source-ranges=130.211.0.0/22,35.191.0.0/16 \
   --target-tags=allow-health-check \
    --rules=tcp:443

Replace FIREWALL_RULE_NAME with a name for the firewall rule.

Configure the load balancer

1. Create an HTTPS health check:

   gcloud compute health-checks create http HTTPS_HEALTH_CHECK_NAME 
   \
       --region= REGION_A 
   \
       --port= HC_PORT 
   

   Replace the following:

   • HTTPS_HEALTH_CHECK_NAME : a name for the health check.
   • HC_PORT : the port for the health check—for example, 443 .
   • REGION_A : the region for the health check.

2. Create a backend service:

   gcloud compute backend-services create BACKEND_SERVICE_NAME 
   \
       --load-balancing-scheme=EXTERNAL_MANAGED \
       --protocol=TCP \
       --port-name= PORT_NAME 
   \
       --health-checks= HTTPS_HEALTH_CHECK_NAME 
   \
       --health-checks-region= REGION_A 
   \
       --region= REGION_A 
   

   Replace the following:

   • BACKEND_SERVICE_NAME : a name for the backend service.
   • PORT_NAME : the port name for the backend service. Use the same named port configured on the instance group—for example, tcp443 .
   • HTTPS_HEALTH_CHECK_NAME : the name of the HTTPS health check.

3. Add the backend instance group to your backend service:

   gcloud compute backend-services add-backend BACKEND_SERVICE_NAME 
   \
       --balancing-mode=UTILIZATION \
       --max-utilization=0.8
       --instance-group= INSTANCE_GROUP_NAME 
   \
       --instance-group-zone= ZONE_A 
   \
       --region= REGION_A 
   

   Replace the following:

   • INSTANCE_GROUP_NAME : the name of the backend instance group.
   • ZONE_A : the zone of the instance group.

4. Create a target TCP proxy.

   gcloud beta compute target-tcp-proxies create TARGET_TCP_PROXY_NAME 
   \
       --load-balancing-scheme=EXTERNAL_MANAGED \
       --proxy-header=NONE \
       --region= REGION_A 
   

   Replace TARGET_TCP_PROXY_NAME with the name of the target TCP proxy.

5. Create a TLS route specification and save it to a YAML file.

     cat <<EOF | tee YAML_FILE_NAME 
    
    name 
    : 
     
     TLS_ROUTE_NAME 
    
    targetProxies 
    : 
    - 
     
    projects/ PROJECT_NUMBER 
   /locations/ REGION_A 
   /targetTcpProxies/ TARGET_TCP_PROXY 
    
    rules 
    : 
    - 
     
    matches 
    : 
     
    - 
     
    sniHost 
    : 
     
    - 
     
    example.com 
     
    action 
    : 
     
    destinations 
    : 
     
    - 
     
    serviceName 
    : 
     
    projects/ PROJECT_NUMBER 
   /locations/ REGION_A 
   /backendServices/ BACKEND_SERVICE_NAME 
    
    EOF 
    
   

   Replace the following:

   • YAML_FILE_NAME : a name for the YAML file—for example, tls-route.yaml .
   • TLS_ROUTE_NAME : a name for the TLS route.
   • PROJECT_NUMBER : the project number.

6. Use the YAML specification file to create the TLS route resource.

   gcloud network-services tls-routes import TLS_ROUTE_NAME 
   \
     --source= YAML_FILE_NAME 
   \
     --location= REGION_A 
   

7. Create the forwarding rule.

   gcloud compute forwarding-rules create FORWARDING_RULE_NAME 
   \
     --load-balancing-scheme=EXTERNAL_MANAGED \
     --network-tier=STANDARD \
     --network= NETWORK 
   \
     --region= REGION_A 
   \
     --target-tcp-proxy= TARGET_TCP_PROXY_NAME 
   \
     --target-tcp-proxy-region= REGION_A 
   \
     --address= IP_ADDRESS 
   \
     --ports= PORT_NUMBER 
   

   Replace the following:

   • FORWARDING_RULE_NAME : a name for the forwarding rule.
   • NETWORK : the name of the network.
   • SUBNET_A : the name of the subnetwork in the same region as the load balancer.
   • IP_ADDRESS : the IP address of the load balancer.
   • PORT_NUMBER : the port used by the forwarding rule—for example, 443 .

Test the load balancer

Now that you have configured your load balancer, you can test sending traffic to the load balancer's IP address.

1. Verify that you can access the HTTPS service through the load balancer.

   curl https://example.com --resolve example.com:443: IP_ADDRESS 
   -k

   You will see the command returning a response from one of the VMs in the managed instance group with the following printed to the console.

   "path": "/",
   "headers": {
     "host": "example.com",
     "user-agent": "curl/7.81.0",
     "accept": "*/*"
   },
   "method": "GET",
   "body": "",
   "fresh": false,
   "hostname": "example.com",
   "ip": "::ffff:10.142.0.2",
   "ips": [],
   "protocol": "https",
   "query": {},
   "subdomains": [],
   "xhr": false,
   "os": {
     "hostname": "0cd3aec9b351"
   },
   "connection": {
     "servername": "example.com"
   }

   You can further verify that if you provide a different SNI hostname that doesn't match a TLS route, or if you don't provide an SNI hostname at all, the request is dropped.

   • Run a test with an SNI hostname that doesn't match example.com, to ensure that the connection is rejected.

   curl https://unknown.com --resolve unknown.com:443: IP_ADDRESS 
   -k

   • Run a test with a plain text connection without TLS, to ensure that the connection is rejected.

   curl example.com:443 --resolve example.com:443: IP_ADDRESS 
   -k

   These commands return the following error.

   curl: (35) OpenSSL SSL_connect: Connection reset by peer in connection

   You'll see the connection_refused error code in the proxyStatus logs when the load balancer refuses such invalid connections.

Enable session affinity

The example configuration creates a backend service without session affinity.

These procedures show you how to update a backend service for the example load balancer created previously so that the backend service uses client IP affinity or generated cookie affinity.

When client IP affinity is enabled, the load balancer directs a particular client's requests to the same backend VM based on a hash created from the client's IP address and the load balancer's IP address (the internal IP address of an internal forwarding rule).

To enable client IP session affinity, complete the following steps.

gcloud compute backend-services update ext-reg-tcp-proxy-bs \
    --region= REGION_A 
\
    --session-affinity=CLIENT_IP

What's next

• Convert proxy Network Load Balancer to IPv6
• External proxy Network Load Balancer overview .
• Proxy Network Load Balancer logging and monitoring .
• Clean up the load balancer setup .