This page shows how to set up proxy and firewall rules for Google Distributed Cloud.
Allowlisting addresses for your proxy
If your organization requires outbound traffic to pass through a proxy server,
allowlist the following addresses in your proxy server. Note that www.googleapis.com
is needed, instead of googleapis.com
:
- dl.google.com 1
- gcr.io
- www.googleapis.com
- accounts.google.com
- anthos.googleapis.com
- anthosgke.googleapis.com
- cloudresourcemanager.googleapis.com
- compute.googleapis.com
- connectgateway.googleapis.com
- container.googleapis.com
- gkeconnect.googleapis.com
- gkehub.googleapis.com
- gkeonprem.googleapis.com
- gkeonprem.mtls.googleapis.com
- iam.googleapis.com
- iamcredentials.googleapis.com
- logging.googleapis.com
- monitoring.googleapis.com
- oauth2.googleapis.com
- opsconfigmonitoring.googleapis.com
- securetoken.googleapis.com
- servicecontrol.googleapis.com
- serviceusage.googleapis.com
- stackdriver.googleapis.com
- storage.googleapis.com
- sts.googleapis.com
- releases.hashicorp.com 2
Notes:
1
dl.google.com
is required by the Google Cloud
SDK installer.
2
If you don't use the Terraform client on your
admin workstation to run commands such as terraform apply
, then you don't
need to allowlist releases.hashicorp.com
. If you do use the Terraform client
on your admin workstation, you can optionally allowlist releases.hashicorp.com
so that you can check if the Terraform client version that you are using is the
latest by running the terraform version
command.
Also, if your vCenter Server has an external IP address, allowlist its address in your proxy server.
Firewall rules
Set up your firewall rules to allow the following traffic.
Firewall rules for IP addresses available in the admin cluster
The IP addresses available in the admin cluster are listed in the IP block file . These IP addresses are used for the admin cluster control-plane node, admin cluster add-on nodes, and the user cluster control-plane node. Because the IP addresses for the admin cluster are not assigned to specific nodes, you must make sure that all of the firewall rules listed in the following table apply to all of the IP addresses available for the admin cluster.
| From |
Source port |
To |
Port |
Protocol |
Description |
|---|---|---|---|---|---|
| Admin cluster control-plane node |
1024 - 65535 |
vCenter Server API |
443 |
TCP/https |
Cluster resizing. |
| Admin cluster add-on nodes |
1024 - 65535 |
vCenter Server API |
443 |
TCP/https |
User cluster lifecycle management. |
| User cluster control-plane node |
1024 - 65535 |
vCenter Server API |
443 |
TCP/https |
Cluster resizing. |
| User cluster control-plane node |
1024 - 65535 |
cloudresourcemanager.googleapis.com gkeconnect.googleapis.com gkehub.googleapis.com |
443 |
TCP/https |
Access is required for hub registration. |
| Cloud Logging Collector, which runs on an admin cluster add-on node |
1024 - 65535 |
oauth2.googleapis.com logging.googleapis.com stackdriver.googleapis.com servicecontrol.googleapis.com storage.googleapis.com www.googleapis.com |
443 |
TCP/https |
|
| Cloud Metadata Collector, which runs on an admin cluster add-on node |
1024 - 65535 |
opsconfigmonitoring.googleapis.com | 443 |
TCP/https |
|
| Cloud Monitoring Collector, which runs on an admin cluster add-on node |
1024 - 65535 |
oauth2.googleapis.com monitoring.googleapis.com stackdriver.googleapis.com servicecontrol.googleapis.com |
443 |
TCP/https |
|
| Admin cluster control-plane node Admin cluster add-on nodes |
1024 - 65535 |
F5 BIG-IP API |
443 |
TCP/https |
|
| User cluster control-plane node |
1024 - 65535 |
F5 BIG-IP API |
443 |
TCP/https |
|
| Admin cluster control-plane node |
1024 - 65535 |
On-premises local Docker registry |
Depends on your registry |
TCP/https |
Required if Google Distributed Cloud is configured to use a local private Docker registry instead of gcr.io. |
| User cluster control-plane node |
1024 - 65535 |
On-premises local Docker registry |
Depends on your registry |
TCP/https |
Required if Google Distributed Cloud is configured to use a local private Docker registry instead of gcr.io. |
| Admin cluster control-plane node |
1024 - 65535 |
gcr.io oauth2.googleapis.com storage.googleapis.com Any *.googleapis.com URL required for the services enabled for the admin cluster |
443 |
TCP/https |
Download images from public Docker registries. Not required if using a private Docker registry. |
| User cluster control-plane node |
1024 - 65535 |
gcr.io oauth2.googleapis.com storage.googleapis.com Any *.googleapis.com URL required for the services enabled for the admin cluster |
443 | TCP/https |
Download images from public Docker registries. Not required if using a private Docker registry. |
| Admin cluster worker nodes |
1024 - 65535 |
Admin cluster worker nodes |
All |
179 - bgp 443 - https 5473 - Calico/Typha 9443 - Envoy metrics 10250 - kubelet node port |
All worker nodes must be layer-2 adjacent and without any firewall. |
| Admin cluster nodes |
1024 - 65535 |
Admin cluster pod CIDR |
all |
any |
External traffic gets SNAT'ed on the first node and sent to pod IP. |
| Admin cluster worker nodes |
all |
User cluster nodes |
22 |
ssh |
API server to kubelet communication over an SSH tunnel. |
| Admin cluster nodes |
1024 - 65535 |
IPs of Seesaw LB VMs of the admin cluster |
20255,20257 |
TCP/http |
LB config push and metrics monitoring. Only needed if you are using Bundled LB Seesaw. |
| Admin cluster nodes |
1024 - 65535 |
Admin cluster nodes |
7946 |
TCP/UDP |
MetalLB health check. Only needed if you are using Bundled LB MetalLB. |
Firewall rules for user cluster nodes
In the user cluster nodes, their IP addresses are listed in the IP block file .
As with the admin cluster nodes, you don't know which IP address will be used for which node. Thus, all of the rules in the user cluster nodes apply to each user cluster node.
| From |
Source port |
To |
Port |
Protocol |
Description |
|---|---|---|---|---|---|
| User cluster worker nodes |
all |
gcr.io oauth2.googleapis.com storage.googleapis.com Any *.googleapis.com URL required for the services enabled for this cluster |
443 |
TCP/https |
Download images from public Docker registries. Not required if using a private Docker registry. |
| User cluster worker nodes |
all |
F5 BIG-IP API |
443 |
TCP/https |
|
| User cluster worker nodes |
all |
VIP of the pushprox server, which runs in the Admin cluster. |
8443 |
TCP/https |
Prometheus traffic. |
| User cluster worker nodes |
all |
User cluster worker nodes |
all |
22 - ssh 179 - bgp 443 - https 5473 - calico-typha 9443 - envoy metrics 10250 - kubelet node port" |
All worker nodes must be layer-2 adjacent and without any firewall. |
| User cluster worker nodes |
all |
User control plane VIP |
443 |
TCP/https |
|
| User cluster worker nodes |
all |
User control plane VIP |
8132 |
GRPC |
Konnectivity connection. |
| User cluster nodes |
1024 - 65535 |
User cluster pod CIDR |
all |
any |
External traffic gets SNAT'ed on the first node and sent to pod IP. |
| Cloud Logging Collector, which runs on a random user cluster worker node |
1024 - 65535 |
oauth2.googleapis.com logging.googleapis.com stackdriver.googleapis.com servicecontrol.googleapis.com www.googleapis.com |
443 |
TCP/https |
|
| Connect agent, which runs on a random user cluster worker node. |
1024 - 65535 |
cloudresourcemanager.googleapis.com gkeconnect.googleapis.com gkehub.googleapis.com www.googleapis.com iam.googleapis.com iamcredentials.googleapis.com oauth2.googleapis.com securetoken.googleapis.com sts.googleapis.com accounts.google.com |
443 |
TCP/https |
Connect traffic. |
| Cloud Metadata Collector, which runs on a random user cluster worker node |
1024 - 65535 |
opsconfigmonitoring.googleapis.com | 443 |
TCP/https |
|
| Cloud Monitoring Collector, which runs on a random user cluster worker node |
1024 - 65535 |
oauth2.googleapis.com Monitoring.googleapis.com stackdriver.googleapis.com servicecontrol.googleapis.com |
443 |
TCP/https |
|
| User cluster nodes |
1024 - 65535 |
IPs of Seesaw LB VMs of the user cluster |
20255,20257 |
TCP/http |
LB config push and metrics monitoring. Only needed if you are using Bundled LB Seesaw. |
| Users cluster nodes with enableLoadBalancer=true |
1024 - 65535 |
Users cluster nodes with enableLoadBalancer=true |
7946 |
TCP/UDP |
MetalLB health check. Only needed if you are using Bundled LB MetalLB. |
| User cluster network |
all |
User cluster control plane VIP |
443 |
TCP/https |
Firewall rules for remaining components
These rules apply to all other components not listed in the tables for the admin cluster and user cluster nodes.
From
Source port
To
Port
Protocol
Description
Admin cluster pod CIDR
1024 - 65535
Admin cluster pod CIDR
all
any
Inter-pod traffic does L2 forwarding directly using source and destination IP within Pod CIDR.
Admin cluster pod CIDR
1024 - 65535
Admin cluster nodes
all
any
Return traffic of external traffic.
User cluster pod CIDR
1024 - 65535
User cluster pod CIDR
all
any
Inter-pod traffic does L2 forwarding directly using source and destination IP within Pod CIDR.
User cluster pod CIDR
1024 - 65535
User cluster nodes
all
any
Return traffic of external traffic.
Clients and application end users
all
VIP of Istio ingress
80, 443
TCP
End user traffic to the ingress service of a user cluster.
Jump server to deploy the admin workstation
ephemeral port range
ESXi VMkernel (mgt) IPs of hosts in target cluster
443
TCP/https
Admin workstation
32768- 60999
cloudresourcemanager.googleapis.com
oauth2.googleapis.com
storage.googleapis.com
Any *.googleapis.com URL required for the services enabled for this cluster
443
TCP/https
Download Docker images from public Docker registries.
Admin workstation
Admin cluster add-on node
32768- 60999
cloudresourcemanager.googleapis.com
compute.googleapis.com
iam.googleapis.com
oauth2.googleapis.com
serviceusage.googleapis.com
storage.googleapis.com
Any *.googleapis.com URL required for the services enabled for the admin or user clusters
VIPs of user clusters' Kubernetes API servers
VIP of the admin cluster's Kubernetes API server
vCenter Server API
F5 BIG-IP API
443
TCP/https
Preflight checks (validation).
- Admin workstation: When you create, update, upgrade, or delete
clusters using
gkectl. - Admin cluster add-on node: When you create, update, upgrade, or delete user clusters using the Google Cloud console.
Admin workstation
32768- 60999
vCenter Server API
F5 BIG-IP API
443
TCP/https
Admin cluster create.
User cluster create.
Admin workstation
32768- 60999
ESXi VMkernel (mgt) IPs of hosts in target cluster
443
TCP/https
The admin workstation uploads the OVA to the datastore through the ESXi hosts.
Admin workstation
32768- 60999
Node IP of Admin Cluster control-plane VM
443
TCP/https
Admin cluster create.
Admin workstation
Admin cluster add-on node
32768- 60999
VIP of the admin cluster's Kubernetes API server
VIPs of user clusters' Kubernetes API servers
443
TCP/https
Admin cluster create.
User cluster create.
User cluster update.
User cluster delete.
Admin workstation
32768- 60999
Admin cluster control-plane node and worker nodes
443
TCP/https
Admin cluster create.
Control plane upgrades.
Admin workstation
32768- 60999
All admin cluster nodes and all user cluster nodes
443
TCP/https
Network validation as part of the gkectl check-config
command.
Admin workstation
32768- 60999
VIP of the admin cluster's Istio ingress
VIP of user clusters' Istio ingress
443
TCP/https
Network validation as part of the gkectl check-config
command.
Admin workstation
32768- 60999
IPs of Seesaw LB VMs in both admin and user clusters
Seesaw LB VIPs of both admin and user clusters
20256,20258
TCP/http/gRPC
Health check of LBs. Only needed if you are using Bundled LB Seesaw.
Admin workstation
32768- 60999
Node IP of the cluster control plane
22
TCP
Required if you need SSH access from the admin workstation to the admin cluster control plane.
LB VM IPs
32768- 60999
node IPs of the corresponding cluster
10256: node health check30000 - 32767: healthCheckNodePort
TCP/http
Node health check. healthCheckNodePort is for services with externalTrafficPolicy set to Local. Only needed if you are using Bundled LB Seesaw.
F5 Self-IP
1024 - 65535
All admin and all user cluster nodes
30000 - 32767
any
For the data plane traffic that F5 BIG-IP load balances via a virtual server VIP to the node ports on the Kubernetes cluster nodes.
Typically the F5 self-ip is on the same network/subnet as the Kubernetes cluster nodes.
