Infrastructure controls

• under:organizations/ORGANIZATION_ID
• under:folders/FOLDER_ID
• under:projects/PROJECT_ID
• projects/PROJECT_ID/zones/ZONE/instances/INSTANCE_NAME .

Unless needed, prevent the creation of Compute Engine instances with public IP addresses. The compute.vmExternalIpAccess list constraint defines the set of Compute Engine VM instances that can have external IP addresses.

Prevent Compute Engine instances from having external IP addresses to drastically reduce their exposure to the internet. Any instance with an external IP address is immediately discoverable and becomes a direct target for automated scans, brute-force attacks, and attempts to exploit vulnerabilities. Instead, require instances to use private IP addresses and manage access through controlled, authenticated, and logged pathways like the Identity-Aware Proxy (IAP) tunnel or a bastion host.

Adopting this deny-by-default posture is a foundational security best practice that helps minimize your attack surface and enforces a zero-trust approach to your network. This constraint isn't retroactive.

The compute.vmExternalIpAccess list constraint lets you restrict external access to virtual machines by not assigning external IP addresses. Configure this list constraint to deny all external IP addresses to virtual machines.

The cloudfunctions.requireVPCConnector boolean constraint requires that administrators specify a Serverless VPC Access connector when they deploy a Cloud Run function. When enforced, functions must specify a connector.

• Set a message storage policy using the Resource Location Restriction ( gcp.resourceLocations ) organization policy constraint.
• Configure a message storage policy when creating a topic. For example:

  gcloud pubsub topics create TOPIC_ID \--message-storage-policy-allowed-regions=REGION1, REGION2

Turn off external IP addresses for administrative and monitoring tasks that are related to Dataflow jobs. Instead, configure access to your Dataflow worker VMs using SSH.

Enable Private Google Access and specify one of the following options in your Dataflow job:

• --usePublicIps=false and --network=NETWORK-NAME
• --subnetwork=SUBNETWORK-NAME

Where:

• NETWORK-NAME : The name of your Compute Engine network.
• SUBNETWORK-NAME : The name of your Compute Engine subnetwork.

Network tags are text attributes that attach to Compute Engine VMs such as Dataflow worker VMs. Network tags let you make VPC network firewall rules and some custom static routes applicable to specific VM instances. Dataflow supports adding network tags to all worker VMs that run a particular Dataflow job.

By default, the Google Kubernetes Engine (GKE) cluster control plane and nodes have internet routable addresses that can be accessed from any IP address. Restrict network access to the control plane by using a DNS-based endpoint and by creating private clusters. The control plane is the management center for a Kubernetes cluster, and exposing it to the internet makes it a prime target for attackers. This configuration makes the control plane private and removes it from the internet.

Restricting control plane access helps ensure that only trusted devices within your organization's private network can manage the cluster, drastically reducing the risk of an external attack.

When you create firewall rules, use the principle of least privilege to provide access only for the required purpose. Ensure that your firewall rules don't conflict with the GKE default firewall rules when possible.

Use Google Groups for role-based access control (RBAC), which also lets you integrate with your existing user account management practices, such as revoking access when someone leaves your organization. Google Groups for RBAC helps provide efficient management of cluster access using Identity and Access Management (IAM) and Google Groups, which is suitable for most organizations that use Google Groups.

Enable Shielded GKE Nodes for cryptographic verification of the nodes in your cluster. Shielded GKE Nodes provide strong, verifiable node identity and integrity. Enable Shielded GKE Nodes when creating or updating clusters. Where possible, use Shielded GKE Nodes with secure boot to also authenticate the boot components of your node VMs during the boot process. Don't use secure boot if you need third-party unsigned kernel modules.

Shielded GKE Nodes is enabled by default when you create a cluster.

Use Container-Optimized OS to implement a hardened and managed container OS. General-purpose operating systems include many extra programs that aren't needed to run containers and therefore create a larger, unnecessary target for attackers. Container-Optimized OS is a minimal, locked-down operating system that significantly reduces this attack surface by including only what is necessary. As a managed OS, Container-Optimized OS also has security patches that are automatically applied by Google, which help ensure critical vulnerabilities are fixed and reduces your operational workload.

An image that includes Container-Optimized OS with containerd (cos_containerd) has containerd as the main container runtime directly integrated with Kubernetes. containerd is the core runtime component of Docker and is designed to deliver core container functionality for the Kubernetes Container Runtime Interface (CRI). It is significantly less complex than the full Docker daemon, and therefore has a smaller attack surface.

Use Workload Identity Federation for GKE to securely authenticate to Google Cloud APIs from Google Kubernetes Engine (GKE) workloads. Workload Identity Federation for GKE provides a simpler and safer alternative to using service account keys.

Use GKE Sandbox to provide an extra layer of security to help prevent untrusted code from affecting the host kernel on your Google Kubernetes Engine (GKE) cluster nodes. GKE Sandbox enhances workload isolation for untrusted or sensitive workloads, providing an additional layer of protection against container escape attacks.

Disable kubelet read-only port 10255 and use the more secure port 10250 instead. Kubernetes doesn't perform any authentication or authorization checks on this port. The kubelet serves the same endpoints on the more secure, authenticated port 10250.

You can only disable the insecure kubelet read-only port in GKE version 1.26.4-gke.500 or later.

Create separate namespaces or clusters for each team and environment to implement least-privilege access to Kubernetes. Assign cost centers and appropriate labels to each namespace for accountability and chargeback. Only give developers the level of access to their namespace that they need to deploy and manage their application, especially in production. Map out the tasks that your users need to undertake against the cluster and define the permissions that they require to do each task.

Assign the appropriate Identity and Access Management (IAM) roles for Google Kubernetes Engine (GKE) to groups and users to provide permissions at the project level and use RBAC to grant permissions on a cluster and namespace level.

By default, all pods in a cluster can communicate with each other. Control pod-to-pod communication as needed for your workloads. Restricting network access to services makes it much more difficult for attackers to move laterally within your cluster, and also offers services some protection against accidental or deliberate denial of service.

There are two ways to control traffic:

• Use Cloud Service Mesh or Istio for load balancing, service authorization, throttling, quota, metrics and more.
• Use Kubernetes network policies. Choose this option if you're looking for the basic access control functionality that's exposed by Kubernetes.

Admission controllers are plugins that govern and enforce how the cluster is used. Enable them to use some of the more advanced security features of Kubernetes because they are an important part of the defense in depth approach to hardening your cluster. By default, pods in Kubernetes can operate with capabilities beyond what they require. Use admission controls to restrict the pod's capabilities to only those that are required for that workload.

GKE supports numerous controls for restricting your pods to execute with capabilities that are explicitly granted. For example, Policy Controller is available for clusters in fleets. Kubernetes also has the built-in PodSecurity admission controller that lets you enforce the Pod Security Standards in individual clusters. Policy Controller is a feature of GKE that lets you enforce and validate security on GKE clusters at scale by using declarative policies.

Certain Kubernetes workloads, especially system workloads, have permission to self-modify. For example, some workloads vertically autoscale themselves. While convenient, self-modification can allow an attacker who has already compromised a node to escalate further in the cluster. For example, an attacker could have a workload on the node change itself to run as a more privileged service account that exists in the same namespace.

Don't grant workloads the permission to modify themselves by default. When self-modification is necessary, limit permissions by applying Gatekeeper or Policy Controller constraints, such as NoUpdateServiceAccount from the open source Gatekeeper library, which provides several useful security policies.

When you deploy policies, allow the controllers that manage the cluster lifecycle to bypass the policies. Controllers must make changes to the cluster, such as applying cluster upgrades. For example, if you deploy the NoUpdateServiceAccount policy on GKE, you must set the following parameters in the constraint:

Audit your cluster configurations for deviations from your defined settings. Settings covered in these best practices, as well as other common misconfigurations, can be automatically checked using Security Command Center.

Use Binary Authorization to make sure trusted images are deployed to Google Kubernetes Engine (GKE) and Cloud Run. Binary Authorization helps ensure that only verified and trusted container images can be deployed in your clusters, strengthening software supply chain security.