This document explains how you can configure reliable communication by assigning one or more persistent IP addresses to specific Pods in your Google Kubernetes Engine (GKE) clusters. For applications that require high availability, you can also configure these persistent IP addresses to support fast failover.
In certain cases, where you are running a custom Network Address Translation (NAT) solution, you might want a static persistent IP address for both outgoing and incoming connections, either when the NAT solution initiates the connection or when it receives it. You might also want to have control over the IP addresses allocated to the application for managing how it interacts with other systems or how it handles specific types of requests based on your business requirements.
By default, the Pod uses its interface IP addresses for egressing traffic. Interface IP addresses change when the Pod restarts or gets moved. To have more control over routing communication, you can configure persistent IP addresses manually for your Pods in GKE.
These IP addresses can be either external IP addresses to communicate over the internet or internal IP addresses to communicate within your Google Cloud network. You can either use Google-provided IP addresses or bring your own IP addresses (BYOIP).
By configuring persistent IP addresses for Pods in GKE, you can map application and business logic to allow specific Pods to send and receive traffic to or from any of the persistent IP addresses.
The following diagram illustrates how a Pod with multiple network interfaces can use a Persistent IP address from a secondary network while still communicating on the default network:
Terminology and concepts
This page uses the following concepts:
Gateway classes
Gateway classes , which manage your persistent IP address assignments, come in the following classes:
- gke-persistent-regional-external-managedfor external IP addresses
- gke-persistent-regional-internal-managedfor internal (Google Cloud-only) IP addresses
- gke-persistent-fast-regional-external-managedfor external IP addresses with fast failover
-
gke-persistent-fast-regional-internal-managedfor internal (Google Cloud-only) IP addresses with fast failover
Gateway classes work within specific regions. Gateway classes offer basic IP address management and are focused on Layer 3 (L3) network routing.
Gateway objects
Gateway objects
serve as the central point for managing and configuring
persistent IP addresses. Gateway objects in GKE manage a pool
of persistent IP addresses. They list these addresses and define rules for how
these IP addresses can be assigned to GKEIPRoute
.
Listener
A Listener is a part of your GKE Gateway configuration that controls which Pods across Gateway namespaces can use the persistent IP addresses held by the Gateway. The Listener lets you customize access for flexibility and security. Each Listener needs a unique name and lets you filter access by namespace (all, label-based, or only the Gateway's namespace).
GKEIPRoute Object
The GKEIPRoute
object is a custom resource which you configure to
assign a persistent IP address to a specific Pod in your GKE
cluster. You can use the status section of the GKEIPRoute
object to monitor
your persistent IP address setup, which provides key information through the
following fields:
-
Pod
The Pod field shows you the exact name of the Pod linked to the persistent IP addresses. A single Pod can use multiple persistent IP addresses.
-
Conditions
The Conditions field informs whether your external IP address setup is working correctly and can also help diagnose issues if your configuration is not valid. There are four conditions:
-
Accepted:denotes if theGKEIPRouteresource specification is valid. If your configuration has errors, theAcceptedcondition isFalsewith a reason. -
GCPReady:denotes that Google Cloud has prepared all necessary resources. Errors during the Google Cloud resource provisioning process are reflected in theGCPReadycondition's status. -
DPV2Ready:denotes the status of datapath programming, such as the datapath is ready and programmed to allow network connections on the configured persistent IP addresses. -
Ready:denotes that your persistent IP address setup is valid and functional. The Pods can be reached on the persistent IP addresses provided you have configured your application to use it. This is set toTruewhen all the other three preceding conditions are alsoTrue.
-
-
nodeSelectorWhen using fast failover, you must use the
nodeSelectorfield to specify which nodes are preconfigured for fast failover. This ensures that when a failover occurs, you can quickly bring the new Pod online on a designated node with the persistent IP address. You must ensure that you schedule your Pods with persistent IP addresses on these nodes. Typically, you do this by using the Pod'snodeSelectorto match labels on the nodes. Use thenodeSelectorfield when configuring fast failover, and ensure that the specified labels don't select more than 64 nodes.
Reaction Modes
Reaction Modes determine how the system behaves when the Pod linked to a persistent IP address undergoes changes, such as moving across nodes or when a newly-created matching Pod becomes available. You can use reaction modes to keep your persistent IP addresses usable even as your Pods change.
Reaction modes are one of:
-
ReadyCondition
In ReadyCondition mode, the persistent IP address system prioritizes Pod health. The persistent IP address system only assigns IP addresses to Pods that match your specified labels and have passed Kubernetes health probes, signaling their
Readystatus asTrue. This mode is ideal for applications where it's crucial that the Pod receiving the persistent IP address is fully prepared to handle incoming and outgoing traffic. -
Exists
The Exists mode prioritizes the presence of a Pod. The persistent IP address attaches to a Pod if that Pod matches the labels in your configuration and has been scheduled onto a specific node in your cluster. This means the Pod exists and has a designated place to run. This mode is well-suited for scenarios where rapid assignment of the persistent IP address takes priority over strict readiness, or in environments like development and testing where immediate connectivity might be more important than full application health.
StatefulSets
StatefulSets are a type of Kubernetes workload designed for applications that need stable identifiers and persistent storage. Pods within a StatefulSet have predictable names (For example: my-app-0, my-app-1).
Deployments
Deployments are a type of Kubernetes workload for managing stateless applications where Pods are generally interchangeable. Pod names within Deployments aren't fully predictable.
Use cases
Persistent IP addresses for GKE Pods address several use cases for network and security service providers running network-related applications on GKE.
Persistent IP addresses for GKE Pods address the following use cases:
- Control over NAT:by assigning persistent IP addresses to Pods running network functions, you get granular control over the source IP addresses used for outbound traffic. This lets you to integrate your proprietary NAT logic.
- Dedicated IP address pools:dedicated IP addresses let you to match specific addresses to individual 5G Core Pods, ensuring compatibility with specialized vendor software.
- Reliable Traffic Flows:since return traffic needs to be routed back through the same network function, persistent IP addresses ensure that external systems recognize and respond to the correct Pod without breaks in communication.
- High availability for time-sensitive applications:for applications with strict uptime requirements, such as in the telecommunications industry, you can use persistent IP addresses with fast failover to quickly redirect traffic to a healthy Pod in the event of a failure, which can minimize the downtime to only a few seconds.
Benefits
Persistent IP addresses for GKE Pods provide you the following benefits:
- External Identity:if you give a Pod an external persistent IP address, external systems can consistently reach that Pod, even if it gets restarted or moved within the cluster. This is helpful for services that need an externally discoverable endpoint.
- Reliable Communication:applications that depend on other resources with specific IP addresses can reliably establish connections using persistent IP addresses. Persistent IP addresses are important for legacy systems or applications with hardcoded IP address dependencies.
- Legacy migrations: legacy migrations can assist in migrating applications that rely on having specific IP addresses during the transition process.
- BYOIP: BYOIP lets you maintain control over specific IP address ranges you already own by using them within your GKE clusters.
- Improved application uptime: with the fast failover capability, you can significantly improve the uptime of your critical applications by minimizing downtime to only a few seconds in the event of a Pod failure.
