In a Google Distributed Cloud implementation, the control-plane VM for an admin cluster has two attached disks:
The boot disk has the operating system for the VM.
The data disk has credentials and the etcd database, which stores the state of
the admin cluster. That is, the data disk stores all of the Kubernetes objects for
the admin cluster.
This page shows how to recover when the control-plane VM is lost or the boot
disk is compromised. For example:
The boot disk becomes read-only due to spam journal logs.
The Docker overlay filesystem gets corrupted.
This page does not cover recovery of the data disk. For instructions on how to
recover the data disk, seeRestoring an admin cluster.
ADMIN_CLUSTER_CONFIGwith the path of your admin cluster
configuration file.
ADMIN_CLUSTER_KUBECONFIGwith the path of your admin cluster's
kubeconfig file.
Notes
The admin cluster's control-plane VM is cloned into a VM template, which has
all the information needed to re-create the VM. Thegkectl repair admin-mastercommand uses the VM template to create a new VM. Then it attaches a new
boot disk and the existing data disk.
If your cluster nodes get their addresses from a DHCP server, the new VM might
have a different IP address from the original VM.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-09-04 UTC."],[[["\u003cp\u003eIn Google Distributed Cloud, the admin cluster's control-plane VM uses a boot disk for the operating system and a data disk for credentials and the etcd database, which stores the Kubernetes objects.\u003c/p\u003e\n"],["\u003cp\u003eThis page details the recovery process when the control-plane VM is lost or the boot disk is compromised, but it does not include data disk recovery.\u003c/p\u003e\n"],["\u003cp\u003eThe \u003ccode\u003egkectl repair admin-master\u003c/code\u003e command, along with the admin cluster configuration and kubeconfig file paths, can be used to repair the admin cluster's control-plane VM.\u003c/p\u003e\n"],["\u003cp\u003eRepairing the control-plane VM involves creating a new VM from a template, attaching a new boot disk, and then attaching the existing data disk.\u003c/p\u003e\n"]]],[],null,["# Repairing the admin cluster's control-plane VM\n\n\u003cbr /\u003e\n\nIn a Google Distributed Cloud implementation, the control-plane VM for an admin cluster has two attached disks:\n\n- The boot disk has the operating system for the VM.\n\n- The data disk has credentials and the etcd database, which stores the state of\n the admin cluster. That is, the data disk stores all of the Kubernetes objects for\n the admin cluster.\n\nThis page shows how to recover when the control-plane VM is lost or the boot\ndisk is compromised. For example:\n\n- The boot disk becomes read-only due to spam journal logs.\n- The Docker overlay filesystem gets corrupted.\n\nThis page does not cover recovery of the data disk. For instructions on how to\nrecover the data disk, see\n[Restoring an admin cluster](/anthos/clusters/docs/on-prem/1.10/how-to/back-up-and-restore-admin-cluster-with-gkectl).\n\nRepairing the control-plane VM\n------------------------------\n\n| **Warning:** Don't run `gkectl repair admin-master` after a failed admin upgrade attempt. Instead, [resume the admin upgrade](/anthos/clusters/docs/on-prem/1.10/how-to/upgrading#about_resume_admin).\n\nTo repair the admin cluster's control-plane VM: \n\n```\ngkectl repair admin-master --config ADMIN_CLUSTER_CONFIG --kubeconfig ADMIN_CLUSTER_KUBECONFIG\n```\n\nReplace:\n\n- \u003cvar translate=\"no\"\u003eADMIN_CLUSTER_CONFIG\u003c/var\u003e with the path of your admin cluster\n configuration file.\n\n- \u003cvar translate=\"no\"\u003eADMIN_CLUSTER_KUBECONFIG\u003c/var\u003e with the path of your admin cluster's\n kubeconfig file.\n\nNotes\n-----\n\nThe admin cluster's control-plane VM is cloned into a VM template, which has\nall the information needed to re-create the VM. The `gkectl repair admin-master`\ncommand uses the VM template to create a new VM. Then it attaches a new\nboot disk and the existing data disk.\n\nIf your cluster nodes get their addresses from a DHCP server, the new VM might\nhave a different IP address from the original VM."]]
