If you have problems with updating Google Distributed Cloud, the following sections might help you to troubleshoot the issue. For more information on what settings can be updated, see What can and cannot be updated in clusters .
Update timeout
The update timeout is dynamically calculated based on the resources to update. However, the calculation isn't always accurate. When the update times out, errors similar to the following are displayed:
-
In the user cluster:
Failed to update the cluster:...timed out waiting for the condition... -
In the admin cluster:
Failed to update the admin cluster:...timed out waiting for the condition...
This kind of timeout error can be safely ignored and you can retry the update command. If you retry the command and it times out again with the same error message, reach out to Cloud Customer Care .
Update contains multiple changes
The gkectl update admin
and gkectl update cluster
commands don't allow
updating multiple settings in one command. When the config contains a diff with
multiple settings being changed, and an error similar to the following example
is returned:
Update
summary
for
cluster
X
:
antiAffinityGroups
:
enabled
to
be
set
to
true
from
false
& config
.
AAGSpec
{
-
Enabled
:
false
,
+
Enabled
:
true
,
}
user
master
cpu
to
be
set
to
5
from
4
config
.
NodePoolProps
{
Role
:
"master"
,
MachineType
:
"standard-master"
,
-
CPUs
:
4
,
+
CPUs
:
5
,
MemoryMB
:
8192
,
Replicas
:
3
,
...
//
2
identical
fields
Labels
:
nil
,
NodeTaints
:
nil
,
-
Vsphere
:
nil
,
& config
.
NodePoolVsphereSpec
{
Datastore
:
"lifecycle-workloads1-datastore1"
},
+
Vsphere
:
nil
,
BootDiskSizeGB
:
nil
,
OSImageType
:
""
,
...
//
5
identical
fields
}
Exit
with
error
:
Failed
to
update
the
cluster
:
the
update
contains
multiple
changes
.
Please
update
only
one
feature
at
a
time
This error could happen for various reasons, including the following:
- A mistake or misconfiguration.
- You ran
gkectl upgradebefore with the configuration diff, and expected the changes to be applied.-
gkectl upgradedoesn't apply any configuration diffs except the version bump.
-
- You edited the configuration for another feature update before, but forgot to
run the
gkectl updatecommand.
If you encounter this behaviour, review the diff in the error message and update
the required settings one by one with multiple gkectl update
commands. To help
identify changes, you can use gkectl get-config
to generate configuration files from a cluster
and view the existing state and configuration.
Unsupported changes
The gkectl update cluster
and gkectl update admin
commands ignore
unsupported changes, and display error messages similar to the following
examples:
detected unsupported changes: (-current +desired)
...
- AdvancedNetworking: &true,
+ AdvancedNetworking: &false,
...
, which will be ignored
If you encounter this behaviour, review the diff in the error message and perform the following actions:
- If the change is unintended, edit the config YAML file and update with only
the correct, intended changes.
- In the previous example, if you didn't intend to disable
AdvancedNetworking, setadvancedNetworking: truein the config YAML file.
- In the previous example, if you didn't intend to disable
- If the change is intended, the error indicates that the change isn't
supported. Perform one of the following actions:
- Recreate the cluster if applicable.
- Reach out to Google Support .
OS image doesn't exist
The gkectl update cluster
and gkectl update admin
commands might fail with OS Images
preflight check failures similar to the following examples:
-
In the user cluster:
- Validation Category : OS Images - [ FAILURE ] User cluster OS images exist : os images [ xxxx ] don ' t exist , please run ` gkectl prepare ` to upload os images . -
In the admin cluster:
- Validation Category : OS Images - [ FAILURE ] Admin cluster OS images exist : os images [ xxxx ] don ' t exist , please run ` gkectl prepare ` to upload os images .
These errors can happen if the OS image was removed unexpectedly in your vCenter environment, such as by a periodical cleanup job.
To re-import the import OS images, run the gkectl prepare
command, as
follows:
gkectl prepare \ --bundle-path /var/lib/gke/bundles/gke-onprem-vsphere- TARGET_VERSION .tgz \ --kubeconfig ADMIN_CLUSTER_KUBECONFIG \ --skip-upload-container-images
Not enough datastore free space for new node pools
When you add new node pools, the gkectl update cluster
command might fail with VSphere Datastore FreeSpace
preflight check errors similar to the following
example:
-
[
FAILURE
]
VSphere
Datastore
FreeSpace
:
vCenter
datastore
:
xxxx
insufficient
FreeSpace
,
requires
at
least
xxx
GB
This failure indicates that the datastore doesn't have sufficient free space to run the new node pools. Use one of the following options helps to provide space for the operation to succeed:
- Free up space from the datastore.
- Configure a different
nodePools[].vsphere.datastoredatastore for the node pool.
What's next
If you need additional assistance, reach out to Cloud Customer Care .
You can also see Getting support for more information about support resources, including the following:
- Requirements for opening a support case.
- Tools to help you troubleshoot, such as logs and metrics.
- Supported components , versions , and features of Google Distributed Cloud for VMware (software only).
