Troubleshoot errors in Cluster Director

This document explains how to resolve common issues and errors that you might encounter when creating or modifying a cluster in Cluster Director. If your error isn't listed in this document, then see the following troubleshooting documentation for Compute Engine resources, Cloud Storage resources, or Google Cloud networking services:

• Troubleshoot creating, updating, and deleting virtual machine (VM) instances

• Troubleshoot Compute Engine reservation consumption

• Troubleshoot Filestore instance mounting issues

• Troubleshoot Managed Lustre instances

• Troubleshoot Cloud Storage bucket issues

• Troubleshoot common networking issues

• Troubleshoot VPC policy and access problems

General cluster errors

This section describes common errors related to resource capacity, quotas, or location during cluster creation or modification.

Insufficient quota error

This error occurs when you attempt to create virtual machine (VM) instances by using the Flex-start, Spot, or On-demand consumption options, but your project lacks sufficient quota for the requested resources.

Error messages:

  RESOURCE_NAME 
creation failed: Quota QUOTA_NAME 
exceeded. Limit QUOTA_LIMIT 
in region REGION 
 

  RESOURCE_NAME 
creation failed: Quota QUOTA_NAME 
exceeded. Limit QUOTA_LIMIT 
in zone ZONE 
 

 Resource exhausted (HTTP 429): QUOTA_EXCEEDED 

Resolution: Check that your project has sufficient quota for the resources that you want to request, and then retry your cluster creation or modification request. For more information about managing quota, see View and manage quotas .

Resource unavailability error

This error occurs when a specified compute resource isn't available in the region or zone where you are attempting to create your cluster.

Error messages:

 ZONE_RESOURCE_POOL_EXHAUSTED 

 The zone 'projects/ PROJECT_ID 
/zones/ ZONE 
' does not have enough resources available to fulfill the request. Try a different zone, or try again later. 

 A MACHINE_TYPE 
VM instance with RESOURCE_ATTACHMENT 
is currently unavailable in the ZONE 
zone. 

Resolution: All compute resources that you specify when creating a cluster must be available. To resolve this issue, do the following:

• If you want to use the Flex-start, Spot, or On-demand consumption option, then try one of the following:

  • Create the cluster in a different region or zone.

  • Create the cluster at a later time when resources become available.

  • Create the cluster by using different compute or storage configuration.

• Otherwise, for a higher guarantee of capacity, reserve capacity by using one of the following methods. Then, if your reservation request is approved, try creating your cluster again at the request's start time.

  • Use future reservations for blocks of capacity

  • Use future reservations in calendar mode

Region or zone availability error

This error occurs when your requested compute resource isn't available in the specified region or zone.

Error messages:

 notFound 

 does not exist in zone 

Resolution: Select a region or zone that supports your requested compute resources. To review available regions and zones in Compute Engine, see Available regions and zones .

A4X resource pool exhausted

This error occurs when your cluster attempts to use more A4X subblocks than you have reserved, which happens when the number of VMs in your nodeset isn't a multiple of 18. Because Compute Engine provisions A4X capacity in indivisible sub-blocks to create 18 VMs, any remaining capacity in a sub-block can't be used by other nodesets in your cluster.

Error messages:

 ZONE_RESOURCE_POOL_EXHAUSTED 

 ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS 

Resolution: Modify your cluster so that the total number of VMs for each nodeset with A4X VMs is a multiple of 18. This total is the sum of the static node count and the dynamic node count. For instructions, see Modify a cluster .