Console
    Start free

    Create a MIG that adds GPU VMs all at once

    This document explains how to create a managed instance group (MIG) to add GPU virtual machine (VM) instances all at once.

    The MIG uses the bulk mode of its target size policy to add the VMs all at once. It also uses the flex-start provisioning model to improve obtainability of resources. For more information about the flex-start provisioning model, see About provisioning models .

    Another method to add GPU VMs all at once in a MIG is by creating resize requests. To determine whether bulk mode or resize request is more suitable for your workload, see the Comparison of bulk mode and resize request .

    If your workload doesn't require the MIG to creates VMs all at once but you still want to improve resource obtainability, use the flex-start provisioning model as described in Create a MIG with Flex-start VMs .

    Before you begin

    • To make sure that you have sufficient GPU quota for the resources you're requesting, check your GPU quota .
    • To understand quota consumption, read GPU VMs and preemptible allocation quotas .
    • If you haven't already, set up authentication . Authentication verifies your identity for access to Google Cloud services and APIs. To run code or samples from a local development environment, you can authenticate to Compute Engine by selecting one of the following options:

      Select the tab for how you plan to use the samples on this page:

      Console

      When you use the Google Cloud console to access Google Cloud services and APIs, you don't need to set up authentication.

      gcloud

      1. Install the Google Cloud CLI. After installation, initialize the Google Cloud CLI by running the following command:

        gcloud  
init

        If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity .

      2. Set a default region and zone .

      REST

      To use the REST API samples on this page in a local development environment, you use the credentials you provide to the gcloud CLI.

        Install the Google Cloud CLI.

        If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity .

      For more information, see Authenticate for using REST in the Google Cloud authentication documentation.

    Limitations

    The following limitations apply:

    Create a MIG and add GPU VMs all at once

    To create the GPU VMs all at once, do the following:

    1. Create an instance template , which is required to create a MIG. The MIG then uses this template to create each VM in the group. In the template, specify the configurations for the flex-start provisioning model.

      For more information about instance templates, see About instance templates .

    2. Create a MIG for bulk provisioning of VMs . During the MIG creation, enable the bulk provisioning capability in the MIG by setting the target size policy mode to BULK . In BULK mode, the MIG attempts to create all VMs at once to meet the target size that you set.

    Create an instance template

    Create an instance template that specifies a GPU machine series . Then, use the template to create a MIG .

    Note: If you want to run data science or machine learning workloads, consider using a Deep Learning VM image when you create an instance template. Deep Learning VM Images is a set of prepackaged VM images that comes with machine learning frameworks and essential tools. For more information about these images, see Choose an image in the Deep Learning VM Images documentation.

    Permissions required for this task

    To perform this task, you must have the following permissions :

    Console

    1. Go to the Instance templatespage.

      Go to Instance templates

    2. Click Create instance template. The Create an instance templatepage opens.

    3. In the Namefield, enter a name for the instance template.

    4. In the Machine configurationsection, do the following:

      1. Click the GPUstab.

      2. In the GPU typelist, select the GPU type.

      3. In the Number of GPUslist, select the number of GPUs.

      4. In the Machine typesection, select a machine type.

    5. In the Provisioning modelsection, do the following:

      1. In the VM provisioning modellist, select Flex-start.

      2. To set a run duration for the VMs created through the instance template, in the Enter number of hoursfield, enter the number of hours. The value must be between one hour ( 1 ) and seven days ( 168 ).

    6. Optional: To change the default value boot disk type or image, in the Boot disksection, click Change. Then, follow the prompts to change the boot disk.

    7. Click Create.

    gcloud

    Create an instance template by using the instance-templates create command :

    gcloud compute instance-templates create INSTANCE_TEMPLATE_NAME 
\
    --image-project= IMAGE_PROJECT 
\
    --image-family= IMAGE_FAMILY 
\
    --instance-termination-action=DELETE \
    --instance-template-region= REGION 
\
    --machine-type= MACHINE_TYPE 
\
    --maintenance-policy=TERMINATE \
    --max-run-duration= RUN_DURATION 
\
    --provisioning-model=FLEX_START \
    --reservation-affinity=none

    Replace the following:

    • INSTANCE_TEMPLATE_NAME : the name of the instance template.

    • IMAGE_PROJECT : the image project that contains the image—for example, debian-cloud . For more information about the supported image projects, see Public images .

    • IMAGE_FAMILY : an image family . This specifies the most recent, non-deprecated OS image. For example, if you specify debian-12 , the latest version in the Debian 12 image family is used. For more information about using image families, see Image families best practices .

    • REGION : the region in which to create the instance template.

    • MACHINE_TYPE : a GPU machine type . If you specify an N1 machine type, then include the --accelerator flag to specify the number and type of GPUs to attach to your VMs.

    • RUN_DURATION : the duration you want the requested VMs to run. You must format the value as the number of days, hours, minutes, or seconds followed by d , h , m , or s respectively. For example, specify 30m for 30 minutes or 1d2h3m4s for one day, two hours, three minutes, and four seconds. The value must be between 10 minutes and seven days.

    REST

    Create an instance template by making a POST request to the instanceTemplates.insert method :

    POST https://compute.googleapis.com/compute/v1/projects/ PROJECT_ID 
/regions/ REGION 
/instanceTemplates

{
  "name": " INSTANCE_TEMPLATE_NAME 
",
  "properties": {
    "disks": [
      {
        "boot": true,
        "initializeParams": {
          "sourceImage": "projects/ IMAGE_PROJECT 
/global/images/ IMAGE 
}
      }
    ],
    "machineType": " MACHINE_TYPE 
",
    "networkInterfaces": [
      {
        "network": "global/networks/default"
      }
    ],
    "reservationAffinity": {
      "consumeReservationType": "NO_RESERVATION"
    },
    "scheduling": {
      "instanceTerminationAction": "DELETE",
      "maxRunDuration": {
        "seconds": RUN_DURATION 
},
      "onHostMaintenance": "TERMINATE",
      "provisioningModel": "FLEX_START"
    }
  }
}

    Replace the following:

    • PROJECT_ID : the ID of the project in which you want to create the instance template.

    • REGION : the region in which to create the instance template.

    • INSTANCE_TEMPLATE_NAME : the name of the instance template.

    • IMAGE_PROJECT : the image project that contains the image—for example, debian-cloud . For more information about the supported image projects, see Public images .

    • IMAGE : specify one of the following:

      • A specific version of the OS image—for example, debian-12-bookworm-v20240617 .

      • An image family , which must be formatted as family/ IMAGE_FAMILY . This specifies the most recent, non-deprecated OS image. For example, if you specify family/debian-12 , the latest version in the Debian 12 image family is used. For more information about using image families, see Image families best practices .

    • MACHINE_TYPE : a GPU machine type . If you specify an N1 machine type, then include the guestAccelerators field to specify the number and type of GPUs to attach to your VMs.

    • RUN_DURATION : the duration, in seconds, you want the requested VMs to run before the MIG automatically deletes them. The value must be between 600 , which is 600 seconds (10 minutes), and 604800 , which is 604,800 seconds (seven days).

    After you create the instance template, you can view it to see its ID and review its instance properties.

    Create a MIG and add GPU VMs all at once

    Create a MIG as described in this section. When enabling bulk provisioning in the MIG, you must turn off repairs .

    Console

    1. Go to the Instance groupspage.

      Go to Instance groups

    2. Click Create instance group. The Create instance grouppage opens.

    3. In the Namefield, enter a name for the MIG.

    4. Before you select an instance template, you must turn off repairs and set the location as follows:

      1. To turn off repairs, do the following:
        1. In the VM instance lifecyclesection, clear the Allow repair of VMs in an alternate zonecheckbox.
        2. In the Default action on failurelist, select No action.
      2. To set the Locationsection, do the following:
        • To create a zonal MIG, select Single zone.
        • To create a regional MIG, select Multiple zones. In the Target distribution shape, select Any single zone. In the dialog that appears, select Disable instance redistribution.

    5. Go back to the Instance templatefield. In the Instance templatelist, select the instance template that you created in the previous section.

    6. In the Number of instancesfield, specify the number of instances that you want to create all at once.

    7. Select the Obtain VMs all at once with bulk target size policycheckbox.

    8. Click Create.

    gcloud

    To create a MIG with bulk provisioning capability, use the instance-groups managed create command .

    • To create a zonal MIG, run the following command:

       gcloud compute instance-groups managed create INSTANCE_GROUP_NAME 
\
    --default-action-on-vm-failure=do-nothing \
    --size= SIZE 
\
    --target-size-policy-mode=bulk \
    --template= INSTANCE_TEMPLATE_URL 
\
    --zone= ZONE 
\

    • To create a regional MIG, run the following command:

       gcloud compute instance-groups managed create INSTANCE_GROUP_NAME 
\
    --default-action-on-vm-failure=do-nothing \
    --size= SIZE 
\
    --target-size-policy-mode=bulk \
    --template= INSTANCE_TEMPLATE_URL 
\
    --zones= LIST_OF_ZONES 
\

    Replace the following:

    • INSTANCE_GROUP_NAME : the name of the MIG.
    • SIZE : the number of VMs in the MIG.
    • INSTANCE_TEMPLATE_URL : the URL of the instance template that you want to use to create instances in the MIG. The URL can contain either the ID or name of the instance template. Specify one of the following values:
      • For a regional instance template: projects/ PROJECT_ID /regions/ REGION /instanceTemplates/ INSTANCE_TEMPLATE_ID
      • For a global instance template: INSTANCE_TEMPLATE_ID
    • ZONE : the zone where the MIG creates VMs.
    • LIST_OF_ZONES : a comma separated list of zones where the regional MIG creates VMs. For example, us-central1-a, us-central1-b, us-central1-c .

    REST

    To create a MIG that has bulk creation enabled, make a POST request as follows. In the request body, set the targetSizePolicy.mode field to bulk .

    If the instance template that you use specifies the flex-start provisioning model, then you must turn off repairs in the MIG by including the "instanceLifecyclePolicy": {"defaultActionOnFailure": "DO_NOTHING"} field in the following requests.

    • To create a zonal MIG, make a POST request to the instanceGroupManagers.insert method .

       POST https://compute.googleapis.com/compute/v1/projects/ PROJECT_ID 
/zones/ ZONE 
/instanceGroupManagers

{
  "name": " INSTANCE_GROUP_NAME 
",
  "targetSize": SIZE 
,
  "targetSizePolicy": {
    "mode": "bulk"
  },
  "instanceLifecyclePolicy": {
      "defaultActionOnFailure": "DO_NOTHING"
  },
  "versions": [
    {
      "instanceTemplate": " INSTANCE_TEMPLATE_URL 
"
    }
  ]
}

    • To create a regional MIG, make a POST request to the regionInstanceGroupManagers.insert method .

       POST https://compute.googleapis.com/compute/v1/projects/ PROJECT_ID 
/regions/ REGION 
/instanceGroupManagers

{
  "name": " INSTANCE_GROUP_NAME 
",
  "targetSize": SIZE 
,
  "targetSizePolicy": {
    "mode": "bulk"
  },
  "instanceLifecyclePolicy": {
      "defaultActionOnFailure": "DO_NOTHING"
  },
  "versions": [
    {
      "instanceTemplate": " INSTANCE_TEMPLATE_URL 
"
    }
  ]
}

    Replace the following:

    • PROJECT_ID :the project ID.
    • ZONE :the zone where the MIG creates VMs.
    • REGION :the region where the MIG creates VMs.
    • INSTANCE_GROUP_NAME :the name of the MIG.
    • SIZE :the number of VMs in the MIG.
    • INSTANCE_TEMPLATE_URL : the URL of the instance template that you want to use to create instances in the MIG. The URL can contain either the ID or name of the instance template. Specify one of the following values:
      • For a regional instance template: projects/ PROJECT_ID /regions/ REGION /instanceTemplates/ INSTANCE_TEMPLATE_ID
      • For a global instance template: INSTANCE_TEMPLATE_ID

    What's next

    Create a Mobile Website
    View Site in Mobile | Classic
    Share by: