This document describes how to create a TPU virtual machine (VM) instance. A TPU VM is also called a single-host TPU slice.
To create a group of single-host slices, see Create a MIG with single-host TPU slices .
Before you begin
- If you haven't already, set up authentication
.
Authentication verifies your identity for access to Google Cloud services and APIs. To run
code or samples from a local development environment, you can authenticate to
Compute Engine by selecting one of the following options:
-
Install the Google Cloud CLI. After installation, initialize the Google Cloud CLI by running the following command:
gcloud init
If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity .
- Set a default region and zone .
-
Prerequisites
Before you proceed to create a TPU instance, you must do the following:
-
Choose the TPU version: select the TPU version that is suitable for your workload. For a list of TPU versions by workload type, see Recommended TPU versions by workload types .
-
Validate TPU availability in your preferred location: TPUs are available in specific Google Cloud regions. To use a TPU version, ensure its availability in your preferred region. For a list of TPU locations, see TPU availability .
-
Ensure that your project has sufficient TPU quota: if you are creating a TPU instance with on-demand or Spot VMs, you must have sufficient TPU quota available in the region that you want to use. Creating a TPU instance that consumes a TPU reservation doesn't require any TPU quota as quota is used when the reservation was created. For a list of TPU quota names, see TPU quota and for instructions on how to view the quota, see View and manage quotas
-
Choose a TPU consumption option: select a consumption option that best fits your workload, its duration, and your cost needs. For a list of consumption option availability by TPU versions, see TPU consumption options .
Create a TPU VM instance
The parameters you use to create a TPU VM instance depend on the consumption option you are using: on-demand, Spot, reservation-bound, or flex-start. For more information, see About VM provisioning models .
Create an on-demand TPU VM instance
To create an on-demand
TPU VM instance, use the gcloud compute instances create
command
:
gcloud
compute
instances
create
TPU_NAME
\
--machine-type =
MACHINE_TYPE
\
--image-family =
IMAGE_FAMILY
\
--image-project =
IMAGE_PROJECT
\
--zone =
ZONE
\
--maintenance-policy =
TERMINATE
Replace the following placeholders:
- TPU_NAME : A name for your TPU VM.
- MACHINE_TYPE
: The machine type
for the TPU VM (for example
ct6e-standard-8t). - IMAGE_FAMILY
: The OS image family
for the TPU VM. If you want to install a specific OS version, use the
--imageflag. For more information about OS images, see OS images . - IMAGE_PROJECT
: The project that contains the OS image.
For TPU images, this is
ubuntu-os-accelerator-images. - ZONE
: The zone
for the TPU VM (for example
us-central1-b).
Create a TPU Spot VM instance
To create a TPU Spot VM instance, use the gcloud compute instances create
command
with the --provisioning-model=SPOT
flag:
gcloud
compute
instances
create
TPU_NAME
\
--machine-type =
MACHINE_TYPE
\
--image-family =
IMAGE_FAMILY
\
--image-project =
IMAGE_PROJECT
\
--zone =
ZONE
\
--provisioning-model =
SPOT
\
--instance-termination-action =
DELETE
\
--maintenance-policy =
TERMINATE
Replace the following placeholders:
- TPU_NAME : A name for your TPU VM.
- MACHINE_TYPE
: The machine type
for the TPU VM (for example
ct6e-standard-8t). - IMAGE_FAMILY
: The OS image family
for the TPU VM. If you want to install a specific OS version, use the
--imageflag. For more information about OS images, see OS images . - IMAGE_PROJECT
: The project that contains the OS image.
For TPU images, this is
ubuntu-os-accelerator-images. - ZONE
: The zone
for the TPU VM (for example
us-central1-b).
Create a TPU VM instance using a reservation
To create a TPU VM instance using the reservation-bound
consumption option,
use the gcloud compute instances create
command
with the --reservation-affinity=specific
and --reservation
flags:
gcloud
compute
instances
create
TPU_NAME
\
--machine-type =
MACHINE_TYPE
\
--image-family =
IMAGE_FAMILY
\
--image-project =
IMAGE_PROJECT
\
--zone =
ZONE
\
--provisioning-model =
reservation-bound
\
--reservation-affinity =
specific
\
--reservation =
RESERVATION_NAME
\
--instance-termination-action =
DELETE
\
--maintenance-policy =
TERMINATE
Replace the following placeholders:
- TPU_NAME : A name for your TPU VM.
- MACHINE_TYPE
: The machine type
for the TPU VM (for example
ct6e-standard-8t). - IMAGE_FAMILY
: The OS image family
for the TPU VM. If you want to install a specific OS version, use the
--imageflag. For more information about OS images, see OS images . - IMAGE_PROJECT
: The project that contains the OS image.
For TPU images, this is
ubuntu-os-accelerator-images. - ZONE
: The zone
for the TPU VM (for example
us-central1-b). - RESERVATION_NAME : The name of your reservation.
Create a TPU Flex-start VM
To create a TPU Flex-start
VM instance, use the gcloud compute instances create
command
with the --provisioning-model=FLEX_START
flag:
gcloud
compute
instances
create
TPU_NAME
\
--machine-type =
MACHINE_TYPE
\
--image-family =
IMAGE_FAMILY
\
--image-project =
IMAGE_PROJECT
\
--zone =
ZONE
\
--provisioning-model =
FLEX_START
\
--max-run-duration =
MAX_RUN_DURATION
\
--request-valid-for-duration =
VALID_DURATION
\
--instance-termination-action =
DELETE
\
--maintenance-policy =
TERMINATE
Replace the following placeholders:
- TPU_NAME : A name for your TPU VM.
- MACHINE_TYPE
: The machine type
for the TPU VM (for example
ct6e-standard-8t). - IMAGE_FAMILY
: The OS image family
for the TPU VM. If you want to install a specific OS version, use the
--imageflag. For more information about OS images, see OS images . - IMAGE_PROJECT
: The project that contains the OS image.
For TPU images, this is
ubuntu-os-accelerator-images. - ZONE
: The zone
for the TPU VM (for example
us-central1-b). - MAX_RUN_DURATION
: The maximum run duration for the TPU VM (for example
6h). - VALID_DURATION : The maximum duration for which the request is valid.
For more information about creating Compute Engine instances, see Create and start a Compute Engine instance .
Limitations
-
Lifecycle operations: You can't stop, start, resume, or suspend TPU instances. To change configurations that require a restart or to stop incurring charges, you must delete the instances.
-
Update VM details: You can update the properties that only require a refresh (
REFRESH) action. If a configuration requires a restart (RESTART), then you must recreate the instance.
