Frequently Asked Questions - Cloud TPU

This document contains a list of frequently asked questions about Cloud TPUs. It is broken up into sections:

Framework independent FAQs - questions about using Cloud TPUs regardless of what ML framework you are using.
JAX FAQS - questions about using Cloud TPUs with JAX.
PyTorch FAQs - questions about using Cloud TPUs with PyTorch.

Framework independent FAQs

How do I check which process is using the TPU on a Cloud TPU VM?

Run tpu-info on the Cloud TPU VM to print the process ID and other information about the process using the TPU. See supported metrics for the metrics and their corresponding definitions.

   
tpu-info

The output from tpu-info is similar to the following:

   
TPU  
Chips  
┏━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━┓  
┃  
Chip  
┃  
Type  
┃  
Devices  
┃  
PID  
┃  
┡━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━┩  
│  
/dev/accel0  
│  
TPU  
v4  
chip  
│  
 1 
  
│  
 130007 
  
│  
│  
/dev/accel1  
│  
TPU  
v4  
chip  
│  
 1 
  
│  
 130007 
  
│  
│  
/dev/accel2  
│  
TPU  
v4  
chip  
│  
 1 
  
│  
 130007 
  
│  
│  
/dev/accel3  
│  
TPU  
v4  
chip  
│  
 1 
  
│  
 130007 
  
│  
└─────────────┴─────────────┴─────────┴────────┘  
TPU  
Runtime  
Utilization  
┏━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓  
┃  
Device  
┃  
Memory  
usage  
┃  
Duty  
cycle  
┃  
┡━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩  
│  
 0 
  
│  
 0 
.00  
GiB  
/  
 31 
.75  
GiB  
│  
 0 
.00%  
│  
│  
 1 
  
│  
 0 
.00  
GiB  
/  
 31 
.75  
GiB  
│  
 0 
.00%  
│  
│  
 2 
  
│  
 0 
.00  
GiB  
/  
 31 
.75  
GiB  
│  
 0 
.00%  
│  
│  
 3 
  
│  
 0 
.00  
GiB  
/  
 31 
.75  
GiB  
│  
 0 
.00%  
│  
└────────┴──────────────────────┴────────────┘  
TensorCore  
Utilization  
┏━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┓  
┃  
Chip  
ID  
┃  
TensorCore  
Utilization  
┃  
┡━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━┩  
│  
 0 
  
│  
 0 
.00%  
│  
│  
 1 
  
│  
 0 
.00%  
│  
│  
 3 
  
│  
 0 
.00%  
│  
│  
 2 
  
│  
 0 
.00%  
 | 
  
└─────────┴────────────────────────┘  
Buffer  
Transfer  
Latency  
┏━━━━━━━━━━━━━┳━━━━━┳━━━━━┳━━━━━┳━━━━━━┓  
┃  
Buffer  
Size  
┃  
P50  
┃  
P90  
┃  
P95  
┃  
P999  
┃  
┡━━━━━━━━━━━━━╇━━━━━╇━━━━━╇━━━━━╇━━━━━━┩  
│  
8MB+  
 | 
  
0us  
│  
0us  
│  
0us  
│  
0us  
 | 
  
└─────────────┴─────┴─────┴─────┴──────┘

How do I add a persistent disk volume to a Cloud TPU VM?

For more information, see Add a persistent disk to a TPU VM .

What storage options are supported or recommended for training with TPU VM?

For more information, see Cloud TPU storage options .

JAX FAQs

How do I know if the TPU is being used by my program?

There are a few ways to double check JAX is using the TPU:

Use the jax.devices() function. For example:

 assert jax.devices()[0].platform == 'tpu'

Profile your program and verify the profile contains TPU operations. For more information, see Profiling JAX programs

For more information, see JAX FAQ

Pytorch FAQs

How do I know if the TPU is being used by my program?

You can run following python commands:

 >>> import 
  
 torch_xla.core.xla_model 
  
 as 
  
 xm 
>>> xm 
 . 
 get_xla_supported_devices 
 ( 
 devkind 
 = 
 "TPU" 
 )

And verify if you can see any TPU devices.