Stay organized with collectionsSave and categorize content based on your preferences.
Connecting to Cloud Storage Buckets
This page introducesCloud Storageas an option for storing your
machine learning data and training output, and describes how to give your
Cloud TPU access to the data objects on Cloud Storage.
Before you begin
You need a Cloud TPUservice accountin order to access a Cloud Storage
bucket.
Create a Cloud TPU service account for your project.
Follow the instructions found in either theCloud TPU quickstart guidesorManage TPUsto configure your Google Cloud project and create your Cloud TPU
VM and TPU resources.
Write data to Cloud Storage
Console
Go to the Cloud Storage page on the Google Cloud console.
regionis the region where you created the Cloud TPU.
For more information about regions and TPU availability, seeTPU regions and zones.
bucket-nameis the name of the bucket you want to create.
Use thegcloud storage cpcommand to write files to the Cloud Storage bucket:
gcloudstoragecp-rlocal-data-dirgs://bucket-name
wherelocal-data-diris a local path to your data.
For example:$HOME/your-data
Give your Cloud TPU access to Cloud Storage
You need to give your Cloud TPU read and write access to your
Cloud Storage objects. To do that, you must grant the required access to
the Cloud TPU service account used by the Cloud TPU.
The following sections show how to find the Cloud TPU service account and
grant the necessary access.
SelectGrant accessto add a new permission and type the complete service account
name in theNew principalsedit box.
If you arereadingfrom this bucket, you must authorize the TPU Service
Account to read from the resource. Do this by granting the Service
Account theStorage Legacy > Storage Legacy Bucket Readerrole.
If you arewritingto this bucket, you must authorize the TPU Service
Account to write to the resource. Do this by granting the Service
Account theStorage Legacy > Storage Legacy Bucket Writerrole.
CLI
If you arereadingfrom this bucket, grant read permission for
the Cloud TPU service account:
If you want to grant broader permissions instead of granting access to each
bucket explicitly, you can grant the Identity Access Management (IAM) Storage
Admin role to the Cloud TPU service account.
Click theGrant accessbutton to add principals to the project.
Enter the names of the Cloud TPU service account in thePrincipalstext box.
Click theRolesdropdown list.
Enable the following roles:
Project > Viewer
Storage > Storage Admin
Cloud Storage FUSE
Cloud Storage FUSE allows you to
mount and access Cloud Storage buckets as local file systems. This allows
applications to read and write objects in your bucket using standard
file system semantics.
See theCloud Storage FUSE documentationfor details about how Cloud Storage FUSE works and a description of how
Cloud Storage FUSE operations map to Cloud Storage operations. You can find
additional information about how to use Cloud Storage FUSE, such as how to
install the gcsfuse CLI and mounting buckets onGitHub.
Clean up
Disconnect from the Cloud TPU, if you have not already done so:
(vm)$exit
In your Cloud Shell, delete the Cloud TPU:
gcloudcomputetpustpu-vmdeletetpu-name--zone=zone
Verify the VM has been deleted by runninggcloud compute tpus tpu-vm list. The
deletion might take several minutes.
gcloudcomputetpustpu-vmlist--zone=zone
A response like the following indicates your instances have been
successfully deleted.
Listed0items
Run the following command to delete the Cloud Storage bucket and its
contents, replacingbucket-namewith the name of the bucket you
created:
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-09-04 UTC."],[],[],null,["# Connecting to Cloud Storage Buckets\n===================================\n\nThis page introduces [Cloud Storage](/storage/docs) as an option for storing your\nmachine learning data and training output, and describes how to give your\nCloud TPU access to the data objects on Cloud Storage.\n\nBefore you begin\n----------------\n\nYou need a Cloud TPU [service account](/iam/docs/service-accounts)\nin order to access a Cloud Storage\nbucket.\n| **Note:** Once you set up a service account for your project, it stays set while the project is active; you don't have to set up the service account every time you run a training workload in the same project.\n\n1. Create a Cloud TPU service account for your project.\n\n ```bash\n gcloud beta services identity create --service tpu.googleapis.com --project project-id\n ```\n\n The command returns a Cloud TPU service account with following format: \n\n ```\n service-project-number@cloud-tpu.iam.gserviceaccount.com\n ```\n | **Important:** This command creates a [service agent](/iam/docs/service-agents), a special type of Google-managed service account which is not listed in your Service accounts page in the Google Cloud console.\n2. Follow the instructions found in either the\n [Cloud TPU quickstart guides](/tpu/docs/quickstart) or\n [Manage TPUs](/tpu/docs/managing-tpus-tpu-vm)\n to configure your Google Cloud project and create your Cloud TPU\n VM and TPU resources.\n\nWrite data to Cloud Storage\n---------------------------\n\n**Note:** Set up the Cloud Storage bucket in the same location (region) that you set up the Cloud TPU. See [Create buckets](/storage/docs/creating-buckets) for all options available for managing storage buckets. \n\n### Console\n\n1. Go to the Cloud Storage page on the Google Cloud console.\n\n [Go to the Cloud Storage page](https://console.cloud.google.com/storage/browser)\n2. Create a new bucket, specifying the following options:\n\n - A unique name of your choosing.\n - Default storage class: `Standard`\n - Location: The region where you created the Cloud TPU. For more information about regions and TPU availability, see [TPU regions and zones](/tpu/docs/regions-zones).\n\n### CLI\n\n1. Use the `gcloud storage buckets create` command to create a Cloud Storage bucket:\n\n ```bash\n gcloud storage buckets create gs://bucket-name --location region\n ```\n\n where:\n - \u003cvar translate=\"no\"\u003eregion\u003c/var\u003e is the region where you created the Cloud TPU.\n For more information about regions and TPU availability, see\n [TPU regions and zones](/tpu/docs/regions-zones).\n\n - \u003cvar translate=\"no\"\u003ebucket-name\u003c/var\u003e is the name of the bucket you want to create.\n\n2. Use the `gcloud storage cp` command to write files to the Cloud Storage bucket:\n\n ```bash\n gcloud storage cp -r local-data-dir gs://bucket-name\n ```\n\n where \u003cvar translate=\"no\"\u003elocal-data-dir\u003c/var\u003e is a local path to your data.\n For example:`$HOME/your-data`\n\nGive your Cloud TPU access to Cloud Storage\n-------------------------------------------\n\nYou need to give your Cloud TPU read and write access to your\nCloud Storage objects. To do that, you must grant the required access to\nthe Cloud TPU service account used by the Cloud TPU.\nThe following sections show how to find the Cloud TPU service account and\ngrant the necessary access.\n\n### Authorize the Cloud TPU service account\n\nThe recommended way to authorize the Cloud TPU service account is by using\n[fine-grained access control lists (ACLs)](#fine-grained-acl). You can also grant\nbroader permissions using [IAM permissions](#iam-permissions).\n\n#### Using fine-grained ACLs for Cloud TPU (Recommended)\n\nIf you store training data on Cloud Storage, the Cloud TPU\nservice account needs read and write permission on the bucket. \n\n### Console\n\n1. Go to the Cloud Storage browser page to view the buckets you own.\n\n [Go to the Cloud Storage browser](https://console.cloud.google.com/storage/browser)\n\n \u003cbr /\u003e\n\n2. Select the bucket whose ACL you want to modify.\n\n3. Select the **Permissions** tab.\n\n4. Select **Grant access** to add a new permission and type the complete service account\n name in the **New principals** edit box.\n\n5. If you are *reading* from this bucket, you must authorize the TPU Service\n Account to read from the resource. Do this by granting the Service\n Account the `Storage Legacy \u003e Storage Legacy Bucket Reader` role.\n\n6. If you are *writing* to this bucket, you must authorize the TPU Service\n Account to write to the resource. Do this by granting the Service\n Account the `Storage Legacy \u003e Storage Legacy Bucket Writer` role.\n\n### CLI\n\n1. If you are *reading* from this bucket, grant read permission for\n the Cloud TPU service account:\n\n \u003cbr /\u003e\n\n ```bash\n gcloud storage buckets add-iam-policy-binding gs://bucket-name --member=serviceAccount:service-account --role=roles/storage.objectViewer\n \n ```\n\n \u003cbr /\u003e\n\n2. If you are *writing* to this bucket, grant write permission for\n the Cloud TPU service account:\n\n \u003cbr /\u003e\n\n ```bash\n gcloud storage buckets add-iam-policy-binding gs://bucket-name --member=serviceAccount:service-account --role=roles/storage.objectCreator\n ```\n\n \u003cbr /\u003e\n\n#### Using IAM permissions for Cloud TPU (Alternative)\n\nIf you want to grant broader permissions instead of granting access to each\nbucket explicitly, you can grant the Identity Access Management (IAM) Storage\nAdmin role to the Cloud TPU service account.\n\n1. [Go to IAM](https://console.cloud.google.com/iam-admin/iam)\n\n2. Click the **Grant access** button to add principals to the project.\n\n3. Enter the names of the Cloud TPU service account in the\n **Principals** text box.\n\n4. Click the **Roles** dropdown list.\n\n5. Enable the following roles:\n\n - **Project \\\u003e Viewer**\n\n - **Storage \\\u003e Storage Admin**\n\n### Cloud Storage FUSE\n\nCloud Storage FUSE allows you to\nmount and access Cloud Storage buckets as local file systems. This allows\napplications to read and write objects in your bucket using standard\nfile system semantics.\n\nSee the [Cloud Storage FUSE documentation](/storage/docs/gcs-fuse)\nfor details about how Cloud Storage FUSE works and a description of how\nCloud Storage FUSE operations map to Cloud Storage operations. You can find\nadditional information about how to use Cloud Storage FUSE, such as how to\ninstall the gcsfuse CLI and mounting buckets on\n[GitHub.](https://github.com/GoogleCloudPlatform/gcsfuse/tree/master/docs)\n\nClean up\n--------\n\n1. Disconnect from the Cloud TPU, if you have not already done so:\n\n ```bash\n (vm)$ exit\n ```\n2. In your Cloud Shell, delete the Cloud TPU:\n\n ```bash\n gcloud compute tpus tpu-vm delete tpu-name --zone=zone\n ```\n3. Verify the VM has been deleted by running `gcloud compute tpus tpu-vm list`. The\n deletion might take several minutes.\n\n ```bash\n gcloud compute tpus tpu-vm list --zone=zone\n ```\n\n A response like the following indicates your instances have been\n successfully deleted. \n\n Listed 0 items\n\n4. Run the following command to delete the Cloud Storage bucket and its\n contents, replacing \u003cvar translate=\"no\"\u003ebucket-name\u003c/var\u003e with the name of the bucket you\n created:\n\n ```bash\n gcloud storage rm --recursive gs://bucket-name\n ```\n\nWhat's next\n-----------\n\n- For more information about creating Cloud Storage buckets and writing data to those buckets, see the [Cloud Storage create buckets\n documentation](/storage/docs/creating-buckets).\n- For more information about service accounts, see the [authentication overview](/docs/authentication)."]]
