Copy data to or from instances

This page shows you how to copy data from:

• Cloud Storage buckets to Filestore file shares
• Computers in your environment to Filestore file shares
• Filestore file shares to Cloud Storage buckets

We recommend the following options for transferring large volumes of data:

• Transfer Appliance : If you're looking to transfer data to Google Cloud offline or from an area with poor internet connectivity.
• Storage Transfer Service : If you want to transfer data to Google Cloud from an online data source, such as another cloud product.
• Transfer service for on-premises data : If you want to transfer data from on-premises to Google Cloud.

Before you begin

To follow these instructions, you must install the gcloud CLI .

Copy data from a Cloud Storage bucket to a Filestore file share

To copy data from a Cloud Storage bucket to a Filestore file share that's mounted on a Compute Engine VM instance, use the gcloud storage rsync command. This command works on both Linux and Windows VMs. To copy data on Linux VM instances, you can also use standard commands like rsync .

gcloud storage rsync gs://data /mnt/filer --recursive

 gcloud storage rsync gs://data Z:\ --recursive 

Copy data from a computer in your environment to a Filestore file share

To copy data from a computer in your environment to a Filestore file share that is mounted on a Compute Engine Linux VM instance, use the gcloud compute scp command . The gcloud compute scp command doesn't work on Windows VM instances.

1. Use SSH to connect to the computer that contains the data you want to copy.

2. Check whether you have the gcloud CLI installed by running the following command:

    which gcloud 
   

   If the gcloud CLI is installed, the command returns the installation path. If no installation path is returned, follow the instructions at Installing gcloud CLI to install it.

3. Copy data to the mounted Filestore file share:

    gcloud compute scp local-data-path 
    client-name 
   :/ mount-directory 
   --project= project-id 
   --zone= zone 
    
   

   where:

   • local-data-path is the path of a local file or directory that contains the data you want to copy to the instance. If you want to copy data from other subdirectories, use the --recurse flag after local-data-path .
   • client-name is the name of the Compute Engine VM instance that the Filestore file share is mounted on.
   • mount-directory is the directory on the client where the Filestore file share is mounted.
   • project-id is the ID of the project that contains the client VM instance. If you have set the gcloud default project by using gcloud config set project project-id , you can skip this flag and use the default project instead.
   • zone is the zone in which the Compute Engine VM instance resides. If you have set the gcloud default zone by using gcloud config set compute/zone zone , you can skip this flag and use default zone instead.

Example

The following command copies data from the local /etc/acme/data directory and its subdirectories to the /mnt/filer directory on the nfs-client VM instance, in the myproject project, in zone us-central1-c :

gcloud compute scp /etc/acme/data --recurse nfs-client:/mnt/filer --project myproject --zone us-central1-c

Copy data from a Filestore file share to a Cloud Storage bucket

You can copy data from a Filestore file share that is mounted on a Compute Engine VM instance to a Cloud Storage bucket by using the gcloud storage rsync command.

Before proceeding, first make sure that the Compute Engine VM can write to Cloud Storage:

1. In the Google Cloud console, go to the VM Instances page.

   Go to the Compute Engine instances page

2. Choose a Compute Engine VM instance that has the source Filestore file share mounted.

3. Check whether the instance has Read Writescope:

   1. Click the instance name to open the instance details page.
   2. Expand Cloud API access scopesand see if Storagehas Read Writescope.

If the Compute Engine VM doesn't have Read Writescope, perform the following steps:

1. Stop the Compute Engine instance by clicking Stop.
2. When the instance has stopped, click Editat the top of the page.
3. Under Access scopes, select Set access for each API.
4. Under Storage, select Read Write, and click Save.
5. When the instance finishes updating, click Startat the top of the page.

To copy data from a Filestore file share to a Cloud Storage bucket, perform the following:

 gcloud storage rsync /mnt/filer gs://archive --recursive 

 gcloud storage rsync Z:\ gs://archive --recursive 

Troubleshooting

Copying data from Cloud Storage to a Filestore instance using the gcloud CLI can be slow. For detailed information on how to improve performance, see Improve performance across Google Cloud resources .