This page describes how to use cross-bucket replication , which uses Storage Transfer Service to copy new and updated objects asynchronously from a source bucket to a destination bucket. When you use cross-bucket replication, you create and manage replication jobs, which are a type of job in Storage Transfer Service.
Before you begin
To use cross-bucket replication, you must complete prerequisite steps. For instructions, refer to Before you begin in the Storage Transfer Service documentation.
Create a replication job
Console
When you create a cross-bucket replication job using the
Google Cloud console, the Storage Transfer Service service agent is
automatically granted the storage.legacyBucketOwner
role on the source
bucket and the storage.legacyBucketWriter
role on the destination
bucket.
When using the Google Cloud console, you can create a replication job for existing buckets or for new buckets during the bucket creation process.
To create a replication job for a new bucket, follow the instructions for creating a new bucket .
To create a replication job for an existing bucket, complete the following steps:
- In the Google Cloud console, go to the Cloud Storage Buckets
page.
-
In the list of buckets, click the name of the source bucket whose objects you want to replicate.
-
On the Bucket detailspage, click the Configurationtab.
-
Locate the Cross-bucket replicationoption and click Edit.
-
In the Edit cross-bucket replicationpane that opens, click Add a destination.
-
In the Choose a destinationsection, select a destination bucket, then click Next.
-
In the Choose replication settingssection, do the following:
-
Optional: To filter objects to replicate by object name prefix, select the Replicate objects based on prefixcheckbox in the Choose which objects to replicatesection.
-
To include objects by prefix, enter a prefix in the Include objects with prefixsection, then click Add a prefix.
-
To exclude objects by prefix, enter a prefix in the Exclude objects with prefixsection, then click Add a prefix.
-
-
Optional: To set a storage class for replicated objects, select a storage class from the menu in the Set storage class for replicated objectssection.
If you skip this step, replicated objects use the destination bucket's storage class by default.
- Click Save.
-
Command line
For Google Cloud CLI instructions on how to create a replication job, refer to Create a replication job using the Google Cloud CLI .
REST APIs
JSON API
For JSON API instructions on how to create a replication job, refer to Create a replication job using the JSON API .
To check the status of the replication job, view Cloud Logging for Storage Transfer Service logs .
List replication jobs
Console
You cannot list replication jobs using the Google Cloud console. See View replication job for instructions on how to view a single replication job at a time.
Command line
For Google Cloud CLI instructions on how to list replication jobs, refer to List replication jobs using the Google Cloud CLI .
REST APIs
JSON API
For JSON API instructions on how to list replication jobs, refer to List replication jobs using the JSON API .
View a replication job
Console
- In the Google Cloud console, go to the Cloud Storage Buckets
page.
-
In the list of buckets, click the name of the source bucket whose cross-bucket replication job you want to view.
-
On the Bucket detailspage, click the Configurationtab.
-
Locate the Cross-bucket replicationoption and click Edit.
The Edit cross-bucket replicationpane appears, which displays the replication job for each destination bucket.
On the Bucketspage, you can view the Replicationcolumn, which displays whether a bucket has a Turbo replication job or a cross-bucket replication job running. For instructions on displaying the Replicationcolumn, see Show columns .
Command line
For Google Cloud CLI instructions on how to view a replication job, refer to View a replication job using the Google Cloud CLI .
REST APIs
JSON API
For JSON API instructions on how to view a replication job, refer to View a replication job using the JSON API .
Update a replication job
You can update the following fields of a replication job:
-
The description of the replication job
-
The configuration for running a replication job
-
The configuration of notifications published to Pub/Sub
-
The logging behavior for replication job operations
-
The status of the replication job (whether it's enabled, disabled, or deleted)
Console
When using the Google Cloud console, you can only update a replication job by pausing or unpausing the job.
- In the Google Cloud console, go to the Cloud Storage Buckets
page.
-
In the list of buckets, click the name of the source bucket that you want to pause or unpause replicating.
-
On the Bucket detailspage, click the Configurationtab.
-
Locate the Cross-bucket replicationoption and click Edit.
-
In the Edit cross-bucket replicationpane that appears, click Pauseor Unpausenext to the replication job you want to update.
Command line
For Google Cloud CLI instructions on how to update a replication job, refer to Update a replication job using the Google Cloud CLI .
REST APIs
JSON API
For JSON API instructions on how to update a replication job, refer to Update a replication job using the JSON API .
Delete a replication job
Console
- In the Google Cloud console, go to the Cloud Storage Buckets
page.
-
In the list of buckets, click the name of the source bucket you want to stop replicating.
-
On the Bucket detailspage, click the Configurationtab.
-
Locate the Cross-bucket replicationoption and click Edit.
-
In the Edit cross-bucket replicationpane that appears, click Deletenext to the replication job you want to delete.
-
In the dialogue that appears, click Confirm.
Command line
For Google Cloud CLI instructions on how to delete a replication job, refer to Delete a replication job using the Google Cloud CLI .
REST APIs
JSON API
For JSON API instructions on how to delete a replication job, refer to Delete a replication job using the JSON API .
