Console
    Contact Us Start free

    Compose objects

    Overview

    This page shows you how to compose Cloud Storage objects into a single object. A compose request takes between 1 and 32 objects and creates a new, composite object. The composite object is a concatenation of the source objects in the order they were specified in the request.

    Note the following when composing objects:

    • The source objects are unaffected by the composition process. If they are meant to be temporary, you must delete them once you've successfully completed the composition.
    • Because other storage classes are subject to early deletion fees , you should always use Standard storage for temporary objects.
    • Frequent deletions associated with compose can increase your storage bill significantly if your bucket has data protection features enabled. Consider disabling soft delete on buckets with high rates of compose operations. With Object Versioning , specify the object versions when deleting source objects to permanently delete them and avoid their becoming noncurrent objects.

    Required roles

    To get the permissions that you need to compose objects, ask your administrator to grant you the Storage Object User ( roles/storage.objectUser ) IAM role on the bucket. This predefined role contains the permissions required to compose objects. To see the exact permissions that are required, expand the Required permissionssection:

    Required permissions

    • storage.objects.create
    • storage.objects.delete
      • This permission is only required if you want to give the object you compose the same name as an object that already exists in the bucket.
    • storage.objects.get
    • storage.objects.list
      • This permission is only required if you want to use wildcards to compose objects with a common prefix without having to list each object separately in your Google Cloud CLI command.

    If you want to set a retention configuration for the object you compose, you'll also need the storage.objects.setRetention permission. To get this permission, ask your administrator to grant you the Storage Object Admin ( roles/storage.objectAdmin ) role instead of the Storage Object User ( roles/storage.objectUser ) role.

    You can also get these permissions with other predefined roles or custom roles .

    For information about granting roles on buckets, see Use IAM with buckets .

    Create a composite object

    Command line

    Use the gcloud storage objects compose command:

    gcloud storage objects compose gs:// BUCKET_NAME 
/ SOURCE_OBJECT_1 
gs:// BUCKET_NAME 
/ SOURCE_OBJECT_2 
gs:// BUCKET_NAME 
/ COMPOSITE_OBJECT_NAME

    Where:

    • BUCKET_NAME is the name of the bucket that contains the source objects.
    • SOURCE_OBJECT_1 and SOURCE_OBJECT_2 are the names of the source objects to use in the object composition.
    • COMPOSITE_OBJECT_NAME is the name you are giving to the result of the object composition.

    Client libraries

    C++

    For more information, see the Cloud Storage C++ API reference documentation .

    To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for client libraries .

      namespace 
  
 gcs 
  
 = 
  
 :: 
 google 
 :: 
 cloud 
 :: 
 storage 
 ; 
 using 
  
 :: 
 google 
 :: 
 cloud 
 :: 
 StatusOr 
 ; 
 []( 
 gcs 
 :: 
 Client 
  
 client 
 , 
  
 std 
 :: 
 string 
  
 const 
&  
 bucket_name 
 , 
  
 std 
 :: 
 string 
  
 const 
&  
 destination_object_name 
 , 
  
 std 
 :: 
 vector<gcs 
 :: 
 ComposeSourceObject 
>  
 const 
&  
 compose_objects 
 ) 
  
 { 
  
 StatusOr<gcs 
 :: 
 ObjectMetadata 
>  
 composed_object 
  
 = 
  
 client 
 . 
 ComposeObject 
 ( 
  
 bucket_name 
 , 
  
 compose_objects 
 , 
  
 destination_object_name 
 ); 
  
 if 
  
 ( 
 ! 
 composed_object 
 ) 
  
 throw 
  
 std 
 :: 
 move 
 ( 
 composed_object 
 ). 
 status 
 (); 
  
 std 
 :: 
 cout 
 << 
 "Composed new object " 
 << 
 composed_object 
 - 
> name 
 () 
 << 
 " in bucket " 
 << 
 composed_object 
 - 
> bucket 
 () 
 << 
 " 
 \n 
 Full metadata: " 
 << 
 * 
 composed_object 
 << 
 " 
 \n 
 " 
 ; 
 }

    C#

    For more information, see the Cloud Storage C# API reference documentation .

    To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for client libraries .

      using 
  
 Google.Apis.Storage.v1.Data 
 ; 
 using 
  
  Google.Cloud.Storage.V1 
 
 ; 
 using 
  
 System 
 ; 
 using 
  
 System.Collections.Generic 
 ; 
 public 
  
 class 
  
 ComposeObjectSample 
 { 
  
 public 
  
 void 
  
 ComposeObject 
 ( 
  
 string 
  
 bucketName 
  
 = 
  
 "your-bucket-name" 
 , 
  
 string 
  
 firstObjectName 
  
 = 
  
 "your-first-object-name" 
 , 
  
 string 
  
 secondObjectName 
  
 = 
  
 "your-second-object-name" 
 , 
  
 string 
  
 targetObjectName 
  
 = 
  
 "new-composite-object-name" 
 ) 
  
 { 
  
 var 
  
 storage 
  
 = 
  
  StorageClient 
 
 . 
  Create 
 
 (); 
  
 var 
  
 sourceObjects 
  
 = 
  
 new 
  
 List<ComposeRequest 
 . 
 SourceObjectsData 
>  
 { 
  
 new 
  
 ComposeRequest 
 . 
 SourceObjectsData 
  
 { 
  
 Name 
  
 = 
  
 firstObjectName 
  
 }, 
  
 new 
  
 ComposeRequest 
 . 
 SourceObjectsData 
  
 { 
  
 Name 
  
 = 
  
 secondObjectName 
  
 } 
  
 }; 
  
 //You could add as many sourceObjects as you want here, up to the max of 32. 
  
 storage 
 . 
 Service 
 . 
 Objects 
 . 
 Compose 
 ( 
 new 
  
 ComposeRequest 
  
 { 
  
 SourceObjects 
  
 = 
  
 sourceObjects 
 , 
  
 Destination 
  
 = 
  
 new 
  
 Google 
 . 
 Apis 
 . 
 Storage 
 . 
 v1 
 . 
 Data 
 . 
 Object 
  
 { 
  
 ContentType 
  
 = 
  
 "text/plain" 
  
 } 
  
 }, 
  
 bucketName 
 , 
  
 targetObjectName 
 ). 
 Execute 
 (); 
  
 Console 
 . 
 WriteLine 
 ( 
 $"New composite file {targetObjectName} was created in bucket {bucketName}" 
  
 + 
  
 $" by combining {firstObjectName} and {secondObjectName}." 
 ); 
  
 } 
 }

    Go

    For more information, see the Cloud Storage Go API reference documentation .

    To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for client libraries .

      import 
  
 ( 
  
 "context" 
  
 "fmt" 
  
 "io" 
  
 "time" 
  
 "cloud.google.com/go/storage" 
 ) 
 // composeFile composes source objects to create a composite object. 
 func 
  
 composeFile 
 ( 
 w 
  
 io 
 . 
  Writer 
 
 , 
  
 bucket 
 , 
  
 object1 
 , 
  
 object2 
 , 
  
 toObject 
  
 string 
 ) 
  
 error 
  
 { 
  
 // bucket := "bucket-name" 
  
 // object1 := "object-name-1" 
  
 // object2 := "object-name-2" 
  
 // toObject := "object-name-3" 
  
 ctx 
  
 := 
  
 context 
 . 
 Background 
 () 
  
 client 
 , 
  
 err 
  
 := 
  
 storage 
 . 
 NewClient 
 ( 
 ctx 
 ) 
  
 if 
  
 err 
  
 != 
  
 nil 
  
 { 
  
 return 
  
 fmt 
 . 
 Errorf 
 ( 
 "storage.NewClient: %w" 
 , 
  
 err 
 ) 
  
 } 
  
 defer 
  
 client 
 . 
 Close 
 () 
  
 ctx 
 , 
  
 cancel 
  
 := 
  
 context 
 . 
 WithTimeout 
 ( 
 ctx 
 , 
  
 time 
 . 
 Second 
 * 
 10 
 ) 
  
 defer 
  
 cancel 
 () 
  
 src1 
  
 := 
  
 client 
 . 
  Bucket 
 
 ( 
 bucket 
 ). 
  Object 
 
 ( 
 object1 
 ) 
  
 src2 
  
 := 
  
 client 
 . 
  Bucket 
 
 ( 
 bucket 
 ). 
  Object 
 
 ( 
 object2 
 ) 
  
 dst 
  
 := 
  
 client 
 . 
  Bucket 
 
 ( 
 bucket 
 ). 
  Object 
 
 ( 
 toObject 
 ) 
  
 // ComposerFrom takes varargs, so you can put as many objects here 
  
 // as you want. 
  
 _ 
 , 
  
 err 
  
 = 
  
 dst 
 . 
  ComposerFrom 
 
 ( 
 src1 
 , 
  
 src2 
 ). 
 Run 
 ( 
 ctx 
 ) 
  
 if 
  
 err 
  
 != 
  
 nil 
  
 { 
  
 return 
  
 fmt 
 . 
 Errorf 
 ( 
 "ComposerFrom: %w" 
 , 
  
 err 
 ) 
  
 } 
  
 fmt 
 . 
 Fprintf 
 ( 
 w 
 , 
  
 "New composite object %v was created by combining %v and %v\n" 
 , 
  
 toObject 
 , 
  
 object1 
 , 
  
 object2 
 ) 
  
 return 
  
 nil 
 }

    Java

    For more information, see the Cloud Storage Java API reference documentation .

    To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for client libraries .

      import 
  
 com.google.cloud.storage. Blob 
 
 ; 
 import 
  
 com.google.cloud.storage. BlobInfo 
 
 ; 
 import 
  
 com.google.cloud.storage. Storage 
 
 ; 
 import 
  
 com.google.cloud.storage. StorageOptions 
 
 ; 
 public 
  
 class 
 ComposeObject 
  
 { 
  
 public 
  
 static 
  
 void 
  
 composeObject 
 ( 
  
 String 
  
 bucketName 
 , 
  
 String 
  
 firstObjectName 
 , 
  
 String 
  
 secondObjectName 
 , 
  
 String 
  
 targetObjectName 
 , 
  
 String 
  
 projectId 
 ) 
  
 { 
  
 // The ID of your GCP project 
  
 // String projectId = "your-project-id"; 
  
 // The ID of your GCS bucket 
  
 // String bucketName = "your-unique-bucket-name"; 
  
 // The ID of the first GCS object to compose 
  
 // String firstObjectName = "your-first-object-name"; 
  
 // The ID of the second GCS object to compose 
  
 // String secondObjectName = "your-second-object-name"; 
  
 // The ID to give the new composite object 
  
 // String targetObjectName = "new-composite-object-name"; 
  
  Storage 
 
  
 storage 
  
 = 
  
  StorageOptions 
 
 . 
 newBuilder 
 (). 
 setProjectId 
 ( 
 projectId 
 ). 
 build 
 (). 
  getService 
 
 (); 
  
 // Optional: set a generation-match precondition to avoid potential race 
  
 // conditions and data corruptions. The request returns a 412 error if the 
  
 // preconditions are not met. 
  
  Storage 
 
 . 
 BlobTargetOption 
  
 precondition 
 ; 
  
 if 
  
 ( 
 storage 
 . 
  get 
 
 ( 
 bucketName 
 , 
  
 tar get 
ObjectName 
 ) 
  
 == 
  
 null 
 ) 
  
 { 
  
 // For a target object that does not yet exist, set the DoesNotExist precondition. 
  
 // This will cause the request to fail if the object is created before the request runs. 
  
 precondition 
  
 = 
  
  Storage 
 
 . 
 BlobTargetOption 
 . 
 doesNotExist 
 (); 
  
 } 
  
 else 
  
 { 
  
 // If the destination already exists in your bucket, instead set a generation-match 
  
 // precondition. This will cause the request to fail if the existing object's generation 
  
 // changes before the request runs. 
  
 precondition 
  
 = 
  
  Storage 
 
 . 
 BlobTargetOption 
 . 
 generationMatch 
 ( 
  
 storage 
 . 
  get 
 
 ( 
 bucketName 
 , 
  
 tar get 
ObjectName 
 ). 
 getGeneration 
 ()); 
  
 } 
  
  Storage 
 
 . 
  ComposeRequest 
 
  
 composeRequest 
  
 = 
  
  Storage 
 
 . 
 ComposeRequest 
 . 
 newBuilder 
 () 
  
 // addSource takes varargs, so you can put as many objects here as you want, up to the 
  
 // max of 32 
  
 . 
  addSource 
 
 ( 
 firstObjectName 
 , 
  
 secondObjectName 
 ) 
  
 . 
 setTarget 
 ( 
  BlobInfo 
 
 . 
 newBuilder 
 ( 
 bucketName 
 , 
  
 targetObjectName 
 ). 
 build 
 ()) 
  
 . 
 setTargetOptions 
 ( 
 precondition 
 ) 
  
 . 
 build 
 (); 
  
  Blob 
 
  
 compositeObject 
  
 = 
  
 storage 
 . 
  compose 
 
 ( 
  compose 
 
Request ); 
  
 System 
 . 
 out 
 . 
 println 
 ( 
  
 "New composite object " 
  
 + 
  
 compositeObject 
 . 
 getName 
 () 
  
 + 
  
 " was created by combining " 
  
 + 
  
 firstObjectName 
  
 + 
  
 " and " 
  
 + 
  
 secondObjectName 
 ); 
  
 } 
 }

    Node.js

    For more information, see the Cloud Storage Node.js API reference documentation .

    To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for client libraries .

      /** 
 * TODO(developer): Uncomment the following lines before running the sample. 
 */ 
 // The ID of your GCS bucket 
 // const bucketName = 'your-unique-bucket-name'; 
 // The ID of the first GCS file to compose 
 // const firstFileName = 'your-first-file-name'; 
 // The ID of the second GCS file to compose 
 // const secondFileName = 'your-second-file-name'; 
 // The ID to give the new composite file 
 // const destinationFileName = 'new-composite-file-name'; 
 // Imports the Google Cloud client library 
 const 
  
 { 
 Storage 
 } 
  
 = 
  
 require 
 ( 
 ' @google-cloud/storage 
' 
 ); 
 // Creates a client 
 const 
  
 storage 
  
 = 
  
 new 
  
 Storage 
 (); 
 async 
  
 function 
  
 composeFile 
 () 
  
 { 
  
 const 
  
 bucket 
  
 = 
  
 storage 
 . 
 bucket 
 ( 
 bucketName 
 ); 
  
 const 
  
 sources 
  
 = 
  
 [ 
 firstFileName 
 , 
  
 secondFileName 
 ]; 
  
 // Optional: 
  
 // Set a generation-match precondition to avoid potential race conditions 
  
 // and data corruptions. The request to compose is aborted if the object's 
  
 // generation number does not match your precondition. For a destination 
  
 // object that does not yet exist, set the ifGenerationMatch precondition to 0 
  
 // If the destination object already exists in your bucket, set instead a 
  
 // generation-match precondition using its generation number. 
  
 const 
  
 combineOptions 
  
 = 
  
 { 
  
 ifGenerationMatch 
 : 
  
 destinationGenerationMatchPrecondition 
 , 
  
 }; 
  
 await 
  
 bucket 
 . 
  combine 
 
 ( 
 sources 
 , 
  
 destinationFileName 
 , 
  
 combineOptions 
 ); 
  
 console 
 . 
 log 
 ( 
  
 `New composite file 
 ${ 
 destinationFileName 
 } 
 was created by combining 
 ${ 
 firstFileName 
 } 
 and 
 ${ 
 secondFileName 
 } 
 ` 
  
 ); 
 } 
 composeFile 
 (). 
 catch 
 ( 
 console 
 . 
 error 
 );

    PHP

    For more information, see the Cloud Storage PHP API reference documentation .

    To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for client libraries .

      use Google\Cloud\Storage\StorageClient; 
 /** 
 * Compose two objects into a single target object. 
 * 
 * @param string $bucketName The name of your Cloud Storage bucket. 
 *        (e.g. 'my-bucket') 
 * @param string $firstObjectName The name of the first GCS object to compose. 
 *        (e.g. 'my-object-1') 
 * @param string $secondObjectName The name of the second GCS object to compose. 
 *        (e.g. 'my-object-2') 
 * @param string $targetObjectName The name of the object to be created. 
 *        (e.g. 'composed-my-object-1-my-object-2') 
 */ 
 function compose_file(string $bucketName, string $firstObjectName, string $secondObjectName, string $targetObjectName): void 
 { 
 $storage = new StorageClient(); 
 $bucket = $storage->bucket($bucketName); 
 // In this example, we are composing only two objects, but Cloud Storage supports 
 // composition of up to 32 objects. 
 $objectsToCompose = [$firstObjectName, $secondObjectName]; 
 $targetObject = $bucket->compose($objectsToCompose, $targetObjectName, [ 
 'destination' => [ 
 'contentType' => 'application/octet-stream' 
 ] 
 ]); 
 if ($targetObject->exists()) { 
 printf( 
 'New composite object %s was created by combining %s and %s', 
 $targetObject->name(), 
 $firstObjectName, 
 $secondObjectName 
 ); 
 } 
 }

    Python

    For more information, see the Cloud Storage Python API reference documentation .

    To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for client libraries .

      from 
  
 google.cloud 
  
 import 
  storage 
 
 def 
  
 compose_file 
 ( 
 bucket_name 
 , 
 first_blob_name 
 , 
 second_blob_name 
 , 
 destination_blob_name 
 ): 
  
 """Concatenate source blobs into destination blob.""" 
 # bucket_name = "your-bucket-name" 
 # first_blob_name = "first-object-name" 
 # second_blob_name = "second-blob-name" 
 # destination_blob_name = "destination-object-name" 
 storage_client 
 = 
  storage 
 
 . 
  Client 
 
 () 
 bucket 
 = 
 storage_client 
 . 
  bucket 
 
 ( 
 bucket_name 
 ) 
 destination 
 = 
 bucket 
 . 
 blob 
 ( 
 destination_blob_name 
 ) 
 destination 
 . 
  content_type 
 
 = 
 "text/plain" 
 # Note sources is a list of Blob instances, up to the max of 32 instances per request 
 sources 
 = 
 [ 
 bucket 
 . 
 blob 
 ( 
 first_blob_name 
 ), 
 bucket 
 . 
 blob 
 ( 
 second_blob_name 
 )] 
 # Optional: set a generation-match precondition to avoid potential race conditions 
 # and data corruptions. The request to compose is aborted if the object's 
 # generation number does not match your precondition. For a destination 
 # object that does not yet exist, set the if_generation_match precondition to 0. 
 # If the destination object already exists in your bucket, set instead a 
 # generation-match precondition using its generation number. 
 # There is also an `if_source_generation_match` parameter, which is not used in this example. 
 destination_generation_match_precondition 
 = 
 0 
 destination 
 . 
  compose 
 
 ( 
 sources 
 , 
 if_generation_match 
 = 
 destination_generation_match_precondition 
 ) 
 print 
 ( 
 "New composite object 
 {} 
 in the bucket 
 {} 
 was created by combining 
 {} 
 and 
 {} 
 " 
 . 
 format 
 ( 
 destination_blob_name 
 , 
 bucket_name 
 , 
 first_blob_name 
 , 
 second_blob_name 
 ) 
 ) 
 return 
 destination

    Ruby

    For more information, see the Cloud Storage Ruby API reference documentation .

    To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for client libraries .

      def 
  
 compose_file 
  
 bucket_name 
 :, 
  
 first_file_name 
 :, 
  
 second_file_name 
 :, 
  
 destination_file_name 
 : 
  
 # The ID of your GCS bucket 
  
 # bucket_name = "your-unique-bucket-name" 
  
 # The ID of the first GCS object to compose 
  
 # first_file_name = "your-first-file-name" 
  
 # The ID of the second GCS object to compose 
  
 # second_file_name = "your-second-file-name" 
  
 # The ID to give the new composite object 
  
 # destination_file_name = "new-composite-file-name" 
  
 require 
  
 "google/cloud/storage" 
  
 storage 
  
 = 
  
 Google 
 :: 
 Cloud 
 :: 
  Storage 
 
 . 
  new 
 
  
 bucket 
  
 = 
  
 storage 
 . 
 bucket 
  
 bucket_name 
 , 
  
 skip_lookup 
 : 
  
 true 
  
 destination 
  
 = 
  
 bucket 
 . 
  compose 
 
  
 [ 
 first_file_name 
 , 
  
 second_file_name 
 ] 
 , 
  
 destination_file_name 
  
 do 
  
 | 
 f 
 | 
  
 f 
 . 
 content_type 
  
 = 
  
 "text/plain" 
  
 end 
  
 puts 
  
 "Composed new file 
 #{ 
 destination 
 . 
 name 
 } 
 in the bucket 
 #{ 
 bucket_name 
 } 
 " 
  
 \ 
  
 "by combining 
 #{ 
 first_file_name 
 } 
 and 
 #{ 
 second_file_name 
 } 
 " 
 end

    REST APIs

    JSON API

    1. Have gcloud CLI installed and initialized , which lets you generate an access token for the Authorization header.

    2. Create a JSON file that contains the following information:

       { 
  
 "sourceObjects" 
 : 
  
 [ 
  
 { 
  
 "name" 
 : 
  
 " SOURCE_OBJECT_1 
" 
  
 }, 
  
 { 
  
 "name" 
 : 
  
 " SOURCE_OBJECT_2 
" 
  
 } 
  
 ], 
  
 "destination" 
 : 
  
 { 
  
 "contentType" 
 : 
  
 " COMPOSITE_OBJECT_CONTENT_TYPE 
" 
  
 } 
 }

      Where:

      • SOURCE_OBJECT_1 and SOURCE_OBJECT_2 are the names of the source objects to use in the object composition.
      • COMPOSITE_OBJECT_CONTENT_TYPE is the Content-Type of the resulting composite object.

    3. Use cURL to call the JSON API with a POST Object request:

      curl -X POST --data-binary @ JSON_FILE_NAME 
\
  -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  -H "Content-Type: application/json" \
  "https://storage.googleapis.com/storage/v1/b/ BUCKET_NAME 
/o/ COMPOSITE_OBJECT_NAME 
/compose"

      Where:

      • JSON_FILE_NAME is the name of the file you created in the previous step.
      • BUCKET_NAME is the name of the bucket that contains the source objects.
      • COMPOSITE_OBJECT_NAME is the name you are giving to the result of the object composition.

    If successful, the response is an object resource for the resulting composite object.

    XML API

    1. Have gcloud CLI installed and initialized , which lets you generate an access token for the Authorization header.

    2. Create an XML file that contains the following information:

        
< ComposeReques 
 t 
>  
< Compo 
 nent 
>  
< Name> SOURCE_OBJECT_ 
 
 1 
< /Name 
>  
< /Compo 
 nent 
>  
< Compo 
 nent 
>  
< Name> SOURCE_OBJECT_ 
 
 2 
< /Name 
>  
< /Compo 
 nent 
>  
< /ComposeReques 
 t 
>

      Where:

      • SOURCE_OBJECT_1 and SOURCE_OBJECT_2 are the names of the source objects to use in the object composition.

    3. Use cURL to call the XML API with a PUT Object request that includes the compose query string parameter:

      curl -X PUT --data-binary @ XML_FILE_NAME 
\
  -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  -H "Content-Type: COMPOSITE_OBJECT_CONTENT_TYPE 
" \
  "https://storage.googleapis.com/ BUCKET_NAME 
/ COMPOSITE_OBJECT_NAME 
?compose"

      Where:

      • XML_FILE_NAME is the name of the file you created in the previous step.
      • COMPOSITE_OBJECT_CONTENT_TYPE is the Content-Type of the resulting composite object.
      • BUCKET_NAME is the name of the bucket that contains the source objects.
      • COMPOSITE_OBJECT_NAME is the name you are giving to the result of the object composition.

    If successful, an empty response body is returned.

    What's next

    Design a Mobile Site
    View Site in Mobile | Classic
    Share by: