Transition from gsutil to gcloud storage

Notes

Notes

Simulated folders created using the Google Cloud console are recognized as 0-byte placeholder objects. gcloud storage cp can be used to copy simulated folders, whereas gsutil cp cannot.

When handling errors, gcloud storage cp attempts to copy all resources, even if one of the resources is invalid or doesn't exist. gsutil cp might stop the entire operation as soon as it encounters an invalid resource.

When downloading an object, gcloud storage cp creates any missing local directories that are specified in the destination path. gsutil cp fails if the destination directory doesn't exist.

Notes

gsutil hash uses additive flags to include specific hashes, such as -c or -m . gcloud storage hash uses subtractive flags to exclude hashes, such as --skip-md5 or --skip-crc32c .

Notes

The combination of adding, updating, and removing labels in a single gsutil label ch command can behave differently than the equivalent operations using gcloud storage buckets update . Scripts relying on specific read-modify-write behavior of gsutil should be tested carefully after transitioning to gcloud storage .

Notes

When a wildcard is used to list objects in a project, gsutil ls returns a flat list of objects, whereas gcloud storage ls returns objects grouped by bucket name.

When using gcloud storage ls -L to return detailed listings as output, the following behaviors apply:

• The labels for each piece of metadata (also known as a "key") has the first letter of each word capitalized.
• If a piece of metadata for an object doesn't have a value, gcloud storage omits that line from the output.

gcloud storage always outputs datetimes in UTC.

gcloud storage returns any error messages at the end of the command's output.

Notes

If an encryption_key is set in your boto configuration file, gsutil rewrite -k applies the key to the objects. If encryption_key isn't set, the equivalent of gsutil rewrite -k is gcloud storage objects update --clear-encryption-key . This removes any customer-supplied encryption keys (CSEK) or customer-managed encryption keys (CMEK) from the object. The object is then encrypted with the bucket's default KMS key or with Google-managed encryption.

gsutil rewrite includes logic to skip rewriting an object if the requested transformation wouldn't change its state (for example, if the object is already in a target storage class). gcloud storage objects update might not perform the same checks, potentially leading to unnecessary operations.

Notes

The gsutil aliases del , delete , and remove aren't supported.

There's a known issue where gsutil and gcloud storage behave differently when the -l and -r flags are used together. Scripts that pipe a list of directories to be recursively deleted should be thoroughly tested before transitioning from using gsutil to gcloud storage .

Notes

By default, gsutil rsync considers an object unchanged if its size and last modification time match between the source and destination. It only performs a more thorough checksum comparison if an object is missing its modification time, or if you manually force it to with the -c flag. gcloud storage rsync also starts by checking a file's size and last modification time; however, if the object sizes match but the modification times differ or are missing, it automatically performs a checksum comparison.

gcloud storage rsync performs operations in parallel by default for better performance. gsutil rsync runs sequentially unless the top-level -m flag is used.

gsutil rsync follows symbolic links by default. gcloud storage rsync ignores them by default unless the --no-ignore-symlinks flag is used.

Notes

gsutil setmeta uses a single, repeatable -h flag to specify which metadata to set or remove. gcloud storage objects update uses distinct flags for each metadata field (for example: --content-type , --clear-cache-control , or --update-custom-metadata ).

In gsutil setmeta , the -h flag is used to set the full header name (for example: -h "x-goog-meta-icecreamflavor:vanilla" ). In gcloud storage objects update , the --update-custom-metadata flag is used instead and takes key-value pairs (for example: --update-custom-metadata=icecreamflavor=vanilla ). To remove a custom metadata field, gsutil setmeta uses -h "x-goog-meta-icecreamflavor" while gcloud storage objects update uses --remove-custom-metadata=icecreamflavor .

Notes

The output format of gcloud storage objects list --stat differs from the output format of gsutil stat , with known issues related to inconsistent spacing. Scripts that were written to parse the output of gsutil stat might require adjustments to handle the new output format.