Console
    Contact Us Start free

    Use inventory reports

    This page describes how to create and manage inventory report configurations and generated inventory reports, so you can get a high level summary of all your objects' metadata in a given bucket. For an overview of inventory reports, see the inventory reports overview documentation .

    Before you begin

    Get required roles

    In order to get the required permissions for creating and managing inventory reports, ask your administrator to grant you the following IAM roles on the project or source and destination buckets with which you'll be managing inventory reports.

    • To create and manage inventory report configurations:

      • roles/storage.admin on the source bucket and destination bucket

      • roles/storageinsights.admin on the project

    • To read and download inventory reports:

      • roles/storage.objectViewer on the destination bucket

      • roles/storageinsights.viewer on the project

    These predefined roles contain the permissions required to create and manage inventory reports and configurations. To see the exact permissions that are required, expand the Required permissionssection:

    Required permissions

    To create and manage inventory report configurations:

    • storage.buckets.get on the source bucket
    • storage.objects.list on the source bucket
    • storage.buckets.getObjectInsights on the source bucket
    • storage.buckets.get on the destination bucket
    • storage.objects.create on the destination bucket
    • storageinsights.reportConfigs.delete on the project
    • storageinsights.reportConfigs.get on the project
    • storageinsights.reportConfigs.create on the project
    • storageinsights.reportConfigs.list on the project
    • storageinsights.reportConfigs.update on the project

    To read and download inventory reports:

    • storage.objects.get on the destination bucket
    • storageinsights.reportDetails.get on the project
    • storageinsights.reportDetails.list on the project

    You might also be able to get these permissions with other predefined roles . To see which roles are associated with which permissions, refer to IAM roles for Cloud Storage .

    For instructions on using roles to control access to buckets, see Use IAM . For instructions on using roles to control access to projects, see Manage access .

    Enable the Storage Insights API

    Console

    To enable the storageinsights.googleapis.com API, follow the instructions in Enabling services .

    Command line

    To enable the Storage Insights API in your current project, run the following command:

    gcloud services enable storageinsights.googleapis.com

    For more details on how to enable services for a Google Cloud project, see Enabling and disabling services .

    REST APIs

    JSON API

    Use the Google Cloud console or the Google Cloud CLI to enable the Storage Insights API.

    Create an inventory report configuration

    Console

    To create an inventory report configuration, complete the following steps:

    1. In the Google Cloud console, go to the Cloud Storage Buckets page.

      Go to Buckets

    2. In the list of buckets, click the name of the bucket you'd like to use as the source bucket .

    3. On the Bucket detailspage, click the Inventory reportstab.

    4. Click Create report configuration.

    5. In the Identify your report configurationsection, create a display name for the inventory report configuration. The display name can contain up to 256 characters.

    6. In the Choose metadata fieldssection, select the metadata fields you want to include in your inventory reports.

    7. Click Continue.

    8. In the Choose a file formatsection, select the file format in which you'd like to generate inventory reports.

    9. In the Choose a destination bucketsection, select the bucket you'd like to use as a destination bucket .

      In the Enter destination path (optional)section, you can optionally specify a destination path where inventory reports will be generated.

    10. In the Scheduling optionssection, specify the frequency, start date, and end date for generating reports.

    11. Click Create.

      The Report configuration detailspage is displayed. The metadata of generated inventory reports will appear on the page.

    Command line

    1. To create an inventory report configuration, run the gcloud storage insights inventory-reports create command:

      gcloud storage insights inventory-reports create SOURCE_BUCKET_URL 
\
  --csv-separator= SEPARATOR 
\
  --csv-delimiter= DELIMITER 
\ --[no]-csv-header 
\ --parquet 
\
  --display-name= DISPLAY_NAME 
\
  --destination= DESTINATION_PATH 
\
  --metadata-fields= METADATA_FIELD 
\
  --schedule-starts= START_DATE 
\
  --schedule-repeats= FREQUENCY 
\
  --schedule-repeats-until= END_DATE

      Replace:

      • SOURCE_BUCKET_URL with the URL of the source bucket. For example, gs://my_example_source_bucket .

      • SEPARATOR with the character used to separate the records in the inventory report CSV file. Must be either \n or \r\n . Default value is \n . Optional. If the --csv-separator flag is used, --parquet cannot be used.

      • DELIMITER with the delimiter that separates the fields in the inventory report CSV file. The value can include one character and cannot be the same value as SEPARATOR . Default value is , . Optional. If --csv-delimiter is used, --parquet cannot be used.

      • --[no]-csv-header with the flag that indicates whether or not headers are included in the inventory report CSV file. Use --csv-header to include headers and --no-csv-header to exclude headers. If either flag is used, --parquet cannot be used.

      • DISPLAY_NAME with the editable name of the inventory report configuration. Optional.

      • --parquet with the flag that generates inventory reports in Apache Parquet format instead of CSV. If used, --csv-delimiter , --csv-separator , and --[no-]csv-header cannot be used.

      • DESTINATION_PATH with the bucket or folder in the bucket where inventory reports will be generated. For example, gs://my_example_destination_bucket or gs://my_example_destination_bucket/path/to/inventory/report .

        When you specify a path to a folder within a bucket, you can use keywords in the path, which are substituted with their corresponding values when the report is generated. This enables you to generate reports in a Hive partitioned format so you can load or query the data in BigQuery without additional processing.

      • METADATA_FIELD with a comma-separated list of metadata fields you want included in the inventory report.

      • START_DATE with the UTC date on which you want to start generating inventory reports. For example, 2022-01-15 .

      • FREQUENCY with how often you want inventory reports to be generated. Values are daily or weekly .

      • END_DATE with the UTC date after which you want to stop generating inventory reports. Must be a value after the START_DATE . For example, if you specify 2022-02-15 , inventory reports will no longer be generated as of February 16, 2022.

    Client libraries

    Java

    For more information, see the Cloud Storage Java API reference documentation .

    To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for client libraries .

      import 
  
 com.google.cloud.storageinsights.v1. CSVOptions 
 
 ; 
 import 
  
 com.google.cloud.storageinsights.v1. CloudStorageDestinationOptions 
 
 ; 
 import 
  
 com.google.cloud.storageinsights.v1. CloudStorageFilters 
 
 ; 
 import 
  
 com.google.cloud.storageinsights.v1. CreateReportConfigRequest 
 
 ; 
 import 
  
 com.google.cloud.storageinsights.v1. FrequencyOptions 
 
 ; 
 import 
  
 com.google.cloud.storageinsights.v1. LocationName 
 
 ; 
 import 
  
 com.google.cloud.storageinsights.v1. ObjectMetadataReportOptions 
 
 ; 
 import 
  
 com.google.cloud.storageinsights.v1. ReportConfig 
 
 ; 
 import 
  
 com.google.cloud.storageinsights.v1. StorageInsightsClient 
 
 ; 
 import 
  
 com.google.common.collect.ImmutableList 
 ; 
 import 
  
 com.google.type. Date 
 
 ; 
 import 
  
 java.io.IOException 
 ; 
 public 
  
 class 
 CreateInventoryReportConfig 
  
 { 
  
 public 
  
 static 
  
 void 
  
 createInventoryReportConfig 
 ( 
  
 String 
  
 projectId 
 , 
  
 String 
  
 bucketLocation 
 , 
  
 String 
  
 sourceBucket 
 , 
  
 String 
  
 destinationBucket 
 ) 
  
 throws 
  
 IOException 
  
 { 
  
 try 
  
 ( 
  StorageInsightsClient 
 
  
 storageInsightsClient 
  
 = 
  
  StorageInsightsClient 
 
 . 
 create 
 ()) 
  
 { 
  
  ReportConfig 
 
  
 reportConfig 
  
 = 
  
  ReportConfig 
 
 . 
 newBuilder 
 () 
  
 . 
  setDisplayName 
 
 ( 
 "Example inventory report configuration" 
 ) 
  
 . 
  setFrequencyOptions 
 
 ( 
  
  FrequencyOptions 
 
 . 
 newBuilder 
 () 
  
 . 
  setFrequency 
 
 ( 
  FrequencyOptions 
 
 . 
 Frequency 
 . 
 WEEKLY 
 ) 
  
 . 
  setStartDate 
 
 ( 
  Date 
 
 . 
 newBuilder 
 (). 
 setDay 
 ( 
 15 
 ). 
 setMonth 
 ( 
 8 
 ). 
 setYear 
 ( 
 3022 
 ). 
 build 
 ()) 
  
 . 
  setEndDate 
 
 ( 
  Date 
 
 . 
 newBuilder 
 (). 
 setDay 
 ( 
 15 
 ). 
 setMonth 
 ( 
 9 
 ). 
 setYear 
 ( 
 3022 
 ). 
 build 
 ()) 
  
 . 
 build 
 ()) 
  
 . 
  setCsvOptions 
 
 ( 
  
  CSVOptions 
 
 . 
 newBuilder 
 () 
  
 . 
  setDelimiter 
 
 ( 
 "," 
 ) 
  
 . 
  setRecordSeparator 
 
 ( 
 "\n" 
 ) 
  
 . 
  setHeaderRequired 
 
 ( 
 true 
 ) 
  
 . 
 build 
 ()) 
  
 . 
  setObjectMetadataReportOptions 
 
 ( 
  
  ObjectMetadataReportOptions 
 
 . 
 newBuilder 
 () 
  
 . 
  addAllMetadataFields 
 
 ( 
 ImmutableList 
 . 
 of 
 ( 
 "project" 
 , 
  
 "name" 
 , 
  
 "bucket" 
 )) 
  
 . 
  setStorageFilters 
 
 ( 
  
  CloudStorageFilters 
 
 . 
 newBuilder 
 (). 
 setBucket 
 ( 
 sourceBucket 
 ). 
 build 
 ()) 
  
 . 
  setStorageDestinationOptions 
 
 ( 
  
  CloudStorageDestinationOptions 
 
 . 
 newBuilder 
 () 
  
 . 
 setBucket 
 ( 
 destinationBucket 
 ) 
  
 . 
 build 
 ()) 
  
 . 
 build 
 ()) 
  
 . 
 build 
 (); 
  
  CreateReportConfigRequest 
 
  
 request 
  
 = 
  
  CreateReportConfigRequest 
 
 . 
 newBuilder 
 () 
  
 . 
 setParent 
 ( 
  LocationName 
 
 . 
 of 
 ( 
 projectId 
 , 
  
 bucketLocation 
 ). 
 toString 
 ()) 
  
 . 
 setReportConfig 
 ( 
 reportConfig 
 ) 
  
 . 
 build 
 (); 
  
  ReportConfig 
 
  
 response 
  
 = 
  
 storageInsightsClient 
 . 
 createReportConfig 
 ( 
 request 
 ); 
  
 System 
 . 
 out 
 . 
 println 
 ( 
 "Created inventory report config with name " 
  
 + 
  
 response 
 . 
  getName 
 
 ()); 
  
 } 
  
 } 
 }

    REST APIs

    JSON API

    To create an inventory report configuration, complete the following steps:

    1. Have gcloud CLI installed and initialized , which lets you generate an access token for the Authorization header.

    2. Create a JSON file that contains the following information, where all fields are required unless noted otherwise:

       { 
  
 "display_name" 
 : 
  
 " DISPLAY_NAME 
" 
 , 
  
 "frequency_options" 
 : 
  
 { 
  
 "frequency" 
 : 
  
 " FREQUENCY 
" 
 , 
  
 "start_date" 
 : 
  
 " START_DATE 
" 
 , 
  
 "end_date" 
 : 
  
 " END_DATE 
" 
 , 
  
 }, 
  
 "csv_options" 
 : 
  
 { 
  
 "record_separator" 
 : 
  
 " RECORD_SEPARATOR 
" 
 , 
  
 "delimiter" 
 : 
  
 " DELIMITER 
" 
 , 
  
 "header_required" 
 : 
  
  HEADER_REQUIRED 
 
  
 }, 
  
 "object_metadata_report_options" 
 : 
  
 { 
  
 "metadata_fields" 
 : 
  
 [ 
  
 " METADATA_FIELD 
" 
 , 
  
 ... 
  
 ], 
  
 "storage_filters" 
 : 
  
 { 
  
 "bucket" 
 : 
  
 " SOURCE_BUCKET_NAME 
" 
  
 }, 
  
 "storage_destination_options" 
 : 
  
 { 
  
 "bucket" 
 : 
  
 " DESTINATION_BUCKET_NAME 
" 
 , 
  
 "destination_path" 
 : 
  
 " DESTINATION_PATH 
" 
  
 } 
  
 } 
 }

      Replace:

      • DISPLAY_NAME with the editable display name of the inventory report configuration. Can contain up to 256 characters.

      • FREQUENCY with the frequency for generating inventory reports. Values are DAILY or WEEKLY .

      • START_DATE with the UTC date on which you want to start generating inventory reports. Cannot be the current date or any past date. Value must be an object containing the keys day , month , and year . For example: {"day": 15, "month": 8, "year": 2022} .

      • END_DATE with the UTC date after which you want to stop generating inventory reports. Value must be an object containing the keys day , month , and year . For example, if you specify {"day": 15, "month": 9, "year": 2022} , inventory reports are no longer generated as of September 16, 2022.

      • RECORD_SEPARATOR with the character used to separate the records in the inventory report CSV file. Must be either \n or \r\n . Default value is \n . This field is optional.

      • DELIMITER with the delimiter that separates the fields in the inventory report CSV file. The value can include one character and cannot be the same as the RECORD_SEPARATOR . Default value is , . This field is optional.

      • HEADER_REQUIRED with the boolean that indicates whether or not headers are included in the CSV file. This field is optional.

      • SOURCE_BUCKET_NAME with the name of the source bucket that contains the objects you want to generate inventory reports for. For example, my_example_bucket .

      • METADATA_FIELD with a comma-separated list of metadata fields you want included in the inventory report.

      • DESTINATION_BUCKET_NAME with the name of the destination bucket where generated inventory reports are generated and stored. For example, my_example_destination_bucket .

      • DESTINATION_PATH with the path in the destination bucket where generated inventory reports are generated. This field is optional.

        When you specify a path to a folder within a bucket, you can use keywords in the path, which are substituted with their corresponding values when the report is generated. This enables you to generate reports in a Hive partitioned format so you can load or query the data in BigQuery without additional processing.

      For example, the following code sample creates an inventory report configuration named "Example inventory report configuration" that generates reports at a weekly frequency:

       { 
  
 "display_name" 
 : 
  
 "Example inventory report configuration" 
 , 
  
 "frequency_options" 
 : 
  
 { 
  
 "frequency" 
 : 
  
 "WEEKLY" 
 , 
  
 "start_date" 
 : 
  
 { 
  
 "day" 
 : 
  
 15 
 , 
  
 "month" 
 : 
  
 8 
 , 
  
 "year" 
 : 
  
 2022 
  
 }, 
  
 "end_date" 
 : 
  
 { 
  
 "day" 
 : 
  
 15 
 , 
  
 "month" 
 : 
  
 9 
 , 
  
 "year" 
 : 
  
 2022 
  
 }, 
  
 }, 
  
 "csv_options" 
 : 
  
 { 
  
 "record_separator" 
 : 
  
 "\n" 
 , 
  
 "delimiter" 
 : 
  
 "," 
 , 
  
 "header_required" 
 : 
  
 true 
  
 }, 
  
 "object_metadata_report_options" 
 : 
  
 { 
  
 "metadata_fields" 
 : 
  
 [ 
  
 "project" 
 , 
  
 "name" 
 , 
  
 "bucket" 
  
 ], 
  
 "storage_filters" 
 : 
  
 { 
  
 "bucket" 
 : 
  
 "example_source_bucket" 
  
 }, 
  
 "storage_destination_options" 
 : 
  
 { 
  
 "bucket" 
 : 
  
 "example_destination_bucket" 
  
 } 
  
 } 
 }

    3. To apply the inventory report configuration, use cURL to call the JSON API with an Insert ReportConfig request:

      curl -X POST --data-binary @ JSON_FILE_NAME 
\
"https://storageinsights.googleapis.com/v1/projects/ YOUR_PROJECT 
/locations/ LOCATION 
/reportConfigs" \
  --header "Authorization: Bearer $(gcloud auth print-access-token)" \
  --header "Accept: application/json" \
  --header "Content-Type: application/json" \

      Replace:

      • JSON_FILE_NAME with the path to the JSON file you created in the previous step.

      • YOUR_PROJECT with the name of the project in which your source and destination buckets reside.

      • LOCATION with the location of the source and destination buckets. For example, us-west1 .

    A project-level service agent is automatically created the first time you set up an inventory report configuration. The service agent follows the naming format service- PROJECT_NUMBER @gcp-sa-storageinsights.iam.gserviceaccount.com and appears on the IAM page of the Google Cloud console when you select the Include Google-provided role grants checkbox.

    To enable Storage Insights to generate and write inventory reports, ask your administrator to grant the service agent the following IAM roles:

    • roles/storage.insightsCollectorService on the source bucket, which includes the storage.buckets.getObjectInsights and storage.buckets.get permissions
    • roles/storage.objectCreator on the destination bucket, which includes the storage.objects.create permission

    For instructions on granting roles, see Use IAM . You can grant the roles/storage.insightsCollectorService role by using the Google Cloud console or the Google Cloud CLI. For example:

    gcloud storage buckets add-iam-policy-binding SOURCE_BUCKET_URL 
\
    --member=serviceAccount:service- PROJECT_NUMBER 
@gcp-sa-storageinsights.iam.gserviceaccount.com \
    --role=roles/storage.insightsCollectorService

    Once the service agent is granted the required permissions, it takes up to 24 hours to write the first inventory report to the destination bucket.

    Edit an inventory report configuration

    Once an inventory report configuration has been created, you can modify certain properties of the configuration.

    Console

    To edit an inventory report configuration, follow the steps below:

    1. In the Google Cloud console, go to the Cloud Storage Buckets page.

      Go to Buckets

    2. In the list of buckets, click the name of the source bucket that contains the inventory report configuration you want to edit.

    3. In your source bucket's Bucket detailspage, click the Inventory reportstab.

    4. Click the configuration name of the inventory report configuration you want to edit.

    5. In the Report configuration detailspage that appears, edit the properties you desire by using the Editbutton.

    Command line

    To edit an inventory report configuration, follow the steps below:

    1. To find the name of the inventory report configuration you want to edit, list all the inventory configurations in the source bucket by using the gcloud storage insights inventory-reports list command:

      gcloud storage insights inventory-reports list \
  --source= SOURCE_BUCKET 
\
  --filter= EXPRESSION 
\
  --page-size= SIZE 
\
  --sort-by= FIELD 
\
  --format="yaml(name)"

      Replace:

      • SOURCE_BUCKET with the URL of the source bucket that contains the inventory report configuration.

      • EXPRESSION with a boolean filter to apply to each resource item to be listed. If the expression evaluates True, the item is listed. For more details and examples of filter expressions, run $ gcloud topic filters .

      • SIZE with the maximum number of resources per page. The default is 50.

      • FIELD with a comma-separated list of resource field key names to sort by. The default order is ascending. Prefix a field with ~ for descending order on that field.

      1. Edit the inventory report configuration fields you want to update by using the gcloud storage insights inventory-reports update command. The following example updates the inventory report configuration to generate inventory reports daily in Apache Parquet format:

        gcloud storage insights inventory-reports update CONFIG_NAME 
\
--parquet \
--schedule-repeats="daily"

    Client libraries

    Java

    For more information, see the Cloud Storage Java API reference documentation .

    To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for client libraries .

      import 
  
 com.google.cloud.storageinsights.v1. ReportConfig 
 
 ; 
 import 
  
 com.google.cloud.storageinsights.v1. ReportConfigName 
 
 ; 
 import 
  
 com.google.cloud.storageinsights.v1. StorageInsightsClient 
 
 ; 
 import 
  
 com.google.cloud.storageinsights.v1. UpdateReportConfigRequest 
 
 ; 
 import 
  
 com.google.protobuf. FieldMask 
 
 ; 
 import 
  
 java.io.IOException 
 ; 
 public 
  
 class 
 EditInventoryReportConfig 
  
 { 
  
 public 
  
 static 
  
 void 
  
 editInventoryReportConfig 
 ( 
  
 String 
  
 projectId 
 , 
  
 String 
  
 location 
 , 
  
 String 
  
 inventoryReportConfigUuid 
 ) 
  
 throws 
  
 IOException 
  
 { 
  
 try 
  
 ( 
  StorageInsightsClient 
 
  
 storageInsightsClient 
  
 = 
  
  StorageInsightsClient 
 
 . 
 create 
 ()) 
  
 { 
  
  ReportConfigName 
 
  
 name 
  
 = 
  
  ReportConfigName 
 
 . 
 of 
 ( 
 projectId 
 , 
  
 location 
 , 
  
 inventoryReportConfigUuid 
 ); 
  
  ReportConfig 
 
  
 reportConfig 
  
 = 
  
 storageInsightsClient 
 . 
 getReportConfig 
 ( 
 name 
 ); 
  
 // Set any other fields you want to update here 
  
  ReportConfig 
 
  
 updatedReportConfig 
  
 = 
  
 reportConfig 
 . 
  toBuilder 
 
 (). 
  setDisplayName 
 
 ( 
 "Updated Display Name" 
 ). 
 build 
 (); 
  
 storageInsightsClient 
 . 
 updateReportConfig 
 ( 
  
  UpdateReportConfigRequest 
 
 . 
 newBuilder 
 () 
  
 // Add any fields that you want to update to the update mask, in snake case 
  
 . 
 setUpdateMask 
 ( 
  FieldMask 
 
 . 
 newBuilder 
 (). 
  addPaths 
 
 ( 
 "display_name" 
 ) 
  
 . 
 build 
 ()) 
  
 . 
 setReportConfig 
 ( 
 updatedReportConfig 
 ). 
 build 
 ()); 
  
 System 
 . 
 out 
 . 
 println 
 ( 
 "Edited inventory report config with name " 
  
 + 
  
 name 
 ); 
  
 } 
  
 } 
 }

    REST APIs

    JSON API

    To edit an inventory report configuration, complete the following steps:

    1. Have gcloud CLI installed and initialized , which lets you generate an access token for the Authorization header.

    2. Get the name of the inventory report configuration you want to edit by using cURL to call the JSON API with a Get ReportConfig request:

      curl --request GET \
"https://storageinsights.googleapis.com/v1/projects/ YOUR_PROJECT 
/locations/ LOCATION 
/reportConfigs?" \
  --header "Authorization: Bearer $(gcloud auth print-access-token)" \
  --header "Accept: application/json" \
  --header "Content-Type: application/json" \

      Replace:

      • YOUR_PROJECT with the name of the project in which your source and destination buckets reside.

      • LOCATION with the location of the source and destination buckets. For example, us-west1 .

      If you want to retrieve all inventory report configurations in a specific source bucket, append the filter query parameter to your request:

      curl --request GET \
"https://storageinsights.googleapis.com/v1/projects/ YOUR_PROJECT 
/locations/ LOCATION 
/reportConfigs?filter=objectMetadataReportOptions.storageFilters.bucket= BUCKET_NAME 
" \
  --header "Authorization: Bearer $(gcloud auth print-access-token)" \
  --header "Accept: application/json" \
  --header "Content-Type: application/json" \

      Replace:

      • YOUR_PROJECT with the name of the project in which your source and destination buckets reside.
      • LOCATION with the location of the source and destination buckets. For example, us-west1 .
      • BUCKET_NAME with the name of the source bucket that contains the inventory report configurations you want to edit.

    3. Create a JSON file that includes changes to the properties you want to edit. The following example updates the inventory report configuration to generate inventory reports daily in Apache Parquet format:

       { 
  
 "ReportConfig" 
 : 
  
 { 
  
 "frequency_options" 
 : 
  
 { 
  
 "frequency" 
 : 
  
 "DAILY" 
  
 }, 
  
 "parquet_options" 
 : 
  
 { 
  
 } 
  
 }

    4. To apply the inventory report configuration, use cURL to call the JSON API with a Patch ReportConfig request:

      curl --request PATCH \
"https://storageinsights.googleapis.com/v1/projects/ YOUR_PROJECT 
/locations/ LOCATION 
/reportConfigs/ REPORT_CONFIG_UUID 
" \
  --header "Authorization: Bearer $(gcloud auth print-access-token)" \
  --header "Accept: application/json" \
  --header "Content-Type: application/json" \
  --data-binary "@JSON_FILE_NAME.json" \

      Replace:

      • YOUR_PROJECT with the name of the project in which your source and destination buckets reside.

      • LOCATION with the location of the source and destination buckets. For example, us-west1 .

      • REPORT_CONFIG_UUID with the auto-generated UUID of the inventory report configuration.

    List inventory report configurations

    Console

    To list the inventory report configurations in a source bucket, complete the steps below:

    1. In the Google Cloud console, go to the Cloud Storage Buckets page.

      Go to Buckets

    2. In the list of buckets, click the name of the source bucket that contains the inventory report configurations you want to view.

    3. On the Bucket detailspage, click the Inventory reportstab.

      Inventory report configurations in the source bucket appear.

    Command line

    You can list all the inventory configurations in a source bucket by using the gcloud storage insights inventory-reports list command:

    gcloud storage insights inventory-reports list \
  --source= SOURCE_BUCKET 
\
  --filter= EXPRESSION 
\
  --page-size= SIZE 
\
  --sort-by= FIELD 
\
  --format="yaml(name)"

    Replace:

    • SOURCE_BUCKET with the URL of the source bucket that contains the inventory report configuration.

    • EXPRESSION with a boolean filter to apply to each resource item to be listed. If the expression evaluates True, the item is listed. For more details and examples of filter expressions, run $ gcloud topic filters .

    • SIZE with the maximum number of resources per page. The default is 50.

    • FIELD with a comma-separated list of resource field key names to sort by. The default order is ascending. Prefix a field with ~ for descending order on that field.

    Client libraries

    Java

    For more information, see the Cloud Storage Java API reference documentation .

    To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for client libraries .

      import 
  
 com.google.cloud.storageinsights.v1. LocationName 
 
 ; 
 import 
  
 com.google.cloud.storageinsights.v1. ReportConfig 
 
 ; 
 import 
  
 com.google.cloud.storageinsights.v1. StorageInsightsClient 
 
 ; 
 import 
  
 java.io.IOException 
 ; 
 public 
  
 class 
 ListInventoryReportConfigs 
  
 { 
  
 public 
  
 static 
  
 void 
  
 listInventoryReportConfigs 
 ( 
 String 
  
 projectId 
 , 
  
 String 
  
 location 
 ) 
  
 throws 
  
 IOException 
  
 { 
  
 try 
  
 ( 
  StorageInsightsClient 
 
  
 storageInsightsClient 
  
 = 
  
  StorageInsightsClient 
 
 . 
 create 
 ()) 
  
 { 
  
 System 
 . 
 out 
 . 
 println 
 ( 
  
 "Printing inventory report configs in project " 
  
 + 
  
 projectId 
  
 + 
  
 " and location " 
  
 + 
  
 location 
 ); 
  
 for 
  
 ( 
  ReportConfig 
 
  
 config 
  
 : 
  
 storageInsightsClient 
  
 . 
 listReportConfigs 
 ( 
  LocationName 
 
 . 
 of 
 ( 
 projectId 
 , 
  
 location 
 )) 
  
 . 
 iterateAll 
 ()) 
  
 { 
  
 System 
 . 
 out 
 . 
 println 
 ( 
 config 
 . 
 getName 
 ()); 
  
 } 
  
 } 
  
 } 
 }

    REST APIs

    JSON API

    1. Have gcloud CLI installed and initialized , which lets you generate an access token for the Authorization header.

    2. You can list all inventory report configurations in a project for a given location by using a request to list inventory report configurations :

      curl --request GET \
"https://storageinsights.googleapis.com/v1/projects/ YOUR_PROJECT 
/locations/ LOCATION 
/reportConfigs?" \
  --header "Authorization: Bearer $(gcloud auth print-access-token)" \
  --header "Accept: application/json" \
  --header "Content-Type: application/json" \

      Replace:

      • YOUR_PROJECT with the name of the project in which your source and destination buckets reside.

      • LOCATION with the location of the source and destination buckets. For example, us-west1 .

      If you want to list all inventory report configurations in a specific source bucket, append the filter query parameter to your request:

      curl --request GET \
"https://storageinsights.googleapis.com/v1/projects/ YOUR_PROJECT 
/locations/ LOCATION 
/reportConfigs?filter=objectMetadataReportOptions.storageFilters.bucket= BUCKET_NAME 
" \
  --header "Authorization: Bearer $(gcloud auth print-access-token)" \
  --header "Accept: application/json" \
  --header "Content-Type: application/json" \

      Replace:

      • YOUR_PROJECT with the name of the project in which your source and destination buckets reside.

      • LOCATION with the location of the source and destination buckets. For example, us-west1 .

      • BUCKET_NAME with the name of the source bucket that contains the inventory report configurations you want to list.

    Download an inventory report

    Console

    Download individual reports

    Inventory reports are generated and stored as objects in the destination bucket, so you can download them as you would download a regular object.

    To download an inventory report, complete the following steps:

    1. In the Google Cloud console, go to the Cloud Storage Buckets page.

      Go to Buckets

    2. In the list of buckets, click the name of the destination bucket that contains the inventory report you want to download.

    3. On the Bucket detailspage, make sure the Objectstab is selected.

    4. Click Downloadassociated with the inventory report you want to download.

    If you don't know the destination bucket, you can also download an inventory report through the inventory report configuration from which it was generated:

    1. In the Google Cloud console, go to the Cloud Storage Buckets page.

      Go to Buckets

    2. In the list of buckets, click the name of the source bucket containing the inventory report configuration that generated the report you want to download.

    3. On the Bucket detailspage, click the configuration name of the inventory report configuration.

    4. On the Report configuration detailspage that appears, navigate to the Inventory report historysection, then click the destination object path of the inventory report you want to download.

      The Bucket detailspage appears for the destination bucket that contains the inventory report.

    5. Click Downloadassociated with the inventory report you want to download.

    Download report shards

    To download an inventory report that's been split into one or more shards , complete the following steps:

    1. In the Google Cloud console, go to the Cloud Storage Buckets page.

      Go to Buckets

    2. In the list of buckets, click the name of the destination bucket you specified when you created the inventory report configuration.

    3. On the Bucket detailspage, check for the presence of a manifest file . The presence of a manifest file indicates that all the shards of an inventory report have been generated.

      An example manifest file name is fc95c52f-157a-494f-af4a-d4a53a69ba66_2022-11-30T00:00_manifest.json .

    4. In the destination bucket, click Downloadassociated with the manifest file. Note the names of the shard files you want to download from the report_shards_file_names field.

    5. In the destination bucket, click Downloadassociated with the shard files you want to download.

    Command line

    Download individual reports

    To download an inventory report, complete the following steps:

    1. To list all the inventory reports that have been generated by an inventory report configuration and retrieve their REPORT_DETAIL_ID , use the gcloud storage insights inventory-reports details list command:

      gcloud storage insights inventory-reports details list CONFIG_NAME 
\
  --filter= EXPRESSION 
\
  --page-size= SIZE 
\
  --sort-by= FIELD

      Replace:

      • CONFIG_NAME with the unique name of the inventory report configuration, in the format projects/ PROJECT /locations/ LOCATION /reportConfigs/ REPORT_CONFIG_UUID .

      • EXPRESSION with a boolean filter to apply to each resource item to be listed. If the expression evaluates True, then that item is listed. For more details and examples of filter expressions, run $ gcloud topic filters .

      • SIZE with the maximum number of resources per page. The default is 50.

      • FIELD with a comma-separated list of resource field key names to sort by. The default order is ascending. Prefix a field with ~ for descending order on that field.

      If successful, the command returns output similar to the following:

      REPORT_DETAIL_ID            SNAPSHOT_TIME
Report_2023-04-10T00-00     2023-04-10T00:53:03Z
Report_2023-04-12T00-00     2023-04-12T00:52:54Z
Report_2023-04-05T00-00     2023-04-05T00:53:01Z

    2. To download an inventory report, you need to first retrieve the reportPathPrefix property of the ReportDetail object. To get the reportPathPrefix of a report, use the gcloud storage insights inventory-reports details describe command:

      gcloud storage insights inventory-reports details describe REPORT_DETAIL_NAME

      Replace REPORT_DETAIL_NAME with the name of the inventory report in the format projects/ PROJECT /locations/ LOCATION /reportConfigs/ REPORT_CONFIG_UUID /reportDetails/ REPORT_DETAIL_ID .

    Download report shards

    To download an inventory report that's been split into one or more shards , complete the following steps:

    1. Download the inventory report manifest file by using the gcloud storage cp command:

      gcloud storage cp gs:// BUCKET_NAME 
/ MANIFEST_FILE_NAME 
 DOWNLOAD_PATH

      Replace:

      • BUCKET_NAME with the name of the destination bucket.

      • MANIFEST_FILE_NAME with the name of the manifest file in the destination bucket, in the following naming convention:

         REPORT_CONFIG_UUID 
_ TARGET_DATETIME 
_manifest.json

        Where:

        • REPORT_CONFIG_UUID is the auto-generated UUID of the inventory report configuration that generated the report shards you want to download.

        • TARGET_DATETIME is the datetime at which the inventory report was generated.

        For example, fc95c52f-157a-494f-af4a-d4a53a69ba66_2022-11-30T00:00_manifest.json .

      • DOWNLOAD_PATH with the path to your file system where you want to save the inventory report to. For example, ./example_report.csv .

    2. To download a shard file, use the gcloud storage cp command :

      gcloud storage cp gs:// BUCKET_NAME 
/ SHARD_FILE_NAME 
 DOWNLOAD_PATH

      Replace:

      • BUCKET_NAME with the name of the destination bucket.

      • SHARD_FILE_NAME with the URL-encoded name of the shard file you want to download. For example, fc95c52f-157a-494f-af4a-d4a53a69ba66_2022-11-30T00:54_0.csv .

      • DOWNLOAD_PATH with the path to your file system where you want to save the inventory report to. For example, ./example_report.csv .

    Client libraries

    Java

    For more information, see the Cloud Storage Java API reference documentation .

    To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for client libraries .

      import 
  
 com.google.cloud.storageinsights.v1. ReportConfig 
 
 ; 
 import 
  
 com.google.cloud.storageinsights.v1. ReportConfigName 
 
 ; 
 import 
  
 com.google.cloud.storageinsights.v1. ReportDetail 
 
 ; 
 import 
  
 com.google.cloud.storageinsights.v1. StorageInsightsClient 
 
 ; 
 import 
  
 java.io.IOException 
 ; 
 public 
  
 class 
 GetInventoryReportNames 
  
 { 
  
 public 
  
 static 
  
 void 
  
 getInventoryReportNames 
 ( 
  
 String 
  
 projectId 
 , 
  
 String 
  
 location 
 , 
  
 String 
  
 reportConfigUuid 
 ) 
  
 throws 
  
 IOException 
  
 { 
  
 try 
  
 ( 
  StorageInsightsClient 
 
  
 storageInsightsClient 
  
 = 
  
  StorageInsightsClient 
 
 . 
 create 
 ()) 
  
 { 
  
  ReportConfig 
 
  
 config 
  
 = 
  
 storageInsightsClient 
 . 
 getReportConfig 
 ( 
  
  ReportConfigName 
 
 . 
 of 
 ( 
 projectId 
 , 
  
 location 
 , 
  
 reportConfigUuid 
 )); 
  
 String 
  
 extension 
  
 = 
  
 config 
 . 
  hasCsvOptions 
 
 () 
  
 ? 
  
 "csv" 
  
 : 
  
 "parquet" 
 ; 
  
 System 
 . 
 out 
 . 
 println 
 ( 
  
 "You can use the Google Cloud Storage Client " 
  
 + 
  
 "to download the following objects from Google Cloud Storage:" 
 ); 
  
 for 
  
 ( 
  ReportDetail 
 
  
 reportDetail 
  
 : 
  
 storageInsightsClient 
 . 
 listReportDetails 
 ( 
 config 
 . 
  getName 
 
 ()). 
 iterateAll 
 ()) 
  
 { 
  
 for 
  
 ( 
 long 
  
 index 
  
 = 
  
 reportDetail 
 . 
 getShardsCount 
 () 
  
 - 
  
 1 
 ; 
  
 index 
  
> = 
  
 0 
 ; 
  
 index 
 -- 
 ) 
  
 { 
  
 System 
 . 
 out 
 . 
 println 
 ( 
 reportDetail 
 . 
 getReportPathPrefix 
 () 
  
 + 
  
 index 
  
 + 
  
 "." 
  
 + 
  
 extension 
 ); 
  
 } 
  
 } 
  
 } 
  
 } 
 }

    REST APIs

    JSON API

    Download individual reports

    To download an inventory report, complete the following steps:

    1. Have gcloud CLI installed and initialized , which lets you generate an access token for the Authorization header.

    2. List all the inventory reports generated by an inventory report configuration by using cURL to call the JSON API with a request to list inventory report details :

      curl --request GET \
"https://storageinsights.googleapis.com/v1/projects/ YOUR_PROJECT 
/locations/ LOCATION 
/reportConfigs/ REPORT_CONFIG_UUID 
/reportDetails/" \
  --header "Authorization: Bearer $(gcloud auth print-access-token)" \
  --header "Accept: application/json" \
  --header "Content-Type: application/json" \

      Replace:

      • YOUR_PROJECT with the name of the project in which your source and destination buckets reside.

      • LOCATION with the location of the source and destination buckets. For example, us-west1 .

      • REPORT_CONFIG_UUID with the auto-generated UUID of the inventory report configuration that generated the report you want to download.

    3. To get the details of an individual report, use cURL to call the JSON API with a Get ReportDetails request:

      curl --request GET \
"https://storageinsights.googleapis.com/v1/projects/ YOUR_PROJECT 
/locations/ LOCATION 
/reportConfigs/ REPORT_CONFIG_UUID 
/reportDetails/ REPORT_DETAIL_ID 
" \
  --header "Authorization: Bearer $(gcloud auth print-access-token)" \
  --header "Accept: application/json" \
  --header "Content-Type: application/json" \

      Replace:

      • YOUR_PROJECT with the name of the project in which your source and destination buckets reside.

      • LOCATION with the location of the source and destination buckets. For example, us-west1 .

      • REPORT_CONFIG_UUID with the auto-generated UUID of the inventory report configuration that generated the report you want to download.

      • REPORT_DETAIL_ID with the name of the inventory report you want to download.

    Download report shards

    To download an inventory report that's been split into one or more shards , complete the following steps:

    1. Have gcloud CLI installed and initialized , which lets you generate an access token for the Authorization header.

    2. To download the manifest file of the inventory report, use cURL to call the JSON API with a GET Object request:

      curl -X GET \
  --header "Authorization: Bearer $(gcloud auth print-access-token)" \
  --header "Accept: application/json" \
  --header "Content-Type: application/json" \
  "https://storage.googleapis.com/storage/v1/b/ BUCKET_NAME 
/o/ MANIFEST_FILE_NAME 
?alt=media" \

      Replace:

      • BUCKET_NAME with the name of the destination bucket.

      • MANIFEST_FILE_NAME with the name of the manifest file in the destination bucket, in the following naming convention:

         REPORT_CONFIG_UUID 
_ TARGET_DATETIME 
_manifest.json

        Where:

        • REPORT_CONFIG_UUID is the auto-generated UUID of the inventory report configuration that generated the report shards you want to download.

        • TARGET_DATETIME is the date on which the inventory report was generated.

      For example, fc95c52f-157a-494f-af4a-d4a53a69ba66_2022-11-30T00:00_manifest.json .

      The report_shards_file_names field of the manifest file contains the names of the inventory report shards you can download.

    3. To download the inventory report shard files, use curl to call the Cloud Storage JSON API with a Get Object request:

      curl -X GET \
  --header "Authorization: Bearer $(gcloud auth print-access-token)" \
  -o " DOWNLOAD_PATH 
" \
  "https://storage.googleapis.com/storage/v1/b/ BUCKET_NAME 
/o/ SHARD_FILE_NAME 
?alt=media"

      Replace:

      • DOWNLOAD_PATH with the path to your local file system where you want to save your object. For example, Desktop/dog.png .

      • BUCKET_NAME with the name of the destination bucket containing the inventory report. For example, my-bucket .

      • SHARD_FILE_NAME with the URL-encoded name of the shard file you want to download. For example, fc95c52f-157a-494f-af4a-d4a53a69ba66_2022-11-30T00:54_0.csv .

    Delete an inventory report configuration

    Console

    To delete an inventory report configuration, complete the following steps:

    1. In the Google Cloud console, go to the Cloud Storage Buckets page.

      Go to Buckets

    2. In the list of buckets, click the name of the source bucket that contains the inventory report configuration you want to delete.

    3. In your source bucket's Bucket detailspage, click the Inventory reportstab.

    4. Click the configuration name of the inventory report configuration you want to delete.

    5. In the Report configuration detailspage that appears, click Delete.

    Command line

    To delete an inventory report configuration, use the gcloud storage insights inventory-reports delete command:

    gcloud storage insights inventory-reports delete CONFIG_NAME 
--force

    Replace CONFIG_NAME with the unique name of the inventory report configuration, in the format projects/ PROJECT /locations/ LOCATION /reportConfigs/ REPORT_CONFIG_UUID .

    The --force flag deletes the metadata for all inventory reports generated by the given inventory report configuration. The inventory report objects themselves will not be deleted.

    Client libraries

    Java

    For more information, see the Cloud Storage Java API reference documentation .

    To authenticate to Cloud Storage, set up Application Default Credentials. For more information, see Set up authentication for client libraries .

      import 
  
 com.google.cloud.storageinsights.v1. ReportConfigName 
 
 ; 
 import 
  
 com.google.cloud.storageinsights.v1. StorageInsightsClient 
 
 ; 
 import 
  
 java.io.IOException 
 ; 
 public 
  
 class 
 DeleteInventoryReportConfig 
  
 { 
  
 public 
  
 static 
  
 void 
  
 deleteInventoryReportConfig 
 ( 
  
 String 
  
 projectId 
 , 
  
 String 
  
 location 
 , 
  
 String 
  
 inventoryReportConfigUuid 
 ) 
  
 throws 
  
 IOException 
  
 { 
  
 try 
  
 ( 
  StorageInsightsClient 
 
  
 storageInsightsClient 
  
 = 
  
  StorageInsightsClient 
 
 . 
 create 
 ()) 
  
 { 
  
  ReportConfigName 
 
  
 name 
  
 = 
  
  ReportConfigName 
 
 . 
 of 
 ( 
 projectId 
 , 
  
 location 
 , 
  
 inventoryReportConfigUuid 
 ); 
  
 storageInsightsClient 
 . 
 deleteReportConfig 
 ( 
 name 
 ); 
  
 System 
 . 
 out 
 . 
 println 
 ( 
 "Deleted inventory report config with name " 
  
 + 
  
 name 
 ); 
  
 } 
  
 } 
 }

    REST APIs

    JSON API

    To delete an inventory report configuration, complete the following steps:

    1. Have gcloud CLI installed and initialized , which lets you generate an access token for the Authorization header.

    2. To delete an inventory report configuration, use cURL to call the JSON API with a Delete ReportConfig request:

      curl --request DELETE \
'https://storageinsights.googleapis.com/v1/projects/ YOUR_PROJECT 
/locations/ LOCATION 
/reportConfigs/ REPORT_CONFIG_UUID 
?force=true' \
  --header 'Authorization: Bearer $(gcloud auth print-access-token)' \
  --header 'Accept: application/json' \
  --header 'Content-Type: application/json' \

      Replace:

      • YOUR_PROJECT with the name of the project in which your source and destination buckets reside.

      • LOCATION with the location of the source and destination buckets. For example, us-west1 .

      • REPORT_CONFIG_UUID with the auto-generated UUID of the inventory report configuration you want to delete.

    The force query parameter deletes the metadata for all inventory reports generated by the given inventory report configuration. The inventory report objects themselves will not be deleted.

    Advanced: Generate inventory reports in a Hive partitioned format

    When you create an inventory report configuration, you can specify a destination path, which is a location in your destination bucket where inventory reports will be generated. This enables you to generate inventory reports in a Hive partitioned format .

    You specify a destination path by using placeholder keywords . Keywords in the destination path are substituted with their corresponding values when the inventory report is generated. For example, the destination path config={{report-config-id}}/date={{date}} can resolve to config=1A34-F2E456-12B456-1C3D/date=2022-05-20 .

    Destination path keywords

    You can specify the keywords report-config-id , date , or datetime , enclosed within {{ and }}, where:

    • report-config-id is the UUID of the inventory report configuration.

    • date is the date on which the inventory report is generated. Follows the ISO 8601 format.

    • datetime is the date and time on which the inventory report is generated. Follows the ISO 8601 format.

    Keywords can be specified by using any of the following tools:

    • When using the Google Cloud console, specify keywords in the Enter destination path (optional)section. This section appears when you create an inventory report configuration .

    • When using the Google Cloud CLI, specify placeholder keywords by using the --destination flag.

    • When using the JSON API, specify placeholder keywords in the storage_destination_options.destination_path field of the ReportConfig object.

    Access control and security

    The following recommendations and considerations relate to the security and usage of inventory reports and configurations:

    • We recommend that users with the roles/storage.admin role also have the storageinsights.reportConfigs.* permissions so they can maintain control over the usage of all resources in their buckets, such as the reading of objects and their metadata.

    • We recommend that you limit access to your inventory report resources by granting the storageinsights.reportConfigs.* permissions only to necessary individuals.

    • Once an inventory report configuration has been created, it continues to generate inventory reports even if the user who created the configuration no longer has the required permissions. To stop generating inventory reports, you can edit the end date in the inventory report configuration or delete the configuration entirely.

    What's next

    • If you encounter issues with generating inventory reports, refer to Troubleshooting for help.
    Create a Mobile Website
    View Site in Mobile | Classic
    Share by: