Use inventory reports

Console

Enable the Storage Insights API.

Enable the API

Command line

Enable the Storage Insights API:

gcloud  
services  
 enable 
  
storageinsights.googleapis.com

REST APIs

1. Have gcloud CLI installed and initialized , which lets you generate an access token for the Authorization header.

2. To enable the Storage Insights API, use cURL to call the services.enable method:

   curl -X POST -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     https://serviceusage.googleapis.com/v1/projects/ PROJECT_ID 
   /services/storageinsights.googleapis.com:enable

   Replace:

   PROJECT_ID with the ID of the project you want to enable the API for.

Console

To create an inventory report configuration, complete the following steps:

1. In the Google Cloud console, go to the Cloud Storage Buckets page.
   

   Go to Buckets

2. In the list of buckets, click the name of the bucket you'd like to use as the source bucket .

3. On the Bucket detailspage, click the Inventory reportstab.

4. Click Create report configuration.

5. In the Identify your report configurationsection, create a display name for the inventory report configuration. The display name can contain up to 256 characters.

6. In the Choose metadata fieldssection, select the metadata fields you want to include in your inventory reports.

7. Click Continue.

8. In the Choose a file formatsection, select the file format in which you'd like to generate inventory reports.

9. In the Choose a destination bucketsection, select the bucket you'd like to use as a destination bucket .

   In the Enter destination path (optional)section, you can optionally specify a destination path where inventory reports will be generated.

10. In the Scheduling optionssection, specify the frequency, start date, and end date for generating reports.

11. Click Create.

    The Report configuration detailspage is displayed. The metadata of generated inventory reports will appear on the page.

Command line

1. To create an inventory report configuration, run the gcloud storage insights inventory-reports create command:

   gcloud storage insights inventory-reports create SOURCE_BUCKET_URL 
   \
     --csv-separator= SEPARATOR 
   \
     --csv-delimiter= DELIMITER 
   \ --[no]-csv-header 
   \ --parquet 
   \
     --display-name= DISPLAY_NAME 
   \
     --destination= DESTINATION_PATH 
   \
     --metadata-fields= METADATA_FIELD 
   \
     --schedule-starts= START_DATE 
   \
     --schedule-repeats= FREQUENCY 
   \
     --schedule-repeats-until= END_DATE 
   

   Replace:

   • SOURCE_BUCKET_URL with the URL of the source bucket. For example, gs://my_example_source_bucket .

   • SEPARATOR with the character used to separate the records in the inventory report CSV file. Must be either \n or \r\n . Default value is \n . Optional. If the --csv-separator flag is used, --parquet cannot be used.

   • DELIMITER with the delimiter that separates the fields in the inventory report CSV file. The value can include one character and cannot be the same value as SEPARATOR . Default value is , . Optional. If --csv-delimiter is used, --parquet cannot be used.

   • --[no]-csv-header with the flag that indicates whether or not headers are included in the inventory report CSV file. Use --csv-header to include headers and --no-csv-header to exclude headers. If either flag is used, --parquet cannot be used.

   • DISPLAY_NAME with the editable name of the inventory report configuration. Optional.

   • --parquet with the flag that generates inventory reports in Apache Parquet format instead of CSV. If used, --csv-delimiter , --csv-separator , and --[no-]csv-header cannot be used.

   • DESTINATION_PATH with the bucket or folder in the bucket where inventory reports will be generated. For example, gs://my_example_destination_bucket or gs://my_example_destination_bucket/path/to/inventory/report .

     When you specify a path to a folder within a bucket, you can use keywords in the path, which are substituted with their corresponding values when the report is generated. This lets you generate reports in a Hive partitioned format so you can load or query the data in BigQuery without additional processing.

   • METADATA_FIELD with a comma-separated list of metadata fields you want included in the inventory report.

   • START_DATE with the UTC date on which you want to start generating inventory reports. For example, 2022-01-15 .

   • FREQUENCY with how often you want inventory reports to be generated. Values are daily or weekly .

   • END_DATE with the UTC date after which you want to stop generating inventory reports. Must be a value after the START_DATE . For example, if you specify 2022-02-15 , inventory reports will no longer be generated as of February 16, 2022.

Client libraries

  import 
  
 com.google.cloud.storageinsights.v1. CSVOptions 
 
 ; 
 import 
  
 com.google.cloud.storageinsights.v1. CloudStorageDestinationOptions 
 
 ; 
 import 
  
 com.google.cloud.storageinsights.v1. CloudStorageFilters 
 
 ; 
 import 
  
 com.google.cloud.storageinsights.v1. CreateReportConfigRequest 
 
 ; 
 import 
  
 com.google.cloud.storageinsights.v1. FrequencyOptions 
 
 ; 
 import 
  
 com.google.cloud.storageinsights.v1. LocationName 
 
 ; 
 import 
  
 com.google.cloud.storageinsights.v1. ObjectMetadataReportOptions 
 
 ; 
 import 
  
 com.google.cloud.storageinsights.v1. ReportConfig 
 
 ; 
 import 
  
 com.google.cloud.storageinsights.v1. StorageInsightsClient 
 
 ; 
 import 
  
 com.google.common.collect.ImmutableList 
 ; 
 import 
  
 com.google.type. Date 
 
 ; 
 import 
  
 java.io.IOException 
 ; 
 public 
  
 class 
 CreateInventoryReportConfig 
  
 { 
  
 public 
  
 static 
  
 void 
  
 createInventoryReportConfig 
 ( 
  
 String 
  
 projectId 
 , 
  
 String 
  
 bucketLocation 
 , 
  
 String 
  
 sourceBucket 
 , 
  
 String 
  
 destinationBucket 
 ) 
  
 throws 
  
 IOException 
  
 { 
  
 try 
  
 ( 
  StorageInsightsClient 
 
  
 storageInsightsClient 
  
 = 
  
  StorageInsightsClient 
 
 . 
 create 
 ()) 
  
 { 
  
  ReportConfig 
 
  
 reportConfig 
  
 = 
  
  ReportConfig 
 
 . 
 newBuilder 
 () 
  
 . 
  setDisplayName 
 
 ( 
 "Example inventory report configuration" 
 ) 
  
 . 
  setFrequencyOptions 
 
 ( 
  
  FrequencyOptions 
 
 . 
 newBuilder 
 () 
  
 . 
  setFrequency 
 
 ( 
  FrequencyOptions 
 
 . 
 Frequency 
 . 
 WEEKLY 
 ) 
  
 . 
  setStartDate 
 
 ( 
  Date 
 
 . 
 newBuilder 
 (). 
 setDay 
 ( 
 15 
 ). 
 setMonth 
 ( 
 8 
 ). 
 setYear 
 ( 
 3022 
 ). 
 build 
 ()) 
  
 . 
  setEndDate 
 
 ( 
  Date 
 
 . 
 newBuilder 
 (). 
 setDay 
 ( 
 15 
 ). 
 setMonth 
 ( 
 9 
 ). 
 setYear 
 ( 
 3022 
 ). 
 build 
 ()) 
  
 . 
 build 
 ()) 
  
 . 
  setCsvOptions 
 
 ( 
  
  CSVOptions 
 
 . 
 newBuilder 
 () 
  
 . 
  setDelimiter 
 
 ( 
 "," 
 ) 
  
 . 
  setRecordSeparator 
 
 ( 
 "\n" 
 ) 
  
 . 
  setHeaderRequired 
 
 ( 
 true 
 ) 
  
 . 
 build 
 ()) 
  
 . 
  setObjectMetadataReportOptions 
 
 ( 
  
  ObjectMetadataReportOptions 
 
 . 
 newBuilder 
 () 
  
 . 
  addAllMetadataFields 
 
 ( 
 ImmutableList 
 . 
 of 
 ( 
 "project" 
 , 
  
 "name" 
 , 
  
 "bucket" 
 )) 
  
 . 
  setStorageFilters 
 
 ( 
  
  CloudStorageFilters 
 
 . 
 newBuilder 
 (). 
 setBucket 
 ( 
 sourceBucket 
 ). 
 build 
 ()) 
  
 . 
  setStorageDestinationOptions 
 
 ( 
  
  CloudStorageDestinationOptions 
 
 . 
 newBuilder 
 () 
  
 . 
 setBucket 
 ( 
 destinationBucket 
 ) 
  
 . 
 build 
 ()) 
  
 . 
 build 
 ()) 
  
 . 
 build 
 (); 
  
  CreateReportConfigRequest 
 
  
 request 
  
 = 
  
  CreateReportConfigRequest 
 
 . 
 newBuilder 
 () 
  
 . 
 setParent 
 ( 
  LocationName 
 
 . 
 of 
 ( 
 projectId 
 , 
  
 bucketLocation 
 ). 
 toString 
 ()) 
  
 . 
 setReportConfig 
 ( 
 reportConfig 
 ) 
  
 . 
 build 
 (); 
  
  ReportConfig 
 
  
 response 
  
 = 
  
 storageInsightsClient 
 . 
 createReportConfig 
 ( 
 request 
 ); 
  
 System 
 . 
 out 
 . 
 println 
 ( 
 "Created inventory report config with name " 
  
 + 
  
 response 
 . 
  getName 
 
 ()); 
  
 } 
  
 } 
 }