Export data for role recommendations

The IAM role recommender uses aggregated IAM access data, collected during the usage of services in Google Cloud, to provide recommendations. This data is primarily used for compliance purposes.

This page explains how to export that access data to BigQuery using the BigQuery Data Transfer Service .

If you want to export a snapshot of your insights and recommendations, see Export recommendations to BigQuery .

Before you begin

• Enable the IAM, Resource Manager, Recommender, BigQuery, BigQuery Data Transfer Service, and Pub/Sub APIs.

  Enable the APIs

• Read about role recommendations .

Required permissions

To get the permissions that you need to create a data transfer, ask your administrator to grant you the following IAM roles:

• Data Processing Controls Resource Admin ( roles/dataprocessing.admin ) on your organization
• BigQuery Admin ( roles/bigquery.admin ) on the project that you will export data to
• To publish notifications for your transfer to an existing Pub/Sub topic: Pub/Sub Viewer ( roles/pubsub.viewer ) on the project that you will export data to
• To publish notifications for your topic to a new Pub/Sub topic: Pub/Sub Editor ( roles/pubsub.editor ) on the project that you will export data to

For more information about granting roles, see Manage access to projects, folders, and organizations .

You might also be able to get the required permissions through custom roles or other predefined roles .

Export aggregated IAM access data

To export your projects' aggregated IAM access history to BigQuery, use the Transparency and Control Center to set up a data transfer:

1. In the Google Cloud console, go to the Privacy & Securitypage.

   Go to Privacy & Security

2. Select your organization from the drop-down list, then click Select.

3. Click Transparency & control.

4. In the Data processing grouptable, click IAM.

5. In the Data sourcessection of the page, click add Create transfer.

6. In the Projectfield, click Browse, then select the project that you want to export data to. If the project does not have the BigQuery Data Transfer Service API enabled, click Enable APIand wait until the API is enabled.

7. Click Next.

8. Configure the data transfer:

   1. In the Display namefield, enter a display name for your data transfer.

   2. In Schedule optionssection, choose when the data transfer will start and how often it will run.

      • To choose when to start the transfer, you can leave the default value of Start now, or click Start at a set time.
      • In the Repeatsfield, choose an option for how often to run the transfer. If you choose an option other than Daily, additional options are available. For example, if you choose Weekly, an option appears for you to select the day of the week.
      • For Start date and run time, enter the date and time to start the transfer. If you choose Start now, this option is disabled.

   3. In the Dataset IDfield, choose a BigQuery dataset to export the data to.

      You can export data to an existing dataset, or create a new dataset:

      • To export data to an existing dataset, click the Dataset IDfield, then select a dataset from the drop-down list.

      • To export data to a new dataset, click the Dataset IDfield, click Create new dataset, and fill out the fields in the Create datasetpane:

        1. In the Dataset IDfield, enter an ID for the dataset. Letters, numbers, and underscores are allowed.
        2. From the Data locationdrop-down list, select either United States (US)or European Union (EU).
        3. Optional: Enable table expiration by selecting Enable table expiration.
        4. Optional: Select an encryption method. The default encryption method is Google-managed encryption key. If you select Customer-managed encryption key (CMEK), you must also select a customer-managed key .

      The transfer you set up will be in the same region as the dataset, and cannot be moved.

   4. In the project_numbersfield, enter the project numbers for the projects whose aggregated IAM access data you want to export. If you list multiple project numbers, separate the project numbers with commas. You can export data for up to 10 projects at a time.

      To find a project's number, do the following:

      1. In the Google Cloud console, go to the Settingspage.

         Go to Settings

      2. Select your project.

      3. Copy the project ID from the Project numberfield.

   5. Optional: Enable notifications for your transfer:

      • To enable notifications for failed transfer runs, click the Email notificationstoggle. When you enable this option, the transfer administrator receives an email notification when a transfer run fails.
      • To enable Pub/Sub notifications for your transfer , click Select a Pub/Sub topic, then select or create a topic.

9. Click Done.

10. If prompted, allow IAM Recommender Aggregated Access Transfersaccess to your Google account.

Manage existing data transfers

You can view and manage your transfers in the Transparency and Control Center, or in BigQuery:

• To view all aggregated IAM access data transfers for your organization, use the Transparency and Control Center:

  1. In the Google Cloud console, go to the Privacy & Securitypage.

     Go to Privacy & Security

  2. Select your organization from the drop-down list, then click Select.

  3. Click Transparency & control.

  4. In the Data processing grouptable, click IAM. The Data transferssection of the page lists all aggregated IAM access data transfers for your organization.

  5. To manage an individual transfer, click the transfer's display name.

• To view all data transfers in a project, including aggregated IAM access data transfers, use BigQuery:

  1. In the Google Cloud console, go to the Data transferspage.

     Go to Data transfers

  2. Select the project that you exported data to.

  3. The Data transferspage shows all data transfers for your project, including aggregated IAM access data transfers.

  4. To manage an individual transfer, click the transfer's display name.

What's next

• Learn how to export a snapshot of your recommendations and insights .
• Understand best practices for using role recommendations .
• Find out how to review and apply recommendations .
• Learn how to disable role recommendations .