This guide walks you through inspecting data in Cloud Storage, Firestore in Datastore mode (Datastore), or BigQuery and sending the inspection results to Security Command Center.
To use this feature, your project must belong to an organization, and Security Command Center must be activated at the organization level. Otherwise, Sensitive Data Protection findings won't appear in Security Command Center. For more information, see Check the activation level of Security Command Center .
For BigQuery data, you can additionally perform profiling , which is different from an inspection operation. You can also send data profiles to Security Command Center. For more information, see Publish data profiles to Security Command Center .
Overview
Security Command Center enables you to gather data about, identify, and act on security threats before they can cause business damage or loss. With Security Command Center, you can perform several security-related actions from a single centralized dashboard.
Sensitive Data Protection has built-in integration with Security Command Center. When you use a Sensitive Data Protection action to inspect your Google Cloud storage repositories for sensitive data, it can send results directly to the Security Command Center dashboard. They display next to other security metrics.
By completing the steps in this guide, you do the following:
- Enable Security Command Center and Sensitive Data Protection.
- Set up Sensitive Data Protection to inspect a Google Cloud storage repository—either a Cloud Storage bucket, BigQuery table, or Datastore kind.
- Configure a Sensitive Data Protection scan to send the inspection job results to Security Command Center.
For more information about Security Command Center, see the Security Command Center documentation .
If you want to send the results of discovery scans—not inspection jobs—to Security Command Center, see the documentation for profiling an organization, folder , or project instead.
Costs
In this document, you use the following billable components of Google Cloud:
- Sensitive Data Protection
- Cloud Storage
- BigQuery
- Datastore
To generate a cost estimate based on your projected usage, use the pricing calculator .
Before you begin
Before you can send Sensitive Data Protection scan results to Security Command Center, you must do each of the following:
- Step 1: Set Google Cloud storage repositories.
- Step 2: Set Identity and Access Management (IAM) roles.
- Step 3: Enable Security Command Center.
- Step 4: Enable Sensitive Data Protection.
- Step 5: Enable Sensitive Data Protection as a security source for Security Command Center.
The steps to set up these components are described in the following sections.
Step 1: Set Google Cloud storage repositories
Choose whether you want to scan your own Google Cloud storage repository or an example one. This topic provides instructions for both scenarios.
Scan your own data
If you want to scan your own existing Cloud Storage bucket, BigQuery table, or Datastore kind, first open the project that the repository is in. In subsequent steps, you'll enable both Security Command Center and Sensitive Data Protection for this project and its organization.
After you open the project you want to use, proceed to Step 2 to set up some IAM roles .
Scan sample data
If you want to scan a test set of data, first make sure that you have a billing account set up, and then create a new project. To complete this step, you must have the IAM Project Creatorrole. Learn more about IAM roles .
- If you don't already have billing configured, set up a billing account.
- Go to the New Projectpage in the Google Cloud console.
- On the Billing accountdrop-down list, select the billing account that the project should be billed to.
- On the Organizationdrop-down list, select the organization that you want to create the project in.
- On the Locationdrop-down list, select the organization or folder that you want to create the project in.
Next, download and store the sample data:
- Go to the Cloud Run functions tutorials repository on GitHub .
- Click Clone or download, and then click Download ZIP.
- Extract the zip file that you downloaded.
- Go to the Storage Browser page in the Google Cloud console.
- Click Create bucket.
- On the Create a bucketpage, give the bucket a unique name, and then click Create.
- On the Bucket detailspage, click Upload folder.
- Go to the
dlp-cloud-functions-tutorials-masterfolder that you extracted, open it, and then select thesample_datafolder. Click Uploadto upload the folder's contents to Cloud Storage.
Note the name that you gave the Cloud Storage bucket for later. After the file upload completes, you're ready to continue.
Step 2: Set IAM roles
To use Sensitive Data Protection to send scan results to Security Command Center, you need the Security Center Adminand Sensitive Data Protection Jobs EditorIAM roles. This section describes how to add the roles. To complete this section, you must have the Organization AdministratorIAM role.
-
Go to the IAM page.
- On the View by principalstab, find your Google Account and click edit Edit principal.
-
Add the Security Center Adminand Sensitive Data Protection Jobs Editorroles:
- In the Edit accesspanel, click Add another role.
- In the Select a rolelist, search for Security Center Admin, and select it.
- Click Add another role.
- In the Select a rolelist, search for DLP Jobs Editor, and select it.
- Click Save.
You now have Sensitive Data Protection Jobs Editor and Security Center Admin roles for your organization. These roles let you complete the tasks in the remainder of this topic.
Step 3: Enable Security Command Center
-
Go to the Security Command Center page in the Google Cloud console.
-
On the Organizationdrop-down list, select the organization for which you want to enable Sensitive Data Protection, and then click Select.
-
On the Enable asset discoverypage that appears, select All current and future projects, and then click Enable. A message should display that Sensitive Data Protection is beginning asset discovery.
After asset discovery is complete, Sensitive Data Protection will display your supported Google Cloud assets. Asset discovery might take a few minutes, and you might need to refresh the page to display the assets.
For more information about enabling Security Command Center, see the Security Command Center documentation .
Step 4: Enable Sensitive Data Protection
Enable Sensitive Data Protection for the project you want to scan. The project must be within the same organization for which you've enabled Security Command Center. To enable Sensitive Data Protection using the Google Cloud console:
- In the Google Cloud console, go to the Enable access to APIpage.
- On the toolbar, select the project from Step 1 of this guide. The project must contain the Cloud Storage bucket, BigQuery table, or Datastore kind you want to scan.
- Click Next.
- Click Enable.
Sensitive Data Protection is now enabled for your project.
Step 5: Enable Sensitive Data Protection as an integrated service for Security Command Center
To view Sensitive Data Protection scan findings in Security Command Center, enable Sensitive Data Protectionas an integrated service. For more information, see Add a Google Cloud integrated service in the Security Command Center documentation.
Findings for Sensitive Data Protection are displayed on the Findingspage in Security Command Center.
Configure and run a Sensitive Data Protection inspection scan
In this section, you configure and run a Sensitive Data Protection inspection job.
The inspection job that you configure here instructs Sensitive Data Protection to scan either the sample data stored in Cloud Storage or your own data stored in Cloud Storage, Datastore, or BigQuery. The job configuration that you specify is also where you instruct Sensitive Data Protection to save its scan results to Security Command Center.
Step 1: Note your project identifier
- Go to the Google Cloud console.
- Click Select.
- On the Select fromdrop-down list, select the organization for which you enabled Security Command Center.
- Under ID, copy the project ID for the project that contains the data you want to scan .
- Under Name, click the project to select it.
Step 2: Open APIs Explorer and configure the job
- Go to APIs Explorer on the reference page for the
dlpJobs.createmethod by clicking the following button: - In the parentbox, enter the following, where PROJECT_ID
is
the project ID you noted in Step 1:
projects/ PROJECT_ID
Replace the contents of the Request bodyfield with the following JSON for the kind of data you want to use: sample data in a Cloud Storage bucket, or your own data stored in Cloud Storage, Datastore, or BigQuery.
Sample data
If you created a Cloud Storage bucket to store sample data
,
copy the following JSON and then paste it into the Request
bodyfield. Replace BUCKET_NAME
with the name
that you gave your Cloud Storage bucket:
{
"inspectJob":{
"storageConfig":{
"cloudStorageOptions":{
"fileSet":{
"url":"gs:// BUCKET_NAME
/**"
}
}
},
"inspectConfig":{
"infoTypes":[
{
"name":"EMAIL_ADDRESS"
},
{
"name":"PERSON_NAME"
},
{
"name": "LOCATION"
},
{
"name":"PHONE_NUMBER"
}
],
"includeQuote":true,
"minLikelihood":"UNLIKELY",
"limits":{
"maxFindingsPerRequest":100
}
},
"actions":[
{
"publishSummaryToCscc":{
}
}
]
}
}
Cloud Storage data
To scan your own Cloud Storage bucket, copy the following JSON and paste it into the Request bodyfield.
Replace PATH_NAME
with the path to the location
that you want to scan. To scan recursively, end the path with two asterisks,
for example, gs://path_to_files/**
. To scan a specific directory and
no deeper, end the path with one asterisk, for example, gs://path_to_files/*
.
{
"inspectJob":{
"storageConfig":{
"cloudStorageOptions":{
"fileSet":{
"url":"gs:// PATH_NAME
"
}
}
},
"inspectConfig":{
"infoTypes":[
{
"name":"EMAIL_ADDRESS"
},
{
"name":"PERSON_NAME"
},
{
"name": "LOCATION"
},
{
"name":"PHONE_NUMBER"
}
],
"includeQuote":true,
"minLikelihood":"UNLIKELY",
"limits":{
"maxFindingsPerRequest":100
}
},
"actions":[
{
"publishSummaryToCscc":{
}
}
]
}
}
To learn more about the available scan options, see Inspecting storage and databases for sensitive data .
Datastore data
To scan your own data kept in Datastore, copy the following JSON and paste it into the Request bodyfield.
Replace DATASTORE_KIND
with the name of the
Datastore kind. You can also replace NAMESPACE_ID
and PROJECT_ID
with the namespace and project
identifiers, repectively, or you can remove the "partitionID"
completely
if you want.
{
"inspectJob":{
"storageConfig":{
"datastoreOptions":{
"kind":{
"name":" DATASTORE_KIND
"
},
"partitionId":{
"namespaceId":" NAMESPACE_ID
",
"projectId":" PROJECT_ID
"
}
}
},
"inspectConfig":{
"infoTypes":[
{
"name":"EMAIL_ADDRESS"
},
{
"name":"PERSON_NAME"
},
{
"name": "LOCATION"
},
{
"name":"PHONE_NUMBER"
}
],
"includeQuote":true,
"minLikelihood":"UNLIKELY",
"limits":{
"maxFindingsPerRequest":100
}
},
"actions":[
{
"publishSummaryToCscc":{
}
}
]
}
}
To learn more about the available scan options, see Inspecting storage and databases for sensitive data .
BigQuery data
To scan your own BigQuery table, copy the following JSON and paste it into the Request bodyfield.
Replace PROJECT_ID
, BIGQUERY_DATASET_NAME
, and BIGQUERY_TABLE_NAME
with the project ID and
BigQuery dataset and table names, repectively.
{
"inspectJob":
{
"storageConfig":
{
"bigQueryOptions":
{
"tableReference":
{
"projectId": " PROJECT_ID
",
"datasetId": " BIGQUERY_DATASET_NAME
",
"tableId": " BIGQUERY_TABLE_NAME
"
}
}
},
"inspectConfig":
{
"infoTypes":
[
{
"name": "EMAIL_ADDRESS"
},
{
"name": "PERSON_NAME"
},
{
"name": "LOCATION"
},
{
"name": "PHONE_NUMBER"
}
],
"includeQuote": true,
"minLikelihood": "UNLIKELY",
"limits":
{
"maxFindingsPerRequest": 100
}
},
"actions":
[
{
"publishSummaryToCscc":
{
}
}
]
}
}
To learn more about the available scan options, see Inspecting storage and databases for sensitive data .
Step 3: Execute the request to start the inspection job
After you configure the job by following the preceding steps, click Executeto send the request. If the request is successful, a response appears below the request with a success code and a JSON object that indicates the status of the Sensitive Data Protection job that you created.
Check the status of the Sensitive Data Protection inspection scan
The response to your scan request includes the job ID of your inspection scan
job as the "name"
key, and the current state of the inspection job as the "state"
key. Immediately after you submit the request, the job's state
is "PENDING"
.
After you submit the scan request, the scan of your content begins immediately.
To check the status of the inspection job:
- Go to APIs Explorer on the reference page for the
dlpJobs.getmethod by clicking the following button: - In the namebox, type the name of the job from the JSON response to
the scan request in the following form:
projects/ PROJECT_ID /dlpJobs/ JOB_ID
i-1234567890123456789. - To submit the request, click Execute.
If the response JSON object's "state"
key indicates that the job is "DONE"
,
then the inspection job has finished.
To view the rest of the response JSON, scroll down the page. Under "result"
> "infoTypeStats"
, each information type listed should have a corresponding "count"
. If not, make sure that you entered the JSON accurately, and that the
path or location to your data is correct.
After the inspection job is done, you can continue to the next section of this guide to view scan results in Security Command Center.
Code samples: inspect a Cloud Storage bucket
This example demonstrates how to use the DLP API to create an inspection job that inspects a Cloud Storage bucket and sends findings to Security Command Center.
C#
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries .
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .
Go
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries .
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .
Java
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries .
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .
Node.js
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries .
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .
PHP
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries .
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .
Python
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries .
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .
Code samples: inspect a BigQuery table
This example demonstrates how to use the DLP API to create an inspection job that inspects a BigQuery table and sends findings to Security Command Center.
C#
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries .
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .
Go
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries .
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .
Java
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries .
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .
Node.js
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries .
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .
PHP
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries .
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .
Python
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries .
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .
Code samples: inspect a Datastore kind
This example demonstrates how to use the DLP API to create an inspection job that inspects a Datastore kind and sends findings to Security Command Center.
C#
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries .
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .
Go
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries .
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .
Java
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries .
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .
Node.js
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries .
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .
PHP
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries .
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .
Python
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries .
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .
View Sensitive Data Protection scan results in Security Command Center
Because you instructed Sensitive Data Protection to send its inspection job results to Security Command Center, you can now view the results of the inspection job in Security Command Center:
- In the Google Cloud console, go to the Security Command Center Findingspage.
- Select the organization for which you enabled Security Command Center.
-
In the Query editorfield, enter the following to query for findings from Sensitive Data Protection.
state="ACTIVE" AND NOT mute="MUTED" AND (parent_display_name="Sensitive Data Protection" OR parent_display_name="Cloud Data Loss Prevention")For more information about the query editor, see Edit a findings query in the Google Cloud console .
If any findings were sent from Sensitive Data Protection, the findings appear in the findings list. The list includes all findings from Sensitive Data Protection, which can include findings from inspection jobs and discovery (data profiling) operations.
The instructions provided in this guide only turn on some of Sensitive Data Protection's built-in detectors.
- For more information about what else Sensitive Data Protection can detect, see the InfoTypes reference .
- For information about configuring your own custom infoType detectors, see Creating custom infoType detectors .
Clean up
To avoid incurring charges to your Google Cloud account for the resources used in this topic:
Delete the project
The easiest way to eliminate billing is to delete the project you created while following the instructions provided in this topic.
- In the Google Cloud console, go to the Manage resources page.
- In the project list, select the project that you want to delete, and then click Delete .
- In the dialog, type the project ID, and then click Shut down to delete the project.
If you delete your project using this method, the Sensitive Data Protection job and Cloud Storage bucket you created were also deleted. It's not necessary to follow the instructions in the following sections.
Delete the Sensitive Data Protection job
If you scanned your own data, you need to delete only the inspection job that you created:
- Go to APIs Explorer on the reference page for the
dlpJobs.deletemethod by clicking the following button: - In the namebox, type the name of the job from the JSON response to
the scan request, which has the following form:
projects/ PROJECT_ID /dlpJobs/ JOB_ID
i-1234567890123456789.
If you created additional inspection jobs or if you want to make sure you've deleted the job successfully, you can list all existing jobs:
- Go to APIs Explorer on the reference page for the
dlpJobs.listmethod by clicking the following button: - In the parentbox, type the project identifier in the following form:
projects/ PROJECT_ID - Click Execute.
If there are no jobs listed in the response, you've deleted all jobs. If jobs are listed in the response, repeat the deletion procedure for those jobs.
Delete the Cloud Storage bucket
If you created a new Cloud Storage bucket to hold sample data, delete the bucket:
- Open the Cloud Storage browser .
- In the Cloud Storage browser, select the checkbox next to the name of the bucket you created, and then click Delete.
What's next
- Learn more about the
publishSummaryToCsccaction in Sensitive Data Protection. - Learn more about scanning storage repositories for sensitive data using Sensitive Data Protection.
- Learn how to use Security Command Center .
