Use customer-managed encryption keys (CMEK)

This page describes how to perform tasks related to customer-managed encryption keys (CMEK) for Cloud Firestore with MongoDB compatibility . For more information about CMEK in general, including when and why to enable it, see the Cloud KMS documentation .

Prepare your CMEK keys

Before you can create a CMEK-protected Cloud Firestore with MongoDB compatibility database, you must complete the following steps:

1. Request access to the Cloud Firestore with MongoDB compatibility CMEK feature .
2. Create (or retrieve) a Cloud Firestore with MongoDB compatibility service agent .
3. Create a CMEK key .
4. Configure IAM settings for that key .

Complete these steps for each project that will contain CMEK-protected Cloud Firestore with MongoDB compatibility databases. If you later create a new CMEK key, you must configure IAM settings for that key.

Request access

Before you create a Cloud Firestore with MongoDB compatibility service agent, request access to the CMEK feature by filling in the access request form .

Create a Cloud Firestore with MongoDB compatibility service agent

Before you create a CMEK key, you must have a Cloud Firestore with MongoDB compatibility service agent , which is a type of Google-managed service account that Cloud Firestore with MongoDB compatibility uses to access the key.

Run the services identity create command to create the service agent that Cloud Firestore with MongoDB compatibility uses to access the CMEK key on your behalf. This command creates the service account if it does not already exist, then displays it.

 gcloud  
beta  
services  
identity  
create  
 \ 
  
--service = 
firestore.googleapis.com  
 \ 
  
--project  
 FIRESTORE_PROJECT 
 

Replace FIRESTORE_PROJECT with the project you plan to use for your Cloud Firestore with MongoDB compatibility databases.

The command displays the service agent ID, which is formatted like an email address. Record the output email string, because you'll use it in a later step.

 Service identity created:
service-xxx@gcp-sa-firestore. 

Create a key

You can use a key created directly in Cloud KMS or an externally managed key that you make available with Cloud External Key Manager .

The Cloud KMS key location must be the same as the location of the Cloud Firestore with MongoDB compatibility database that it will be used with.

• For regional database locations , use the same location name for key ring, key, and database because the location names have a one-to-one mapping.

  For example, if you want to create a CMEK-protected database in us-west1 , create a key ring and key in us-west1 .

• For multi-region database locations , use the location name of the KMS multi-region location :

  • Use the Cloud KMS us multi-region location for the Cloud Firestore with MongoDB compatibility nam5 multi-region location.

  • Use the Cloud KMS europe multi-region location for the Cloud Firestore with MongoDB compatibility eur3 multi-region location.

In the Google Cloud project where you want to manage your keys, complete the following:

1. Enable the Cloud KMS API .

2. Create a key ring and a key using one of the following options:

   • Create the key ring and key directly in Cloud KMS .
   • Use an externally managed key. Create the external key and then create an Cloud EKM key to make the key available through Cloud KMS.

Configure IAM settings for the key

 gcloud  
kms  
keys  
add-iam-policy-binding  
 KMS_KEY 
  
 \ 
--keyring  
 KMS_KEYRING 
 \ 
--location  
 KMS_LOCATION 
  
 \ 
--member  
serviceAccount: SERVICE_AGENT_EMAIL 
  
 \ 
--role  
roles/cloudkms.cryptoKeyEncrypterDecrypter  
 \ 
--project  
 KMS_PROJECT 
 

 Updated IAM policy for key KMS_KEY 
.
bindings:
- members:
- serviceAccount:
service-{project-number}@gcp-sa-firestore.
role: roles/cloudkms.cryptoKeyEncrypterDecrypter 

Create a CMEK-enabled database

After your CMEK keys are created and configured, you can create a CMEK-protected database. Existing Cloud Firestore with MongoDB compatibility databases that are protected by Google default encryption can't be converted to use CMEK.

You can choose an encryption type and key only when you create a CMEK-enabled database.

 gcloud  
firestore  
databases  
create  
 \ 
  
--location = 
 FIRESTORE_DATABASE_LOCATION 
  
 \ 
  
--database = 
 DATABASE_ID 
  
 \ 
  
--edition = 
enterprise  
 \ 
  
--kms-key-name = 
 KMS_KEY_NAME 
  
 \ 
  
--project = 
 FIRESTORE_PROJECT 
 

Access a CMEK-protected database

All the read, write, and query operations sent to a CMEK-protected database should function the same as with a Google default encrypted database. For example, you don't need to provide a key for each request.

Restore a CMEK-protected database

Before you restore CMEK-protected database from a backup:

• Decide if you want to restore the database to CMEK encryption, to Google's default encryption (non-CMEK), or to the same encryption as the backup.

• Prepare the key (primary-version) and the key version that you used to encrypt the backup. Enable both the key and the key version.

 gcloud  
firestore  
databases  
restore  
 \ 
--encryption-type = 
 customer-managed-encryption  
 \ 
--kms-key-name = 
 KMS_KEY_NAME 
 

 gcloud  
firestore  
databases  
restore  
 \ 
--encryption-type = 
 google-default-encryption 

 gcloud  
firestore  
databases  
restore  
--encryption-type = 
 use-source-encryption 

 firebase  
firestore:databases:restore  
 \ 
--database  
 DATABASE_ID 
  
 \ 
--backup  
 'projects/ FIRESTORE_PROJECT 
/locations/ FIRESTORE_LOCATION 
/backups/ BACKUP_ID 
' 
  
 \ 
--encryption-type  
 CUSTOMER_MANAGED_ENCRYPTION  
 \ 
--kms-key-name  
projects/ KMS_PROJECT 
/locations/ KMS_LOCATION 
/keyRings/ KMS_KEYRING_ID 
/cryptoKeys/ KMS_KEY_ID 
  
 \ 
--project  
 FIRESTORE_PROJECT 
 

 firebase  
firestore:databases:get  
 DATABASE_ID 
  
--project  
 FIRESTORE_PROJECT 
 

 firebase  
firestore:databases:restore  
 \ 
--database  
 DATABASE_ID 
  
 \ 
--backup  
 'projects/ FIRESTORE_PROJECT 
/locations/ FIRESTORE_LOCATION 
/backups/ BACKUP_ID 
' 
  
 \ 
--encryption-type  
 GOOGLE_DEFAULT_ENCRYPTION  
 \ 
--project  
 FIRESTORE_PROJECT 
 

 firebase  
firestore:databases:restore  
 \ 
--database  
 DATABASE_ID 
  
 \ 
--backup  
 'projects/ FIRESTORE_PROJECT 
/locations/ FIRESTORE_LOCATION 
/backups/ BACKUP_ID 
' 
  
 \ 
--encryption-type  
 USE_SOURCE_ENCRYPTION 

Clone a CMEK-protected database

Before you clone a CMEK-protected database:

• Decide if you want to clone the database to CMEK encryption, to Google's default encryption (non-CMEK), or to the same encryption as the source database.

• Prepare the key (primary-version) and the key version that you used to encrypt the source database. Enable both the key and the key version.

 gcloud  
alpha  
firestore  
databases  
clone  
 \ 
--encryption-type = 
 customer-managed-encryption  
 \ 
--kms-key-name = 
 KMS_KEY_NAME 
 

 projects/ KMS_PROJECT 
/locations/ KMS_LOCATION 
/keyRings/ KMS_KEYRING_ID 
/cryptoKeys/ KMS_KEY_ID 
 

 gcloud  
alpha  
firestore  
databases  
clone  
 \ 
--encryption-type = 
 google-default-encryption 

 gcloud  
alpha  
firestore  
databases  
clone  
 \ 
--encryption-type = 
 use-source-encryption 

View the key in use

 gcloud  
firestore  
databases  
describe  
 \ 
  
--database = 
 DATABASE_ID 
  
 \ 
  
--project = 
 FIRESTORE_PROJECT 
 

  cmekConfig 
 : 
  
 activeKeyVersion 
 : 
  
 - 
  
 projects/PROJECT_ID/locations/us/keyRings/KEYRING_NAME/cryptoKeys/KEY_NAME/cryptoKeyVersions/1 
  
 kmsKeyName 
 : 
  
 projects/PROJECT_ID/locations/us/keyRings/KEYRING_NAME/cryptoKeys/KEY_NAME 
  
 locationId 
 : 
  
 nam5 
  
 name 
 : 
  
 projects/PROJECT_ID/databases/DATABASE_ID 
 

 GET https://firestore.googleapis.com/v1/{name=projects/FIRESTORE_PROJECT/databases/DATABASE_ID} 

 curl  
 'https://firestore.googleapis.com/v1/projects/FIRESTORE_PROJECT/databases/{DATABASE_ID}' 
  
 \ 
-H  
 "Authorization: Bearer 
 $( 
gcloud  
auth  
print-access-token ) 
 " 
  
 \ 
-H  
 "Content-type: application/json" 
 

  { 
  
 "name" 
 : 
  
 "projects/FIRESTORE_PROJECT/databases/{DATABASE_ID}" 
 , 
  
 "locationId" 
 : 
  
 "{FIRESTORE_DATABASE_LOCATION}" 
 , 
  
 "type" 
 : 
  
 "FIRESTORE_NATIVE" 
 , 
  
 "cmekConfig" 
 : 
  
 { 
  
 "kmsKeyName" 
 : 
  
 "projects/{KMS_PROJECT}/locations/{KMS_LOCATION}/keyRings/{KMS_KEYRING_ID}/cryptoKeys/{KMS_KEY_ID}" 
 , 
  
 "activeKeyVersion" 
 : 
  
 [ 
  
 "projects/{KMS_PROJECT}/locations/{KMS_LOCATION}/keyRings/{KMS_KEYRING_ID}/cryptoKeys/{KMS_KEY_ID}/cryptoKeyVersions/1" 
  
 ] 
  
 }, 
  
 ... 
 } 
 

Disable a key

To disable a key associated with a database, complete the following:

1. View the key versions in use for a database
2. Disable those key versions
3. Wait for the change to take effect and check if the data is no longer accessible. Changes typically take effect within minutes, but can take up to 3 hours.

When a key used by a database is disabled, expect to receive a INVALID_ARGUMENT exception with additional details in the error message, for example:

  { 
  
 "error" 
 : 
  
 { 
  
 "code" 
 : 
  
 400 
 , 
  
 "message" 
 : 
  
 "Failed: (InvalidArgument) The customer-managed encryption key required by the requested resource is not accessible. Error reason: projects/{FIRESTORE_PROJECT}/locations/{KMS_LOCATION}/keyRings/{KMS_KEYRING_ID}/cryptoKeys/{KMS_KEY_ID}/cryptoKeyVersions/1 is not enabled, current state is: DISABLED." 
 , 
  
 "status" 
 : 
  
 "INVALID_ARGUMENT" 
 , 
  
 "details" 
 : 
  
 [ 
  
 { 
  
 "@type" 
 : 
  
 "type.googleapis.com/google.rpc.DebugInfo" 
 , 
  
 "detail" 
 : 
  
 "Failed: (InvalidArgument) The customer-managed encryption key required by the requested resource is not accessible. Error reason: projects/{FIRESTORE_PROJECT}/locations/{KMS_LOCATION}/keyRings/{KMS_KEYRING_ID}/cryptoKeys/{KMS_KEY_ID}/cryptoKeyVersions/1 is not enabled, current state is: DISABLED." 
  
 } 
  
 ] 
  
 } 
 } 
 

Enable a key

To re-enable a key associated with a database, complete the following:

1. View the key versions in use for a database
2. Enable those key versions
3. Wait for the change to take effect and check if the data is no longer accessible. Changes typically take effect within minutes, but can take up to 3 hours.

View audit logs for a Cloud KMS key

Before you enable Cloud KMS Data Access audit logs, you should be familiar with Cloud Audit Logs .

Cloud KMS Data Access audit logs show you when Cloud Firestore with MongoDB compatibility or any other products that are configured to use your CMEK key make encrypt or decrypt calls to Cloud KMS. Cloud Firestore with MongoDB compatibility does not issue an encrypt or decrypt call on every data request, but instead maintains a poller that checks the key periodically. The polling results appear in the audit logs.

You can set up and interact with the audit logs in the Google Cloud console :

1. Make sure that logging is enabled for the Cloud KMS API in your project.

2. Go to Cloud Logging in the Google Cloud console.

   Go to Cloud Logging

3. Limit the log entries to your Cloud KMS key by adding the following lines to the Query builder:

    resource.type="cloudkms_cryptokey"
   resource.labels.key_ring_id = KMS_KEYRING 
   resource.labels.crypto_key_id = KMS_KEY 
   resource.labels.location= KMS_LOCATION 
    
   

   Replace the following:

   • KMS_KEY with the name of the CMEK key
   • KMS_KEYRING with the KMS key ring that contains the key
   • KMS_LOCATION with the location of the key and key ring

   The log shows a couple log entries about every five minutes per database. The log entries look similar to these examples:

    Info 2021-03-20 08:02:24.869 EDT Cloudkms.googleapis.com Decrypt projects/cloud-kms-project/locations/us-central1/keyRings/firestore-keys/cryptoKeys/my-cmek-key service-123456789123@gcp-sa-firestore.
   audit_log, method: "Decrypt", principal_email: "service-1234567891011@gcp-sa-firestore."
   
   Info 2021-03-20 08:02:24.913 EDT Cloudkms.googleapis.com Encrypt projects/cloud-kms-project/locations/us-central1/keyRings/firestore-keys/cryptoKeys/my-cmek-key service-123456789123@gcp-sa-firestore.
   audit_log, method: "Encrypt", principal_email: "service-123456789123@gcp-sa-firestore." 
   

See Understanding audit logs for details about interpreting audit logs.

Configure a CMEK organization policy

To specify encryption compliance requirements for Cloud Firestore with MongoDB compatibility databases in your organization, use a CMEK organization policy constraint .

Require CMEK protection

Configure constraints/gcp.restrictNonCmekServices to require CMEK for Cloud Firestore with MongoDB compatibility database creation. Set the constraint to deny and add firestore.googleapis.com to the deny list, for example:

 gcloud  
resource-manager  
org-policies  
deny  
gcp.restrictNonCmekServices  
is:firestore.googleapis.com  
--project = 
 FIRESTORE_PROJECT 
 

Replace FIRESTORE_PROJECT with the project to restrict.

To learn more about configuring organization policies, see Creating and editing policies .

After the policy takes effect, you receive a FAILED_PRECONDITION exception and error message if you try to create a non-CMEK database under the affected project. For example, an exception looks like:

  { 
  
 "error" 
 : 
  
 { 
  
 "code" 
 : 
  
 400 
 , 
  
 "message" 
 : 
  
 "Constraint 'constraints/gcp.restrictNonCmekServices' violated for 'projects/FIRESTORE_PROJECT' attempting to perform the operation 'google.firestore.admin.v1.FirestoreAdmin.CreateDatabase' with violated value 'firestore.googleapis.com'. See https://cloud.google.com/resource-manager/docs/organization-policy/org-policy-constraints for more information." 
 , 
  
 "status" 
 : 
  
 "FAILED_PRECONDITION" 
 , 
  
 "details" 
 : 
  
 [ 
  
 { 
  
 "@type" 
 : 
  
 "type.googleapis.com/google.rpc.PreconditionFailure" 
 , 
  
 "violations" 
 : 
  
 [ 
  
 { 
  
 "type" 
 : 
  
 "constraints/gcp.restrictNonCmekServices" 
 , 
  
 "subject" 
 : 
  
 "orgpolicy:projects/FIRESTORE_PROJECT" 
 , 
  
 "description" 
 : 
  
 "Constraint 'constraints/gcp.restrictNonCmekServices' violated for 'projects/FIRESTORE_PROJECT' attempting to perform the operation 'google.firestore.admin.v1.FirestoreAdmin.CreateDatabase' with violated value 'firestore.googleapis.com'. See https://cloud.google.com/resource-manager/docs/organization-policy/org-policy-constraints for more information." 
  
 } 
  
 ] 
 

Limit the use of keys for CMEK

To limit which Cloud KMS keys are used for CMEK protection, configure the constraints/gcp.restrictCmekCryptoKeyProjects constraint.

As a list constraint, the accepted values are resource hierarchy indicators (for example, projects/PROJECT_ID , under:folders/FOLDER_ID , and under:organizations/ORGANIZATION_ID ). Use this constraint by configuring a list of resource hierarchy indicators and setting the constraint to Allow. This configuration restricts supported services so that CMEK keys can be chosen only from the listed projects, folders, and organizations. Requests to create CMEK-protected resources in configured services don't succeed without a Cloud Firestore with MongoDB compatibility key from one of the allowed resources.

The following example allows only keys from the ALLOWED_KEY_PROJECT_ID for CMEK-protected databases in the specified project:

 gcloud  
resource-manager  
org-policies  
allow  
gcp.restrictCmekCryptoKeyProjects  
 \ 
under:projects/<var>ALLOWED_KEY_PROJECT_ID</var>  
 \ 
--project = 
<var>FIRESTORE_PROJECT</var> 

After the policy takes effect, you receive a FAILED_PRECONDITION exception and an error message if you violate the constraint. An exception looks like the following:

  { 
  
 "error" 
 : 
  
 { 
  
 "code" 
 : 
  
 400 
 , 
  
 "message" 
 : 
  
 "Constraint 'constraints/gcp.restrictCmekCryptoKeyProjects' violated for 'projects/FIRESTORE_PROJECT' attempting to perform the operation 'google.firestore.admin.v1.FirestoreAdmin.CreateDatabase' with violated value 'projects/{NOT_ALLOWED_KEY_PROJECT}'. See https://cloud.google.com/resource-manager/docs/organization-policy/org-policy-constraints for more information." 
 , 
  
 "status" 
 : 
  
 "FAILED_PRECONDITION" 
 , 
  
 "details" 
 : 
  
 [ 
  
 { 
  
 "@type" 
 : 
  
 "type.googleapis.com/google.rpc.PreconditionFailure" 
 , 
  
 "violations" 
 : 
  
 [ 
  
 { 
  
 "type" 
 : 
  
 "constraints/gcp.restrictCmekCryptoKeyProjects" 
 , 
  
 "subject" 
 : 
  
 "orgpolicy:projects/FIRESTORE_PROJECT" 
 , 
  
 "description" 
 : 
  
 "Constraint 'constraints/gcp.restrictCmekCryptoKeyProjects' violated for 'projects/FIRESTORE_PROJECT' attempting to perform the operation 'google.firestore.admin.v1.FirestoreAdmin.CreateDatabase' with violated value 'projects/{NOT_ALLOWED_KEY_PROJECT}'. See https://cloud.google.com/resource-manager/docs/organization-policy/org-policy-constraints for more information." 
  
 } 
  
 ] 
  
 } 
  
 ] 
  
 } 
 } 
 

What's next

• Read the overview of Cloud Firestore with MongoDB compatibility and CMEK