Use secrets from Secret Manager

This page explains how to include sensitive information such as passwords and API keys in Cloud Build.

Secret Manager is a Google Cloud service that securely stores API keys, passwords, and other sensitive data. To include sensitive information in your builds, you can store the information in Secret Manager and then configure your build to access the information from Secret Manager.

Before you begin

• Enable the Cloud Build and Secret Manager APIs.

  Enable the APIs

• To use the command-line examples in this guide, install and configure the Google Cloud CLI .

• Make sure you've stored the secret in Secret Manager. For instructions, see Creating a secret .

  • Note down the secret name and secret version of your secret. You'll need this information to configure Cloud Build to access the secret.

Required IAM permissions

Grant the Secret Manager Secret Accessor ( roles/secretmanager.secretAccessor ) IAM role for the secret to the service account you are using for the build:

1. Open the Secret Manager page in the Google Cloud console:

   Go to the Secret Manager page

2. Select the checkbox of the secret you wish to use in your build.

3. If it is not already open, click Show info panelto open the panel.

4. In the panel, under Permissions, click Add principal.

5. In the New principalsfield, enter the email address of your service account.

6. In the Select a roledrop-down box, select Secret Manager Secret Accessor.

7. Click Save.

Configuring builds to access UTF-8 secrets from Secret Manager

1. In your project root directory, create a Cloud Build config file named cloudbuild.yaml or cloudbuild.json .

2. In the build config file:

   • After all the build steps , add an availableSecrets field to specify the secret version and environment variables to use for your secret. You can include substitution variables in the value of the secretVersion field. You can specify more than one secret in a build.
   • In the build step where you want to specify the secret:
     • Add an entrypoint field pointing to bash to use the bash tool in the build step. This is requiredto refer to the environment variable for the secret.
     • Add a secretEnv field specifying the environment variable.
     • In the args field, add a -c flag as the first argument. Any string you pass after -c is treated as a command. For more information on running bash commands with -c , see the bash documentation .
     • When specifying the secret in the args field, specify it using the environment variable prefixed with $$ .

   The following example build config file shows how to login to Docker using the Docker username and password stored in Secret Manager.

   YAML

     steps 
    : 
    - 
     
    name 
    : 
     
    'gcr.io/cloud-builders/docker' 
     
    entrypoint 
    : 
     
    'bash' 
     
    args 
    : 
     
    [ 
    '-c' 
    , 
     
    'docker 
     
    login 
     
    --username=$$USERNAME 
     
    --password=$$PASSWORD' 
    ] 
     
    secretEnv 
    : 
     
    [ 
    'USERNAME' 
    , 
     
    'PASSWORD' 
    ] 
    availableSecrets 
    : 
     
    secretManager 
    : 
     
    - 
     
    versionName 
    : 
     
    projects/ PROJECT_ID 
   /secrets/ DOCKER_PASSWORD_SECRET_NAME 
   /versions/ DOCKER_PASSWORD_SECRET_VERSION 
    
     
    env 
    : 
     
    'PASSWORD' 
     
    - 
     
    versionName 
    : 
     
    projects/ PROJECT_ID 
   /secrets/ DOCKER_USERNAME_SECRET_NAME 
   /versions/ DOCKER_USERNAME_SECRET_VERSION 
    
     
    env 
    : 
     
    'USERNAME' 
    
   

   JSON

     { 
     
    "steps" 
    : 
     
    [ 
     
    { 
     
    "name" 
    : 
     
    "gcr.io/cloud-builders/docker" 
    , 
     
    "entrypoint" 
    : 
     
    "bash" 
    , 
     
    "args" 
    : 
     
    [ 
     
    "-c" 
    , 
     
    "docker login --username=$$USERNAME --password=$$PASSWORD" 
     
    ], 
     
    "secretEnv" 
    : 
     
    [ 
     
    "USERNAME" 
    , 
     
    "PASSWORD" 
     
    ] 
     
    } 
     
    ], 
     
    "availableSecrets" 
    : 
     
    { 
     
    "secretManager" 
    : 
     
    [{ 
     
    "versionName" 
    : 
     
    "projects/ PROJECT_ID 
   /secrets/ DOCKER_PASSWORD_SECRET_NAME 
   /versions/ DOCKER_PASSWORD_SECRET_VERSION 
   " 
    , 
     
    "env" 
    : 
     
    "PASSWORD" 
     
    }, 
     
    { 
     
    "versionName" 
    : 
     
    "projects/ PROJECT_ID 
   /secrets/ DOCKER_USERNAME_SECRET_NAME 
   /versions/ DOCKER_USERNAME_SECRET_VERSION 
   " 
    , 
     
    "env" 
    : 
     
    "USERNAME" 
     
    }] 
     
    } 
    } 
    
   

   Replace the placeholder values in the above commands with the following:

   • PROJECT_ID : The ID of the Google Cloud project where you've stored your secrets.
   • DOCKER_USERNAME_SECRET_NAME : The secret name corresponding to your Docker username. You can get the secret name from the Secret Managerpage in the Google Cloud console.
   • DOCKER_USERNAME_SECRET_VERSION : The secret version of your Docker username. You can get the secret version by clicking on a secret name on the Secret Managerpage in the Google Cloud console.
   • DOCKER_PASSWORD_SECRET_NAME : The secret name corresponding to your Docker password. You can get the secret name from the Secret Managerpage in the Google Cloud console.
   • DOCKER_PASSWORD_SECRET_VERSION : The secret version of your Docker password. You can get the secret version by clicking on a secret name on the Secret Managerpage in the Google Cloud console.

3. Use the build config file to start a build using the command line or to automate builds using triggers .

Example: Accessing secrets from scripts and processes

secretEnv field adds the value of the secret to the environment and you can access this value via environment variable from scripts or processes:

  steps 
 : 
 - 
  
 name 
 : 
  
 python:slim 
  
 entrypoint 
 : 
  
 python 
  
 args 
 : 
  
 [ 
 'main.py' 
 ] 
  
 secretEnv 
 : 
  
 [ 
 'MYSECRET' 
 ] 
 availableSecrets 
 : 
  
 secretManager 
 : 
  
 - 
  
 versionName 
 : 
  
 projects/$PROJECT_ID/secrets/mySecret/versions/latest 
  
 env 
 : 
  
 'MYSECRET' 
 

  { 
  
 "steps" 
 : 
  
 [ 
  
 { 
  
 "name" 
 : 
  
 "python:slim" 
 , 
  
 "entrypoint" 
 : 
  
 "python" 
 , 
  
 "args" 
 : 
  
 [ 
  
 "main.py" 
  
 ], 
  
 "secretEnv" 
 : 
  
 [ 
  
 "MYSECRET" 
  
 ] 
 } 
 ], 
 "availableSecrets" 
 : 
  
 { 
  
 "secretManager" 
 : 
  
 [ 
  
 { 
  
 "versionName" 
 : 
  
 "projects/$PROJECT_ID/secrets/mySecret/versions/latest" 
 , 
  
 "env" 
 : 
  
 "MYSECRET" 
  
 } 
  
 ] 
 } 
 } 
 

The following contents of main.py prints the first five characters of the secret:

  import 
  
 os 
 print 
 ( 
 os 
 . 
 environ 
 . 
 get 
 ( 
 "MYSECRET" 
 , 
 "Not Found" 
 )[: 
 5 
 ], 
 "..." 
 ) 
 

Example: authenticating to Docker

In some situations, before interacting with Docker images, your build would need to authenticate to Docker. For example, Docker authentication is required for builds to pull private images and push private or public images to Docker Hub. In these cases, you can store your Docker username and password in Secret Manager and then configure Cloud Build to access the username and password from Secret Manager. For instructions on doing this see Interacting with Docker Hub images .

Example: GitHub pull request creation

Another example where you might want to configure your build to access a sensitive information from Secret Manager is for creating a GitHub pull request in response to builds. To do this:

• Create a GitHub token .
• Store the GitHub token in Secret Manager.
• In your build config file:
  • After all the build steps , add an availableSecrets field to specify the secret version and the environment variable to use for the GitHub token.
  • Add a build step to invoke the command to create a GitHub pull request .
• Create a GitHub app trigger and use the build config file to invoke the trigger.

The following example config file shows how to create a GitHub pull request using the GitHub token:

  steps 
 : 
 - 
  
 name 
 : 
  
 'launcher.gcr.io/google/ubuntu1604' 
  
 id 
 : 
  
 Create GitHub pull request 
  
 entrypoint 
 : 
  
 bash 
  
 args 
 : 
  
 - 
  
 -c 
  
 - 
  
 curl -X POST -H "Authorization:Bearer $$GH_TOKEN" -H 'Accept:application/vnd.github.v3+json' https://api.github.com/repos/ GITHUB_USERNAME 
/ REPO_NAME 
/pulls -d '{"head":" HEAD_BRANCH 
","base":" BASE_BRANCH 
", "title":" NEW_PR 
"}' 
  
 secretEnv 
 : 
  
 [ 
 'GH_TOKEN' 
 ] 
 availableSecrets 
 : 
  
 secretManager 
 : 
  
 - 
  
 versionName 
 : 
  
 projects/ PROJECT_ID 
/secrets/ GH_TOKEN_SECRET_NAME 
/versions/latest 
  
 env 
 : 
  
 GH_TOKEN 
 

  { 
  
 "steps" 
 : 
  
 [ 
  
 { 
  
 "name" 
 : 
  
 "launcher.gcr.io/google/ubuntu1604" 
 , 
  
 "id" 
 : 
  
 "Create GitHub pull request" 
 , 
  
 "entrypoint" 
 : 
  
 "bash" 
 , 
  
 "args" 
 : 
  
 [ 
  
 "-c" 
 , 
  
 "curl -X POST -H \"Authorization:Bearer $$GH_TOKEN\" -H 'Accept:application/vnd.github.v3+json' https://api.github.com/repos/ GITHUB_USERNAME 
/ REPO_NAME 
-d '{\"head\":\" HEAD_BRANCH 
\",\"base\":\" BASE_BRANCH 
\", \"title\":\" NEW_PR 
\"}' 
 ], 
 " 
 secre 
 t 
 E 
 n 
 v 
 ": ['GH_TOKEN'] 
 } 
 ], 
 " 
 availableSecre 
 ts 
 ": { 
 " 
 secre 
 t 
 Ma 
 na 
 ger 
 ": [ 
 { 
 " 
 versio 
 n 
 Name 
 ": " 
 projec 
 ts 
 / PROJECT_ID 
/secre 
 ts 
 / GH_TOKEN_SECRET_NAME 
/versio 
 ns 
 /la 
 test 
 ", 
 " 
 e 
 n 
 v 
 ": " 
 GH_TOKEN" 
 } 
 ] 
 } 
 } 
 

Replace the placeholder values in the above commands with the following:

• PROJECT_ID : The ID of the Google Cloud project where you've stored your secrets.
• GITHUB_USERNAME : The GitHub username of the repository owner.
• REPO_NAME : The name of the GitHub repository.
• HEAD_BRANCH : The name of the branch where the changes are implemented. For cross-repository pull requests in the same network, namespace head with a user like this: username:branch .
• BASE_BRANCH : The name of the branch you want the changes pulled into. This should be an existing branch on the current repository. You cannot submit a pull request to one repository that requests a merge to a base of another repository.
• GH_TOKEN_SECRET_NAME : The secret name corresponding to your GitHub token.
• NEW_PR : The new pull request you want to create.

Configuring builds to access non-UTF-8 secrets from Secret Manager

1. In your build config file, add a build step to access the secret version in Secret Manager and store it in a file. The following build step accesses secret-name and stores it in a file named decrypted-data.txt :

   YAML

     steps 
    : 
    - 
     
    name 
    : 
     
    gcr.io/cloud-builders/gcloud 
     
    entrypoint 
    : 
     
    'bash' 
     
    args 
    : 
     
    [ 
     
    '-c' 
    , 
     
    "gcloud 
     
    secrets 
     
    versions 
     
    access 
     
    latest 
     
    --secret= secret-name 
    
     
    --format='get(payload.data)' 
     
    | 
     
    tr 
     
    '_-' 
     
    '/+' 
     
    | 
     
    base64 
     
    -d 
    > 
     decrypted-data.txt 
    
   "  
    ] 
    
   

   JSON

     { 
     
    "steps" 
    : 
     
    [ 
     
    { 
     
    "name" 
    : 
     
    "gcr.io/cloud-builders/gcloud" 
    , 
     
    "entrypoint" 
    : 
     
    "bash" 
    , 
     
    "args" 
    : 
     
    [ 
     
    "-c" 
    , 
     
    "gcloud secrets versions access latest --secret= secret-name 
   --format='get(payload.data)' | tr '_-' '/+' | base64 -d > decrypted-data.txt 
   " 
     
    ] 
     
    } 
     
    ] 
    } 
    
   

2. Use the file with the decrypted data in a build step. The following code snippet uses decrypted-data.txt to login to a private Docker registry:

   YAML

     steps 
    : 
    - 
     
    name 
    : 
     
    gcr.io/cloud-builders/gcloud 
     
    entrypoint 
    : 
     
    'bash' 
     
    args 
    : 
     
    [ 
     
    '-c' 
    , 
     
    "gcloud 
     
    secrets 
     
    versions 
     
    access 
     
    latest 
     
    --secret= secret-name 
    
     
    --format='get(payload.data)' 
     
    | 
     
    tr 
     
    '_-' 
     
    '/+' 
     
    | 
     
    base64 
     
    -d 
    > 
     decrypted-data.txt 
    
   "  
    ] 
    - 
     
    name 
    : 
     
    gcr.io/cloud-builders/docker 
     
    entrypoint 
    : 
     
    'bash' 
     
    args 
    : 
     
    [ 
     
    '-c' 
    , 
     
    'docker 
     
    login 
     
    --username= my-user 
    
     
    --password-stdin 
    < 
     decrypted-data.txt 
    
   ' ] 
    
   

   JSON

     { 
     
    "steps" 
    : 
     
    [ 
     
    { 
     
    "name" 
    : 
     
    "gcr.io/cloud-builders/gcloud" 
    , 
     
    "entrypoint" 
    : 
     
    "bash" 
    , 
     
    "args" 
    : 
     
    [ 
     
    "-c" 
    , 
     
    "gcloud secrets versions access latest --secret= secret-name 
   --format='get(payload.data)' | tr '_-' '/+' | base64 -d > password.txt" 
     
    ] 
     
    }, 
     
    { 
     
    "name" 
    : 
     
    "gcr.io/cloud-builders/docker" 
    , 
     
    "entrypoint" 
    : 
     
    "bash" 
    , 
     
    "args" 
    : 
     
    [ 
     
    "-c" 
    , 
     
    "docker login --username=my-user --password-stdin < decrypted-data.txt 
   " 
     
    ] 
     
    } 
     
    ] 
    } 
    
   

3. Use the build config file to start a build using the command line or to automate builds using triggers .

What's next

• Learn how to use encrypted credentials in builds .
• Learn how to access private GitHub repositories .