Use custom organization policies

This page shows you how to use Organization Policy Service custom constraints to restrict specific operations on the following Google Cloud resources:

• bigquerydatatransfer.googleapis.com/TransferConfig

To learn more about Organization Policy, see Custom organization policies .

About organization policies and constraints

The Google Cloud Organization Policy Service gives you centralized, programmatic control over your organization's resources. As the organization policy administrator , you can define an organization policy, which is a set of restrictions called constraints that apply to Google Cloud resources and descendants of those resources in the Google Cloud resource hierarchy . You can enforce organization policies at the organization, folder, or project level.

Organization Policy provides built-in managed constraints for various Google Cloud services. However, if you want more granular, customizable control over the specific fields that are restricted in your organization policies, you can also create custom constraints and use those custom constraints in an organization policy.

Policy inheritance

By default, organization policies are inherited by the descendants of the resources on which you enforce the policy. For example, if you enforce a policy on a folder, Google Cloud enforces the policy on all projects in the folder. To learn more about this behavior and how to change it, refer to Hierarchy evaluation rules .

Benefits

You can use a custom organization policy to allow or deny specific operations on BigQuery Data Transfer Service transfer configurations in order to meet your organization's compliance and security requirements. If a request to create or update a transfer configuration fails to satisfy custom constraints set by your organization policy, then the request fails, and an error is returned to the caller.

Limitations

• To specify a data source in your custom constraint, use the resource.dataSourceId field along with the value of your data source. For a list of supported values for resource.dataSourceId , call the dataSources.list method .

Before you begin

Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Go to project selector

Verify that billing is enabled for your Google Cloud project .

Install the Google Cloud CLI.

If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity .

To initialize the gcloud CLI, run the following command:

gcloud  
init

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Go to project selector

Verify that billing is enabled for your Google Cloud project .

Install the Google Cloud CLI.

If you're using an external identity provider (IdP), you must first sign in to the gcloud CLI with your federated identity .

To initialize the gcloud CLI, run the following command:

gcloud  
init

1. Ensure that you know your organization ID .

Required roles

To get the permissions that you need to manage organization policies, ask your administrator to grant you the following IAM roles:

• Organization Policy Administrator ( roles/orgpolicy.policyAdmin ) on the organization resource
• Create or update a BigQuery Data Transfer Service transfer configuration: BigQuery Admin ( roles/bigquery.admin ) on the project resource

For more information about granting roles, see Manage access to projects, folders, and organizations .

These predefined roles contain the permissions required to manage organization policies. To see the exact permissions that are required, expand the Required permissionssection:

You might also be able to get these permissions with custom roles or other predefined roles .

Create a custom constraint

A custom constraint is defined in a YAML file by the resources, methods, conditions, and actions that are supported by the service on which you are enforcing the organization policy. Conditions for your custom constraints are defined using Common Expression Language (CEL) . For more information about how to build conditions in custom constraints using CEL, see the CEL section of Creating and managing custom constraints .

To create a custom constraint, create a YAML file using the following format:

  name 
 : 
  
 organizations/ ORGANIZATION_ID 
/customConstraints/ CONSTRAINT_NAME 
 
 resourceTypes 
 : 
 - 
  
  RESOURCE_NAME 
 
 methodTypes 
 : 
 - 
  
 CREATE 
 - 
  
 UPDATE 
 condition 
 : 
  
 " CONDITION 
" 
 actionType 
 : 
  
  ACTION 
 
 displayName 
 : 
  
  DISPLAY_NAME 
 
 description 
 : 
  
  DESCRIPTION 
 
 

Replace the following:

• ORGANIZATION_ID : your organization ID, such as 123456789 .

• CONSTRAINT_NAME : the name you want for your new custom constraint. A custom constraint must start with custom. , and can only include uppercase letters, lowercase letters, or numbers. For example, custom.dtsEnableEmailNotification . The maximum length of this field is 70 characters.

• RESOURCE_NAME : the fully qualified name of the Google Cloud resource containing the object and field you want to restrict. For example, bigquerydatatransfer.googleapis.com/TransferConfig .

• CONDITION : a CEL condition that is written against a representation of a supported service resource. This field has a maximum length of 1000 characters. See Supported resources for more information about the resources available to write conditions against. For example, resource.emailPreferences.enableFailureEmail == true .

• ACTION : the action to take if the condition is met. Possible values are ALLOW and DENY .

• DISPLAY_NAME : a human-friendly name for the constraint. This field has a maximum length of 200 characters.

• DESCRIPTION : a human-friendly description of the constraint to display as an error message when the policy is violated. This field has a maximum length of 2000 characters.

For more information about how to create a custom constraint, see Defining custom constraints .

Set up a custom constraint

gcloud  
org-policies  
set-custom-constraint  
 CONSTRAINT_PATH 

gcloud  
org-policies  
list-custom-constraints  
--organization = 
 ORGANIZATION_ID 

Enforce a custom organization policy

  
 name 
 : 
  
 projects/ PROJECT_ID 
/policies/ CONSTRAINT_NAME 
 
  
 spec 
 : 
  
 rules 
 : 
  
 - 
  
 enforce 
 : 
  
 true 
  

  
gcloud  
org-policies  
set-policy  
 POLICY_PATH 
  

Test the custom organization policy

The following example creates a custom constraint and policy that require that email notifications are enabled for all new transfer configurations in a specific project.

Before you begin, you should know the following:

• Your organization ID
• A project ID

Create the constraint

1. Save the following file as constraint-dts-enable-email.yaml :

     name 
    : 
     
    organizations/ ORGANIZATION_ID 
   /customConstraints/custom.dtsEnableEmailNotification 
    resourceTypes 
    : 
    - 
     
    bigquerydatatransfer.googleapis.com/TransferConfig 
    methodTypes 
    : 
    - 
     
    CREATE 
    condition 
    : 
     
    resource.emailPreferences.enableFailureEmail == true 
    actionType 
    : 
     
    ALLOW 
    displayName 
    : 
     
    The BigQuery Data Transfer Service always enables email notications 
    description 
    : 
     
    The BigQuery Data Transfer Service always enables email notications on data transfer failures. 
    
   

   Replace ORGANIZATION_ID with your organization ID .

   This defines a constraint that checks whether email notifications are enabled for a new transfer configuration. If they are not enabled, the operation is denied.

2. Apply the constraint:

    gcloud  
   org-policies  
   set-custom-constraint  
   ~/constraint-dts-enable-email.yaml 
   

3. Verify that the constraint exists:

    gcloud  
   org-policies  
   list-custom-constraints  
   --organization = 
    ORGANIZATION_ID 
    
   

   The output is similar to the following:

    CUSTOM_CONSTRAINT                  ACTION_TYPE  METHOD_TYPES   RESOURCE_TYPES                                      DISPLAY_NAME
   custom.dtsEnableEmailNotification  ALLOW        CREATE         bigquerydatatransfer.googleapis.com/TransferConfig  The BigQuery Data Transfer Service always enables email notications
   ... 
   

Create the policy

1. Save the following file as policy-dts-enable-email.yaml :

     name 
    : 
     
    projects/ PROJECT_ID 
   /policies/custom.dtsEnableEmailNotification 
    spec 
    : 
     
    rules 
    : 
     
    - 
     
    enforce 
    : 
     
    true 
    
   

   Replace PROJECT_ID with your project ID.

2. Apply the policy:

    gcloud  
   org-policies  
   set-policy  
   ~/policy-dts-enable-email.yaml 
   

3. Verify that the policy exists:

    gcloud  
   org-policies  
   list  
   --project = 
    PROJECT_ID 
    
   

   The output is similar to the following:

    CONSTRAINT                         LIST_POLICY  BOOLEAN_POLICY        ETAG
   custom.dtsEnableEmailNotification  -            SET                   CPyxlbgGENDL3tEC- 
   

After you apply the policy, wait for about two minutes for Google Cloud to start enforcing the policy.

Test the policy

In the project, try to create a BigQuery Data Transfer Service transfer configuration with the email notification turned off.

The output is the following:

 Operation denied by custom org policy: ["customConstraints/custom.dtsEnableEmailNotification": "The BigQuery Data Transfer Service always enables email notications on data transfer failures."]. 

Example custom organization policies for common use cases

This table provides syntax examples for some common custom constraints.

  
 name 
 : 
  
 organizations/ ORGANIZATION_ID 
/customConstraints/custom.denyDtsAzureTransfers 
  
 resourceTypes 
 : 
  
 - 
  
 bigquerydatatransfer.googleapis.com/TransferConfig 
  
 methodTypes 
 : 
  
 - 
  
 CREATE 
  
 condition 
 : 
  
 resource.dataSourceId == "azure_blob_storage" 
  
 actionType 
 : 
  
 DENY 
  
 displayName 
 : 
  
 Deny data transfers from Azure Blob Storage to BigQuery 
  
 description 
 : 
  
 Disallow creating data transfer configurations from Azure Blob Storage. 
  

  
 name 
 : 
  
 organizations/ ORGANIZATION_ID 
/customConstraints/custom.dtsNoManualSchedule 
  
 resourceTypes 
 : 
  
 - 
  
 bigquerydatatransfer.googleapis.com/TransferConfig 
  
 methodTypes 
 : 
  
 - 
  
 CREATE 
  
 - 
  
 UPDATE 
  
 condition 
 : 
  
 resource.scheduleOptions.disableAutoScheduling == false 
  
 actionType 
 : 
  
 ALLOW 
  
 displayName 
 : 
  
 Transfer configurations always enable auto-scheduling 
  
 description 
 : 
  
 Always enable auto-scheduling for BigQuery Data Transfer Service transfer configurations. 
  

  
 name 
 : 
  
 organizations/ ORGANIZATION_ID 
/customConstraints/custom.dtsGoogleAdsConstraint 
  
 resourceTypes 
 : 
  
 - 
  
 bigquerydatatransfer.googleapis.com/TransferConfig 
  
 methodTypes 
 : 
  
 - 
  
 CREATE 
  
 - 
  
 UPDATE 
  
 condition 
 : 
  
 resource.dataSourceId == "google_ads" && resource.dataRefreshWindowDays < 3 
  
 actionType 
 : 
  
 DENY 
  
 displayName 
 : 
  
 Google Ads transfers data refresh window must be greater than three days 
  
 description 
 : 
  
 Disallow creating Google Ads data transfer configurations whose data refresh window is less than three days. 
  

BigQuery Data Transfer Service supported resources

What's next

• Learn more about Organization Policy Service .
• Learn more about how to create and manage organization policies .
• See the full list of managed organization policy constraints .