Work with slot reservations
The BigQuery Reservation API lets you purchase dedicated slots (called commitments ), create pools of slots (called reservations ), and assign projects, folders, and organizations to those reservations.
Reservations allow you to assign a dedicated number of slots
to a workload. For example, you might not want
a production workload to compete with test workloads for slots. You could
create a reservation named prod
and assign your production workloads to this
reservation. For more information, see Reservations
.
Create reservations
Required permissions
To create a reservation, you need the following Identity and Access Management (IAM) permission:
-
bigquery.reservations.create
on the administration project that maintains ownership of the commitments.
Each of the following predefined IAM roles includes this permission:
-
BigQuery Resource Editor
-
BigQuery Resource Admin
For more information about IAM roles in BigQuery, see Predefined roles and permissions .
Create a reservation with dedicated slots
Select one of the following options:
Console
-
In the Google Cloud console, go to the BigQuery page.
-
In the navigation panel, go to the Capacity managementsection, and then click Create reservation.
-
In the Reservation namefield, enter a name for the reservation.
-
In the Locationdrop-down list, select the location. If you select a BigQuery Omni location , your edition option is limited to the Enterprise edition.
-
In the Editionlist, select the edition. Autoscaling is only available within an edition. For more information, see Introduction to BigQuery editions .
-
In the Max reservation size selectorlist, select the maximum reservation size.
-
Optional: In the Baseline slotsfield, enter the number of baseline slots for the reservation.
The number of available autoscaling slots is determined by subtracting the Baseline slotsvalue from the Max reservation size. For example, if you create a reservation with 100 baseline slots and a max reservation size of 400, your reservation has 300 autoscaling slots. For more information about baseline slots, see Using reservations with baseline and autoscaling slots .
-
To disable idle slot sharing and use only the specified slot capacity, click the Ignore idle slotstoggle.
-
To expand the Advanced settingssection, click the expander arrow.
-
Optional: To set the target job concurrency, click the Override automatic target job concurrencytoggle to on and enter the Target Job Concurrency.
-
The breakdown of slots is displayed in the Cost estimatetable. A summary of the reservation is displayed in the Capacity summarytable.
-
Click Save.
The new reservation is visible in the Slot reservationstab.
SQL
To create a reservation, use the CREATE RESERVATION
DDL statement
.
-
In the Google Cloud console, go to the BigQuerypage.
-
In the query editor, enter the following statement:
CREATE RESERVATION ` ADMIN_PROJECT_ID . region - LOCATION . RESERVATION_NAME ` OPTIONS ( slot_capacity = NUMBER_OF_BASELINE_SLOTS , edition = EDITION , autoscale_max_slots = NUMBER_OF_AUTOSCALING_SLOTS );
Replace the following:
-
ADMIN_PROJECT_ID
: the project ID of the administration project that owns the reservation resource -
LOCATION
: the location of the reservation. If you select a BigQuery Omni location , your edition option is limited to the Enterprise edition. -
RESERVATION_NAME
: the name of the reservationIt must start and end with a lowercase letter or a number and contain only lowercase letters, numbers, and dashes.
-
NUMBER_OF_BASELINE_SLOTS
: the number baseline of slots to allocate to the reservation. You cannot set theslot_capacity
option and theedition
option in the same reservation. -
EDITION
: the edition of the reservation. Assigning a reservation to an edition comes with feature and pricing changes. For more information, see Introduction to BigQuery editions . -
NUMBER_OF_AUTOSCALING_SLOTS
: the number of autoscaling slots assigned to the reservation. This is equal to the value of the max reservation size minus the number of baseline slots.
-
-
Click Run.
For more information about how to run queries, see Run an interactive query .
bq
To create a reservation, use the bq mk
command with the --reservation
flag:
bq mk \ --project_id= ADMIN_PROJECT_ID \ --location= LOCATION \ --reservation \ --slots= NUMBER_OF_BASELINE_SLOTS \ --ignore_idle_slots=false \ --edition= EDITION \ --autoscale_max_slots= NUMBER_OF_AUTOSCALING_SLOTS \ RESERVATION_NAME
Replace the following:
-
ADMIN_PROJECT_ID
: the project ID -
LOCATION
: the location of the reservation. If you select a BigQuery Omni location , your edition option is limited to the Enterprise edition. -
NUMBER_OF_BASELINE_SLOTS
: the number of baseline slots to allocate to the reservation -
RESERVATION_NAME
: the name of the reservation -
EDITION
: the edition of the reservation. Assigning a reservation to an edition comes with feature and pricing changes. For more information, see Introduction to BigQuery editions . -
NUMBER_OF_AUTOSCALING_SLOTS
: the number of autoscaling slots assigned to the reservation. This is equal to the value of the max reservation size minus the number of baseline slots.
For information about the --ignore_idle_slots
flag, see Idle slots
. The default
value is false
.
Terraform
Use the google_bigquery_reservation
resource.
To authenticate to BigQuery, set up Application Default Credentials. For more information, see Set up authentication for client libraries .
The following example creates a reservation named my-reservation
:
To apply your Terraform configuration in a Google Cloud project, complete the steps in the following sections.
Prepare Cloud Shell
- Launch Cloud Shell .
-
Set the default Google Cloud project where you want to apply your Terraform configurations.
You only need to run this command once per project, and you can run it in any directory.
export GOOGLE_CLOUD_PROJECT= PROJECT_ID
Environment variables are overridden if you set explicit values in the Terraform configuration file.
Prepare the directory
Each Terraform configuration file must have its own directory (also called a root module ).
- In Cloud Shell
, create a directory and a new
file within that directory. The filename must have the
.tf
extension—for examplemain.tf
. In this tutorial, the file is referred to asmain.tf
.mkdir DIRECTORY && cd DIRECTORY && touch main.tf
-
If you are following a tutorial, you can copy the sample code in each section or step.
Copy the sample code into the newly created
main.tf
.Optionally, copy the code from GitHub. This is recommended when the Terraform snippet is part of an end-to-end solution.
- Review and modify the sample parameters to apply to your environment.
- Save your changes.
- Initialize Terraform. You only need to do this once per directory.
terraform init
Optionally, to use the latest Google provider version, include the
-upgrade
option:terraform init -upgrade
Apply the changes
- Review the configuration and verify that the resources that Terraform is going to create or
update match your expectations:
terraform plan
Make corrections to the configuration as necessary.
- Apply the Terraform configuration by running the following command and entering
yes
at the prompt:terraform apply
Wait until Terraform displays the "Apply complete!" message.
- Open your Google Cloud project to view the results. In the Google Cloud console, navigate to your resources in the UI to make sure that Terraform has created or updated them.
Python