Stay organized with collectionsSave and categorize content based on your preferences.
Create and use tables
This document describes how to create and usestandard (built-in) tables in
BigQuery. For
information about creating other table types, see the following:
These predefined roles contain
the permissions required to create a table. To see the exact permissions that are
required, expand theRequired permissionssection:
Required permissions
The following permissions are required to create a table:
bigquery.tables.createon the dataset where you're creating the table.
bigquery.tables.getDataon all tables and views that your query references if you're saving query results as a table.
bigquery.jobs.createon the project if you're creating the table by loading data or by saving query results to a table.
bigquery.tables.updateDataon the table if you're appending to or overwriting a table with query results.
When you create a table in BigQuery, the table name must
be unique per dataset. The table name can:
Contain characters with a total of up to 1,024 UTF-8 bytes.
Contain Unicode characters in category L (letter), M (mark), N (number),
Pc (connector, including underscore), Pd (dash), Zs (space). For more
information, seeGeneral Category.
The following are all examples of valid table names:table 01,ग्राहक,00_お客様,étudiant-01.
Some table names and table name prefixes are reserved. If
you receive an error saying that your table name or prefix is
reserved, then select a different name and try again.
If you include multiple dot operators (.) in a sequence, the duplicate
operators are implicitly stripped.
For example, this:project_name....dataset_name..table_name
Becomes this:project_name.dataset_name.table_name
Create tables
You can create a table in BigQuery in the following ways:
Manually by using the Google Cloud console or the bq command-line toolbq mkcommand.
Programmatically by calling thetables.insertAPI method.
By using the client libraries.
From query results.
By defining a table that references an external data source.
When you load data.
By using aCREATE TABLEdata definition language (DDL) statement.
Create an empty table with a schema definition
You can create an empty table with a schema definition in the following ways:
Enter the schema using the Google Cloud console.
Provide the schema inline using the bq command-line tool.
Submit a JSON schema file using the bq command-line tool.
importcom.google.cloud.bigquery.BigQuery;importcom.google.cloud.bigquery.BigQueryException;importcom.google.cloud.bigquery.BigQueryOptions;importcom.google.cloud.bigquery.Schema;importcom.google.cloud.bigquery.StandardTableDefinition;importcom.google.cloud.bigquery.TableDefinition;importcom.google.cloud.bigquery.TableId;importcom.google.cloud.bigquery.TableInfo;// Sample to create a table without schemapublicclassCreateTableWithoutSchema{publicstaticvoidmain(String[]args){// TODO(developer): Replace these variables before running the sample.StringdatasetName="MY_DATASET_NAME";StringtableName="MY_TABLE_NAME";createTableWithoutSchema(datasetName,tableName);}publicstaticvoidcreateTableWithoutSchema(StringdatasetName,StringtableName){try{// Initialize client that will be used to send requests. This client only needs to be created// once, and can be reused for multiple requests.BigQuerybigquery=BigQueryOptions.getDefaultInstance().getService();TableIdtableId=TableId.of(datasetName,tableName);TableDefinitiontableDefinition=StandardTableDefinition.of(Schema.of());TableInfotableInfo=TableInfo.newBuilder(tableId,tableDefinition).build();bigquery.create(tableInfo);System.out.println("Table created successfully");}catch(BigQueryExceptione){System.out.println("Table was not created. \n"+e.toString());}}}
Create a table from a query result
To create a table from a query result, write the results to a destination table.
Create a table that references an external data source
An external data source is a data source that you can query directly from
BigQuery, even though the data is not stored in
BigQuery storage. For example, you might have data in a
different Google Cloud database, in files in Cloud Storage, or in a
different cloud product altogether that you would like to analyze in
BigQuery, but that you aren't prepared to migrate.
When you load data into BigQuery, you can load data into a new
table or partition, you can append data to an existing table or partition, or
you can overwrite a table or partition. You don't need to create an empty
table before loading data into it. You can create the new table and load your
data at the same time.
When you load data into BigQuery, you can supply the table
or partition schema, or for supported data formats, you can use schemaauto-detection.
You can create a table with one or moreObjectRefcolumns in order to store metadata
about unstructured data that is related to the other structured data in the
table. For example, in a products table, you could create anObjectRefcolumn
to store product image information along with the other product data. The
unstructured data itself is stored in Cloud Storage, and is made available
in BigQuery by using anobject table.
To configure access to tables and views, you can grant an
IAM role to an entity at the following levels, listed in
order of range of resources allowed (largest to smallest):
Access with any resource protected by IAM is additive. For
example, if an entity does not have access at the high level such as a project,
you could grant the entity access at the dataset level, and then the entity will
have access to the tables and views in the dataset. Similarly, if the entity
does not have access at the high level or the dataset level, you could grant the
entity access at the table or view level.
Granting IAM roles at a higher level in theGoogle Cloud
resource hierarchysuch as the project, folder, or organization level gives the entity access to a
broad set of resources. For example, granting a role to an entity at the project
level gives that entity permissions that apply to all datasets throughout the
project.
Granting a role at the dataset level specifies the operations an entity is
allowed to perform on tables and views in that specific dataset, even if the
entity does not have access at a higher level. For information on configuring
dataset-level access controls, seeControlling access to datasets.
Granting a role at the table or view level specifies the operations an entity is
allowed to perform on specific tables and views, even if the entity does not
have access at a higher level. For information on configuring table-level access
controls, seeControlling access to tables and views.
You can also createIAM custom roles.
If you create a custom role, the permissions you grant depend on the specific
operations you want the entity to be able to perform.
You can't set a "deny" permission on any resource protected by
IAM.
At a minimum, to get information about tables, you must be grantedbigquery.tables.getpermissions. The following predefined IAM
roles includebigquery.tables.getpermissions:
bigquery.metadataViewer
bigquery.dataViewer
bigquery.dataOwner
bigquery.dataEditor
bigquery.admin
In addition, if a user hasbigquery.datasets.createpermissions, when that
user creates a dataset, they are grantedbigquery.dataOwneraccess to it.bigquery.dataOwneraccess gives the user the ability to retrieve table
metadata.
For more information on IAM roles and permissions in
BigQuery, seeAccess control.
INFORMATION_SCHEMAis a series of views that provide access to metadata
about datasets, routines, tables, views, jobs, reservations, and streaming data.
You can query the following views to get table information:
Use theINFORMATION_SCHEMA.TABLESandINFORMATION_SCHEMA.TABLE_OPTIONSviews to retrieve metadata about tables and views in a project.
Use theINFORMATION_SCHEMA.COLUMNSandINFORMATION_SCHEMA.COLUMN_FIELD_PATHSviews to retrieve metadata about the
columns (fields) in a table.
Use theINFORMATION_SCHEMA.TABLE_STORAGEviews to retrieve metadata
about current and historical storage usage by a table.
TheTABLESandTABLE_OPTIONSviews also contain high-level
information about views. For detailed information, query theINFORMATION_SCHEMA.VIEWSview
instead.
TABLESview
When you query theINFORMATION_SCHEMA.TABLESview, the query results contain
one row for each table or view in a dataset. For detailed information about
views, query theINFORMATION_SCHEMA.VIEWSviewinstead.
TheINFORMATION_SCHEMA.TABLESview has the following schema:
Column name
Data type
Value
table_catalog
STRING
The project ID of the project that contains the dataset.
table_schema
STRING
The name of the dataset that contains the table or view. Also referred
to as thedatasetId.
table_name
STRING
The name of the table or view. Also referred to as thetableId.
Fortable clonesandtable snapshots,
the base table's project. Applicable only to
tables withtable_typeset toCLONEorSNAPSHOT.
base_table_schema
STRING
Fortable clonesandtable snapshots,
the base table's dataset. Applicable only to tables withtable_typeset toCLONEorSNAPSHOT.
base_table_name
STRING
Fortable clonesandtable snapshots,
the base table's name. Applicable only to tables withtable_typeset toCLONEorSNAPSHOT.
snapshot_time_ms
TIMESTAMP
Fortable clonesandtable snapshots,
the time when thecloneorsnapshotoperation was run on the base table to create this table. Iftime travelwas used, then this
field contains the time travel timestamp. Otherwise, thesnapshot_time_msfield is the same as thecreation_timefield. Applicable only to
tables withtable_typeset toCLONEorSNAPSHOT.
Formaterialized view replicas,
the status of the replication from the base materialized view to the
materialized view replica; one of the following:
REPLICATION_STATUS_UNSPECIFIED
ACTIVE: Replication is active with no errors
SOURCE_DELETED: The source materialized view has
been deleted
PERMISSION_DENIED: The source materialized view
hasn't beenauthorizedon the dataset that contains the source Amazon S3
BigLake tables used in the query that created the
materialized view.
UNSUPPORTED_CONFIGURATION: There is an issue with
the replica'sprerequisitesother than source materialized view authorization.
replication_error
STRING
Ifreplication_statusindicates a replication issue for amaterialized view replica,replication_errorprovides further details about the issue.
The status of the sync between the primary and secondary replicas forcross-region
replicationanddisaster recoverydatasets. ReturnsNULLif the replica is a primary replica or the dataset doesn't use replication.
upsert_stream_apply_watermark
TIMESTAMP
For tables that use change data capture (CDC), the time when row
modifications were last applied. For more information, seeMonitor table upsert operation progress.
Examples
Example 1:
The following example retrieves table metadata for all of the tables in the
dataset namedmydataset. The metadata that's
returned is for all types of tables inmydatasetin your default project.
mydatasetcontains the following tables:
mytable1: a standard BigQuery table
myview1: a BigQuery view
To run the query against a project other than your default project, add the
project ID to the dataset in the following format:`project_id`.dataset.INFORMATION_SCHEMA.view;
for example,`myproject`.mydataset.INFORMATION_SCHEMA.TABLES.
The following example retrieves table metadata for all tables of typeCLONEorSNAPSHOTfrom theINFORMATION_SCHEMA.TABLESview. The metadata returned
is for tables inmydatasetin your default project.
To run the query against a project other than your default project, add the
project ID to the dataset in the following format:`project_id`.dataset.INFORMATION_SCHEMA.view;
for example,`myproject`.mydataset.INFORMATION_SCHEMA.TABLES.
The following example retrievestable_nameandddlcolumns from theINFORMATION_SCHEMA.TABLESview for thepopulation_by_zip_2010table in thecensus_bureau_usadataset. This dataset is part of the BigQuerypublic dataset program.
Because the table you're querying is in another project, you add the project ID to the dataset in
the following format:`project_id`.dataset.INFORMATION_SCHEMA.view.
In this example, the value is`bigquery-public-data`.census_bureau_usa.INFORMATION_SCHEMA.TABLES.
+------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| table_name | ddl |
+------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| population_by_zip_2010 | CREATE TABLE `bigquery-public-data.census_bureau_usa.population_by_zip_2010` |
| | ( |
| | geo_id STRING OPTIONS(description="Geo code"), |
| | zipcode STRING NOT NULL OPTIONS(description="Five digit ZIP Code Tabulation Area Census Code"), |
| | population INT64 OPTIONS(description="The total count of the population for this segment."), |
| | minimum_age INT64 OPTIONS(description="The minimum age in the age range. If null, this indicates the row as a total for male, female, or overall population."), |
| | maximum_age INT64 OPTIONS(description="The maximum age in the age range. If null, this indicates the row as having no maximum (such as 85 and over) or the row is a total of the male, female, or overall population."), |
| | gender STRING OPTIONS(description="male or female. If empty, the row is a total population summary.") |
| | ) |
| | OPTIONS( |
| | labels=[("freebqcovid", "")] |
| | ); |
+------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
TABLE_OPTIONSview
When you query theINFORMATION_SCHEMA.TABLE_OPTIONSview, the query results
contain one row for each option, for each table or view in a dataset. For
detailed information about
views, query theINFORMATION_SCHEMA.VIEWSviewinstead.
TheINFORMATION_SCHEMA.TABLE_OPTIONSview has the following schema:
Column name
Data type
Value
table_catalog
STRING
The project ID of the project that contains the dataset
table_schema
STRING
The name of the dataset that contains the table or view also referred
to as thedatasetId
table_name
STRING
The name of the table or view also referred to as thetableId
The default lifetime, in days, of all partitions in a partitioned
table
refresh_interval_minutes
FLOAT64
How frequently a materialized view is refreshed
require_partition_filter
BOOL
Whether queries over the table require a partition filter
tags
ARRAY<STRUCT<STRING, STRING>>
Tags attached to a table in a namespaced <key, value> syntax.
For more information, seeTags and
conditional access.
For external tables, the following options are possible:
Options
allow_jagged_rows
BOOL
Iftrue, allow rows that are missing trailing optional
columns.
Applies to CSV data.
allow_quoted_newlines
BOOL
Iftrue, allow quoted data sections that contain newline
characters in the file.
Applies to CSV data.
bigtable_options
STRING
Only required when creating a Bigtable external
table.
Specifies the schema of the Bigtable external table
in JSON format.
For a list of Bigtable table definition options, seeBigtableOptionsin the REST API reference.
column_name_character_map
STRING
Defines the scope of supported column name characters and the
handling behavior of unsupported characters. The default setting isSTRICT, which means unsupported characters cause
BigQuery to throw errors.V1andV2replace any unsupported characters
with underscores.
Supported values include:
STRICT. Enablesflexible column names.
This is the default value. Load jobs with unsupported characters in column
names fail with an error message. To configure the replacement of
unsupported characters with underscores so that the load job succeeds,
specify thedefault_column_name_character_mapconfiguration setting.
Iftrue, use schema inference specifically for
Parquet LIST logical type.
Applies to Parquet data.
enable_logical_types
BOOL
Iftrue, convert Avro logical types into their
corresponding SQL types. For more information, seeLogical types.
Applies to Avro data.
encoding
STRING
The character encoding of the data. Supported values include:UTF8(orUTF-8),ISO_8859_1(orISO-8859-1),UTF-16BE,UTF-16LE,UTF-32BE, orUTF-32LE.
The default value isUTF-8.
Applies to CSV data.
enum_as_string
BOOL
Iftrue, infer Parquet ENUM logical type as STRING
instead of BYTES by default.
Applies to Parquet data.
expiration_timestamp
TIMESTAMP
The time when this table expires. If not specified, the table does
not expire.
Example:"2025-01-01 00:00:00 UTC".
field_delimiter
STRING
The separator for fields in a CSV file.
Applies to CSV data.
format
STRING
The format of the external data.
Supported values forCREATE EXTERNAL TABLEinclude:AVRO,CLOUD_BIGTABLE,CSV,DATASTORE_BACKUP,DELTA_LAKE(preview),GOOGLE_SHEETS,NEWLINE_DELIMITED_JSON(orJSON),ORC,PARQUET.
The valueJSONis equivalent toNEWLINE_DELIMITED_JSON.
hive_partition_uri_prefix
STRING
A common prefix for all source URIs before the partition key encoding
begins. Applies only to hive-partitioned external tables.
Applies to Avro, CSV, JSON, Parquet, and ORC data.
Example:"gs://bucket/path".
file_set_spec_type
STRING
Specifies how to interpret source URIs for load jobs and external tables.
Supported values include:
FILE_SYSTEM_MATCH. Expands source URIs by listing files from the object store. This is the default behavior if FileSetSpecType is not set.
NEW_LINE_DELIMITED_MANIFEST. Indicates that the provided URIs are newline-delimited manifest files, with one URI per line. Wildcard URIs are not supported in the manifest files, and all referenced data files must be in the same bucket as the manifest file.
For example, if you have a source URI of"gs://bucket/path/file"and thefile_set_spec_typeisFILE_SYSTEM_MATCH, then the file is used directly as a data file. If thefile_set_spec_typeisNEW_LINE_DELIMITED_MANIFEST, then each line in the file is interpreted as a URI that points to a data file.
ignore_unknown_values
BOOL
Iftrue, ignore extra values that are not represented
in the table schema, without returning an error.
Applies to CSV and JSON data.
json_extension
STRING
For JSON data, indicates a particular JSON interchange format. If
not specified, BigQuery reads the data as generic JSON
records.
Specifies whether cached metadata is used by operations against the
table, and how fresh the cached metadata must be in order for
the operation to use it.
To disable metadata caching, specify 0. This is the default.
To enable metadata caching, specify aninterval literalvalue between 30 minutes and 7 days. For example, specifyINTERVAL 4 HOURfor a 4 hour staleness interval.
With this value, operations against the table use cached metadata if
it has been refreshed within the past 4 hours. If the cached metadata
is older than that, the operation falls back to retrieving metadata from
Cloud Storage instead.
null_marker
STRING
The string that representsNULLvalues in a CSV file.
Applies to CSV data.
null_markers
ARRAY<STRING>
The list of strings that representNULLvalues in a CSV
file.
Set the value of this option toSIMPLEwhen
creating an object table.
preserve_ascii_control_characters
BOOL
Iftrue, then the embedded ASCII control characters
which are the first 32 characters in the ASCII table, ranging from
'\x00' to '\x1F', are preserved.
Applies to CSV data.
projection_fields
STRING
A list of entity properties to load.
Applies to Datastore data.
quote
STRING
The string used to quote data sections in a CSV file. If your data
contains quoted newline characters, also set theallow_quoted_newlinesproperty totrue.
Applies to CSV data.
reference_file_schema_uri
STRING
User provided reference file with the table schema.
Iftrue, all queries over this table require a partition
filter that can be used to eliminate partitions when reading data.
Applies only to hive-partitioned external tables.
Applies to Avro, CSV, JSON, Parquet, and ORC data.
sheet_range
STRING
Range of a Google Sheets spreadsheet to query from.
Applies to Google Sheets data.
Example:"sheet1!A1:B20",
skip_leading_rows
INT64
The number of rows at the top of a file to skip when reading the
data.
Applies to CSV and Google Sheets data.
source_column_match
STRING
This controls the strategy used to match loaded columns to the
schema.
If this value is unspecified, then the default is based on how the
schema is provided. If autodetect is enabled, then the default behavior
is to match columns by name. Otherwise, the default is to match columns
by position. This is done to keep the behavior backward-compatible.
Supported values include:
POSITION: matches by position. This option assumes
that the columns are ordered the same way as the schema.
NAME: matches by name. This option reads the header
row as column names and reorders columns to match the field names in
the schema. Column names are read from the last skipped row based on
theskip_leading_rowsproperty.
tags
<ARRAY<STRUCT<STRING, STRING>>>
An array of IAM tags for the table, expressed as
key-value pairs. The key should be thenamespaced key name,
and the value should be theshort name.
time_zone
STRING
Default time zone that will apply when parsing timestamp values that
have no specific time zone.
If this value is not present, the timestamp values without specific
time zone is parsed using default time zone UTC.
Applies to CSV and JSON data.
date_format
STRING
Format elementsthat define how the DATE values are formatted in the input files (for
example,MM/DD/YYYY).
If this value is present, this format is the only compatible DATE
format.Schema autodetectionwill also decide DATE column type based on this format instead of the
existing format.
If this value is not present, the DATE field is parsed with thedefault formats.
Applies to CSV and JSON data.
datetime_format
STRING
Format elementsthat define how the DATETIME values are formatted in the input files
(for example,MM/DD/YYYY HH24:MI:SS.FF3).
If this value is present, this format is the only compatible DATETIME
format.Schema autodetectionwill also decide DATETIME column type based on this format instead of
the existing format.
If this value is not present, the DATETIME field is parsed with thedefault formats.
Applies to CSV and JSON data.
time_format
STRING
Format elementsthat define how the TIME values are formatted in the input files (for
example,HH24:MI:SS.FF3).
If this value is present, this format is the only compatible TIME
format.Schema autodetectionwill also decide TIME column type based on this format instead of the
existing format.
If this value is not present, the TIME field is parsed with thedefault formats.
Applies to CSV and JSON data.
timestamp_format
STRING
Format elementsthat define how the TIMESTAMP values are formatted in the input files
(for example,MM/DD/YYYY HH24:MI:SS.FF3).
If this value is present, this format is the only compatible
TIMESTAMP format.Schema autodetectionwill also decide TIMESTAMP column type based on this format instead of
the existing format.
If this value is not present, the TIMESTAMP field is parsed with thedefault formats.
Applies to CSV and JSON data.
uris
For external tables, including object tables, that aren't
Bigtable tables:
ARRAY<STRING>
An array of fully qualified URIs for the external data locations.
Each URI can contain one
asterisk (*)wildcard character,
which must come after the bucket name. When you specifyurisvalues that target multiple files, all of those
files must share a compatible schema.
The following example retrieves the default table expiration times for all
tables inmydatasetin your default project (myproject) by querying theINFORMATION_SCHEMA.TABLE_OPTIONSview.
To run the query against a project other than your default project, add the
project ID to the dataset in the following format:`project_id`.dataset.INFORMATION_SCHEMA.view;
for example,`myproject`.mydataset.INFORMATION_SCHEMA.TABLE_OPTIONS.
The following example retrieves metadata about all tables inmydatasetthat
contain test data. The query uses the values in thedescriptionoption to find
tables that contain "test" anywhere in the description.mydatasetis in your
default project —myproject.
To run the query against a project other than your default project, add the
project ID to the dataset in the following format:`project_id`.dataset.INFORMATION_SCHEMA.view;
for example,`myproject`.mydataset.INFORMATION_SCHEMA.TABLE_OPTIONS.
The value is the generation expression used to define the column
if the column is an automatically
generated embedding column; otherwise the value isNULL.
is_stored
STRING
The value isYESif the column is an automatically
generated embedding column; otherwise, the value isNULL.
is_hidden
STRING
YESorNOdepending on whether the column is
a pseudo column such as _PARTITIONTIME or _PARTITIONDATE.
is_updatable
STRING
The value is alwaysNULL.
is_system_defined
STRING
YESorNOdepending on whether the column is
a pseudo column such as _PARTITIONTIME or _PARTITIONDATE.
If aSTRINGorARRAY<STRING>is passed
in, the collation specification is returned if it exists; otherwiseNULLis returned.
column_default
STRING
Thedefault valueof the
column if it exists; otherwise, the value isNULL.
rounding_mode
STRING
The mode of rounding that's used for values written to the field if its
type is a parameterizedNUMERICorBIGNUMERIC;
otherwise, the value isNULL.
data_policies.name
STRING
The list of data policies that are attached to the column to control access and masking. This field is in (Preview).
policy_tags
ARRAY<STRING>
The list of policy tags that are attached to the column.
Examples
The following example retrieves metadata from theINFORMATION_SCHEMA.COLUMNSview for thepopulation_by_zip_2010table in thecensus_bureau_usadataset. This dataset is part of the BigQuerypublic dataset program.
Because the table you're querying is in another project, thebigquery-public-dataproject, you add the project ID to the dataset in the
following format:`project_id`.dataset.INFORMATION_SCHEMA.view;
for example,`bigquery-public-data`.census_bureau_usa.INFORMATION_SCHEMA.TABLES.
The following column is excluded from the query results:
If aSTRING,ARRAY<STRING>, orSTRINGfield in aSTRUCTis passed in, the
collation specification is returned if it exists; otherwise,NULLis returned.
rounding_mode
STRING
The mode of rounding that's used when applying precision and scale to+
parameterizedNUMERICorBIGNUMERICvalues;
otherwise, the value isNULL.
data_policies.name
STRING
The list of data policies that are attached to the column to control access and masking. This field is in (Preview).
policy_tags
ARRAY<STRING>
The list of policy tags that are attached to the column.
Examples
The following example retrieves metadata from theINFORMATION_SCHEMA.COLUMN_FIELD_PATHSview for thecommitstable in thegithub_reposdataset.
This dataset is part of the BigQuerypublic dataset program.
Because the table you're querying is in another project, thebigquery-public-dataproject, you add the project ID to the dataset in the
following format:`project_id`.dataset.INFORMATION_SCHEMA.view;
for example,`bigquery-public-data`.github_repos.INFORMATION_SCHEMA.COLUMN_FIELD_PATHS.
Thecommitstable contains the following nested and nested and repeated
columns:
author: nestedRECORDcolumn
committer: nestedRECORDcolumn
trailer: nested and repeatedRECORDcolumn
difference: nested and repeatedRECORDcolumn
To view metadata about theauthoranddifferencecolumns, run the following query.
TheTABLE_STORAGEandTABLE_STORAGE_BY_ORGANIZATIONviews have the following
schema:
Column name
Data type
Value
project_id
STRING
The project ID of the project that contains the dataset.
project_number
INT64
The project number of the project that contains the dataset.
table_catalog
STRING
The project ID of the project that contains the dataset.
table_schema
STRING
The name of the dataset that contains the table or materialized view,
also referred to as thedatasetId.
table_name
STRING
The name of the table or materialized view, also referred to as thetableId.
creation_time
TIMESTAMP
The creation time of the table.
total_rows
INT64
The total number of rows in the table or materialized view.
total_partitions
INT64
The number of partitions present in the table or materialized view.
Unpartitioned tables return 0.
total_logical_bytes
INT64
Total number of logical (uncompressed) bytes in the table or
materialized view.
active_logical_bytes
INT64
Number of logical (uncompressed) bytes that are younger than 90 days.
long_term_logical_bytes
INT64
Number of logical (uncompressed) bytes that are older than 90 days.
current_physical_bytes
INT64
Total number of physical bytes for the current storage of the
table across all partitions.
total_physical_bytes
INT64
Total number of physical (compressed) bytes used for storage,
including active, long-term, and time-travel (deleted or changed data)
bytes. Fail-safe (deleted or changed data retained after the time-travel
window) bytes aren't included.
active_physical_bytes
INT64
Number of physical (compressed) bytes younger than 90 days,
including time-travel (deleted or changed data) bytes.
long_term_physical_bytes
INT64
Number of physical (compressed) bytes older than 90 days.
time_travel_physical_bytes
INT64
Number of physical (compressed) bytes used by time-travel storage
(deleted or changed data).
storage_last_modified_time
TIMESTAMP
The most recent time that data was written to the table. ReturnsNULLif no data exists.
deleted
BOOLEAN
Indicates whether or not the table is deleted.
table_type
STRING
The type of table. For example,BASE TABLE.
managed_table_type
STRING
This column is in Preview. The managed type of the table. For example,NATIVEorBIGLAKE.
fail_safe_physical_bytes
INT64
Number of physical (compressed) bytes used by the fail-safe storage
(deleted or changed data).
last_metadata_index_refresh_time
TIMESTAMP
The last metadata index refresh time of the table.
table_deletion_reason
STRING
Table deletion reason if thedeletedfield is true. The
possible values are as follows:
TABLE_EXPIRATION:table deleted after set expiration time
DATASET_DELETION:dataset deleted by user
USER_DELETED:table was deleted by user
table_deletion_time
TIMESTAMP
The deletion time of the table.
Examples
Example 1:
The following example shows you the total logical bytes billed for the
current project.
The following example shows you how to forecast the price difference per
dataset between logical and physical billing models for the next 30 days.
This example assumes that future storage usage is constant over the next
30 days from the moment the query was run. Note that the forecast is limited to
base tables, it excludes all other types of tables within a dataset.
The prices used in the pricing variables for this query are for
theus-central1region. If you want to run this query for a different region,
update the pricing variables appropriately. SeeStorage pricingfor pricing information.
Open the BigQuery page in the Google Cloud console.
Enter the following GoogleSQL query in theQuery editorbox.INFORMATION_SCHEMArequires GoogleSQL syntax. GoogleSQL
is the default syntax in the Google Cloud console.
DECLAREactive_logical_gib_priceFLOAT64DEFAULT0.02;DECLARElong_term_logical_gib_priceFLOAT64DEFAULT0.01;DECLAREactive_physical_gib_priceFLOAT64DEFAULT0.04;DECLARElong_term_physical_gib_priceFLOAT64DEFAULT0.02;WITHstorage_sizesAS(SELECTtable_schemaASdataset_name,-- LogicalSUM(IF(deleted=false,active_logical_bytes,0))/power(1024,3)ASactive_logical_gib,SUM(IF(deleted=false,long_term_logical_bytes,0))/power(1024,3)ASlong_term_logical_gib,-- PhysicalSUM(active_physical_bytes)/power(1024,3)ASactive_physical_gib,SUM(active_physical_bytes-time_travel_physical_bytes)/power(1024,3)ASactive_no_tt_physical_gib,SUM(long_term_physical_bytes)/power(1024,3)ASlong_term_physical_gib,-- Restorable previously deleted physicalSUM(time_travel_physical_bytes)/power(1024,3)AStime_travel_physical_gib,SUM(fail_safe_physical_bytes)/power(1024,3)ASfail_safe_physical_gib,FROM`region-REGION`.INFORMATION_SCHEMA.TABLE_STORAGE_BY_PROJECTWHEREtotal_physical_bytes+fail_safe_physical_bytes>0-- Base the forecast on base tables only for highest precision resultsANDtable_type='BASE TABLE'GROUPBY1)SELECTdataset_name,-- LogicalROUND(active_logical_gib,2)ASactive_logical_gib,ROUND(long_term_logical_gib,2)ASlong_term_logical_gib,-- PhysicalROUND(active_physical_gib,2)ASactive_physical_gib,ROUND(long_term_physical_gib,2)ASlong_term_physical_gib,ROUND(time_travel_physical_gib,2)AStime_travel_physical_gib,ROUND(fail_safe_physical_gib,2)ASfail_safe_physical_gib,-- Compression ratioROUND(SAFE_DIVIDE(active_logical_gib,active_no_tt_physical_gib),2)ASactive_compression_ratio,ROUND(SAFE_DIVIDE(long_term_logical_gib,long_term_physical_gib),2)ASlong_term_compression_ratio,-- Forecast costs logicalROUND(active_logical_gib*active_logical_gib_price,2)ASforecast_active_logical_cost,ROUND(long_term_logical_gib*long_term_logical_gib_price,2)ASforecast_long_term_logical_cost,-- Forecast costs physicalROUND((active_no_tt_physical_gib+time_travel_physical_gib+fail_safe_physical_gib)*active_physical_gib_price,2)ASforecast_active_physical_cost,ROUND(long_term_physical_gib*long_term_physical_gib_price,2)ASforecast_long_term_physical_cost,-- Forecast costs totalROUND(((active_logical_gib*active_logical_gib_price)+(long_term_logical_gib*long_term_logical_gib_price))-(((active_no_tt_physical_gib+time_travel_physical_gib+fail_safe_physical_gib)*active_physical_gib_price)+(long_term_physical_gib*long_term_physical_gib_price)),2)ASforecast_total_cost_differenceFROMstorage_sizesORDERBY(forecast_active_logical_cost+forecast_active_physical_cost)DESC;
To enable this view, you can set the value ofenable_info_schema_storagetoTRUEon your project or organization. For more information on managing your
configuration, seeManage configuration
settings.
If you haven't configured this setting, you will see the following error:
INFORMATION_SCHEMA.TABLE_STORAGE hasn't been enabled for project <myproject>.
Consider using one of the following SQL statements to enable data collection:
ALTER PROJECT `<myproject>`
SET OPTIONS (`region-<region>.enable_info_schema_storage` = TRUE)
Or to enable for the entire organization:
ALTER ORGANIZATION
SET OPTIONS (`region-<region>.enable_info_schema_storage` = TRUE)
After enabling, please allow around 1 day for the complete historical data to
become available.
Run the SQL statements described in the error message to enable the view.
List tables in a dataset
You can list tables in datasets in the following ways:
At a minimum, to list tables in a dataset, you must be grantedbigquery.tables.listpermissions. The following predefined IAM
roles includebigquery.tables.listpermissions:
bigquery.user
bigquery.metadataViewer
bigquery.dataViewer
bigquery.dataEditor
bigquery.dataOwner
bigquery.admin
For more information on IAM roles and permissions in
BigQuery, seeAccess control.
You can audit the history of BigQuery tables by querying
Cloud Audit Logs in Logs Explorer. These logs help you track when tables were
created, updated, or deleted, and identify the user or service account that made
the changes.
Required permissions
To browse audit logs, you need theroles/logging.privateLogViewerrole. For more information on IAM roles and permissions in
Cloud Logging, seeAccess control with IAM.
Get audit data
You can access audit information from the Google Cloud console,gcloudcommand
line, REST API, and all supported languages using client libraries. The logging
filter shown in the following example can be used regardless of method used.
In the Google Cloud console, go to theLoggingpage.
logName = "projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Factivity"
AND resource.type = "bigquery_dataset"
AND timestamp >= "STARTING_TIMESTAMP"
AND protoPayload.@type = "type.googleapis.com/google.cloud.audit.AuditLog"
AND (
protoPayload.metadata.tableCreation :*
OR protoPayload.metadata.tableChange :*
OR protoPayload.metadata.tableDeletion :*
)
AND protoPayload.resourceName : "projects/PROJECT_ID/datasets/DATASET_ID/tables/"
Replace the following:
PROJECT_ID: the project that contains datasets and tables
you are interested in.
STARTING_TIMESTAMP: the oldest logs that you want to see.
Use ISO 8601 format, such as2025-01-01or2025-02-03T04:05:06Z.
DATASET_ID: the dataset that you want to filter by.
Interpret the results
In the Logs Explorer result pane, expand the entry you're interested in,
and then clickExpand nested fieldsto show the whole message.
The logging entry contains only one of the following objects to indicate
the operation performed:
protoPayload.metadata.tableCreation: a table was created.
protoPayload.metadata.tableChange: table metadata was changed, such as
schema update, description change, or table replacement.
protoPayload.metadata.tableDeletion: a table was deleted.
The content of these objects describes the requested action.
For a detailed description, seeBigQueryAuditMetadata.
Explanation of the query
logName = "projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Factivity":
This line filters for Admin Activity audit logs within your Google Cloud
project. These logs record API calls and actions that modify the
configuration or metadata of your resources.
resource.type = "bigquery_dataset": This narrows the search to events
related to BigQuery datasets, where table operations are
logged.
timestamp >= "STARTING_TIMESTAMP": Filters log entries to
only show those created on or after the specified timestamp.
protoPayload.@type = "type.googleapis.com/google.cloud.audit.AuditLog":
Ensures the log message conforms to the standard Cloud Audit Log structure.
( ... ): This block groups conditions to find different types of table
events, as outlined in the previous section. The:*operator indicates
that the key must be present. If you are interested in only one event,
such as table creation, remove unnecessary conditions from this block.
protoPayload.resourceName : "projects/PROJECT_ID/datasets/DATASET_ID/tables/":
Selects log entries matching tables contained in the specified dataset. The
colon (:) operator performs a substring search.
To filter entries for a single table, replace the condition with the
following one:protoPayload.resourceName = "projects/PROJECT_ID/datasets/DATASET_ID/tables/TABLE_NAME".
To include all tables in all datasets in the specific project, remove this
condition.
If you're new to Google Cloud, create an account to evaluate how
BigQuery performs in real-world
scenarios. New customers also get $300 in free credits to run, test, and
deploy workloads.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2026-05-16 UTC."],[],[]]