Module external_config (3.31.0)

Define classes that describe external data sources.

These are used for both Table.externalDataConfiguration and Job.configuration.query.tableDefinitions.

Classes

BigtableColumn

  BigtableColumn 
 () 
 

Options for a Bigtable column.

BigtableColumnFamily

  BigtableColumnFamily 
 () 
 

Options for a Bigtable column family.

BigtableOptions

  BigtableOptions 
 () 
 

Options that describe how to treat Bigtable tables as BigQuery tables.

CSVOptions

  CSVOptions 
 () 
 

Options that describe how to treat CSV files as BigQuery tables.

ExternalCatalogDatasetOptions

  ExternalCatalogDatasetOptions 
 ( 
 default_storage_location_uri 
 : 
 typing 
 . 
 Optional 
 [ 
 str 
 ] 
 = 
 None 
 , 
 parameters 
 : 
 typing 
 . 
 Optional 
 [ 
 typing 
 . 
 Dict 
 [ 
 str 
 , 
 typing 
 . 
 Any 
 ]] 
 = 
 None 
 , 
 ) 
 

Options defining open source compatible datasets living in the BigQuery catalog. Contains metadata of open source database, schema or namespace represented by the current dataset.

Parameters
Name
Description
default_storage_location_uri
Optional[str]

The storage location URI for all tables in the dataset. Equivalent to hive metastore's database locationUri. Maximum length of 1024 characters. (str)

parameters
Optional[dict[str, Any]]

A map of key value pairs defining the parameters and properties of the open source schema. Maximum size of 2Mib.

ExternalCatalogTableOptions

  ExternalCatalogTableOptions 
 ( 
 connection_id 
 : 
 typing 
 . 
 Optional 
 [ 
 str 
 ] 
 = 
 None 
 , 
 parameters 
 : 
 typing 
 . 
 Optional 
 [ 
 typing 
 . 
 Dict 
 [ 
 str 
 , 
 typing 
 . 
 Any 
 ]] 
 = 
 None 
 , 
 storage_descriptor 
 : 
 typing 
 . 
 Optional 
 [ 
 google 
 . 
 cloud 
 . 
 bigquery 
 . 
 schema 
 . 
 StorageDescriptor 
 ] 
 = 
 None 
 , 
 ) 
 

Metadata about open source compatible table. The fields contained in these options correspond to hive metastore's table level properties.

Parameters
Name
Description
connection_id
Optional[str]

The connection specifying the credentials to be used to read external storage, such as Azure Blob, Cloud Storage, or S3. The connection is needed to read the open source table from BigQuery Engine. The connection_id can have the form .. or projects//locations//connections/ .

parameters
Union[Dict[str, Any], None]

A map of key value pairs defining the parameters and properties of the open source table. Corresponds with hive meta store table parameters. Maximum size of 4Mib.

storage_descriptor
Optional[StorageDescriptor]

A storage descriptor containing information about the physical storage of this table.

ExternalConfig

  ExternalConfig 
 ( 
 source_format 
 ) 
 

Description of an external data source.

Parameter
Name
Description
source_format
ExternalSourceFormat

See source_format .

ExternalSourceFormat

  ExternalSourceFormat 
 () 
 

The format for external data files.

Note that the set of allowed values for external data sources is different than the set used for loading data (see SourceFormat ).

GoogleSheetsOptions

  GoogleSheetsOptions 
 () 
 

Options that describe how to treat Google Sheets as BigQuery tables.

HivePartitioningOptions

  HivePartitioningOptions 
 () 
 

[Beta] Options that configure hive partitioning.

See https://cloud.google.com/bigquery/docs/reference/rest/v2/tables#HivePartitioningOptions
Design a Mobile Site
View Site in Mobile | Classic
Share by: