Index
-
DataprocMetastore(interface) -
DataprocMetastoreFederation(interface) -
AlterMetadataResourceLocationRequest(message) -
AlterMetadataResourceLocationResponse(message) -
AlterTablePropertiesRequest(message) -
AlterTablePropertiesResponse(message) -
AutoscalingConfig(message) -
AuxiliaryVersionConfig(message) -
BackendMetastore(message) -
BackendMetastore.MetastoreType(enum) -
Backup(message) -
Backup.State(enum) -
CancelMigrationRequest(message) -
CancelMigrationResponse(message) -
CompleteMigrationRequest(message) -
CompleteMigrationResponse(message) -
CreateBackupRequest(message) -
CreateFederationRequest(message) -
CreateMetadataImportRequest(message) -
CreateServiceRequest(message) -
DataCatalogConfig(message) -
DatabaseDumpSpec(message) -
DatabaseDumpSpec.Type(enum) -
DataplexConfig(message) -
DeleteBackupRequest(message) -
DeleteFederationRequest(message) -
DeleteMigrationExecutionRequest(message) -
DeleteServiceRequest(message) -
EncryptionConfig(message) -
ErrorDetails(message) -
ExportMetadataRequest(message) -
Federation(message) -
Federation.State(enum) -
GetBackupRequest(message) -
GetFederationRequest(message) -
GetMetadataImportRequest(message) -
GetMigrationExecutionRequest(message) -
GetServiceRequest(message) -
HiveMetastoreConfig(message) -
HiveMetastoreConfig.EndpointProtocol(enum) -
KerberosConfig(message) -
Lake(message) -
LatestBackup(message) -
LatestBackup.State(enum) -
LimitConfig(message) -
ListBackupsRequest(message) -
ListBackupsResponse(message) -
ListFederationsRequest(message) -
ListFederationsResponse(message) -
ListMetadataImportsRequest(message) -
ListMetadataImportsResponse(message) -
ListMigrationExecutionsRequest(message) -
ListMigrationExecutionsResponse(message) -
ListServicesRequest(message) -
ListServicesResponse(message) -
LocationMetadata(message) -
LocationMetadata.CustomRegionMetadata(message) -
LocationMetadata.HiveMetastoreVersion(message) -
LocationMetadata.MultiRegionMetadata(message) -
MaintenanceWindow(message) -
MetadataExport(message) -
MetadataExport.State(enum) -
MetadataImport(message) -
MetadataImport.DatabaseDump(message) -
MetadataImport.DatabaseDump.DatabaseType(enum) -
MetadataImport.State(enum) -
MetadataIntegration(message) -
MetadataManagementActivity(message) -
MigrationExecution(message) -
MigrationExecution.CloudSQLMigrationConfig(message) -
MigrationExecution.CloudSQLMigrationConfig.CdcConfig(message) -
MigrationExecution.CloudSQLMigrationConfig.CloudSQLConnectionConfig(message) -
MigrationExecution.Phase(enum) -
MigrationExecution.State(enum) -
MoveTableToDatabaseRequest(message) -
MoveTableToDatabaseResponse(message) -
MultiRegionConfig(message) -
MultiRegionConfig.CustomRegionConfig(message) -
MultiRegionConfig.RootCACertificate(message) (deprecated) -
NetworkConfig(message) -
NetworkConfig.Consumer(message) -
OperationMetadata(message) -
QueryMetadataRequest(message) -
QueryMetadataResponse(message) -
RemoveIamPolicyRequest(message) -
RemoveIamPolicyResponse(message) -
Restore(message) -
Restore.RestoreType(enum) -
Restore.State(enum) -
RestoreServiceRequest(message) -
ScalingConfig(message) -
ScalingConfig.InstanceSize(enum) -
ScheduledBackup(message) -
Secret(message) -
Service(message) -
Service.DatabaseType(enum) -
Service.ReleaseChannel(enum) -
Service.State(enum) -
Service.Tier(enum) -
StartMigrationRequest(message) -
TelemetryConfig(message) -
TelemetryConfig.LogFormat(enum) -
UpdateFederationRequest(message) -
UpdateMetadataImportRequest(message) -
UpdateServiceRequest(message)
DataprocMetastore
Configures and manages metastore services. Metastore services are fully managed, highly available, autoscaled, autohealing, OSS-native deployments of technical metadata management software. Each metastore service exposes a network endpoint through which metadata queries are served. Metadata queries can originate from a variety of sources, including Apache Hive, Apache Presto, and Apache Spark.
The Dataproc Metastore API defines the following resource model:
- The service works with a collection of Google Cloud projects, named:
/projects/* - Each project has a collection of available locations, named:
/locations/*(a location must refer to a Google Cloudregion) - Each location has a collection of services, named:
/services/* - Dataproc Metastore services are resources with names of the form:
/projects/{project_number}/locations/{location_id}/services/{service_id}
.
rpc AlterMetadataResourceLocation(
AlterMetadataResourceLocationRequest
) returns ( Operation
)
Alter metadata resource location. The metadata resource can be a database, table, or partition. This functionality only updates the parent directory for the respective metadata resource and does not transfer any existing data to the new location.
- Authorization scopes
-
Requires the following OAuth scope:
-
https://www.googleapis.com/auth/cloud-platform
For more information, see the Authentication Overview .
-
rpc AlterTableProperties(
AlterTablePropertiesRequest
) returns ( Operation
)
Alter metadata table properties.
- Authorization scopes
-
Requires the following OAuth scope:
-
https://www.googleapis.com/auth/cloud-platform
For more information, see the Authentication Overview .
-
rpc CancelMigration(
CancelMigrationRequest
) returns ( Operation
)
Cancels the ongoing Managed Migration process.
- Authorization scopes
-
Requires the following OAuth scope:
-
https://www.googleapis.com/auth/cloud-platform
For more information, see the Authentication Overview .
-
rpc CompleteMigration(
CompleteMigrationRequest
) returns ( Operation
)
Completes the managed migration process. The Dataproc Metastore service will switch to using its own backend database after successful migration.
- Authorization scopes
-
Requires the following OAuth scope:
-
https://www.googleapis.com/auth/cloud-platform
For more information, see the Authentication Overview .
-
rpc CreateBackup(
CreateBackupRequest
) returns ( Operation
)
Creates a new backup in a given project and location.
- Authorization scopes
-
Requires the following OAuth scope:
-
https://www.googleapis.com/auth/cloud-platform
For more information, see the Authentication Overview .
-
rpc CreateMetadataImport(
CreateMetadataImportRequest
) returns ( Operation
)
Creates a new MetadataImport in a given project and location.
- Authorization scopes
-
Requires the following OAuth scope:
-
https://www.googleapis.com/auth/cloud-platform
For more information, see the Authentication Overview .
-
rpc CreateService(
CreateServiceRequest
) returns ( Operation
)
Creates a metastore service in a project and location.
- Authorization scopes
-
Requires the following OAuth scope:
-
https://www.googleapis.com/auth/cloud-platform
For more information, see the Authentication Overview .
-
rpc DeleteBackup(
DeleteBackupRequest
) returns ( Operation
)
Deletes a single backup.
- Authorization scopes
-
Requires the following OAuth scope:
-
https://www.googleapis.com/auth/cloud-platform
For more information, see the Authentication Overview .
-
rpc DeleteMigrationExecution(
DeleteMigrationExecutionRequest
) returns ( Operation
)
Deletes a single migration execution.
- Authorization scopes
-
Requires the following OAuth scope:
-
https://www.googleapis.com/auth/cloud-platform
For more information, see the Authentication Overview .
-
rpc DeleteService(
DeleteServiceRequest
) returns ( Operation
)
Deletes a single service.
- Authorization scopes
-
Requires the following OAuth scope:
-
https://www.googleapis.com/auth/cloud-platform
For more information, see the Authentication Overview .
-
rpc ExportMetadata(
ExportMetadataRequest
) returns ( Operation
)
Exports metadata from a service.
- Authorization scopes
-
Requires the following OAuth scope:
-
https://www.googleapis.com/auth/cloud-platform
For more information, see the Authentication Overview .
-
rpc GetBackup(
GetBackupRequest
) returns ( Backup
)
Gets details of a single backup.
- Authorization scopes
-
Requires the following OAuth scope:
-
https://www.googleapis.com/auth/cloud-platform
For more information, see the Authentication Overview .
-
rpc GetMetadataImport(
GetMetadataImportRequest
) returns ( MetadataImport
)
Gets details of a single import.
- Authorization scopes
-
Requires the following OAuth scope:
-
https://www.googleapis.com/auth/cloud-platform
For more information, see the Authentication Overview .
-
rpc GetMigrationExecution(
GetMigrationExecutionRequest
) returns ( MigrationExecution
)
Gets details of a single migration execution.
- Authorization scopes
-
Requires the following OAuth scope:
-
https://www.googleapis.com/auth/cloud-platform
For more information, see the Authentication Overview .
-
rpc GetService(
GetServiceRequest
) returns ( Service
)
Gets the details of a single service.
- Authorization scopes
-
Requires the following OAuth scope:
-
https://www.googleapis.com/auth/cloud-platform
For more information, see the Authentication Overview .
-
rpc ListBackups(
ListBackupsRequest
) returns ( ListBackupsResponse
)
Lists backups in a service.
- Authorization scopes
-
Requires the following OAuth scope:
-
https://www.googleapis.com/auth/cloud-platform
For more information, see the Authentication Overview .
-
rpc ListMetadataImports(
ListMetadataImportsRequest
) returns ( ListMetadataImportsResponse
)
Lists imports in a service.
- Authorization scopes
-
Requires the following OAuth scope:
-
https://www.googleapis.com/auth/cloud-platform
For more information, see the Authentication Overview .
-
rpc ListMigrationExecutions(
ListMigrationExecutionsRequest
) returns ( ListMigrationExecutionsResponse
)
Lists migration executions on a service.
- Authorization scopes
-
Requires the following OAuth scope:
-
https://www.googleapis.com/auth/cloud-platform
For more information, see the Authentication Overview .
-
rpc ListServices(
ListServicesRequest
) returns ( ListServicesResponse
)
Lists services in a project and location.
- Authorization scopes
-
Requires the following OAuth scope:
-
https://www.googleapis.com/auth/cloud-platform
For more information, see the Authentication Overview .
-
rpc MoveTableToDatabase(
MoveTableToDatabaseRequest
) returns ( Operation
)
Move a table to another database.
- Authorization scopes
-
Requires the following OAuth scope:
-
https://www.googleapis.com/auth/cloud-platform
For more information, see the Authentication Overview .
-
rpc QueryMetadata(
QueryMetadataRequest
) returns ( Operation
)
Query Dataproc Metastore metadata.
- Authorization scopes
-
Requires the following OAuth scope:
-
https://www.googleapis.com/auth/cloud-platform
For more information, see the Authentication Overview .
-
rpc RemoveIamPolicy(
RemoveIamPolicyRequest
) returns ( RemoveIamPolicyResponse
)
Removes the attached IAM policies for a resource
- Authorization scopes
-
Requires the following OAuth scope:
-
https://www.googleapis.com/auth/cloud-platform
For more information, see the Authentication Overview .
-
rpc RestoreService(
RestoreServiceRequest
) returns ( Operation
)
Restores a service from a backup.
- Authorization scopes
-
Requires the following OAuth scope:
-
https://www.googleapis.com/auth/cloud-platform
For more information, see the Authentication Overview .
-
rpc StartMigration(
StartMigrationRequest
) returns ( Operation
)
Starts the Managed Migration process.
- Authorization scopes
-
Requires the following OAuth scope:
-
https://www.googleapis.com/auth/cloud-platform
For more information, see the Authentication Overview .
-
rpc UpdateMetadataImport(
UpdateMetadataImportRequest
) returns ( Operation
)
Updates a single import. Only the description field of MetadataImport is supported to be updated.
- Authorization scopes
-
Requires the following OAuth scope:
-
https://www.googleapis.com/auth/cloud-platform
For more information, see the Authentication Overview .
-
rpc UpdateService(
UpdateServiceRequest
) returns ( Operation
)
Updates the parameters of a single service.
- Authorization scopes
-
Requires the following OAuth scope:
-
https://www.googleapis.com/auth/cloud-platform
For more information, see the Authentication Overview .
-
DataprocMetastoreFederation
Configures and manages metastore federation services. Dataproc Metastore Federation Service allows federating a collection of backend metastores like BigQuery, Dataplex Lakes, and other Dataproc Metastores. The Federation Service exposes a gRPC URL through which metadata from the backend metastores are served at query time.
The Dataproc Metastore Federation API defines the following resource model: * The service works with a collection of Google Cloud projects. * Each project has a collection of available locations. * Each location has a collection of federations. * Dataproc Metastore Federations are resources with names of the form: projects/{project_number}/locations/{location_id}/federations/{federation_id}
.
rpc CreateFederation(
CreateFederationRequest
) returns ( Operation
)
Creates a metastore federation in a project and location.
- Authorization scopes
-
Requires the following OAuth scope:
-
https://www.googleapis.com/auth/cloud-platform
For more information, see the Authentication Overview .
-
rpc DeleteFederation(
DeleteFederationRequest
) returns ( Operation
)
Deletes a single federation.
- Authorization scopes
-
Requires the following OAuth scope:
-
https://www.googleapis.com/auth/cloud-platform
For more information, see the Authentication Overview .
-
rpc GetFederation(
GetFederationRequest
) returns ( Federation
)
Gets the details of a single federation.
- Authorization scopes
-
Requires the following OAuth scope:
-
https://www.googleapis.com/auth/cloud-platform
For more information, see the Authentication Overview .
-
rpc ListFederations(
ListFederationsRequest
) returns ( ListFederationsResponse
)
Lists federations in a project and location.
- Authorization scopes
-
Requires the following OAuth scope:
-
https://www.googleapis.com/auth/cloud-platform
For more information, see the Authentication Overview .
-
rpc UpdateFederation(
UpdateFederationRequest
) returns ( Operation
)
Updates the fields of a federation.
- Authorization scopes
-
Requires the following OAuth scope:
-
https://www.googleapis.com/auth/cloud-platform
For more information, see the Authentication Overview .
-
AlterMetadataResourceLocationRequest
Request message for DataprocMetastore.AlterMetadataResourceLocation
.
service
string
Required. The relative resource name of the metastore service to mutate metadata, in the following format:
projects/{project_id}/locations/{location_id}/services/{service_id}
.
Authorization requires the following IAM
permission on the specified resource service
:
-
metastore.services.mutateMetadata
resource_name
string
Required. The relative metadata resource name in the following format.
databases/{database_id}
or databases/{database_id}/tables/{table_id}
or databases/{database_id}/tables/{table_id}/partitions/{partition_id}
location_uri
string
Required. The new location URI for the metadata resource.
AlterMetadataResourceLocationResponse
This type has no fields.
Response message for DataprocMetastore.AlterMetadataResourceLocation
.
AlterTablePropertiesRequest
Request message for DataprocMetastore.AlterTableProperties
.
service
string
Required. The relative resource name of the Dataproc Metastore service that's being used to mutate metadata table properties, in the following format:
projects/{project_id}/locations/{location_id}/services/{service_id}
.
Authorization requires the following IAM
permission on the specified resource service
:
-
metastore.services.mutateMetadata
table_name
string
Required. The name of the table containing the properties you're altering in the following format.
databases/{database_id}/tables/{table_id}
update_mask
A field mask that specifies the metadata table properties that are overwritten by the update. Fields specified in the update_mask
are relative to the resource (not to the full request). A field is overwritten if it is in the mask.
For example, given the target properties:
properties {
a: 1
b: 2
}
And an update properties:
properties {
a: 2
b: 3
c: 4
}
then if the field mask is:
paths: ["properties.b", "properties.c"]
then the result will be:
properties {
a: 1
b: 3
c: 4
}
properties
map<string, string>
A map that describes the desired values to mutate. If update_mask is empty, the properties will not update. Otherwise, the properties only alters the value whose associated paths exist in the update mask
AlterTablePropertiesResponse
This type has no fields.
Response message for DataprocMetastore.AlterTableProperties
.
AutoscalingConfig
Represents the autoscaling configuration of a metastore service.
| Fields | |
|---|---|
autoscaling_factor
|
Output only. The scaling factor of a service with autoscaling enabled. |
autoscaling_enabled
|
Optional. Whether or not autoscaling is enabled for this service. |
limit_config
|
Optional. The LimitConfig of the service. |
AuxiliaryVersionConfig
Configuration information for the auxiliary service versions.
| Fields | |
|---|---|
version
|
The Hive metastore version of the auxiliary service. It must be less than the primary Hive metastore service's version. |
config_overrides
|
A mapping of Hive metastore configuration key-value pairs to apply to the auxiliary Hive metastore (configured in |
network_config
|
Output only. The network configuration contains the endpoint URI(s) of the auxiliary Hive metastore service. |
BackendMetastore
Represents a backend metastore for the federation.
name
string
The relative resource name of the metastore that is being federated. The formats of the relative resource names for the currently supported metastores are listed below:
- BigQuery
-
projects/{project_id}
-
- Dataproc Metastore
-
projects/{project_id}/locations/{location}/services/{service_id}
-
metastore_type
The type of the backend metastore.
MetastoreType
The type of the backend metastore.
| Enums | |
|---|---|
METASTORE_TYPE_UNSPECIFIED
|
The metastore type is not set. |
DATAPLEX
|
The backend metastore is Dataplex. |
BIGQUERY
|
The backend metastore is BigQuery. |
DATAPROC_METASTORE
|
The backend metastore is Dataproc Metastore. |
Backup
The details of a backup resource.
| Fields | |
|---|---|
name
|
Immutable. The relative resource name of the backup, in the following form: |
create_time
|
Output only. The time when the backup was started. |
end_time
|
Output only. The time when the backup finished creating. |
state
|
Output only. The current state of the backup. |
service_revision
|
Output only. The revision of the service at the time of backup. |
description
|
The description of the backup. |
restoring_services[]
|
Output only. Services that are restoring from the backup. |
State
The current state of the backup.
| Enums | |
|---|---|
STATE_UNSPECIFIED
|
The state of the backup is unknown. |
CREATING
|
The backup is being created. |
DELETING
|
The backup is being deleted. |
ACTIVE
|
The backup is active and ready to use. |
FAILED
|
The backup failed. |
RESTORING
|
The backup is being restored. |
CancelMigrationRequest
Request message for DataprocMetastore.CancelMigration
.
service
string
Required. The relative resource name of the metastore service to cancel the ongoing migration to, in the following format:
projects/{project_id}/locations/{location_id}/services/{service_id}
.
Authorization requires the following IAM
permission on the specified resource service
:
-
metastore.migrations.cancel
CancelMigrationResponse
Response message for DataprocMetastore.CancelMigration
.
| Fields | |
|---|---|
migration_execution
|
The relative resource name of the migration execution, in the following form: |
CompleteMigrationRequest
Request message for DataprocMetastore.CompleteMigration
.
service
string
Required. The relative resource name of the metastore service to complete the migration to, in the following format:
projects/{project_id}/locations/{location_id}/services/{service_id}
.
Authorization requires the following IAM
permission on the specified resource service
:
-
metastore.migrations.complete
CompleteMigrationResponse
Response message for DataprocMetastore.CompleteMigration
.
| Fields | |
|---|---|
migration_execution
|
The relative resource name of the migration execution, in the following form: |
CreateBackupRequest
Request message for DataprocMetastore.CreateBackup
.
parent
string
Required. The relative resource name of the service in which to create a backup of the following form:
projects/{project_number}/locations/{location_id}/services/{service_id}
.
Authorization requires the following IAM
permission on the specified resource parent
:
-
metastore.backups.create
backup_id
string
Required. The ID of the backup, which is used as the final component of the backup's name.
This value must be between 1 and 64 characters long, begin with a letter, end with a letter or number, and consist of alpha-numeric ASCII characters or hyphens.
backup
Required. The backup to create. The name
field is ignored. The ID of the created backup must be provided in the request's backup_id
field.
request_id
string
Optional. A request ID. Specify a unique request ID to allow the server to ignore the request if it has completed. The server will ignore subsequent requests that provide a duplicate request ID for at least 60 minutes after the first request.
For example, if an initial request times out, followed by another request with the same request ID, the server ignores the second request to prevent the creation of duplicate commitments.
The request ID must be a valid UUID A zero UUID (00000000-0000-0000-0000-000000000000) is not supported.
CreateFederationRequest
Request message for CreateFederation.
parent
string
Required. The relative resource name of the location in which to create a federation service, in the following form:
projects/{project_number}/locations/{location_id}
.
Authorization requires the following IAM
permission on the specified resource parent
:
-
metastore.federations.create
federation_id
string
Required. The ID of the metastore federation, which is used as the final component of the metastore federation's name.
This value must be between 2 and 63 characters long inclusive, begin with a letter, end with a letter or number, and consist of alpha-numeric ASCII characters or hyphens.
federation
Required. The Metastore Federation to create. The name
field is ignored. The ID of the created metastore federation must be provided in the request's federation_id
field.
request_id
string
Optional. A request ID. Specify a unique request ID to allow the server to ignore the request if it has completed. The server will ignore subsequent requests that provide a duplicate request ID for at least 60 minutes after the first request.
For example, if an initial request times out, followed by another request with the same request ID, the server ignores the second request to prevent the creation of duplicate commitments.
The request ID must be a valid UUID A zero UUID (00000000-0000-0000-0000-000000000000) is not supported.
CreateMetadataImportRequest
Request message for DataprocMetastore.CreateMetadataImport
.
parent
string
Required. The relative resource name of the service in which to create a metastore import, in the following form:
projects/{project_number}/locations/{location_id}/services/{service_id}
.
Authorization requires the following IAM
permission on the specified resource parent
:
-
metastore.imports.create
metadata_import_id
string
Required. The ID of the metadata import, which is used as the final component of the metadata import's name.
This value must be between 1 and 64 characters long, begin with a letter, end with a letter or number, and consist of alpha-numeric ASCII characters or hyphens.
metadata_import
Required. The metadata import to create. The name
field is ignored. The ID of the created metadata import must be provided in the request's metadata_import_id
field.
request_id
string
Optional. A request ID. Specify a unique request ID to allow the server to ignore the request if it has completed. The server will ignore subsequent requests that provide a duplicate request ID for at least 60 minutes after the first request.
For example, if an initial request times out, followed by another request with the same request ID, the server ignores the second request to prevent the creation of duplicate commitments.
The request ID must be a valid UUID A zero UUID (00000000-0000-0000-0000-000000000000) is not supported.
CreateServiceRequest
Request message for DataprocMetastore.CreateService
.
parent
string
Required. The relative resource name of the location in which to create a metastore service, in the following form:
projects/{project_number}/locations/{location_id}
.
Authorization requires the following IAM
permission on the specified resource parent
:
-
metastore.services.create
service_id
string
Required. The ID of the metastore service, which is used as the final component of the metastore service's name.
This value must be between 2 and 63 characters long inclusive, begin with a letter, end with a letter or number, and consist of alpha-numeric ASCII characters or hyphens.
service
Required. The Metastore service to create. The name
field is ignored. The ID of the created metastore service must be provided in the request's service_id
field.
request_id
string
Optional. A request ID. Specify a unique request ID to allow the server to ignore the request if it has completed. The server will ignore subsequent requests that provide a duplicate request ID for at least 60 minutes after the first request.
For example, if an initial request times out, followed by another request with the same request ID, the server ignores the second request to prevent the creation of duplicate commitments.
The request ID must be a valid UUID A zero UUID (00000000-0000-0000-0000-000000000000) is not supported.
DataCatalogConfig
Specifies how metastore metadata should be integrated with the Data Catalog service.
| Fields | |
|---|---|
enabled
|
Optional. Defines whether the metastore metadata should be synced to Data Catalog. The default value is to disable syncing metastore metadata to Data Catalog. |
DatabaseDumpSpec
This type has no fields.
The specification of database dump to import from or export to.
Type
The type of the database dump.
| Enums | |
|---|---|
TYPE_UNSPECIFIED
|
The type of the database dump is unknown. |
MYSQL
|
Database dump is a MySQL dump file. |
AVRO
|
Database dump contains Avro files. |
DataplexConfig
Specifies how metastore metadata should be integrated with the Dataplex service.
| Fields | |
|---|---|
lake_resources
|
A reference to the Lake resources that this metastore service is attached to. The key is the lake resource name. Example: |
DeleteBackupRequest
Request message for DataprocMetastore.DeleteBackup
.
name
string
Required. The relative resource name of the backup to delete, in the following form:
projects/{project_number}/locations/{location_id}/services/{service_id}/backups/{backup_id}
.
Authorization requires the following IAM
permission on the specified resource name
:
-
metastore.backups.delete
request_id
string
Optional. A request ID. Specify a unique request ID to allow the server to ignore the request if it has completed. The server will ignore subsequent requests that provide a duplicate request ID for at least 60 minutes after the first request.
For example, if an initial request times out, followed by another request with the same request ID, the server ignores the second request to prevent the creation of duplicate commitments.
The request ID must be a valid UUID A zero UUID (00000000-0000-0000-0000-000000000000) is not supported.
DeleteFederationRequest
Request message for DeleteFederation.
name
string
Required. The relative resource name of the metastore federation to delete, in the following form:
projects/{project_number}/locations/{location_id}/federations/{federation_id}
.
Authorization requires the following IAM
permission on the specified resource name
:
-
metastore.federations.delete
request_id
string
Optional. A request ID. Specify a unique request ID to allow the server to ignore the request if it has completed. The server will ignore subsequent requests that provide a duplicate request ID for at least 60 minutes after the first request.
For example, if an initial request times out, followed by another request with the same request ID, the server ignores the second request to prevent the creation of duplicate commitments.
The request ID must be a valid UUID A zero UUID (00000000-0000-0000-0000-000000000000) is not supported.
DeleteMigrationExecutionRequest
Request message for DataprocMetastore.DeleteMigrationExecution
.
name
string
Required. The relative resource name of the migrationExecution to delete, in the following form:
projects/{project_number}/locations/{location_id}/services/{service_id}/migrationExecutions/{migration_execution_id}
.
Authorization requires the following IAM
permission on the specified resource name
:
-
metastore.migrations.delete
request_id
string
Optional. A request ID. Specify a unique request ID to allow the server to ignore the request if it has completed. The server will ignore subsequent requests that provide a duplicate request ID for at least 60 minutes after the first request.
For example, if an initial request times out, followed by another request with the same request ID, the server ignores the second request to prevent the creation of duplicate commitments.
The request ID must be a valid UUID A zero UUID (00000000-0000-0000-0000-000000000000) is not supported.
DeleteServiceRequest
Request message for DataprocMetastore.DeleteService
.
name
string
Required. The relative resource name of the metastore service to delete, in the following form:
projects/{project_number}/locations/{location_id}/services/{service_id}
.
Authorization requires the following IAM
permission on the specified resource name
:
-
metastore.services.delete
request_id
string
Optional. A request ID. Specify a unique request ID to allow the server to ignore the request if it has completed. The server will ignore subsequent requests that provide a duplicate request ID for at least 60 minutes after the first request.
For example, if an initial request times out, followed by another request with the same request ID, the server ignores the second request to prevent the creation of duplicate commitments.
The request ID must be a valid UUID A zero UUID (00000000-0000-0000-0000-000000000000) is not supported.
EncryptionConfig
Encryption settings for the service.
| Fields | |
|---|---|
kms_key
|
The fully qualified customer provided Cloud KMS key name to use for customer data encryption, in the following format: |
ErrorDetails
Error details in public error message for DataprocMetastore.QueryMetadata
.
| Fields | |
|---|---|
details
|
Additional structured details about this error. Keys define the failure items. Value describes the exception or details of the item. |
ExportMetadataRequest
Request message for DataprocMetastore.ExportMetadata
.
service
string
Required. The relative resource name of the metastore service to run export, in the following form:
projects/{project_id}/locations/{location_id}/services/{service_id}
.
Authorization requires the following IAM
permission on the specified resource service
:
-
metastore.services.export
request_id
string
Optional. A request ID. Specify a unique request ID to allow the server to ignore the request if it has completed. The server will ignore subsequent requests that provide a duplicate request ID for at least 60 minutes after the first request.
For example, if an initial request times out, followed by another request with the same request ID, the server ignores the second request to prevent the creation of duplicate commitments.
The request ID must be a valid UUID . A zero UUID (00000000-0000-0000-0000-000000000000) is not supported.
database_dump_type
Optional. The type of the database dump. If unspecified, defaults to MYSQL
.
destination
. Required. Destination that metadata is exported to. destination
can be only one of the following:destination_gcs_folder
string
A Cloud Storage URI of a folder, in the format gs://<bucket_name>/<path_inside_bucket>
. A sub-folder <export_folder>
containing exported files will be created below it.
Federation
Represents a federation of multiple backend metastores.
| Fields | |
|---|---|
name
|
Immutable. The relative resource name of the federation, of the form: projects/{project_number}/locations/{location_id}/federations/{federation_id}`. |
create_time
|
Output only. The time when the metastore federation was created. |
update_time
|
Output only. The time when the metastore federation was last updated. |
labels
|
User-defined labels for the metastore federation. |
version
|
Immutable. The Apache Hive metastore version of the federation. All backend metastore versions must be compatible with the federation version. |
backend_metastores
|
A map from |
endpoint_uri
|
Output only. The federation endpoint. |
state
|
Output only. The current state of the federation. |
state_message
|
Output only. Additional information about the current state of the metastore federation, if available. |
uid
|
Output only. The globally unique resource identifier of the metastore federation. |
State
The current state of the federation.
| Enums | |
|---|---|
STATE_UNSPECIFIED
|
The state of the metastore federation is unknown. |
CREATING
|
The metastore federation is in the process of being created. |
ACTIVE
|
The metastore federation is running and ready to serve queries. |
UPDATING
|
The metastore federation is being updated. It remains usable but cannot accept additional update requests or be deleted at this time. |
DELETING
|
The metastore federation is undergoing deletion. It cannot be used. |
ERROR
|
The metastore federation has encountered an error and cannot be used. The metastore federation should be deleted. |
GetBackupRequest
Request message for DataprocMetastore.GetBackup
.
name
string
Required. The relative resource name of the backup to retrieve, in the following form:
projects/{project_number}/locations/{location_id}/services/{service_id}/backups/{backup_id}
.
Authorization requires the following IAM
permission on the specified resource name
:
-
metastore.backups.get
GetFederationRequest
Request message for GetFederation.
name
string
Required. The relative resource name of the metastore federation to retrieve, in the following form:
projects/{project_number}/locations/{location_id}/federations/{federation_id}
.
Authorization requires the following IAM
permission on the specified resource name
:
-
metastore.federations.get
GetMetadataImportRequest
Request message for DataprocMetastore.GetMetadataImport
.
name
string
Required. The relative resource name of the metadata import to retrieve, in the following form:
projects/{project_number}/locations/{location_id}/services/{service_id}/metadataImports/{import_id}
.
Authorization requires the following IAM
permission on the specified resource name
:
-
metastore.imports.get
GetMigrationExecutionRequest
Request message for DataprocMetastore.GetMigrationExecution
.
name
string
Required. The relative resource name of the migration execution to retrieve, in the following form:
projects/{project_number}/locations/{location_id}/services/{service_id}/migrationExecutions/{migration_execution_id}
.
Authorization requires the following IAM
permission on the specified resource name
:
-
metastore.migrations.get
GetServiceRequest
Request message for DataprocMetastore.GetService
.
name
string
Required. The relative resource name of the metastore service to retrieve, in the following form:
projects/{project_number}/locations/{location_id}/services/{service_id}
.
Authorization requires the following IAM
permission on the specified resource name
:
-
metastore.services.get
HiveMetastoreConfig
Specifies configuration information specific to running Hive metastore software as the metastore service.
| Fields | |
|---|---|
version
|
Immutable. The Hive metastore schema version. |
config_overrides
|
A mapping of Hive metastore configuration key-value pairs to apply to the Hive metastore (configured in |
kerberos_config
|
Information used to configure the Hive metastore service as a service principal in a Kerberos realm. To disable Kerberos, use the |
endpoint_protocol
|
The protocol to use for the metastore service endpoint. If unspecified, defaults to |
auxiliary_versions
|
A mapping of Hive metastore version to the auxiliary version configuration. When specified, a secondary Hive metastore service is created along with the primary service. All auxiliary versions must be less than the service's primary version. The key is the auxiliary service name and it must match the regular expression a-z ?. This means that the first character must be a lowercase letter, and all the following characters must be hyphens, lowercase letters, or digits, except the last character, which cannot be a hyphen. |
EndpointProtocol
Protocols available for serving the metastore service endpoint.
| Enums | |
|---|---|
ENDPOINT_PROTOCOL_UNSPECIFIED
|
The protocol is not set. |
THRIFT
|
Use the legacy Apache Thrift protocol for the metastore service endpoint. |
GRPC
|
Use the modernized gRPC protocol for the metastore service endpoint. |
KerberosConfig
Configuration information for a Kerberos principal.
| Fields | |
|---|---|
keytab
|
A Kerberos keytab file that can be used to authenticate a service principal with a Kerberos Key Distribution Center (KDC). |
principal
|
A Kerberos principal that exists in the both the keytab the KDC to authenticate as. A typical principal is of the form |
krb5_config_gcs_uri
|
A Cloud Storage URI that specifies the path to a krb5.conf file. It is of the form |
Lake
Represents a Lake resource
| Fields | |
|---|---|
name
|
The Lake resource name. Example: |
LatestBackup
The details of the latest scheduled backup.
| Fields | |
|---|---|
backup_id
|
Output only. The ID of an in-progress scheduled backup. Empty if no backup is in progress. |
start_time
|
Output only. The time when the backup was started. |
state
|
Output only. The current state of the backup. |
duration
|
Output only. The duration of the backup completion. |
State
The current state of the backup.
| Enums | |
|---|---|
STATE_UNSPECIFIED
|
The state of the backup is unknown. |
IN_PROGRESS
|
The backup is in progress. |
SUCCEEDED
|
The backup completed. |
FAILED
|
The backup failed. |
LimitConfig
Represents the autoscaling limit configuration of a metastore service.
| Fields | |
|---|---|
max_scaling_factor
|
Optional. The highest scaling factor that the service should be autoscaled to. |
min_scaling_factor
|
Optional. The lowest scaling factor that the service should be autoscaled to. |
ListBackupsRequest
Request message for DataprocMetastore.ListBackups
.
parent
string
Required. The relative resource name of the service whose backups to list, in the following form:
projects/{project_number}/locations/{location_id}/services/{service_id}/backups
.
Authorization requires the following IAM
permission on the specified resource parent
:
-
metastore.backups.list
page_size
int32
Optional. The maximum number of backups to return. The response may contain less than the maximum number. If unspecified, no more than 500 backups are returned. The maximum value is 1000; values above 1000 are changed to 1000.
page_token
string
Optional. A page token, received from a previous DataprocMetastore.ListBackups
call. Provide this token to retrieve the subsequent page.
To retrieve the first page, supply an empty page token.
When paginating, other parameters provided to DataprocMetastore.ListBackups
must match the call that provided the page token.
filter
string
Optional. The filter to apply to list results.
order_by
string
Optional. Specify the ordering of results as described in Sorting Order . If not specified, the results will be sorted in the default order.
ListBackupsResponse
Response message for DataprocMetastore.ListBackups
.
| Fields | |
|---|---|
backups[]
|
The backups of the specified service. |
next_page_token
|
A token that can be sent as |
unreachable[]
|
Locations that could not be reached. |
ListFederationsRequest
Request message for ListFederations.
parent
string
Required. The relative resource name of the location of metastore federations to list, in the following form: projects/{project_number}/locations/{location_id}
.
Authorization requires the following IAM
permission on the specified resource parent
:
-
metastore.federations.list
page_size
int32
Optional. The maximum number of federations to return. The response may contain less than the maximum number. If unspecified, no more than 500 services are returned. The maximum value is 1000; values above 1000 are changed to 1000.
page_token
string
Optional. A page token, received from a previous ListFederationServices call. Provide this token to retrieve the subsequent page.
To retrieve the first page, supply an empty page token.
When paginating, other parameters provided to ListFederationServices must match the call that provided the page token.
filter
string
Optional. The filter to apply to list results.
order_by
string
Optional. Specify the ordering of results as described in Sorting Order . If not specified, the results will be sorted in the default order.
ListFederationsResponse
Response message for ListFederations
| Fields | |
|---|---|
federations[]
|
The services in the specified location. |
next_page_token
|
A token that can be sent as |
unreachable[]
|
Locations that could not be reached. |
ListMetadataImportsRequest
Request message for DataprocMetastore.ListMetadataImports
.
parent
string
Required. The relative resource name of the service whose metadata imports to list, in the following form:
projects/{project_number}/locations/{location_id}/services/{service_id}/metadataImports
.
Authorization requires the following IAM
permission on the specified resource parent
:
-
metastore.imports.list
page_size
int32
Optional. The maximum number of imports to return. The response may contain less than the maximum number. If unspecified, no more than 500 imports are returned. The maximum value is 1000; values above 1000 are changed to 1000.
page_token
string
Optional. A page token, received from a previous DataprocMetastore.ListServices
call. Provide this token to retrieve the subsequent page.
To retrieve the first page, supply an empty page token.
When paginating, other parameters provided to DataprocMetastore.ListServices
must match the call that provided the page token.
filter
string
Optional. The filter to apply to list results.
order_by
string
Optional. Specify the ordering of results as described in Sorting Order . If not specified, the results will be sorted in the default order.
ListMetadataImportsResponse
Response message for DataprocMetastore.ListMetadataImports
.
| Fields | |
|---|---|
metadata_imports[]
|
The imports in the specified service. |
next_page_token
|
A token that can be sent as |
unreachable[]
|
Locations that could not be reached. |
ListMigrationExecutionsRequest
Request message for DataprocMetastore.ListMigrationExecutions
.
parent
string
Required. The relative resource name of the service whose migration executions to list, in the following form:
projects/{project_number}/locations/{location_id}/services/{service_id}/migrationExecutions
.
Authorization requires the following IAM
permission on the specified resource parent
:
-
metastore.migrations.list
page_size
int32
Optional. The maximum number of migration executions to return. The response may contain less than the maximum number. If unspecified, no more than 500 migration executions are returned. The maximum value is 1000; values above 1000 are changed to 1000.
page_token
string
Optional. A page token, received from a previous DataprocMetastore.ListMigrationExecutions
call. Provide this token to retrieve the subsequent page.
To retrieve the first page, supply an empty page token.
When paginating, other parameters provided to DataprocMetastore.ListMigrationExecutions
must match the call that provided the page token.
filter
string
Optional. The filter to apply to list results.
order_by
string
Optional. Specify the ordering of results as described in Sorting Order . If not specified, the results will be sorted in the default order.
ListMigrationExecutionsResponse
Response message for DataprocMetastore.ListMigrationExecutions
.
| Fields | |
|---|---|
migration_executions[]
|
The migration executions on the specified service. |
next_page_token
|
A token that can be sent as |
unreachable[]
|
Locations that could not be reached. |
ListServicesRequest
Request message for DataprocMetastore.ListServices
.
parent
string
Required. The relative resource name of the location of metastore services to list, in the following form:
projects/{project_number}/locations/{location_id}
.
Authorization requires the following IAM
permission on the specified resource parent
:
-
metastore.services.list
page_size
int32
Optional. The maximum number of services to return. The response may contain less than the maximum number. If unspecified, no more than 500 services are returned. The maximum value is 1000; values above 1000 are changed to 1000.
page_token
string
Optional. A page token, received from a previous DataprocMetastore.ListServices
call. Provide this token to retrieve the subsequent page.
To retrieve the first page, supply an empty page token.
When paginating, other parameters provided to DataprocMetastore.ListServices
must match the call that provided the page token.
filter
string
Optional. The filter to apply to list results.
order_by
string
Optional. Specify the ordering of results as described in Sorting Order . If not specified, the results will be sorted in the default order.
ListServicesResponse
Response message for DataprocMetastore.ListServices
.
| Fields | |
|---|---|
services[]
|
The services in the specified location. |
next_page_token
|
A token that can be sent as |
unreachable[]
|
Locations that could not be reached. |
LocationMetadata
Metadata about the service in a location.
| Fields | |
|---|---|
supported_hive_metastore_versions[]
|
The versions of Hive Metastore that can be used when creating a new metastore service in this location. The server guarantees that exactly one |
multi_region_metadata
|
The multi-region metadata if the current region is a multi-region. |
custom_region_metadata[]
|
Possible configurations supported if the current region is a custom region. |
CustomRegionMetadata
Metadata about a custom region. This is only populated if the region is a custom region. For single/multi regions, it will be empty.
| Fields | |
|---|---|
required_read_write_regions[]
|
The read-write regions for this custom region. |
optional_read_only_regions[]
|
The read-only regions for this custom region. |
witness_region
|
The Spanner witness region for this custom region. |
HiveMetastoreVersion
A specification of a supported version of the Hive Metastore software.
| Fields | |
|---|---|
version
|
The semantic version of the Hive Metastore software. |
is_default
|
Whether |
MultiRegionMetadata
The metadata for the multi-region that includes the constituent regions. The metadata is only populated if the region is multi-region. For single region or custom dual region, it will be empty.
| Fields | |
|---|---|
constituent_regions[]
|
The regions constituting the multi-region. |
MaintenanceWindow
Maintenance window. This specifies when Dataproc Metastore may perform system maintenance operation to the service.
| Fields | |
|---|---|
hour_of_day
|
The hour of day (0-23) when the window starts. |
day_of_week
|
The day of week, when the window starts. |
MetadataExport
The details of a metadata export operation.
start_time
Output only. The time when the export started.
end_time
Output only. The time when the export ended.
state
Output only. The current state of the export.
database_dump_type
Output only. The type of the database dump.
Union field destination
.
destination
can be only one of the following:
destination_gcs_uri
string
Output only. A Cloud Storage URI of a folder that metadata are exported to, in the form of gs://<bucket_name>/<path_inside_bucket>/<export_folder>
, where <export_folder>
is automatically generated.
State
The current state of the metadata export.
| Enums | |
|---|---|
STATE_UNSPECIFIED
|
The state of the metadata export is unknown. |
RUNNING
|
The metadata export is running. |
SUCCEEDED
|
The metadata export completed successfully. |
FAILED
|
The metadata export failed. |
CANCELLED
|
The metadata export is cancelled. |
MetadataImport
A metastore resource that imports metadata.
name
string
Immutable. The relative resource name of the metadata import, of the form:
projects/{project_number}/locations/{location_id}/services/{service_id}/metadataImports/{metadata_import_id}
.
description
string
The description of the metadata import.
create_time
Output only. The time when the metadata import was started.
update_time
Output only. The time when the metadata import was last updated.
end_time
Output only. The time when the metadata import finished.
state
Output only. The current state of the metadata import.
metadata
. The metadata to be imported. metadata
can be only one of the following:database_dump
Immutable. A database dump from a pre-existing metastore's database.
DatabaseDump
A specification of the location of and metadata about a database dump from a relational database management system.
| Fields | |
|---|---|
database_type
|
The type of the database. |
gcs_uri
|
A Cloud Storage object or folder URI that specifies the source from which to import metadata. It must begin with |
source_database
|
The name of the source database. |
type
|
Optional. The type of the database dump. If unspecified, defaults to |
DatabaseType
The type of the database.
| Enums | |
|---|---|
DATABASE_TYPE_UNSPECIFIED
|
The type of the source database is unknown. |
MYSQL
|
The type of the source database is MySQL. |
State
The current state of the metadata import.
| Enums | |
|---|---|
STATE_UNSPECIFIED
|
The state of the metadata import is unknown. |
RUNNING
|
The metadata import is running. |
SUCCEEDED
|
The metadata import completed successfully. |
UPDATING
|
The metadata import is being updated. |
FAILED
|
The metadata import failed, and attempted metadata changes were rolled back. |
MetadataIntegration
Specifies how metastore metadata should be integrated with external services.
| Fields | |
|---|---|
data_catalog_config
|
Optional. The integration config for the Data Catalog service. |
dataplex_config
|
The integration config for the Dataplex service. |
MetadataManagementActivity
The metadata management activities of the metastore service.
| Fields | |
|---|---|
metadata_exports[]
|
Output only. The latest metadata exports of the metastore service. |
restores[]
|
Output only. The latest restores of the metastore service. |
MigrationExecution
The details of a migration execution resource.
name
string
Output only. The relative resource name of the migration execution, in the following form: projects/{project_number}/locations/{location_id}/services/{service_id}/migrationExecutions/{migration_execution_id}
create_time
Output only. The time when the migration execution was started.
end_time
Output only. The time when the migration execution finished.
state
Output only. The current state of the migration execution.
phase
Output only. The current phase of the migration execution.
state_message
string
Output only. Additional information about the current state of the migration execution.
migration_config
. Configuration information required by Dataproc Metastore for starting the migration process migration_config
can be only one of the following:cloud_sql_migration_config
Configuration information specific to migrating from self-managed hive metastore on Google Cloud using Cloud SQL as the backend database to Dataproc Metastore.
CloudSQLMigrationConfig
Configuration information for migrating from self-managed hive metastore on Google Cloud using Cloud SQL as the backend database to Dataproc Metastore.
| Fields | |
|---|---|
cdc_config
|
Required. Configuration information to start the Change Data Capture (CDC) streams from customer database to backend database of Dataproc Metastore. Dataproc Metastore switches to using its backend database after the cutover phase of migration. |
cloud_sql_connection_config
|
Required. Configuration information to establish customer database connection before the cutover phase of migration |
CdcConfig
Configuration information to start the Change Data Capture (CDC) streams from customer database to backend database of Dataproc Metastore.
| Fields | |
|---|---|
vpc_network
|
Required. Fully qualified name of the Cloud SQL instance's VPC network or the shared VPC network that Datastream will peer to, in the following format: |
subnet_ip_range
|
Required. A /29 CIDR IP range for peering with datastream. |
username
|
Required. The username that the Datastream service should use for the MySQL connection. |
password
|
Required. Input only. The password for the user that Datastream service should use for the MySQL connection. This field is not returned on request. |
reverse_proxy_subnet
|
Required. The URL of the subnetwork resource to create the VM instance hosting the reverse proxy in. More context in https://cloud.google.com/datastream/docs/private-connectivity#reverse-csql-proxy
The subnetwork should reside in the network provided in the request that Datastream will peer to and should be in the same region as Datastream, in the following format. |
bucket
|
Optional. The bucket to write the intermediate stream event data in. The bucket name must be without any prefix like "gs://". See the bucket naming requirements . This field is optional. If not set, the Artifacts Cloud Storage bucket will be used. |
root_path
|
Optional. The root path inside the Cloud Storage bucket. The stream event data will be written to this path. The default value is |
CloudSQLConnectionConfig
Configuration information to establish customer database connection before the cutover phase of migration
| Fields | |
|---|---|
instance_connection_name
|
Required. Cloud SQL database connection name (project_id:region:instance_name) |
ip_address
|
Required. The private IP address of the Cloud SQL instance. |
port
|
Required. The network port of the database. |
hive_database_name
|
Required. The hive database name. |
username
|
Required. The username that Dataproc Metastore service will use to connect to the database. |
password
|
Required. Input only. The password for the user that Dataproc Metastore service will be using to connect to the database. This field is not returned on request. |
proxy_subnet
|
Required. The relative resource name of the subnetwork to deploy the SOCKS5 proxy service in. The subnetwork should reside in a network through which the Cloud SQL instance is accessible. The resource name should be in the format, |
nat_subnet
|
Required. The relative resource name of the subnetwork to be used for Private Service Connect. Note that this cannot be a regular subnet and is used only for NAT. ( https://cloud.google.com/vpc/docs/about-vpc-hosted-services#psc-subnets
) This subnet is used to publish the SOCKS5 proxy service. The subnet size must be at least |
Phase
The current phase of the migration execution.
| Enums | |
|---|---|
PHASE_UNSPECIFIED
|
The phase of the migration execution is unknown. |
REPLICATION
|
Replication phase refers to the migration phase when Dataproc Metastore is running a pipeline to replicate changes in the customer database to its backend database. During this phase, Dataproc Metastore uses the customer database as the hive metastore backend database. |
CUTOVER
|
Cutover phase refers to the migration phase when Dataproc Metastore switches to using its own backend database. Migration enters this phase when customer is done migrating all their clusters/workloads to Dataproc Metastore and triggers CompleteMigration
. |
State
The current state of the migration execution.
| Enums | |
|---|---|
STATE_UNSPECIFIED
|
The state of the migration execution is unknown. |
STARTING
|
The migration execution is starting. |
RUNNING
|
The migration execution is running. |
CANCELLING
|
The migration execution is in the process of being cancelled. |
AWAITING_USER_ACTION
|
The migration execution is awaiting user action. |
SUCCEEDED
|
The migration execution has completed successfully. |
FAILED
|
The migration execution has failed. |
CANCELLED
|
The migration execution is cancelled. |
DELETING
|
The migration execution is being deleted. |
MoveTableToDatabaseRequest
Request message for DataprocMetastore.MoveTableToDatabase
.
service
string
Required. The relative resource name of the metastore service to mutate metadata, in the following format:
projects/{project_id}/locations/{location_id}/services/{service_id}
.
Authorization requires the following IAM
permission on the specified resource service
:
-
metastore.services.mutateMetadata
table_name
string
Required. The name of the table to be moved.
db_name
string
Required. The name of the database where the table resides.
destination_db_name
string
Required. The name of the database where the table should be moved.
MoveTableToDatabaseResponse
This type has no fields.
Response message for DataprocMetastore.MoveTableToDatabase
.
MultiRegionConfig
The multi-region config for the Dataproc Metastore service.
| Fields | |
|---|---|
certificates[]
|
Output only. The list of root CA certificates that a gRPC client uses to connect to a multi-regional Dataproc Metastore service. |
custom_region_config
|
|
CustomRegionConfig
Custom configuration used to specify regions that the metastore service runs in. Currently only supported in the us
multi-region.
| Fields | |
|---|---|
read_write_regions[]
|
Required. The list of read-write regions where the metastore service runs in. These regions should be part (or subset) of the multi-region. |
read_only_regions[]
|
Optional. The list of read-only regions where the metastore service runs in. These regions should be part (or subset) of the multi-region. |
RootCACertificate
A gRPC client must install all root CA certificates to connect to a multi-regional Dataproc Metastore service and achieve failover.
| Fields | |
|---|---|
certificate
|
The root CA certificate in PEM format. The maximum length is 65536 bytes. |
expiration_time
|
The certificate expiration time in timestamp format. |
NetworkConfig
Network configuration for the Dataproc Metastore service.
| Fields | |
|---|---|
consumers[]
|
Immutable. The consumer-side network configuration for the Dataproc Metastore instance. |
custom_routes_enabled
|
Enables custom routes to be imported and exported for the Dataproc Metastore service's peered VPC network. |
Consumer
Contains information of the customer's network configurations.
endpoint_uri
string
Output only. The URI of the endpoint used to access the metastore service.
endpoint_location
string
Output only. The location of the endpoint URI. Format: projects/{project}/locations/{location}
.
Union field vpc_resource
.
vpc_resource
can be only one of the following:
subnetwork
string
Immutable. The subnetwork of the customer project from which an IP address is reserved and used as the Dataproc Metastore service's endpoint. It is accessible to hosts in the subnet and to all hosts in a subnet in the same region and same network. There must be at least one IP address available in the subnet's primary range. The subnet is specified in the following form:
projects/{project_number}/regions/{region_id}/subnetworks/{subnetwork_id}
OperationMetadata
Represents the metadata of a long-running operation.
| Fields | |
|---|---|
create_time
|
Output only. The time the operation was created. |
end_time
|
Output only. The time the operation finished running. |
target
|
Output only. Server-defined resource path for the target of the operation. |
verb
|
Output only. Name of the verb executed by the operation. |
status_message
|
Output only. Human-readable status of the operation, if any. |
requested_cancellation
|
Output only. Identifies whether the caller has requested cancellation of the operation. Operations that have successfully been cancelled have [Operation.error][] value with a |
api_version
|
Output only. API version used to start the operation. |
QueryMetadataRequest
Request message for DataprocMetastore.QueryMetadata
.
service
string
Required. The relative resource name of the metastore service to query metadata, in the following format:
projects/{project_id}/locations/{location_id}/services/{service_id}
.
Authorization requires the following IAM
permission on the specified resource service
:
-
metastore.services.queryMetadata
query
string
Required. A read-only SQL query to execute against the metadata database. The query cannot change or mutate the data.
QueryMetadataResponse
Response message for DataprocMetastore.QueryMetadata
.
| Fields | |
|---|---|
result_manifest_uri
|
The manifest URI is link to a JSON instance in Cloud Storage. This instance manifests immediately along with QueryMetadataResponse. The content of the URI is not retriable until the long-running operation query against the metadata finishes. |
RemoveIamPolicyRequest
Request message for DataprocMetastore.RemoveIamPolicy
.
| Fields | |
|---|---|
resource
|
Required. The relative resource name of the dataplane resource to remove IAM policy, in the following form: |
asynchronous
|
Optional. Removes IAM policy attached to database or table asynchronously when it is set. The default is false. |
RemoveIamPolicyResponse
Response message for DataprocMetastore.RemoveIamPolicy
.
| Fields | |
|---|---|
success
|
True if the policy is successfully removed. |
Restore
The details of a metadata restore operation.
| Fields | |
|---|---|
start_time
|
Output only. The time when the restore started. |
end_time
|
Output only. The time when the restore ended. |
state
|
Output only. The current state of the restore. |
backup
|
Output only. The relative resource name of the metastore service backup to restore from, in the following form: |
type
|
Output only. The type of restore. |
details
|
Output only. The restore details containing the revision of the service to be restored to, in format of JSON. |
backup_location
|
Optional. A Cloud Storage URI specifying where the backup artifacts are stored, in the format |
RestoreType
The type of restore. If unspecified, defaults to METADATA_ONLY
.
| Enums | |
|---|---|
RESTORE_TYPE_UNSPECIFIED
|
The restore type is unknown. |
FULL
|
The service's metadata and configuration are restored. |
METADATA_ONLY
|
Only the service's metadata is restored. |
State
The current state of the restore.
| Enums | |
|---|---|
STATE_UNSPECIFIED
|
The state of the metadata restore is unknown. |
RUNNING
|
The metadata restore is running. |
SUCCEEDED
|
The metadata restore completed successfully. |
FAILED
|
The metadata restore failed. |
CANCELLED
|
The metadata restore is cancelled. |
RestoreServiceRequest
Request message for [DataprocMetastore.Restore][].
service
string
Required. The relative resource name of the metastore service to run restore, in the following form:
projects/{project_id}/locations/{location_id}/services/{service_id}
.
Authorization requires the following IAM
permission on the specified resource service
:
-
metastore.services.restore
backup
string
Optional. The relative resource name of the metastore service backup to restore from, in the following form:
projects/{project_id}/locations/{location_id}/services/{service_id}/backups/{backup_id}
. Mutually exclusive with backup_location
, and exactly one of the two must be set.
Authorization requires the following IAM
permission on the specified resource backup
:
-
metastore.backups.use
backup_location
string
Optional. A Cloud Storage URI specifying the location of the backup artifacts, namely - backup avro files under "avro/", backup_metastore.json and service.json, in the following form:
gs://<path_to_backup>
. Mutually exclusive with backup
, and exactly one of the two must be set.
restore_type
Optional. The type of restore. If unspecified, defaults to METADATA_ONLY
.
request_id
string
Optional. A request ID. Specify a unique request ID to allow the server to ignore the request if it has completed. The server will ignore subsequent requests that provide a duplicate request ID for at least 60 minutes after the first request.
For example, if an initial request times out, followed by another request with the same request ID, the server ignores the second request to prevent the creation of duplicate commitments.
The request ID must be a valid UUID . A zero UUID (00000000-0000-0000-0000-000000000000) is not supported.
ScalingConfig
Represents the scaling configuration of a metastore service.
autoscaling_config
Optional. The autoscaling configuration.
scaling_model
. Represents either a predetermined instance size or a numeric scaling factor. scaling_model
can be only one of the following:instance_size
An enum of readable instance sizes, with each instance size mapping to a float value (e.g. InstanceSize.EXTRA_SMALL = scaling_factor(0.1))
scaling_factor
float
Scaling factor, increments of 0.1 for values less than 1.0, and increments of 1.0 for values greater than 1.0.
InstanceSize
Metastore instance sizes.
| Enums | |
|---|---|
INSTANCE_SIZE_UNSPECIFIED
|
Unspecified instance size |
EXTRA_SMALL
|
Extra small instance size, maps to a scaling factor of 0.1. |
SMALL
|
Small instance size, maps to a scaling factor of 0.5. |
MEDIUM
|
Medium instance size, maps to a scaling factor of 1.0. |
LARGE
|
Large instance size, maps to a scaling factor of 3.0. |
EXTRA_LARGE
|
Extra large instance size, maps to a scaling factor of 6.0. |
ScheduledBackup
This specifies the configuration of scheduled backup.
| Fields | |
|---|---|
enabled
|
Optional. Defines whether the scheduled backup is enabled. The default value is false. |
cron_schedule
|
Optional. The scheduled interval in Cron format, see https://en.wikipedia.org/wiki/Cron The default is empty: scheduled backup is not enabled. Must be specified to enable scheduled backups. |
time_zone
|
Optional. Specifies the time zone to be used when interpreting cron_schedule. Must be a time zone name from the time zone database ( https://en.wikipedia.org/wiki/List_of_tz_database_time_zones) , e.g. America/Los_Angeles or Africa/Abidjan. If left unspecified, the default is UTC. |
next_scheduled_time
|
Output only. The time when the next backups execution is scheduled to start. |
backup_location
|
Optional. A Cloud Storage URI of a folder, in the format |
latest_backup
|
Output only. The details of the latest scheduled backup. |
Secret
A securely stored value.
Union field value
.
value
can be only one of the following:
cloud_secret
string
The relative resource name of a Secret Manager secret version, in the following form:
projects/{project_number}/secrets/{secret_id}/versions/{version_id}
.
Service
A managed metastore service that serves metadata queries.
name
string
Immutable. The relative resource name of the metastore service, in the following format:
projects/{project_number}/locations/{location_id}/services/{service_id}
.
create_time
Output only. The time when the metastore service was created.
update_time
Output only. The time when the metastore service was last updated.
labels
map<string, string>
User-defined labels for the metastore service.
network
string
Immutable. The relative resource name of the VPC network on which the instance can be accessed. It is specified in the following form:
projects/{project_number}/global/networks/{network_id}
.
endpoint_uri
string
Output only. The URI of the endpoint used to access the metastore service.
port
int32
The TCP port at which the metastore service is reached. Default: 9083.
state
Output only. The current state of the metastore service.
state_message
string
Output only. Additional information about the current state of the metastore service, if available.
artifact_gcs_uri
string
Output only. A Cloud Storage URI (starting with gs://
) that specifies where artifacts related to the metastore service are stored.
tier
The tier of the service.
metadata_integration
Optional. The setting that defines how metastore metadata should be integrated with external services and systems.
maintenance_window
The one hour maintenance window of the metastore service. This specifies when the service can be restarted for maintenance purposes in UTC time. Maintenance window is not needed for services with the SPANNER database type.
uid
string
Output only. The globally unique resource identifier of the metastore service.
metadata_management_activity
Output only. The metadata management activities of the metastore service.
release_channel
Immutable. The release channel of the service. If unspecified, defaults to STABLE
.
encryption_config
Immutable. Information used to configure the Dataproc Metastore service to encrypt customer data at rest. Cannot be updated.
network_config
The configuration specifying the network settings for the Dataproc Metastore service.
database_type
Immutable. The database type that the Metastore service stores its data.
telemetry_config
The configuration specifying telemetry settings for the Dataproc Metastore service. If unspecified defaults to JSON
.
scaling_config
Scaling configuration of the metastore service.
multi_region_config
Optional. Specifies the multi-region configuration information for the Hive metastore service.
scheduled_backup
Optional. The configuration of scheduled backup for the metastore service.
deletion_protection
bool
Optional. Indicates if the dataproc metastore should be protected against accidental deletions.
metastore_config
. Configuration properties specific to the underlying metastore service technology (the software that serves metastore queries). metastore_config
can be only one of the following:hive_metastore_config
Configuration information specific to running Hive metastore software as the metastore service.
DatabaseType
The backend database type for the metastore service.
| Enums | |
|---|---|
DATABASE_TYPE_UNSPECIFIED
|
The DATABASE_TYPE is not set. |
MYSQL
|
MySQL is used to persist the metastore data. |
SPANNER
|
Spanner is used to persist the metastore data. |
ReleaseChannel
Release channels bundle features of varying levels of stability. Newer features may be introduced initially into less stable release channels and can be automatically promoted into more stable release channels.
| Enums | |
|---|---|
RELEASE_CHANNEL_UNSPECIFIED
|
Release channel is not specified. |
CANARY
|
The CANARY
release channel contains the newest features, which may be unstable and subject to unresolved issues with no known workarounds. Services using the CANARY
release channel are not subject to any SLAs. |
STABLE
|
The STABLE
release channel contains features that are considered stable and have been validated for production use. |
State
The current state of the metastore service.
| Enums | |
|---|---|
STATE_UNSPECIFIED
|
The state of the metastore service is unknown. |
CREATING
|
The metastore service is in the process of being created. |
ACTIVE
|
The metastore service is running and ready to serve queries. |
SUSPENDING
|
The metastore service is entering suspension. Its query-serving availability may cease unexpectedly. |
SUSPENDED
|
The metastore service is suspended and unable to serve queries. |
UPDATING
|
The metastore service is being updated. It remains usable but cannot accept additional update requests or be deleted at this time. |
DELETING
|
The metastore service is undergoing deletion. It cannot be used. |
ERROR
|
The metastore service has encountered an error and cannot be used. The metastore service should be deleted. |
AUTOSCALING
|
The Dataproc Metastore service 2 is being scaled up or down. |
MIGRATING
|
The metastore service is processing a managed migration. |
Tier
Available service tiers.
| Enums | |
|---|---|
TIER_UNSPECIFIED
|
The tier is not set. |
DEVELOPER
|
The developer tier provides limited scalability and no fault tolerance. Good for low-cost proof-of-concept. |
ENTERPRISE
|
The enterprise tier provides multi-zone high availability, and sufficient scalability for enterprise-level Dataproc Metastore workloads. |
StartMigrationRequest
Request message for DataprocMetastore.StartMigration
.
service
string
Required. The relative resource name of the metastore service to start migrating to, in the following format:
projects/{project_id}/locations/{location_id}/services/{service_id}
.
Authorization requires the following IAM
permission on the specified resource service
:
-
metastore.migrations.start
migration_execution
Required. The configuration details for the migration.
request_id
string
Optional. A request ID. Specify a unique request ID to allow the server to ignore the request if it has completed. The server will ignore subsequent requests that provide a duplicate request ID for at least 60 minutes after the first request.
For example, if an initial request times out, followed by another request with the same request ID, the server ignores the second request to prevent the creation of duplicate commitments.
The request ID must be a valid UUID A zero UUID (00000000-0000-0000-0000-000000000000) is not supported.
TelemetryConfig
Telemetry Configuration for the Dataproc Metastore service.
| Fields | |
|---|---|
log_format
|
The output format of the Dataproc Metastore service's logs. |
LogFormat
| Enums | |
|---|---|
LOG_FORMAT_UNSPECIFIED
|
The LOG_FORMAT is not set. |
LEGACY
|
Logging output uses the legacy textPayload
format. |
JSON
|
Logging output uses the jsonPayload
format. |
UpdateFederationRequest
Request message for UpdateFederation.
update_mask
Required. A field mask used to specify the fields to be overwritten in the metastore federation resource by the update. Fields specified in the update_mask
are relative to the resource (not to the full request). A field is overwritten if it is in the mask.
federation
Required. The metastore federation to update. The server only merges fields in the service if they are specified in update_mask
.
The metastore federation's name
field is used to identify the metastore service to be updated.
Authorization requires the following IAM
permission on the specified resource federation
:
-
metastore.federations.update
request_id
string
Optional. A request ID. Specify a unique request ID to allow the server to ignore the request if it has completed. The server will ignore subsequent requests that provide a duplicate request ID for at least 60 minutes after the first request.
For example, if an initial request times out, followed by another request with the same request ID, the server ignores the second request to prevent the creation of duplicate commitments.
The request ID must be a valid UUID A zero UUID (00000000-0000-0000-0000-000000000000) is not supported.
UpdateMetadataImportRequest
Request message for DataprocMetastore.UpdateMetadataImport
.
update_mask
Required. A field mask used to specify the fields to be overwritten in the metadata import resource by the update. Fields specified in the update_mask
are relative to the resource (not to the full request). A field is overwritten if it is in the mask.
metadata_import
Required. The metadata import to update. The server only merges fields in the import if they are specified in update_mask
.
The metadata import's name
field is used to identify the metastore import to be updated.
Authorization requires the following IAM
permission on the specified resource metadataImport
:
-
metastore.imports.update
request_id
string
Optional. A request ID. Specify a unique request ID to allow the server to ignore the request if it has completed. The server will ignore subsequent requests that provide a duplicate request ID for at least 60 minutes after the first request.
For example, if an initial request times out, followed by another request with the same request ID, the server ignores the second request to prevent the creation of duplicate commitments.
The request ID must be a valid UUID A zero UUID (00000000-0000-0000-0000-000000000000) is not supported.
UpdateServiceRequest
Request message for DataprocMetastore.UpdateService
.
update_mask
Required. A field mask used to specify the fields to be overwritten in the metastore service resource by the update. Fields specified in the update_mask
are relative to the resource (not to the full request). A field is overwritten if it is in the mask.
service
Required. The metastore service to update. The server only merges fields in the service if they are specified in update_mask
.
The metastore service's name
field is used to identify the metastore service to be updated.
Authorization requires the following IAM
permission on the specified resource service
:
-
metastore.services.update
request_id
string
Optional. A request ID. Specify a unique request ID to allow the server to ignore the request if it has completed. The server will ignore subsequent requests that provide a duplicate request ID for at least 60 minutes after the first request.
For example, if an initial request times out, followed by another request with the same request ID, the server ignores the second request to prevent the creation of duplicate commitments.
The request ID must be a valid UUID A zero UUID (00000000-0000-0000-0000-000000000000) is not supported.
