Console
    Start free

    MCP Tools Reference: datastream

    Tool: get_stream

    Get details of the stream specified by the provided resource name parameter.

    • The resource name parameter is in the form: projects/{project name}/locations/{location}/streams/{stream name} , for example: projects/my-project/locations/us-central1/streams/my-streams .

    The following sample demonstrate how to use curl to invoke the get_stream MCP tool.

    Curl Request 
      
curl  
--location  
 'https://datastream.googleapis.com/mcp' 
  
 \ 
--header  
 'content-type: application/json' 
  
 \ 
--header  
 'accept: application/json, text/event-stream' 
  
 \ 
--data  
 '{ 
 "method": "tools/call", 
 "params": { 
 "name": "get_stream", 
 "arguments": { 
 // provide these details according to the tool' 
s  
MCP  
specification  
 } 
  
 } 
,  
 "jsonrpc" 
:  
 "2.0" 
,  
 "id" 
:  
 1 
 } 
 '

    Input Schema

    Request message for getting a stream.

    GetStreamRequest

    JSON representation 
     { 
 "name" 
 : 
 string 
 }
    Fields
    name

    string

    Required. The name of the stream resource to get.

    Output Schema

    A resource representing streaming data from a source to a destination.

    Stream

    JSON representation 
     { 
 "name" 
 : 
 string 
 , 
 "createTime" 
 : 
 string 
 , 
 "updateTime" 
 : 
 string 
 , 
 "labels" 
 : 
 { 
 string 
 : 
 string 
 , 
 ... 
 } 
 , 
 "displayName" 
 : 
 string 
 , 
 "sourceConfig" 
 : 
 { 
 object (  SourceConfig 
 
) 
 } 
 , 
 "destinationConfig" 
 : 
 { 
 object (  DestinationConfig 
 
) 
 } 
 , 
 "state" 
 : 
 enum ( State 
) 
 , 
 "errors" 
 : 
 [ 
 { 
 object (  Error 
 
) 
 } 
 ] 
 , 
 "lastRecoveryTime" 
 : 
 string 
 , 
 "ruleSets" 
 : 
 [ 
 { 
 object (  RuleSet 
 
) 
 } 
 ] 
 , 
 // Union field backfill_strategy 
can be only one of the following: 
 "backfillAll" 
 : 
 { 
 object (  BackfillAllStrategy 
 
) 
 } 
 , 
 "backfillNone" 
 : 
 { 
 object ( BackfillNoneStrategy 
) 
 } 
 // End of list of possible types for union field backfill_strategy 
. 
 // Union field _customer_managed_encryption_key 
can be only one of the 
 // following: 
 "customerManagedEncryptionKey" 
 : 
 string 
 // End of list of possible types for union field 
 // _customer_managed_encryption_key 
. 
 // Union field _satisfies_pzs 
can be only one of the following: 
 "satisfiesPzs" 
 : 
 boolean 
 // End of list of possible types for union field _satisfies_pzs 
. 
 // Union field _satisfies_pzi 
can be only one of the following: 
 "satisfiesPzi" 
 : 
 boolean 
 // End of list of possible types for union field _satisfies_pzi 
. 
 }
    Fields
    name

    string

    Output only. Identifier. The stream's name.

    createTime

    string ( Timestamp format)

    Output only. The creation time of the stream.

    Uses RFC 3339, where generated output will always be Z-normalized and use 0, 3, 6 or 9 fractional digits. Offsets other than "Z" are also accepted. Examples: "2014-10-02T15:01:23Z" , "2014-10-02T15:01:23.045123456Z" or "2014-10-02T15:01:23+05:30" .

    updateTime

    string ( Timestamp format)

    Output only. The last update time of the stream.

    Uses RFC 3339, where generated output will always be Z-normalized and use 0, 3, 6 or 9 fractional digits. Offsets other than "Z" are also accepted. Examples: "2014-10-02T15:01:23Z" , "2014-10-02T15:01:23.045123456Z" or "2014-10-02T15:01:23+05:30" .

    labels

    map (key: string, value: string)

    Labels.

    An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" } .

    displayName

    string

    Required. Display name.

    sourceConfig

    object ( SourceConfig )

    Required. Source connection profile configuration.

    destinationConfig

    object ( DestinationConfig )

    Required. Destination connection profile configuration.

    state

    enum ( State )

    The state of the stream.

    errors[]

    object ( Error )

    Output only. Errors on the Stream.

    lastRecoveryTime

    string ( Timestamp format)

    Output only. If the stream was recovered, the time of the last recovery. Note: This field is currently experimental.

    Uses RFC 3339, where generated output will always be Z-normalized and use 0, 3, 6 or 9 fractional digits. Offsets other than "Z" are also accepted. Examples: "2014-10-02T15:01:23Z" , "2014-10-02T15:01:23.045123456Z" or "2014-10-02T15:01:23+05:30" .

    ruleSets[]

    object ( RuleSet )

    Optional. Rule sets to apply to the stream.

    Union field backfill_strategy . Stream backfill strategy. backfill_strategy can be only one of the following:
    backfillAll

    object ( BackfillAllStrategy )

    Automatically backfill objects included in the stream source configuration. Specific objects can be excluded.

    backfillNone

    object ( BackfillNoneStrategy )

    Do not automatically backfill any objects.

    Union field _customer_managed_encryption_key .

    _customer_managed_encryption_key can be only one of the following:

    customerManagedEncryptionKey

    string

    Immutable. A reference to a KMS encryption key. If provided, it will be used to encrypt the data. If left blank, data will be encrypted using an internal Stream-specific encryption key provisioned through KMS.

    Union field _satisfies_pzs .

    _satisfies_pzs can be only one of the following:

    satisfiesPzs

    boolean

    Output only. Reserved for future use.

    Union field _satisfies_pzi .

    _satisfies_pzi can be only one of the following:

    satisfiesPzi

    boolean

    Output only. Reserved for future use.

    Timestamp

    JSON representation 
     { 
 "seconds" 
 : 
 string 
 , 
 "nanos" 
 : 
 integer 
 }
    Fields
    seconds

    string ( int64 format)

    Represents seconds of UTC time since Unix epoch 1970-01-01T00:00:00Z. Must be between -62135596800 and 253402300799 inclusive (which corresponds to 0001-01-01T00:00:00Z to 9999-12-31T23:59:59Z).
    nanos

    integer

    Non-negative fractions of a second at nanosecond resolution. This field is the nanosecond portion of the duration, not an alternative to seconds. Negative second values with fractions must still have non-negative nanos values that count forward in time. Must be between 0 and 999,999,999 inclusive.

    LabelsEntry

    JSON representation 
     { 
 "key" 
 : 
 string 
 , 
 "value" 
 : 
 string 
 }
    Fields
    key

    string

    value

    string

    SourceConfig

    JSON representation 
     { 
 "sourceConnectionProfile" 
 : 
 string 
 , 
 // Union field source_stream_config 
can be only one of the following: 
 "oracleSourceConfig" 
 : 
 { 
 object (  OracleSourceConfig 
 
) 
 } 
 , 
 "mysqlSourceConfig" 
 : 
 { 
 object (  MysqlSourceConfig 
 
) 
 } 
 , 
 "postgresqlSourceConfig" 
 : 
 { 
 object (  PostgresqlSourceConfig 
 
) 
 } 
 , 
 "sqlServerSourceConfig" 
 : 
 { 
 object (  SqlServerSourceConfig 
 
) 
 } 
 , 
 "salesforceSourceConfig" 
 : 
 { 
 object (  SalesforceSourceConfig 
 
) 
 } 
 , 
 "mongodbSourceConfig" 
 : 
 { 
 object (  MongodbSourceConfig 
 
) 
 } 
 , 
 "spannerSourceConfig" 
 : 
 { 
 object (  SpannerSourceConfig 
 
) 
 } 
 // End of list of possible types for union field source_stream_config 
. 
 }
    Fields
    sourceConnectionProfile

    string

    Required. Source connection profile resource. Format: projects/{project}/locations/{location}/connectionProfiles/{name}

    Union field source_stream_config . Stream configuration that is specific to the data source type. source_stream_config can be only one of the following:
    oracleSourceConfig

    object ( OracleSourceConfig )

    Oracle data source configuration.

    mysqlSourceConfig

    object ( MysqlSourceConfig )

    MySQL data source configuration.

    postgresqlSourceConfig

    object ( PostgresqlSourceConfig )

    PostgreSQL data source configuration.

    sqlServerSourceConfig

    object ( SqlServerSourceConfig )

    SQLServer data source configuration.

    salesforceSourceConfig

    object ( SalesforceSourceConfig )

    Salesforce data source configuration.

    mongodbSourceConfig

    object ( MongodbSourceConfig )

    MongoDB data source configuration.

    spannerSourceConfig

    object ( SpannerSourceConfig )

    Spanner data source configuration.

    OracleSourceConfig

    JSON representation 
     { 
 "includeObjects" 
 : 
 { 
 object (  OracleRdbms 
 
) 
 } 
 , 
 "excludeObjects" 
 : 
 { 
 object (  OracleRdbms 
 
) 
 } 
 , 
 "maxConcurrentCdcTasks" 
 : 
 integer 
 , 
 "maxConcurrentBackfillTasks" 
 : 
 integer 
 , 
 // Union field large_objects_handling 
can be only one of the following: 
 "dropLargeObjects" 
 : 
 { 
 object ( DropLargeObjects 
) 
 } 
 , 
 "streamLargeObjects" 
 : 
 { 
 object ( StreamLargeObjects 
) 
 } 
 // End of list of possible types for union field large_objects_handling 
. 
 // Union field cdc_method 
can be only one of the following: 
 "logMiner" 
 : 
 { 
 object ( LogMiner 
) 
 } 
 , 
 "binaryLogParser" 
 : 
 { 
 object (  BinaryLogParser 
 
) 
 } 
 // End of list of possible types for union field cdc_method 
. 
 }
    Fields
    includeObjects

    object ( OracleRdbms )

    The Oracle objects to include in the stream.

    excludeObjects

    object ( OracleRdbms )

    The Oracle objects to exclude from the stream.

    maxConcurrentCdcTasks

    integer

    Maximum number of concurrent CDC tasks. The number should be non-negative. If not set (or set to 0), the system's default value is used.

    maxConcurrentBackfillTasks

    integer

    Maximum number of concurrent backfill tasks. The number should be non-negative. If not set (or set to 0), the system's default value is used.

    Union field large_objects_handling . The configuration for handle Oracle large objects. large_objects_handling can be only one of the following:
    dropLargeObjects

    object ( DropLargeObjects )

    Drop large object values.

    streamLargeObjects

    object ( StreamLargeObjects )

    Stream large object values.

    Union field cdc_method . Configuration to select the CDC method. cdc_method can be only one of the following:
    logMiner

    object ( LogMiner )

    Use LogMiner.

    binaryLogParser

    object ( BinaryLogParser )

    Use Binary Log Parser.

    OracleRdbms

    JSON representation 
     { 
 "oracleSchemas" 
 : 
 [ 
 { 
 object (  OracleSchema 
 
) 
 } 
 ] 
 }
    Fields
    oracleSchemas[]

    object ( OracleSchema )

    Oracle schemas/databases in the database server.

    OracleSchema

    JSON representation 
     { 
 "schema" 
 : 
 string 
 , 
 "oracleTables" 
 : 
 [ 
 { 
 object (  OracleTable 
 
) 
 } 
 ] 
 }
    Fields
    schema

    string

    The schema name.
    oracleTables[]

    object ( OracleTable )

    Tables in the schema.

    OracleTable

    JSON representation 
     { 
 "table" 
 : 
 string 
 , 
 "oracleColumns" 
 : 
 [ 
 { 
 object (  OracleColumn 
 
) 
 } 
 ] 
 }
    Fields
    table

    string

    The table name.
    oracleColumns[]

    object ( OracleColumn )

    Oracle columns in the schema. When unspecified as part of include/exclude objects, includes/excludes everything.

    OracleColumn

    JSON representation 
     { 
 "column" 
 : 
 string 
 , 
 "dataType" 
 : 
 string 
 , 
 "length" 
 : 
 integer 
 , 
 "precision" 
 : 
 integer 
 , 
 "scale" 
 : 
 integer 
 , 
 "encoding" 
 : 
 string 
 , 
 "primaryKey" 
 : 
 boolean 
 , 
 "nullable" 
 : 
 boolean 
 , 
 "ordinalPosition" 
 : 
 integer 
 }
    Fields
    column

    string

    The column name.
    dataType

    string

    The Oracle data type.
    length

    integer

    Column length.
    precision

    integer

    Column precision.
    scale

    integer

    Column scale.
    encoding

    string

    Column encoding.
    primaryKey

    boolean

    Whether or not the column represents a primary key.
    nullable

    boolean

    Whether or not the column can accept a null value.
    ordinalPosition

    integer

    The ordinal position of the column in the table.

    BinaryLogParser

    JSON representation 
     { 
 // Union field log_file_access 
can be only one of the following: 
 "oracleAsmLogFileAccess" 
 : 
 { 
 object ( OracleAsmLogFileAccess 
) 
 } 
 , 
 "logFileDirectories" 
 : 
 { 
 object (  LogFileDirectories 
 
) 
 } 
 // End of list of possible types for union field log_file_access 
. 
 }
    Fields
    Union field log_file_access . Configuration to specify how the log file should be accessed. log_file_access can be only one of the following:
    oracleAsmLogFileAccess

    object ( OracleAsmLogFileAccess )

    Use Oracle ASM.

    logFileDirectories

    object ( LogFileDirectories )

    Use Oracle directories.

    LogFileDirectories

    JSON representation 
     { 
 "onlineLogDirectory" 
 : 
 string 
 , 
 "archivedLogDirectory" 
 : 
 string 
 }
    Fields
    onlineLogDirectory

    string

    Required. Oracle directory for online logs.
    archivedLogDirectory

    string

    Required. Oracle directory for archived logs.

    MysqlSourceConfig

    JSON representation 
     { 
 "includeObjects" 
 : 
 { 
 object (  MysqlRdbms 
 
) 
 } 
 , 
 "excludeObjects" 
 : 
 { 
 object (  MysqlRdbms 
 
) 
 } 
 , 
 "maxConcurrentCdcTasks" 
 : 
 integer 
 , 
 "maxConcurrentBackfillTasks" 
 : 
 integer 
 , 
 // Union field cdc_method 
can be only one of the following: 
 "binaryLogPosition" 
 : 
 { 
 object ( BinaryLogPosition 
) 
 } 
 , 
 "gtid" 
 : 
 { 
 object ( Gtid 
) 
 } 
 // End of list of possible types for union field cdc_method 
. 
 }
    Fields
    includeObjects

    object ( MysqlRdbms )

    The MySQL objects to retrieve from the source.

    excludeObjects

    object ( MysqlRdbms )

    The MySQL objects to exclude from the stream.

    maxConcurrentCdcTasks

    integer

    Maximum number of concurrent CDC tasks. The number should be non negative. If not set (or set to 0), the system's default value will be used.

    maxConcurrentBackfillTasks

    integer

    Maximum number of concurrent backfill tasks. The number should be non negative. If not set (or set to 0), the system's default value will be used.

    Union field cdc_method . The CDC method to use for the stream. cdc_method can be only one of the following:
    binaryLogPosition

    object ( BinaryLogPosition )

    Use Binary log position based replication.

    gtid

    object ( Gtid )

    Use GTID based replication.

    MysqlRdbms

    JSON representation 
     { 
 "mysqlDatabases" 
 : 
 [ 
 { 
 object (  MysqlDatabase 
 
) 
 } 
 ] 
 }
    Fields
    mysqlDatabases[]

    object ( MysqlDatabase )

    Mysql databases on the server

    MysqlDatabase

    JSON representation 
     { 
 "database" 
 : 
 string 
 , 
 "mysqlTables" 
 : 
 [ 
 { 
 object (  MysqlTable 
 
) 
 } 
 ] 
 }
    Fields
    database

    string

    The database name.
    mysqlTables[]

    object ( MysqlTable )

    Tables in the database.

    MysqlTable

    JSON representation 
     { 
 "table" 
 : 
 string 
 , 
 "mysqlColumns" 
 : 
 [ 
 { 
 object (  MysqlColumn 
 
) 
 } 
 ] 
 }
    Fields
    table

    string

    The table name.
    mysqlColumns[]

    object ( MysqlColumn )

    MySQL columns in the database. When unspecified as part of include/exclude objects, includes/excludes everything.

    MysqlColumn

    JSON representation 
     { 
 "column" 
 : 
 string 
 , 
 "dataType" 
 : 
 string 
 , 
 "length" 
 : 
 integer 
 , 
 "collation" 
 : 
 string 
 , 
 "primaryKey" 
 : 
 boolean 
 , 
 "nullable" 
 : 
 boolean 
 , 
 "ordinalPosition" 
 : 
 integer 
 , 
 "precision" 
 : 
 integer 
 , 
 "scale" 
 : 
 integer 
 }
    Fields
    column

    string

    The column name.
    dataType

    string

    The MySQL data type. Full data types list can be found here: https://dev.mysql.com/doc/refman/8.0/en/data-types.html

    length

    integer

    Column length.
    collation

    string

    Column collation.
    primaryKey

    boolean

    Whether or not the column represents a primary key.
    nullable

    boolean

    Whether or not the column can accept a null value.
    ordinalPosition

    integer

    The ordinal position of the column in the table.
    precision

    integer

    Column precision.
    scale

    integer

    Column scale.

    PostgresqlSourceConfig

    JSON representation 
     { 
 "includeObjects" 
 : 
 { 
 object (  PostgresqlRdbms 
 
) 
 } 
 , 
 "excludeObjects" 
 : 
 { 
 object (  PostgresqlRdbms 
 
) 
 } 
 , 
 "replicationSlot" 
 : 
 string 
 , 
 "publication" 
 : 
 string 
 , 
 "maxConcurrentBackfillTasks" 
 : 
 integer 
 }
    Fields
    includeObjects

    object ( PostgresqlRdbms )

    The PostgreSQL objects to include in the stream.
    excludeObjects

    object ( PostgresqlRdbms )

    The PostgreSQL objects to exclude from the stream.
    replicationSlot

    string

    Required. Immutable. The name of the logical replication slot that's configured with the pgoutput plugin.
    publication

    string

    Required. The name of the publication that includes the set of all tables that are defined in the stream's include_objects.
    maxConcurrentBackfillTasks

    integer

    Maximum number of concurrent backfill tasks. The number should be non negative. If not set (or set to 0), the system's default value will be used.

    PostgresqlRdbms

    JSON representation 
     { 
 "postgresqlSchemas" 
 : 
 [ 
 { 
 object (  PostgresqlSchema 
 
) 
 } 
 ] 
 }
    Fields
    postgresqlSchemas[]

    object ( PostgresqlSchema )

    PostgreSQL schemas in the database server.

    PostgresqlSchema

    JSON representation 
     { 
 "schema" 
 : 
 string 
 , 
 "postgresqlTables" 
 : 
 [ 
 { 
 object (  PostgresqlTable 
 
) 
 } 
 ] 
 }
    Fields
    schema

    string

    The schema name.
    postgresqlTables[]

    object ( PostgresqlTable )

    Tables in the schema.

    PostgresqlTable

    JSON representation 
     { 
 "table" 
 : 
 string 
 , 
 "postgresqlColumns" 
 : 
 [ 
 { 
 object (  PostgresqlColumn 
 
) 
 } 
 ] 
 }
    Fields
    table

    string

    The table name.
    postgresqlColumns[]

    object ( PostgresqlColumn )

    PostgreSQL columns in the schema. When unspecified as part of include/exclude objects, includes/excludes everything.

    PostgresqlColumn

    JSON representation 
     { 
 "column" 
 : 
 string 
 , 
 "dataType" 
 : 
 string 
 , 
 "length" 
 : 
 integer 
 , 
 "precision" 
 : 
 integer 
 , 
 "scale" 
 : 
 integer 
 , 
 "primaryKey" 
 : 
 boolean 
 , 
 "nullable" 
 : 
 boolean 
 , 
 "ordinalPosition" 
 : 
 integer 
 }
    Fields
    column

    string

    The column name.
    dataType

    string

    The PostgreSQL data type.
    length

    integer

    Column length.
    precision

    integer

    Column precision.
    scale

    integer

    Column scale.
    primaryKey

    boolean

    Whether or not the column represents a primary key.
    nullable

    boolean

    Whether or not the column can accept a null value.
    ordinalPosition

    integer

    The ordinal position of the column in the table.

    SqlServerSourceConfig

    JSON representation 
     { 
 "includeObjects" 
 : 
 { 
 object (  SqlServerRdbms 
 
) 
 } 
 , 
 "excludeObjects" 
 : 
 { 
 object (  SqlServerRdbms 
 
) 
 } 
 , 
 "maxConcurrentCdcTasks" 
 : 
 integer 
 , 
 "maxConcurrentBackfillTasks" 
 : 
 integer 
 , 
 // Union field cdc_method 
can be only one of the following: 
 "transactionLogs" 
 : 
 { 
 object ( SqlServerTransactionLogs 
) 
 } 
 , 
 "changeTables" 
 : 
 { 
 object ( SqlServerChangeTables 
) 
 } 
 // End of list of possible types for union field cdc_method 
. 
 }
    Fields
    includeObjects

    object ( SqlServerRdbms )

    The SQLServer objects to include in the stream.

    excludeObjects

    object ( SqlServerRdbms )

    The SQLServer objects to exclude from the stream.

    maxConcurrentCdcTasks

    integer

    Max concurrent CDC tasks.

    maxConcurrentBackfillTasks

    integer

    Max concurrent backfill tasks.

    Union field cdc_method . Configuration to select the CDC read method for the stream. cdc_method can be only one of the following:
    transactionLogs

    object ( SqlServerTransactionLogs )

    CDC reader reads from transaction logs.

    changeTables

    object ( SqlServerChangeTables )

    CDC reader reads from change tables.

    SqlServerRdbms

    JSON representation 
     { 
 "schemas" 
 : 
 [ 
 { 
 object (  SqlServerSchema 
 
) 
 } 
 ] 
 }
    Fields
    schemas[]

    object ( SqlServerSchema )

    SQLServer schemas in the database server.

    SqlServerSchema

    JSON representation 
     { 
 "schema" 
 : 
 string 
 , 
 "tables" 
 : 
 [ 
 { 
 object (  SqlServerTable 
 
) 
 } 
 ] 
 }
    Fields
    schema

    string

    The schema name.
    tables[]

    object ( SqlServerTable )

    Tables in the schema.

    SqlServerTable

    JSON representation 
     { 
 "table" 
 : 
 string 
 , 
 "columns" 
 : 
 [ 
 { 
 object (  SqlServerColumn 
 
) 
 } 
 ] 
 }
    Fields
    table

    string

    The table name.
    columns[]

    object ( SqlServerColumn )

    SQLServer columns in the schema. When unspecified as part of include/exclude objects, includes/excludes everything.

    SqlServerColumn

    JSON representation 
     { 
 "column" 
 : 
 string 
 , 
 "dataType" 
 : 
 string 
 , 
 "length" 
 : 
 integer 
 , 
 "precision" 
 : 
 integer 
 , 
 "scale" 
 : 
 integer 
 , 
 "primaryKey" 
 : 
 boolean 
 , 
 "nullable" 
 : 
 boolean 
 , 
 "ordinalPosition" 
 : 
 integer 
 }
    Fields
    column

    string

    The column name.
    dataType

    string

    The SQLServer data type.
    length

    integer

    Column length.
    precision

    integer

    Column precision.
    scale

    integer

    Column scale.
    primaryKey

    boolean

    Whether or not the column represents a primary key.
    nullable

    boolean

    Whether or not the column can accept a null value.
    ordinalPosition

    integer

    The ordinal position of the column in the table.

    SalesforceSourceConfig

    JSON representation 
     { 
 "includeObjects" 
 : 
 { 
 object (  SalesforceOrg 
 
) 
 } 
 , 
 "excludeObjects" 
 : 
 { 
 object (  SalesforceOrg 
 
) 
 } 
 , 
 "pollingInterval" 
 : 
 string 
 }
    Fields
    includeObjects

    object ( SalesforceOrg )

    The Salesforce objects to retrieve from the source.
    excludeObjects

    object ( SalesforceOrg )

    The Salesforce objects to exclude from the stream.
    pollingInterval

    string ( Duration format)

    Required. Salesforce objects polling interval. The interval at which new changes will be polled for each object. The duration must be from 5 minutes to 24 hours , inclusive.

    A duration in seconds with up to nine fractional digits, ending with ' s '. Example: "3.5s" .

    SalesforceOrg

    JSON representation 
     { 
 "objects" 
 : 
 [ 
 { 
 object (  SalesforceObject 
 
) 
 } 
 ] 
 }
    Fields
    objects[]

    object ( SalesforceObject )

    Salesforce objects in the database server.

    SalesforceObject

    JSON representation 
     { 
 "objectName" 
 : 
 string 
 , 
 "fields" 
 : 
 [ 
 { 
 object (  SalesforceField 
 
) 
 } 
 ] 
 }
    Fields
    objectName

    string

    The object name.
    fields[]

    object ( SalesforceField )

    Salesforce fields. When unspecified as part of include objects, includes everything, when unspecified as part of exclude objects, excludes nothing.

    SalesforceField

    JSON representation 
     { 
 "name" 
 : 
 string 
 , 
 "dataType" 
 : 
 string 
 , 
 "nillable" 
 : 
 boolean 
 }
    Fields
    name

    string

    The field name.
    dataType

    string

    The data type.
    nillable

    boolean

    Indicates whether the field can accept nil values.

    Duration

    JSON representation 
     { 
 "seconds" 
 : 
 string 
 , 
 "nanos" 
 : 
 integer 
 }
    Fields
    seconds

    string ( int64 format)

    Signed seconds of the span of time. Must be from -315,576,000,000 to +315,576,000,000 inclusive. Note: these bounds are computed from: 60 sec/min * 60 min/hr * 24 hr/day * 365.25 days/year * 10000 years
    nanos

    integer

    Signed fractions of a second at nanosecond resolution of the span of time. Durations less than one second are represented with a 0 seconds field and a positive or negative nanos field. For durations of one second or more, a non-zero value for the nanos field must be of the same sign as the seconds field. Must be from -999,999,999 to +999,999,999 inclusive.

    MongodbSourceConfig

    JSON representation 
     { 
 "includeObjects" 
 : 
 { 
 object (  MongodbCluster 
 
) 
 } 
 , 
 "excludeObjects" 
 : 
 { 
 object (  MongodbCluster 
 
) 
 } 
 , 
 "maxConcurrentBackfillTasks" 
 : 
 integer 
 , 
 "jsonMode" 
 : 
 enum ( MongodbJsonMode 
) 
 }
    Fields
    includeObjects

    object ( MongodbCluster )

    The MongoDB collections to include in the stream.
    excludeObjects

    object ( MongodbCluster )

    The MongoDB collections to exclude from the stream.
    maxConcurrentBackfillTasks

    integer

    Optional. Maximum number of concurrent backfill tasks. The number should be non-negative and less than or equal to 50. If not set (or set to 0), the system's default value is used
    jsonMode

    enum ( MongodbJsonMode )

    Optional. MongoDB JSON mode to use for the stream.

    MongodbCluster

    JSON representation 
     { 
 "databases" 
 : 
 [ 
 { 
 object (  MongodbDatabase 
 
) 
 } 
 ] 
 }
    Fields
    databases[]

    object ( MongodbDatabase )

    MongoDB databases in the cluster.

    MongodbDatabase

    JSON representation 
     { 
 "database" 
 : 
 string 
 , 
 "collections" 
 : 
 [ 
 { 
 object (  MongodbCollection 
 
) 
 } 
 ] 
 }
    Fields
    database

    string

    The database name.
    collections[]

    object ( MongodbCollection )

    Collections in the database.

    MongodbCollection

    JSON representation 
     { 
 "collection" 
 : 
 string 
 , 
 "fields" 
 : 
 [ 
 { 
 object (  MongodbField 
 
) 
 } 
 ] 
 }
    Fields
    collection

    string

    The collection name.
    fields[]

    object ( MongodbField )

    Fields in the collection.

    MongodbField

    JSON representation 
     { 
 "field" 
 : 
 string 
 }
    Fields
    field

    string

    The field name.

    SpannerSourceConfig

    JSON representation 
     { 
 "changeStreamName" 
 : 
 string 
 , 
 "spannerRpcPriority" 
 : 
 enum ( SpannerRpcPriority 
) 
 , 
 "fgacRole" 
 : 
 string 
 , 
 "maxConcurrentCdcTasks" 
 : 
 integer 
 , 
 "maxConcurrentBackfillTasks" 
 : 
 integer 
 , 
 "includeObjects" 
 : 
 { 
 object (  SpannerDatabase 
 
) 
 } 
 , 
 "excludeObjects" 
 : 
 { 
 object (  SpannerDatabase 
 
) 
 } 
 , 
 "backfillDataBoostEnabled" 
 : 
 boolean 
 }
    Fields
    changeStreamName

    string

    Required. Immutable. The change stream name to use for the stream.
    spannerRpcPriority

    enum ( SpannerRpcPriority )

    Optional. The RPC priority to use for the stream.
    fgacRole

    string

    Optional. The FGAC role to use for the stream.
    maxConcurrentCdcTasks

    integer

    Optional. Maximum number of concurrent CDC tasks.
    maxConcurrentBackfillTasks

    integer

    Optional. Maximum number of concurrent backfill tasks.
    includeObjects

    object ( SpannerDatabase )

    Optional. The Spanner objects to retrieve from the data source. If some objects are both included and excluded, an error will be thrown.
    excludeObjects

    object ( SpannerDatabase )

    Optional. The Spanner objects to avoid retrieving. If some objects are both included and excluded, an error will be thrown.
    backfillDataBoostEnabled

    boolean

    Optional. Whether to use Data Boost for Spanner backfills. Defaults to false if not set.

    SpannerDatabase

    JSON representation 
     { 
 "schemas" 
 : 
 [ 
 { 
 object (  SpannerSchema 
 
) 
 } 
 ] 
 }
    Fields
    schemas[]

    object ( SpannerSchema )

    Optional. Spanner schemas in the database.

    SpannerSchema

    JSON representation 
     { 
 "schema" 
 : 
 string 
 , 
 "tables" 
 : 
 [ 
 { 
 object (  SpannerTable 
 
) 
 } 
 ] 
 }
    Fields
    schema

    string

    Required. The schema name.
    tables[]

    object ( SpannerTable )

    Optional. Spanner tables in the schema.

    SpannerTable

    JSON representation 
     { 
 "table" 
 : 
 string 
 , 
 "columns" 
 : 
 [ 
 { 
 object (  SpannerColumn 
 
) 
 } 
 ] 
 }
    Fields
    table

    string

    Required. The table name.
    columns[]

    object ( SpannerColumn )

    Optional. Spanner columns in the table.

    SpannerColumn

    JSON representation 
     { 
 "column" 
 : 
 string 
 , 
 "dataType" 
 : 
 string 
 , 
 "isPrimaryKey" 
 : 
 boolean 
 , 
 "ordinalPosition" 
 : 
 string 
 }
    Fields
    column

    string

    Required. The column name.
    dataType

    string

    Optional. Spanner data type.
    isPrimaryKey

    boolean

    Optional. Whether or not the column is a primary key.
    ordinalPosition

    string ( int64 format)

    Optional. The ordinal position of the column in the table.

    DestinationConfig

    JSON representation 
     { 
 "destinationConnectionProfile" 
 : 
 string 
 , 
 // Union field destination_stream_config 
can be only one of the following: 
 "gcsDestinationConfig" 
 : 
 { 
 object (  GcsDestinationConfig 
 
) 
 } 
 , 
 "bigqueryDestinationConfig" 
 : 
 { 
 object (  BigQueryDestinationConfig 
 
) 
 } 
 // End of list of possible types for union field destination_stream_config 
. 
 }
    Fields
    destinationConnectionProfile

    string

    Required. Destination connection profile resource. Format: projects/{project}/locations/{location}/connectionProfiles/{name}

    Union field destination_stream_config . Stream configuration that is specific to the data destination type. destination_stream_config can be only one of the following:
    gcsDestinationConfig

    object ( GcsDestinationConfig )

    A configuration for how data should be loaded to Cloud Storage.

    bigqueryDestinationConfig

    object ( BigQueryDestinationConfig )

    BigQuery destination configuration.

    GcsDestinationConfig

    JSON representation 
     { 
 "path" 
 : 
 string 
 , 
 "fileRotationMb" 
 : 
 integer 
 , 
 "fileRotationInterval" 
 : 
 string 
 , 
 // Union field file_format 
can be only one of the following: 
 "avroFileFormat" 
 : 
 { 
 object ( AvroFileFormat 
) 
 } 
 , 
 "jsonFileFormat" 
 : 
 { 
 object (  JsonFileFormat 
 
) 
 } 
 // End of list of possible types for union field file_format 
. 
 }
    Fields
    path

    string

    Path inside the Cloud Storage bucket to write data to.

    fileRotationMb

    integer

    The maximum file size to be saved in the bucket.

    fileRotationInterval

    string ( Duration format)

    The maximum duration for which new events are added before a file is closed and a new file is created. Values within the range of 15-60 seconds are allowed.

    A duration in seconds with up to nine fractional digits, ending with ' s '. Example: "3.5s" .

    Union field file_format . File Format that the data should be written in. file_format can be only one of the following:
    avroFileFormat

    object ( AvroFileFormat )

    AVRO file format configuration.

    jsonFileFormat

    object ( JsonFileFormat )

    JSON file format configuration.

    JsonFileFormat

    JSON representation 
     { 
 "schemaFileFormat" 
 : 
 enum ( SchemaFileFormat 
) 
 , 
 "compression" 
 : 
 enum ( JsonCompression 
) 
 }
    Fields
    schemaFileFormat

    enum ( SchemaFileFormat )

    The schema file format along JSON data files.
    compression

    enum ( JsonCompression )

    Compression of the loaded JSON file.

    BigQueryDestinationConfig

    JSON representation 
     { 
 "dataFreshness" 
 : 
 string 
 , 
 "blmtConfig" 
 : 
 { 
 object (  BlmtConfig 
 
) 
 } 
 , 
 // Union field dataset_config 
can be only one of the following: 
 "singleTargetDataset" 
 : 
 { 
 object (  SingleTargetDataset 
 
) 
 } 
 , 
 "sourceHierarchyDatasets" 
 : 
 { 
 object (  SourceHierarchyDatasets 
 
) 
 } 
 // End of list of possible types for union field dataset_config 
. 
 // Union field write_mode 
can be only one of the following: 
 "merge" 
 : 
 { 
 object ( Merge 
) 
 } 
 , 
 "appendOnly" 
 : 
 { 
 object ( AppendOnly 
) 
 } 
 // End of list of possible types for union field write_mode 
. 
 }
    Fields
    dataFreshness

    string ( Duration format)

    The guaranteed data freshness (in seconds) when querying tables created by the stream. Editing this field will only affect new tables created in the future, but existing tables will not be impacted. Lower values mean that queries will return fresher data, but may result in higher cost.

    A duration in seconds with up to nine fractional digits, ending with ' s '. Example: "3.5s" .

    blmtConfig

    object ( BlmtConfig )

    Optional. Big Lake Managed Tables (BLMT) configuration.

    Union field dataset_config . Target dataset(s) configuration. dataset_config can be only one of the following:
    singleTargetDataset

    object ( SingleTargetDataset )

    Single destination dataset.

    sourceHierarchyDatasets

    object ( SourceHierarchyDatasets )

    Source hierarchy datasets.

    Union field write_mode .

    write_mode can be only one of the following:

    merge

    object ( Merge )

    The standard mode

    appendOnly

    object ( AppendOnly )

    Append only mode

    SingleTargetDataset

    JSON representation 
     { 
 "datasetId" 
 : 
 string 
 }
    Fields
    datasetId

    string

    The dataset ID of the target dataset. DatasetIds allowed characters: https://cloud.google.com/bigquery/docs/reference/rest/v2/datasets#datasetreference .

    SourceHierarchyDatasets

    JSON representation 
     { 
 "datasetTemplate" 
 : 
 { 
 object (  DatasetTemplate 
 
) 
 } 
 , 
 // Union field _project_id 
can be only one of the following: 
 "projectId" 
 : 
 string 
 // End of list of possible types for union field _project_id 
. 
 }
    Fields
    datasetTemplate

    object ( DatasetTemplate )

    The dataset template to use for dynamic dataset creation.

    Union field _project_id .

    _project_id can be only one of the following:

    projectId

    string

    Optional. The project id of the BigQuery dataset. If not specified, the project will be inferred from the stream resource.

    DatasetTemplate

    JSON representation 
     { 
 "location" 
 : 
 string 
 , 
 "datasetIdPrefix" 
 : 
 string 
 , 
 "kmsKeyName" 
 : 
 string 
 }
    Fields
    location

    string

    Required. The geographic location where the dataset should reside. See https://cloud.google.com/bigquery/docs/locations for supported locations.
    datasetIdPrefix

    string

    If supplied, every created dataset will have its name prefixed by the provided value. The prefix and name will be separated by an underscore. i.e.

    _ .
    kmsKeyName

    string

    Describes the Cloud KMS encryption key that will be used to protect destination BigQuery table. The BigQuery Service Account associated with your project requires access to this encryption key. i.e. projects/{project}/locations/{location}/keyRings/{key_ring}/cryptoKeys/{cryptoKey}. See https://cloud.google.com/bigquery/docs/customer-managed-encryption for more information.

    BlmtConfig

    JSON representation 
     { 
 "bucket" 
 : 
 string 
 , 
 "rootPath" 
 : 
 string 
 , 
 "connectionName" 
 : 
 string 
 , 
 "fileFormat" 
 : 
 enum ( FileFormat 
) 
 , 
 "tableFormat" 
 : 
 enum ( TableFormat 
) 
 }
    Fields
    bucket

    string

    Required. The Cloud Storage bucket name.
    rootPath

    string

    The root path inside the Cloud Storage bucket.
    connectionName

    string

    Required. The bigquery connection. Format: {project}.{location}.{name}

    fileFormat

    enum ( FileFormat )

    Required. The file format.
    tableFormat

    enum ( TableFormat )

    Required. The table format.

    BackfillAllStrategy

    JSON representation 
     { 
 // Union field excluded_objects 
can be only one of the following: 
 "oracleExcludedObjects" 
 : 
 { 
 object (  OracleRdbms 
 
) 
 } 
 , 
 "mysqlExcludedObjects" 
 : 
 { 
 object (  MysqlRdbms 
 
) 
 } 
 , 
 "postgresqlExcludedObjects" 
 : 
 { 
 object (  PostgresqlRdbms 
 
) 
 } 
 , 
 "sqlServerExcludedObjects" 
 : 
 { 
 object (  SqlServerRdbms 
 
) 
 } 
 , 
 "salesforceExcludedObjects" 
 : 
 { 
 object (  SalesforceOrg 
 
) 
 } 
 , 
 "mongodbExcludedObjects" 
 : 
 { 
 object (  MongodbCluster 
 
) 
 } 
 , 
 "spannerExcludedObjects" 
 : 
 { 
 object (  SpannerDatabase 
 
) 
 } 
 // End of list of possible types for union field excluded_objects 
. 
 }
    Fields
    Union field excluded_objects . List of objects to exclude. excluded_objects can be only one of the following:
    oracleExcludedObjects

    object ( OracleRdbms )

    Oracle data source objects to avoid backfilling.

    mysqlExcludedObjects

    object ( MysqlRdbms )

    MySQL data source objects to avoid backfilling.

    postgresqlExcludedObjects

    object ( PostgresqlRdbms )

    PostgreSQL data source objects to avoid backfilling.

    sqlServerExcludedObjects

    object ( SqlServerRdbms )

    SQLServer data source objects to avoid backfilling

    salesforceExcludedObjects

    object ( SalesforceOrg )

    Salesforce data source objects to avoid backfilling

    mongodbExcludedObjects

    object ( MongodbCluster )

    MongoDB data source objects to avoid backfilling

    spannerExcludedObjects

    object ( SpannerDatabase )

    Spanner data source objects to avoid backfilling.

    Error

    JSON representation 
     { 
 "reason" 
 : 
 string 
 , 
 "errorUuid" 
 : 
 string 
 , 
 "message" 
 : 
 string 
 , 
 "errorTime" 
 : 
 string 
 , 
 "details" 
 : 
 { 
 string 
 : 
 string 
 , 
 ... 
 } 
 }
    Fields
    reason

    string

    A title that explains the reason for the error.
    errorUuid

    string

    A unique identifier for this specific error, allowing it to be traced throughout the system in logs and API responses.
    message

    string

    A message containing more information about the error that occurred.
    errorTime

    string ( Timestamp format)

    The time when the error occurred.

    Uses RFC 3339, where generated output will always be Z-normalized and use 0, 3, 6 or 9 fractional digits. Offsets other than "Z" are also accepted. Examples: "2014-10-02T15:01:23Z" , "2014-10-02T15:01:23.045123456Z" or "2014-10-02T15:01:23+05:30" .
    details

    map (key: string, value: string)

    Additional information about the error.

    An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" } .

    DetailsEntry

    JSON representation 
     { 
 "key" 
 : 
 string 
 , 
 "value" 
 : 
 string 
 }
    Fields
    key

    string

    value

    string

    RuleSet

    JSON representation 
     { 
 "customizationRules" 
 : 
 [ 
 { 
 object (  CustomizationRule 
 
) 
 } 
 ] 
 , 
 "objectFilter" 
 : 
 { 
 object (  ObjectFilter 
 
) 
 } 
 }
    Fields
    customizationRules[]

    object ( CustomizationRule )

    Required. List of customization rules to apply.
    objectFilter

    object ( ObjectFilter )

    Required. Object filter to apply the customization rules to.

    CustomizationRule

    JSON representation 
     { 
 // Union field rule 
can be only one of the following: 
 "bigqueryPartitioning" 
 : 
 { 
 object (  BigQueryPartitioning 
 
) 
 } 
 , 
 "bigqueryClustering" 
 : 
 { 
 object (  BigQueryClustering 
 
) 
 } 
 // End of list of possible types for union field rule 
. 
 }
    Fields
    Union field rule . The rule to apply. rule can be only one of the following:
    bigqueryPartitioning

    object ( BigQueryPartitioning )

    BigQuery partitioning rule.

    bigqueryClustering

    object ( BigQueryClustering )

    BigQuery clustering rule.

    BigQueryPartitioning

    JSON representation 
     { 
 "requirePartitionFilter" 
 : 
 boolean 
 , 
 // Union field partitioning 
can be only one of the following: 
 "integerRangePartition" 
 : 
 { 
 object (  IntegerRangePartition 
 
) 
 } 
 , 
 "timeUnitPartition" 
 : 
 { 
 object (  TimeUnitPartition 
 
) 
 } 
 , 
 "ingestionTimePartition" 
 : 
 { 
 object (  IngestionTimePartition 
 
) 
 } 
 // End of list of possible types for union field partitioning 
. 
 }
    Fields
    requirePartitionFilter

    boolean

    Optional. If true, queries over the table require a partition filter.

    Union field partitioning . Partitioning to apply on the table. partitioning can be only one of the following:
    integerRangePartition

    object ( IntegerRangePartition )

    Integer range partitioning.

    timeUnitPartition

    object ( TimeUnitPartition )

    Time unit column partitioning.

    ingestionTimePartition

    object ( IngestionTimePartition )

    Ingestion time partitioning.

    IntegerRangePartition

    JSON representation 
     { 
 "column" 
 : 
 string 
 , 
 "start" 
 : 
 string 
 , 
 "end" 
 : 
 string 
 , 
 "interval" 
 : 
 string 
 }
    Fields
    column

    string

    Required. The partitioning column.
    start

    string ( int64 format)

    Required. The starting value for range partitioning (inclusive).
    end

    string ( int64 format)

    Required. The ending value for range partitioning (exclusive).
    interval

    string ( int64 format)

    Required. The interval of each range within the partition.

    TimeUnitPartition

    JSON representation 
     { 
 "column" 
 : 
 string 
 , 
 "partitioningTimeGranularity" 
 : 
 enum ( PartitioningTimeGranularity 
) 
 }
    Fields
    column

    string

    Required. The partitioning column.
    partitioningTimeGranularity

    enum ( PartitioningTimeGranularity )

    Optional. Partition granularity.

    IngestionTimePartition

    JSON representation 
     { 
 "partitioningTimeGranularity" 
 : 
 enum ( PartitioningTimeGranularity 
) 
 }
    Fields
    partitioningTimeGranularity

    enum ( PartitioningTimeGranularity )

    Optional. Partition granularity

    BigQueryClustering

    JSON representation 
     { 
 "columns" 
 : 
 [ 
 string 
 ] 
 }
    Fields
    columns[]

    string

    Required. Column names to set as clustering columns.

    ObjectFilter

    JSON representation 
     { 
 // Union field filter_type 
can be only one of the following: 
 "sourceObjectIdentifier" 
 : 
 { 
 object (  SourceObjectIdentifier 
 
) 
 } 
 // End of list of possible types for union field filter_type 
. 
 }
    Fields
    Union field filter_type . The filter to apply. filter_type can be only one of the following:
    sourceObjectIdentifier

    object ( SourceObjectIdentifier )

    Specific source object identifier.

    SourceObjectIdentifier

    JSON representation 
     { 
 // Union field source_identifier 
can be only one of the following: 
 "oracleIdentifier" 
 : 
 { 
 object (  OracleObjectIdentifier 
 
) 
 } 
 , 
 "mysqlIdentifier" 
 : 
 { 
 object (  MysqlObjectIdentifier 
 
) 
 } 
 , 
 "postgresqlIdentifier" 
 : 
 { 
 object (  PostgresqlObjectIdentifier 
 
) 
 } 
 , 
 "sqlServerIdentifier" 
 : 
 { 
 object (  SqlServerObjectIdentifier 
 
) 
 } 
 , 
 "salesforceIdentifier" 
 : 
 { 
 object (  SalesforceObjectIdentifier 
 
) 
 } 
 , 
 "mongodbIdentifier" 
 : 
 { 
 object (  MongodbObjectIdentifier 
 
) 
 } 
 , 
 "spannerIdentifier" 
 : 
 { 
 object (  SpannerObjectIdentifier 
 
) 
 } 
 // End of list of possible types for union field source_identifier 
. 
 }
    Fields
    Union field source_identifier . The identifier for an object in the data source. source_identifier can be only one of the following:
    oracleIdentifier

    object ( OracleObjectIdentifier )

    Oracle data source object identifier.

    mysqlIdentifier

    object ( MysqlObjectIdentifier )

    Mysql data source object identifier.

    postgresqlIdentifier

    object ( PostgresqlObjectIdentifier )

    PostgreSQL data source object identifier.

    sqlServerIdentifier

    object ( SqlServerObjectIdentifier )

    SQLServer data source object identifier.

    salesforceIdentifier

    object ( SalesforceObjectIdentifier )

    Salesforce data source object identifier.

    mongodbIdentifier

    object ( MongodbObjectIdentifier )

    MongoDB data source object identifier.

    spannerIdentifier

    object ( SpannerObjectIdentifier )

    Spanner data source object identifier.

    OracleObjectIdentifier

    JSON representation 
     { 
 "schema" 
 : 
 string 
 , 
 "table" 
 : 
 string 
 }
    Fields
    schema

    string

    Required. The schema name.
    table

    string

    Required. The table name.

    MysqlObjectIdentifier

    JSON representation 
     { 
 "database" 
 : 
 string 
 , 
 "table" 
 : 
 string 
 }
    Fields
    database

    string

    Required. The database name.
    table

    string

    Required. The table name.

    PostgresqlObjectIdentifier

    JSON representation 
     { 
 "schema" 
 : 
 string 
 , 
 "table" 
 : 
 string 
 }
    Fields
    schema

    string

    Required. The schema name.
    table

    string

    Required. The table name.

    SqlServerObjectIdentifier

    JSON representation 
     { 
 "schema" 
 : 
 string 
 , 
 "table" 
 : 
 string 
 }
    Fields
    schema

    string

    Required. The schema name.
    table

    string

    Required. The table name.

    SalesforceObjectIdentifier

    JSON representation 
     { 
 "objectName" 
 : 
 string 
 }
    Fields
    objectName

    string

    Required. The object name.

    MongodbObjectIdentifier

    JSON representation 
     { 
 "database" 
 : 
 string 
 , 
 "collection" 
 : 
 string 
 }
    Fields
    database

    string

    Required. The database name.
    collection

    string

    Required. The collection name.

    SpannerObjectIdentifier

    JSON representation 
     { 
 "schema" 
 : 
 string 
 , 
 "table" 
 : 
 string 
 }
    Fields
    schema

    string

    Optional. The schema name.
    table

    string

    Required. The table name.

    Tool Annotations

    Destructive Hint: ❌ | Idempotent Hint: ✅ | Read Only Hint: ✅ | Open World Hint: ❌
    Create a Mobile Website
    View Site in Mobile | Classic
    Share by: