- Resource: Collection
- DataConnector
- State
- DateTime
- TimeZone
- SourceEntity
- SyncMode
- DestinationConfig
- Destination
- ActionConfig
- EndUserConfig
- Tenant
- FederatedConfig
- ConnectorMode
- BlockingReason
- IdentityScheduleConfig
- BAPConfig
- ConnectorMode
- ConnectorType
- AlertPolicyConfig
- AlertEnrollment
- EnrollStates
- RealtimeSyncConfig
- StreamingError
- StreamingErrorReason
- Methods
Resource: Collection
Collection is a container for configuring resources and access to a set of DataStores.
JSON representation |
---|
{
"name"
:
string
,
"displayName"
:
string
,
"dataConnector"
:
{
object (
|
Fields | |
---|---|
name
|
Immutable. The full resource name of the Collection. Format: This field must be a UTF-8 encoded string with a length limit of 1024 characters. |
displayName
|
Required. The Collection display name. This field must be a UTF-8 encoded string with a length limit of 128 characters. Otherwise, an INVALID_ARGUMENT error is returned. |
dataConnector
|
Output only. The data connector, if present, manages the connection for data stores in the Collection. To set up the connector, use This output only field contains a subset of the To get more details about a data connector, use the |
createTime
|
Output only. Timestamp the Uses RFC 3339, where generated output will always be Z-normalized and use 0, 3, 6 or 9 fractional digits. Offsets other than "Z" are also accepted. Examples: |
DataConnector
Manages the connection to external data sources for all data stores grouped under a Collection
. It's a singleton resource of Collection
. The initialization is only supported through DataConnectorService.SetUpDataConnector
method, which will create a new Collection
and initialize its DataConnector
.
JSON representation |
---|
{ "name" : string , "state" : enum ( |
name
string
Output only. The full resource name of the data Connector. Format: projects/*/locations/*/collections/*/dataConnector
.
state
enum (
State
)
Output only. state of the connector.
dataSource
string
Required. The name of the data source. Supported values: salesforce
, jira
, confluence
, bigquery
.
params
object (
Struct
format)
Required. params needed to access the source in the format of (Key, value) pairs.
Required parameters for all data sources:
- Key:
instance_uri
- value: type STRING. The uri to access the data source.
Required parameters for sources that support OAUTH, i.e. salesforce
:
- Key:
clientId
- value: type STRING. The client id for the third party service provider to identify your application.
- Key:
clientSecret
- value:type STRING. The client secret generated by the third party authorization server.
- Key:
accessToken
- value: type STRING. OAuth token for UCS to access to the protected resource.
- Key:
refreshToken
- value: type STRING. OAuth refresh token for UCS to obtain a new access token without user interaction.
Required parameters for sources that support basic API token auth, i.e. jira
, confluence
:
- Key:
userAccount
- value: type STRING. The username or email with the source.
- Key:
apiToken
- value: type STRING. The API token generated for the source account, that is used for authenticating anywhere where you would have used a password.
Example:
{
"instance_uri": "https://xxx.atlassian.net",
"userAccount": "xxxx.xxx@xxx.com",
"apiToken": "test-token"
}
Optional parameter to specify the authorization type to use for multiple authorization types support:
- Key:
authType
- value: type STRING. The authorization type for the data source. Supported values:
BASIC_AUTH
,OAUTH
,OAUTH_ACCESS_TOKEN
,OAUTH_TWO_LEGGED
,OAUTH_JWT_BEARER
,OAUTH_PASSWORD_GRANT
,JWT
,API_TOKEN
,FEDERATED_CREDENTIAL
.
removeParamKeys[]
string
Optional. Specifies keys to be removed from the 'params' field. This is only active when 'params' is included in the 'updateMask' in an UpdateDataConnectorRequest. Deletion takes precedence if a key is both in 'removeParamKeys' and present in the 'params' field of the request.
refreshInterval
string (
Duration
format)
Required. The refresh interval for data sync. If duration is set to 0, the data will be synced in real time. The streaming feature is not supported yet. The minimum is 30 minutes and maximum is 7 days. When the refresh interval is set to the same value as the incremental refresh interval, incremental sync will be disabled.
A duration in seconds with up to nine fractional digits, ending with ' s
'. Example: "3.5s"
.
nextSyncTime
object (
DateTime
)
Defines the scheduled time for the next data synchronization. This field requires hour , minute, and timeZone from the IANA time Zone Database . This is utilized when the data connector has a refresh interval greater than 1 day. When the hours or minutes are not specified, we will assume a sync time of 0:00. The user must provide a time zone to avoid ambiguity.
entities[]
object (
SourceEntity
)
List of entities from the connected data source to ingest.
createTime
string (
Timestamp
format)
Output only. Timestamp the DataConnector
was created at.
Uses RFC 3339, where generated output will always be Z-normalized and use 0, 3, 6 or 9 fractional digits. Offsets other than "Z" are also accepted. Examples: "2014-10-02T15:01:23Z"
, "2014-10-02T15:01:23.045123456Z"
or "2014-10-02T15:01:23+05:30"
.
latestPauseTime
string (
Timestamp
format)
Output only. The most recent timestamp when this DataConnector
was paused, affecting all functionalities such as data synchronization. Pausing a connector has the following effects: - All functionalities, including data synchronization, are halted. - Any ongoing data synchronization job will be canceled. - No future data synchronization runs will be scheduled nor can be triggered.
Uses RFC 3339, where generated output will always be Z-normalized and use 0, 3, 6 or 9 fractional digits. Offsets other than "Z" are also accepted. Examples: "2014-10-02T15:01:23Z"
, "2014-10-02T15:01:23.045123456Z"
or "2014-10-02T15:01:23+05:30"
.
lastSyncTime
string (
Timestamp
format)
Output only. For periodic connectors only, the last time a data sync was completed.
Uses RFC 3339, where generated output will always be Z-normalized and use 0, 3, 6 or 9 fractional digits. Offsets other than "Z" are also accepted. Examples: "2014-10-02T15:01:23Z"
, "2014-10-02T15:01:23.045123456Z"
or "2014-10-02T15:01:23+05:30"
.
updateTime
string (
Timestamp
format)
Output only. Timestamp the DataConnector
was last updated.
Uses RFC 3339, where generated output will always be Z-normalized and use 0, 3, 6 or 9 fractional digits. Offsets other than "Z" are also accepted. Examples: "2014-10-02T15:01:23Z"
, "2014-10-02T15:01:23.045123456Z"
or "2014-10-02T15:01:23+05:30"
.
errors[]
object (
Status
)
Output only. The errors from initialization or from the latest connector run.
syncMode
enum (
SyncMode
)
The data synchronization mode supported by the data connector.
kmsKeyName
string
Input only. The KMS key to be used to protect the DataStores managed by this connector.
Must be set for requests that need to comply with CMEK Org Policy protections.
If this field is set and processed successfully, the DataStores created by this connector will be protected by the KMS key.
destinationConfigs[]
object (
DestinationConfig
)
Optional. Any target destinations used to connect to third-party services.
actionState
enum (
State
)
Output only. state of the action connector. This reflects whether the action connector is initializing, active or has encountered errors.
actionConfig
object (
ActionConfig
)
Optional. Action configurations to make the connector support actions.
endUserConfig
object (
EndUserConfig
)
Optional. Any params and credentials used specifically for EUA connectors.
federatedConfig
object (
FederatedConfig
)
Optional. Any params and credentials used specifically for hybrid connectors supporting FEDERATED mode. This field should only be set if the connector is a hybrid connector and we want to enable FEDERATED mode.
connectorModes[]
enum (
ConnectorMode
)
Optional. The modes enabled for this connector. Default state is CONNECTOR_MODE_UNSPECIFIED.
staticIpEnabled
boolean
Optional. Whether customer has enabled static IP addresses for this connector.
staticIpAddresses[]
string
Output only. The static IP addresses used by this connector.
blockingReasons[]
enum (
BlockingReason
)
Output only. User actions that must be completed before the connector can start syncing data.
identityRefreshInterval
(deprecated)
string (
Duration
format)
The refresh interval to sync the Access Control List information for the documents ingested by this connector. If not set, the access control list will be refreshed at the default interval of 30 minutes. The identity refresh interval can be at least 30 minutes and at most 7 days.
A duration in seconds with up to nine fractional digits, ending with ' s
'. Example: "3.5s"
.
identityScheduleConfig
object (
IdentityScheduleConfig
)
The configuration for the identity data synchronization runs. This contains the refresh interval to sync the Access Control List information for the documents ingested by this connector.
privateConnectivityProjectId
string
Output only. The tenant project id associated with private connectivity connectors. This project must be allowlisted by in order for the connector to function.
bapConfig
object (
BAPConfig
)
Optional. The configuration for establishing a BAP connection.
connectorType
enum (
ConnectorType
)
Output only. The type of connector. Each source can only map to one type. For example, salesforce, confluence and jira have THIRD_PARTY connector type. It is not mutable once set by system.
realtimeState
enum (
State
)
Output only. real-time sync state
alertPolicyConfigs[]
object (
AlertPolicyConfig
)
Optional. The connector level alert config.
realtimeSyncConfig
object (
RealtimeSyncConfig
)
Optional. The configuration for realtime sync.
aclEnabled
boolean
Optional. Whether the connector will be created with an ACL config. Currently this field only affects Cloud Storage and BigQuery connectors.
createEuaSaas
boolean
Optional. Whether the END USER AUTHENTICATION connector is created in SaaS.
autoRunDisabled
boolean
Optional. Indicates whether the connector is disabled for auto run. It can be used to pause periodical and real time sync. Update: with the introduction of incrementalSyncDisabled, autoRunDisabled is used to pause/disable only full syncs
incrementalSyncDisabled
boolean
Optional. Indicates whether incremental syncs are paused for this connector. This is independent of autoRunDisabled. Applicable to only 3P connectors. When the refresh interval is set to the same value as the incremental refresh interval, incremental sync will be disabled, i.e. set to true.
incrementalRefreshInterval
string (
Duration
format)
Optional. The refresh interval specifically for incremental data syncs. If unset, incremental syncs will use the default from env, set to 3hrs. The minimum is 30 minutes and maximum is 7 days. Applicable to only 3P connectors. When the refresh interval is set to the same value as the incremental refresh interval, incremental sync will be disabled.
A duration in seconds with up to nine fractional digits, ending with ' s
'. Example: "3.5s"
.
hybridIngestionDisabled
boolean
Optional. If the connector is a hybrid connector, determines whether ingestion is enabled and appropriate resources are provisioned during connector creation. If the connector is not a hybrid connector, this field is ignored.
State
The state of connector.
Enums | |
---|---|
STATE_UNSPECIFIED
|
Default value. |
CREATING
|
The connector is being set up. |
ACTIVE
|
The connector is successfully set up and awaiting next sync run. |
FAILED
|
The connector is in error. The error details can be found in DataConnector.errors
. If the error is unfixable, the DataConnector
can be deleted by [CollectionService.DeleteCollection] API. |
RUNNING
|
The connector is actively syncing records from the data source. |
WARNING
|
The connector has completed a sync run, but encountered non-fatal errors. |
INITIALIZATION_FAILED
|
Connector initialization failed. Potential causes include runtime errors or issues in the asynchronous pipeline, preventing the request from reaching downstream services (except for some connector types). |
UPDATING
|
Connector is in the process of an update. |
DateTime
Represents civil time (or occasionally physical time).
This type can represent a civil time in one of a few possible ways:
- When utcOffset is set and timeZone is unset: a civil time on a calendar day with a particular offset from UTC.
- When timeZone is set and utcOffset is unset: a civil time on a calendar day in a particular time zone.
- When neither timeZone nor utcOffset is set: a civil time on a calendar day in local time.
The date is relative to the Proleptic Gregorian Calendar.
If year, month, or day are 0, the DateTime is considered not to have a specific year, month, or day respectively.
This type may also be used to represent a physical time if all the date and time fields are set and either case of the time_offset
oneof is set. Consider using timestamp
message for physical time instead. If your use case also would like to store the user's timezone, that can be done in another field.
This type is more flexible than some applications may want. Make sure to document and validate your application's limitations.
JSON representation |
---|
{ "year" : integer , "month" : integer , "day" : integer , "hours" : integer , "minutes" : integer , "seconds" : integer , "nanos" : integer , // Union field |
year
integer
Optional. year of date. Must be from 1 to 9999, or 0 if specifying a datetime without a year.
month
integer
Optional. Month of year. Must be from 1 to 12, or 0 if specifying a datetime without a month.
day
integer
Optional. Day of month. Must be from 1 to 31 and valid for the year and month, or 0 if specifying a datetime without a day.
hours
integer
Optional. Hours of day in 24 hour format. Should be from 0 to 23, defaults to 0 (midnight). An API may choose to allow the value "24:00:00" for scenarios like business closing time.
minutes
integer
Optional. Minutes of hour of day. Must be from 0 to 59, defaults to 0.
seconds
integer
Optional. Seconds of minutes of the time. Must normally be from 0 to 59, defaults to 0. An API may allow the value 60 if it allows leap-seconds.
nanos
integer
Optional. Fractions of seconds in nanoseconds. Must be from 0 to 999,999,999, defaults to 0.
time_offset
. Optional. Specifies either the UTC offset or the time zone of the DateTime. Choose carefully between them, considering that time zone data may change in the future (for example, a country modifies their DST start/end dates, and future DateTimes in the affected range had already been stored). If omitted, the DateTime is considered to be in local time. time_offset
can be only one of the following:utcOffset
string (
Duration
format)
UTC offset. Must be whole seconds, between -18 hours and +18 hours. For example, a UTC offset of -4:00 would be represented as { seconds: -14400 }.
A duration in seconds with up to nine fractional digits, ending with ' s
'. Example: "3.5s"
.
timeZone
object (
TimeZone
)
time zone.
TimeZone
Represents a time zone from the IANA time Zone Database .
JSON representation |
---|
{ "id" : string , "version" : string } |
Fields | |
---|---|
id
|
IANA time Zone Database time zone. For example "America/New_York". |
version
|
Optional. IANA time Zone Database version number. For example "2019a". |
SourceEntity
Represents an entity in the data source. For example, the Account
object in Salesforce.
JSON representation |
---|
{ "entityName" : string , "keyPropertyMappings" : { string : string , ... } , "dataStore" : string , "params" : { object } , "startingSchema" : { object ( |
entityName
string
The name of the entity. Supported values by data source:
- Salesforce:
Lead
,Opportunity
,Contact
,Account
,Case
,Contract
,Campaign
- Jira:
Issue
- Confluence:
Content
,Space
keyPropertyMappings
map (key: string, value: string)
Attributes for indexing. Key: Field name. value: The key property to map a field to, such as title
, and description
. Supported key properties:
-
title
: The title for data record. This would be displayed on search results. -
description
: The description for data record. This would be displayed on search results.
dataStore
string
Output only. The full resource name of the associated data store for the source entity. Format: projects/*/locations/*/collections/*/dataStores/*
. When the connector is initialized by the DataConnectorService.SetUpDataConnector
method, a DataStore
is automatically created for each source entity.
params
object (
Struct
format)
The parameters for the entity to facilitate data ingestion. E.g. for BigQuery connectors:
- Key:
document_id_column
- value: type STRING. The value of the column id.
startingSchema
object (
Schema
)
Optional. The start schema to use for the DataStore
created from this SourceEntity. If unset, a default vertical specialized schema will be used.
This field is only used by [SetUpDataConnector][] API, and will be ignored if used in other APIs. This field will be omitted from all API responses including [collections.getDataConnector][] API. To retrieve a schema of a DataStore
, use SchemaService.GetSchema
API instead.
The provided schema will be validated against certain rules on schema. Learn more from this doc .
healthcareFhirConfig
object (
HealthcareFhirConfig
)
Optional. Configuration for HEALTHCARE_FHIR
vertical.
SyncMode
Defines the data synchronization mode supported by the data connector.
Enums | |
---|---|
PERIODIC
|
The connector will sync data periodically based on the refreshInterval. Use it with autoRunDisabled to pause the periodic sync, or indicate a one-time sync. |
STREAMING
|
The data will be synced in real time. |
UNSPECIFIED
|
Connector that doesn't ingest data will have this value |
DestinationConfig
Defines target endpoints used to connect to third-party sources.
JSON representation |
---|
{
"key"
:
string
,
"destinations"
:
[
{
object (
|
Fields | |
---|---|
key
|
Optional. Unique destination identifier that is supported by the connector. |
destinations[]
|
Optional. The destinations for the corresponding key. |
params
|
Optional. Additional parameters for this destination config. |
Destination
Defines a target endpoint
JSON representation |
---|
{ "port" : integer , // Union field |
port
integer
Optional. Target port number accepted by the destination.
destination
. The endpoint destination. destination
can be only one of the following:host
string
Publicly routable host.
ActionConfig
Informations to support actions on the connector.
JSON representation |
---|
{ "isActionConfigured" : boolean , "actionParams" : { object } , "serviceName" : string , "useStaticSecrets" : boolean } |
isActionConfigured
boolean
Output only. The connector contains the necessary parameters and is configured to support actions.
actionParams
object (
Struct
format)
Required. params needed to support actions in the format of (Key, value) pairs.
Required parameters for sources that support OAUTH, i.e. gmail
, google_calendar
, jira
, workday
, salesforce
, confluence
:
- Key:
clientId
- value: type STRING. The client id for the service provider to identify your application.
- Key:
clientSecret
- value:type STRING. The client secret generated by the application's authorization server.
serviceName
string
Optional. The Service Directory resource name (projects/*/locations/*/namespaces/*/services/*) representing a VPC network endpoint used to connect to the data source's instance_uri
, defined in DataConnector.params
.
Required when VPC Service Controls are enabled.
useStaticSecrets
boolean
Optional. Whether to use static secrets for the connector. If true, the secrets provided in the actionParams will be ignored.
EndUserConfig
Any params and credentials used specifically for EUA connectors.
JSON representation |
---|
{
"authParams"
:
{
object
}
,
"additionalParams"
:
{
object
}
,
"tenant"
:
{
object (
|
Fields | |
---|---|
authParams
|
Optional. Any authentication parameters specific to EUA connectors. |
additionalParams
|
Optional. Any additional parameters needed for EUA. |
tenant
|
Optional. The tenant project the connector is connected to. |
Tenant
Tenant information for a connector source. This includes some of the same information stored in the Credential message, but is limited to only what is needed to provide a list of accessible tenants to the user.
JSON representation |
---|
{ "id" : string , "uri" : string , "displayName" : string } |
Fields | |
---|---|
id
|
The tenant's instance id. Examples: Jira ("8594f221-9797-5f78-1fa4-485e198d7cd0"), Slack ("T123456"). |
uri
|
The URI of the tenant, if applicable. For example, the URI of a Jira instance is https://my-jira-instance.atlassian.net , and a Slack tenant does not have a URI. |
displayName
|
Optional display name for the tenant, e.g. "My Slack Team". |
FederatedConfig
Any params and credentials used specifically for hybrid connectors supporting FEDERATED mode.
JSON representation |
---|
{ "authParams" : { object } , "additionalParams" : { object } } |
Fields | |
---|---|
authParams
|
Optional. Any authentication parameters specific to FEDERATED. |
additionalParams
|
Optional. Any additional parameters needed for FEDERATED. |
ConnectorMode
The modes enabled for this connector. INGESTION, ACTIONS, FEDERATED, OR EUA
Enums | |
---|---|
CONNECTOR_MODE_UNSPECIFIED
|
Connector mode unspecified. |
DATA_INGESTION
|
Connector utilized for data ingestion. |
ACTIONS
|
Connector utilized for actions. |
FEDERATED
|
Connector utilized for federated search. |
EUA
|
Connector utilized for End User Authentication. |
FEDERATED_AND_EUA
|
Hybrid connector utilized for federated search and End User Authentication. |
BlockingReason
User actions that must be completed before the connector can start syncing data.
Enums | |
---|---|
BLOCKING_REASON_UNSPECIFIED
|
Default value. |
ALLOWLIST_STATIC_IP
|
Connector requires customer to allowlist static IPs. |
ALLOWLIST_IN_SERVICE_ATTACHMENT
|
Connector requires customer to allowlist our project in their service attachment. |
ALLOWLIST_SERVICE_ACCOUNT
|
Connector requires customer to allowlist service account in their project. |
IdentityScheduleConfig
The configuration for the identity data synchronization runs.
JSON representation |
---|
{
"refreshInterval"
:
string
,
"nextSyncTime"
:
{
object (
|
Fields | |
---|---|
refreshInterval
|
Optional. The refresh interval to sync the Access Control List information for the documents ingested by this connector. If not set, the access control list will be refreshed at the default interval of 30 minutes. The identity refresh interval can be at least 30 minutes and at most 7 days. A duration in seconds with up to nine fractional digits, ending with ' |
nextSyncTime
|
Optional. The UTC time when the next data sync is expected to start for the data Connector. Customers are only able to specify the hour and minute to schedule the data sync. This is utilized when the data connector has a refresh interval greater than 1 day. |
BAPConfig
The configuration for the BAP connector.
JSON representation |
---|
{
"supportedConnectorModes"
:
[
enum (
|
Fields | |
---|---|
supportedConnectorModes[]
|
Required. The supported connector modes for the associated BAP connection. |
ConnectorMode
Use cases supported by this BAP connector.
Enums | |
---|---|
CONNECTOR_MODE_UNSPECIFIED
|
Connector mode unspecified. |
DATA_INGESTION
|
Connector utilized for data ingestion. |
ACTIONS
|
Connector utilized for Actions |
END_USER_AUTHENTICATION
|
Connector utilized for End User Authentication. |
ConnectorType
The type of connector.
Enums | |
---|---|
CONNECTOR_TYPE_UNSPECIFIED
|
Default value. |
THIRD_PARTY
|
Third party connector to connector to third party application. |
GCP_FHIR
|
data connector connects between FHIR store and VAIS datastore. |
BIG_QUERY
|
Big query connector. |
GCS
|
Google Cloud Storage connector. |
GOOGLE_MAIL
|
Gmail connector. |
GOOGLE_CALENDAR
|
Google Calendar connector. |
GOOGLE_DRIVE
|
Google Drive connector. |
NATIVE_CLOUD_IDENTITY
|
Native Cloud Identity connector for people search powered by People API. |
THIRD_PARTY_FEDERATED
|
Federated connector, it is a third party connector that doesn't ingestion data, and search is powered by third party application's API. |
THIRD_PARTY_EUA
|
Connector utilized for End User Authentication features. |
GCNV
|
Google Cloud NetApp Volumes connector. |
AlertPolicyConfig
The connector level alert config.
JSON representation |
---|
{
"alertPolicyName"
:
string
,
"alertEnrollments"
:
[
{
object (
|
Fields | |
---|---|
alertPolicyName
|
Immutable. The fully qualified resource name of the AlertPolicy. |
alertEnrollments[]
|
Optional. The enrollment states of each alert. |
AlertEnrollment
The alert enrollment status.
JSON representation |
---|
{
"alertId"
:
string
,
"enrollState"
:
enum (
|
Fields | |
---|---|
alertId
|
Immutable. The ID of an alert. |
enrollState
|
Required. The enrollment status of a customer. |
EnrollStates
The enrollment status enum for the alert.
Enums | |
---|---|
ENROLL_STATES_UNSPECIFIED
|
Default value. Used for customers who have not responded to the alert policy. |
ENROLLED
|
Customer is enrolled in this policy. |
DECLINED
|
Customer declined this policy. |
RealtimeSyncConfig
The configuration for realtime sync to store additional params for realtime sync.
JSON representation |
---|
{
"webhookUri"
:
string
,
"realtimeSyncSecret"
:
string
,
"streamingError"
:
{
object (
|
Fields | |
---|---|
webhookUri
|
Optional. Webhook url for the connector to specify additional params for realtime sync. |
realtimeSyncSecret
|
Optional. The ID of the Secret Manager secret used for webhook secret. |
streamingError
|
Optional. Streaming error details. |
StreamingError
Streaming error details.
JSON representation |
---|
{ "streamingErrorReason" : enum ( |
Fields | |
---|---|
streamingErrorReason
|
Optional. Streaming error. |
error
|
Optional. Error details. |
StreamingErrorReason
Broad types of streaming errors.
Enums | |
---|---|
STREAMING_ERROR_REASON_UNSPECIFIED
|
Streaming error reason unspecified. |
STREAMING_SETUP_ERROR
|
Some error occurred while setting up resources for realtime sync. |
STREAMING_SYNC_ERROR
|
Some error was encountered while running realtime sync for the connector. |
INGRESS_ENDPOINT_REQUIRED
|
Ingress endpoint is required when setting up realtime sync in private connectivity. |
Methods |
|
---|---|
|
Deletes a Collection
. |
|
Gets a Collection
. |
|
Gets the DataConnector
. |
|
Gets a list of Collection
s. |
|
Updates a Collection
. |
|
Updates a DataConnector
. |