- Resource: Topic
- MessageStoragePolicy
- SchemaSettings
- Encoding
- State
- IngestionDataSourceSettings
- AwsKinesis
- State
- CloudStorage
- State
- TextFormat
- AvroFormat
- PubSubAvroFormat
- AzureEventHubs
- State
- AwsMsk
- State
- ConfluentCloud
- State
- PlatformLogsSettings
- Severity
- Methods
Resource: Topic
A topic resource.
JSON representation |
---|
{ "name" : string , "labels" : { string : string , ... } , "messageStoragePolicy" : { object ( |
Fields | |
---|---|
name
|
Required. Name of the topic. Format is |
labels
|
Optional. See Creating and managing labels . An object containing a list of |
kmsKeyName
|
Optional. The resource name of the Cloud KMS CryptoKey to be used to protect access to messages published on this topic. The expected format is |
schemaSettings
|
Optional. Settings for validating messages published against a schema. |
satisfiesPzs
|
Optional. Reserved for future use. This field is set only in responses from the server; it is ignored if it is set in any requests. |
state
|
Output only. An output-only field indicating the state of the topic. |
ingestionDataSourceSettings
|
Optional. Settings for ingestion from a data source into this topic. |
MessageStoragePolicy
A policy constraining the storage of messages published to the topic.
JSON representation |
---|
{ "allowedPersistenceRegions" : [ string ] , "enforceInTransit" : boolean } |
Fields | |
---|---|
allowedPersistenceRegions[]
|
Optional. A list of IDs of Google Cloud regions where messages that are published to the topic may be persisted in storage. Messages published by publishers running in non-allowed Google Cloud regions (or running outside of Google Cloud altogether) are routed for storage in one of the allowed regions. An empty list means that no regions are allowed, and is not a valid configuration. |
enforceInTransit
|
Optional. If true, |
SchemaSettings
Settings for validating messages published against a schema.
JSON representation |
---|
{
"schema"
:
string
,
"encoding"
:
enum (
|
Fields | |
---|---|
schema
|
Required. The name of the schema that messages published should be validated against. Format is |
encoding
|
Optional. The encoding of messages validated against |
firstRevisionId
|
Optional. The minimum (inclusive) revision allowed for validating messages. If empty or not present, allow any revision to be validated against last_revision or any revision created before. |
lastRevisionId
|
Optional. The maximum (inclusive) revision allowed for validating messages. If empty or not present, allow any revision to be validated against first_revision or any revision created after. |
Encoding
Possible encoding types for messages.
Enums | |
---|---|
ENCODING_UNSPECIFIED
|
Unspecified |
JSON
|
JSON encoding |
BINARY
|
Binary encoding, as defined by the schema type. For some schema types, binary encoding may not be available. |
State
The state of the topic.
Enums | |
---|---|
STATE_UNSPECIFIED
|
Default value. This value is unused. |
ACTIVE
|
The topic does not have any persistent errors. |
INGESTION_RESOURCE_ERROR
|
Ingestion from the data source has encountered a permanent error. See the more detailed error state in the corresponding ingestion source configuration. |
IngestionDataSourceSettings
Settings for an ingestion data source on a topic.
JSON representation |
---|
{ "platformLogsSettings" : { object ( |
platformLogsSettings
object (
PlatformLogsSettings
)
Optional. Platform Logs settings. If unset, no Platform Logs will be generated.
source
. Only one source type can have settings set. source
can be only one of the following:awsKinesis
object (
AwsKinesis
)
Optional. Amazon Kinesis Data Streams.
cloudStorage
object (
CloudStorage
)
Optional. Cloud Storage.
azureEventHubs
object (
AzureEventHubs
)
Optional. Azure Event Hubs.
awsMsk
object (
AwsMsk
)
Optional. Amazon MSK.
confluentCloud
object (
ConfluentCloud
)
Optional. Confluent Cloud.
AwsKinesis
Ingestion settings for Amazon Kinesis Data Streams.
JSON representation |
---|
{
"state"
:
enum (
|
Fields | |
---|---|
state
|
Output only. An output-only field that indicates the state of the Kinesis ingestion source. |
streamArn
|
Required. The Kinesis stream ARN to ingest data from. |
consumerArn
|
Required. The Kinesis consumer ARN to used for ingestion in Enhanced Fan-Out mode. The consumer must be already created and ready to be used. |
awsRoleArn
|
Required. AWS role ARN to be used for Federated Identity authentication with Kinesis. Check the Pub/Sub docs for how to set up this role and the required permissions that need to be attached to it. |
gcpServiceAccount
|
Required. The GCP service account to be used for Federated Identity authentication with Kinesis (via a |
State
Possible states for ingestion from Amazon Kinesis Data Streams.
Enums | |
---|---|
STATE_UNSPECIFIED
|
Default value. This value is unused. |
ACTIVE
|
Ingestion is active. |
KINESIS_PERMISSION_DENIED
|
Permission denied encountered while consuming data from Kinesis. This can happen if: - The provided awsRoleArn
does not exist or does not have the appropriate permissions attached. - The provided awsRoleArn
is not set up properly for Identity Federation using gcpServiceAccount
. - The Pub/Sub SA is not granted the iam.serviceAccounts.getOpenIdToken
permission on gcpServiceAccount
. |
PUBLISH_PERMISSION_DENIED
|
Permission denied encountered while publishing to the topic. This can happen if the Pub/Sub SA has not been granted the appropriate publish permissions |
STREAM_NOT_FOUND
|
The Kinesis stream does not exist. |
CONSUMER_NOT_FOUND
|
The Kinesis consumer does not exist. |
CloudStorage
Ingestion settings for Cloud Storage.
JSON representation |
---|
{ "state" : enum ( |
state
enum (
State
)
Output only. An output-only field that indicates the state of the Cloud Storage ingestion source.
bucket
string
Optional. Cloud Storage bucket. The bucket name must be without any prefix like "gs://". See the bucket naming requirements .
minimumObjectCreateTime
string (
Timestamp
format)
Optional. Only objects with a larger or equal creation timestamp will be ingested.
Uses RFC 3339, where generated output will always be Z-normalized and uses 0, 3, 6 or 9 fractional digits. Offsets other than "Z" are also accepted. Examples: "2014-10-02T15:01:23Z"
, "2014-10-02T15:01:23.045123456Z"
or "2014-10-02T15:01:23+05:30"
.
matchGlob
string
Optional. Glob pattern used to match objects that will be ingested. If unset, all objects will be ingested. See the supported patterns .
input_format
. Defaults to text format. input_format
can be only one of the following:textFormat
object (
TextFormat
)
Optional. Data from Cloud Storage will be interpreted as text.
avroFormat
object (
AvroFormat
)
Optional. Data from Cloud Storage will be interpreted in Avro format.
pubsubAvroFormat
object (
PubSubAvroFormat
)
Optional. It will be assumed data from Cloud Storage was written via Cloud Storage subscriptions .
State
Possible states for ingestion from Cloud Storage.
Enums | |
---|---|
STATE_UNSPECIFIED
|
Default value. This value is unused. |
ACTIVE
|
Ingestion is active. |
CLOUD_STORAGE_PERMISSION_DENIED
|
Permission denied encountered while calling the Cloud Storage API. This can happen if the Pub/Sub SA has not been granted the appropriate permissions : - storage.objects.list: to list the objects in a bucket. - storage.objects.get: to read the objects in a bucket. - storage.buckets.get: to verify the bucket exists. |
PUBLISH_PERMISSION_DENIED
|
Permission denied encountered while publishing to the topic. This can happen if the Pub/Sub SA has not been granted the appropriate publish permissions |
BUCKET_NOT_FOUND
|
The provided Cloud Storage bucket doesn't exist. |
TOO_MANY_OBJECTS
|
The Cloud Storage bucket has too many objects, ingestion will be paused. |
TextFormat
Configuration for reading Cloud Storage data in text format. Each line of text as specified by the delimiter will be set to the data
field of a Pub/Sub message.
JSON representation |
---|
{ "delimiter" : string } |
Fields | |
---|---|
delimiter
|
Optional. When unset, '\n' is used. |
AvroFormat
This type has no fields.
Configuration for reading Cloud Storage data in Avro binary format. The bytes of each object will be set to the data
field of a Pub/Sub message.
PubSubAvroFormat
This type has no fields.
Configuration for reading Cloud Storage data written via Cloud Storage subscriptions . The data and attributes fields of the originally exported Pub/Sub message will be restored when publishing.
AzureEventHubs
Ingestion settings for Azure Event Hubs.
JSON representation |
---|
{
"state"
:
enum (
|
Fields | |
---|---|
state
|
Output only. An output-only field that indicates the state of the Event Hubs ingestion source. |
resourceGroup
|
Optional. Name of the resource group within the azure subscription. |
namespace
|
Optional. The name of the Event Hubs namespace. |
eventHub
|
Optional. The name of the Event Hub. |
clientId
|
Optional. The client id of the Azure application that is being used to authenticate Pub/Sub. |
tenantId
|
Optional. The tenant id of the Azure application that is being used to authenticate Pub/Sub. |
subscriptionId
|
Optional. The Azure subscription id. |
gcpServiceAccount
|
Optional. The GCP service account to be used for Federated Identity authentication. |
State
Possible states for managed ingestion from Event Hubs.
Enums | |
---|---|
STATE_UNSPECIFIED
|
Default value. This value is unused. |
ACTIVE
|
Ingestion is active. |
EVENT_HUBS_PERMISSION_DENIED
|
Permission denied encountered while consuming data from Event Hubs. This can happen when clientId
, or tenantId
are invalid. Or the right permissions haven't been granted. |
PUBLISH_PERMISSION_DENIED
|
Permission denied encountered while publishing to the topic. |
NAMESPACE_NOT_FOUND
|
The provided Event Hubs namespace couldn't be found. |
EVENT_HUB_NOT_FOUND
|
The provided Event Hub couldn't be found. |
SUBSCRIPTION_NOT_FOUND
|
The provided Event Hubs subscription couldn't be found. |
RESOURCE_GROUP_NOT_FOUND
|
The provided Event Hubs resource group couldn't be found. |
AwsMsk
Ingestion settings for Amazon MSK.
JSON representation |
---|
{
"state"
:
enum (
|
Fields | |
---|---|
state
|
Output only. An output-only field that indicates the state of the Amazon MSK ingestion source. |
clusterArn
|
Required. The Amazon Resource Name (ARN) that uniquely identifies the cluster. |
topic
|
Required. The name of the topic in the Amazon MSK cluster that Pub/Sub will import from. |
awsRoleArn
|
Required. AWS role ARN to be used for Federated Identity authentication with Amazon MSK. Check the Pub/Sub docs for how to set up this role and the required permissions that need to be attached to it. |
gcpServiceAccount
|
Required. The GCP service account to be used for Federated Identity authentication with Amazon MSK (via a |
State
Possible states for managed ingestion from Amazon MSK.
Enums | |
---|---|
STATE_UNSPECIFIED
|
Default value. This value is unused. |
ACTIVE
|
Ingestion is active. |
MSK_PERMISSION_DENIED
|
Permission denied encountered while consuming data from Amazon MSK. |
PUBLISH_PERMISSION_DENIED
|
Permission denied encountered while publishing to the topic. |
CLUSTER_NOT_FOUND
|
The provided MSK cluster wasn't found. |
TOPIC_NOT_FOUND
|
The provided topic wasn't found. |
ConfluentCloud
Ingestion settings for Confluent Cloud.
JSON representation |
---|
{
"state"
:
enum (
|
Fields | |
---|---|
state
|
Output only. An output-only field that indicates the state of the Confluent Cloud ingestion source. |
bootstrapServer
|
Required. The address of the bootstrap server. The format is url:port. |
clusterId
|
Required. The id of the cluster. |
topic
|
Required. The name of the topic in the Confluent Cloud cluster that Pub/Sub will import from. |
identityPoolId
|
Required. The id of the identity pool to be used for Federated Identity authentication with Confluent Cloud. See https://docs.confluent.io/cloud/current/security/authenticate/workload-identities/identity-providers/oauth/identity-pools.html#add-oauth-identity-pools . |
gcpServiceAccount
|
Required. The GCP service account to be used for Federated Identity authentication with |
State
Possible states for managed ingestion from Confluent Cloud.
Enums | |
---|---|
STATE_UNSPECIFIED
|
Default value. This value is unused. |
ACTIVE
|
Ingestion is active. |
CONFLUENT_CLOUD_PERMISSION_DENIED
|
Permission denied encountered while consuming data from Confluent Cloud. |
PUBLISH_PERMISSION_DENIED
|
Permission denied encountered while publishing to the topic. |
UNREACHABLE_BOOTSTRAP_SERVER
|
The provided bootstrap server address is unreachable. |
CLUSTER_NOT_FOUND
|
The provided cluster wasn't found. |
TOPIC_NOT_FOUND
|
The provided topic wasn't found. |
PlatformLogsSettings
Settings for Platform Logs produced by Pub/Sub.
JSON representation |
---|
{
"severity"
:
enum (
|
Fields | |
---|---|
severity
|
Optional. The minimum severity level of Platform Logs that will be written. |
Severity
Severity levels of Platform Logs.
Enums | |
---|---|
SEVERITY_UNSPECIFIED
|
Default value. Logs level is unspecified. Logs will be disabled. |
DISABLED
|
Logs will be disabled. |
DEBUG
|
Debug logs and higher-severity logs will be written. |
INFO
|
Info logs and higher-severity logs will be written. |
WARNING
|
Warning logs and higher-severity logs will be written. |
ERROR
|
Only error logs will be written. |
Methods |
|
---|---|
|
Creates the given topic with the given name. |
|
Deletes the topic with the given name. |
|
Gets the configuration of a topic. |
|
Gets the access control policy for a resource. |
|
Lists matching topics. |
|
Updates an existing topic by updating the fields specified in the update mask. |
|
Adds one or more messages to the topic. |
|
Sets the access control policy on the specified resource. |
|
Returns permissions that a caller has on the specified resource. |