A Pub/Sub schema is an optional feature that you can use to enforce the format of the data field in a Pub/Sub message.
A schema creates a contract between the publisher and subscriber about the format of the messages. Pub/Sub enforces this format. Schemas facilitate inter-team consumption of data streams in your organization by creating a central authority for message types and permissions. A Pub/Sub message schema defines the names and data types for the fields in a message.
You can create a schema and associate it with a topic to enforce the schema for published messages. If a specific message does not conform to the schema, the message is not published. You can also create additional revisions for a schema.
Types of schemas
You can create a schema in Pub/Sub by using one of the following frameworks:
For example, the following schema defines the inventory of a warehouse, first in the Avro format and then in the Protocol Buffer format.
Apache Avro format
{
"type"
:
"record"
,
"name"
:
"Avro"
,
"fields"
:
[
{
"name"
:
"ProductName"
,
"type"
:
"string"
,
"default"
:
""
},
{
"name"
:
"SKU"
,
"type"
:
"int"
,
"default"
:
0
},
{
"name"
:
"InStock"
,
"type"
:
"boolean"
,
"default"
:
false
}
]
}
Protocol Buffer format
syntax
=
"proto3"
;
message
ProtocolBuffer
{
string
product_name
=
1
;
int32
SKU
=
2
;
bool
in_stock
=
3
;
}
Important information about schemas
The following is some important information regarding creating and managing Pub/Sub schemas.
-
For a schema to work with Pub/Sub, you must define only one top-level type . Import statements that reference other types are not supported.
-
You can associate the same schema to multiple topics.
-
You can manually test whether a message validates against a schema.
-
Don't include sensitive information such as personally identifiable information (PII) or security data in your schema field names.
Quotas and limits for schemas
Schemas have the following limits:
-
The schema definition field cannot exceed 300 KB in size.
-
A project can have a maximum of 10,000 schemas.
-
A single schema cannot have more than 20 revisions.
For information about schemas and limits related to Pub/Sub, see Pub/Sub quotas and limits .
