Schema overview

A Pub/Sub schema is an optional feature that you can use to enforce the format of the data field in a Pub/Sub message.

A schema creates a contract between the publisher and subscriber about the format of the messages. Pub/Sub enforces this format. Schemas facilitate inter-team consumption of data streams in your organization by creating a central authority for message types and permissions. A Pub/Sub message schema defines the names and data types for the fields in a message.

You can create a schema and associate it with a topic to enforce the schema for published messages. If a specific message does not conform to the schema, the message is not published. You can also create additional revisions for a schema.

Types of schemas

You can create a schema in Pub/Sub by using one of the following frameworks:

For example, the following schema defines the inventory of a warehouse, first in the Avro format and then in the Protocol Buffer format.

Apache Avro format

  { 
  
 "type" 
  
 : 
  
 "record" 
 , 
  
 "name" 
  
 : 
  
 "Avro" 
 , 
  
 "fields" 
  
 : 
  
 [ 
  
 { 
  
 "name" 
  
 : 
  
 "ProductName" 
 , 
  
 "type" 
  
 : 
  
 "string" 
 , 
  
 "default" 
 : 
  
 "" 
  
 }, 
  
 { 
  
 "name" 
  
 : 
  
 "SKU" 
 , 
  
 "type" 
  
 : 
  
 "int" 
 , 
  
 "default" 
 : 
  
 0 
  
 }, 
  
 { 
  
 "name" 
  
 : 
  
 "InStock" 
 , 
  
 "type" 
  
 : 
  
 "boolean" 
 , 
  
 "default" 
 : 
  
 false 
  
 } 
  
 ] 
 }

Protocol Buffer format

  syntax 
  
 = 
  
 "proto3" 
 ; 
 message 
  
 ProtocolBuffer 
  
 { 
  
 string 
  
 product_name 
  
 = 
  
 1 
 ; 
  
 int32 
  
 SKU 
  
 = 
  
 2 
 ; 
  
 bool 
  
 in_stock 
  
 = 
  
 3 
 ; 
 }

Important information about schemas

The following is some important information regarding creating and managing Pub/Sub schemas.

For a schema to work with Pub/Sub, you must define only one top-level type . Import statements that reference other types are not supported.
You can associate the same schema to multiple topics.
You can manually test whether a message validates against a schema.
Don't include sensitive information such as personally identifiable information (PII) or security data in your schema field names.

Quotas and limits for schemas

Schemas have the following limits:

The schema definition field cannot exceed 300 KB in size.
A project can have a maximum of 10,000 schemas.
A single schema cannot have more than 20 revisions.

For information about schemas and limits related to Pub/Sub, see Pub/Sub quotas and limits .