Create a topic with SMTs

This document explains how to create a Pub/Sub topic with Single Message Transforms (SMTs).

Topic SMTs allow for lightweight modifications to message data and attributes directly within Pub/Sub. This feature enables data cleaning, filtering, or format conversion before the messages are published to the topic.

To create a topic with SMTs, you can use the Google Cloud console, the Google Cloud CLI, the client library, or the Pub/Sub API.

Before you begin

Learn about the Pub/Sub service and its terminology .
Learn about SMTs .

Required roles and permissions

To get the permissions that you need to create a topic with SMTs, ask your administrator to grant you the Pub/Sub Editor ( roles/pubsub.editor ) IAM role on your project. For more information about granting roles, see Manage access to projects, folders, and organizations .

This predefined role contains the permissions required to create a topic with SMTs. To see the exact permissions that are required, expand the Required permissionssection:

Required permissions

The following permissions are required to create a topic with SMTs:

Grant the create a topic permission on the project: pubsub.topics.create

You might also be able to get these permissions with custom roles or other predefined roles .

You can configure access control at the project level and at the individual resource level.

Create a topic with SMTs

Before you create a topic with SMTs, review the documentation for Properties of a topic .

The following samples assume that you want to create a topic with this User Defined Function (UDF) SMT. For more information about UDFs, see the UDFs overview .

  function 
  
 redactSSN 
 ( 
 message 
 , 
  
 metadata 
 ) 
  
 { 
  
 const 
  
 data 
  
 = 
  
 JSON 
 . 
 parse 
 ( 
 message 
 . 
 data 
 ); 
  
 delete 
  
 data 
 [ 
 'ssn' 
 ]; 
  
 message 
 . 
 data 
  
 = 
  
 JSON 
 . 
 stringify 
 ( 
 data 
 ); 
  
 return 
  
 message 
 ; 
 }

Console

In the Google Cloud console, go to the Pub/Sub Topicspage.

Go to Topics
Click Create topic .

The Create topic page opens.
In the Topic ID field, enter an ID for your topic. For more information about naming topics, see the naming guidelines .
Under Transforms , click Add a transform .
Enter a function name. For example: redactSSN .
If you don't want to use the SMT with your topic immediately, click the Disable transform option. This will still save the SMT, but it will not be executed as messages flow through your topic.

Enter a new transform. For example:

 function 
  
 redactSSN 
 ( 
 message 
 , 
  
 metadata 
 ) 
  
 { 
  
 const 
  
 data 
  
 = 
  
 JSON 
 . 
 parse 
 ( 
 message 
 . 
 data 
 ); 
  
 delete 
  
 data 
 [ 
 'ssn' 
 ]; 
  
 message 
 . 
 data 
  
 = 
  
 JSON 
 . 
 stringify 
 ( 
 data 
 ); 
  
 return 
  
 message 
 ; 
 }

Pub/Sub provides a validate function that lets you validate an SMT. Click Validate to validate the transform.
If you want to add another transform, click Add a transform .
To arrange all the SMTs in a specific order, you can use the up and down arrows. To remove an SMT, click the delete button.
Pub/Sub provides a test function that lets you check the result of running the SMT on a sample message. To test the SMTs, click Test transform .
In the Test transform window, select the function that you want to test.
In the Input message window, enter a sample message.
If you want to add message attributes, click Add an attribute and enter one or more key-value pairs.
Click Test . The result of applying the SMTs on the message is displayed.
Close the window to stop testing SMTs on sample messages.
Click Create to create the topic.

gcloud

In the Google Cloud console, activate Cloud Shell.

Activate Cloud Shell

At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

Pub/Sub provides a validate function that lets you validate an SMT. Run the gcloud pubsub message-transforms validate command:

gcloud  
pubsub  
message-transforms  
validate  
--message-transform-file = 
 TRANSFORM_FILE

Replace the following:

TRANSFORM_FILE : The path to the YAML or JSON file containing a single SMT.

Here is an example of a YAML transform file:

 - 
  
 javascriptUdf 
 : 
  
 code 
 : 
  
 > 
  
 function redactSSN(message, metadata) { 
  
 const data = JSON.parse(message.data); 
  
 delete data['ssn']; 
  
 message.data = JSON.stringify(data); 
  
 return message; 
  
 } 
  
 functionName 
 : 
  
 redactSSN

Pub/Sub provides a test function that lets you check the result of running one or more SMTs on a sample message. Run the gcloud pubsub message-transforms test command:

gcloud  
pubsub  
message-transforms  
 test 
  
--message-transforms-file = 
 TRANSFORMS_FILE 
  
--message = 
 MESSAGE 
  
--attributes = 
 ATTRIBUTES

Replace the following:

TRANSFORMS_FILE : The path to the YAML or JSON file containing one or more SMTs.

Here is an example of a YAML transforms file:

 - 
  
 javascriptUdf 
 : 
  
 code 
 : 
  
 > 
  
 function redactSSN(message, metadata) { 
  
 const data = JSON.parse(message.data); 
  
 delete data['ssn']; 
  
 message.data = JSON.stringify(data); 
  
 return message; 
  
 } 
  
 functionName 
 : 
  
 redactSSN

MESSAGE : Message body with which to test the SMT(s).
ATTRIBUTES : Message attributes with which to test the SMT(s).

To create the topic, run the gcloud pubsub topics create command:

gcloud  
pubsub  
topics  
create  
 TOPIC_ID 
  
 \ 
  
--message-transforms-file = 
 TRANSFORMS_FILE

Replace the following:

TOPIC_ID : The ID or name of the topic you want to create. For guidelines on how to name a topic, see Resource names . The name of a topic is immutable.

TRANSFORMS_FILE : The path to the YAML or JSON file containing one or more SMTs.

Here is an example of a YAML transforms file:

 - 
  
 javascriptUdf 
 : 
  
 code 
 : 
  
 > 
  
 function redactSSN(message, metadata) { 
  
 const data = JSON.parse(message.data); 
  
 delete data['ssn']; 
  
 message.data = JSON.stringify(data); 
  
 return message; 
  
 } 
  
 functionName 
 : 
  
 redactSSN

Java

Before trying this sample, follow the Java setup instructions in the Pub/Sub quickstart using client libraries . For more information, see the Pub/Sub Java API reference documentation .

To authenticate to Pub/Sub, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .

  import 
  
 com.google.api.gax.rpc. AlreadyExistsException 
 
 ; 
 import 
  
 com.google.cloud.pubsub.v1. TopicAdminClient 
 
 ; 
 import 
  
 com.google.pubsub.v1. JavaScriptUDF 
 
 ; 
 import 
  
 com.google.pubsub.v1. MessageTransform 
 
 ; 
 import 
  
 com.google.pubsub.v1. Topic 
 
 ; 
 import 
  
 com.google.pubsub.v1. TopicName 
 
 ; 
 import 
  
 java.io.IOException 
 ; 
 public 
  
 class 
 CreateTopicWithSmtExample 
  
 { 
  
 public 
  
 static 
  
 void 
  
 main 
 ( 
 String 
 ... 
  
 args 
 ) 
  
 throws 
  
 Exception 
  
 { 
  
 // TODO(developer): Replace these variables before running the sample. 
  
 String 
  
 projectId 
  
 = 
  
 "your-project-id" 
 ; 
  
 String 
  
 topicId 
  
 = 
  
 "your-topic-id" 
 ; 
  
 createTopicWithSmtExample 
 ( 
 projectId 
 , 
  
 topicId 
 ); 
  
 } 
  
 public 
  
 static 
  
 void 
  
 createTopicWithSmtExample 
 ( 
 String 
  
 projectId 
 , 
  
 String 
  
 topicId 
 ) 
  
 throws 
  
 IOException 
  
 { 
  
  TopicName 
 
  
 topicName 
  
 = 
  
  TopicName 
 
 . 
 of 
 ( 
 projectId 
 , 
  
 topicId 
 ); 
  
 // UDF that removes the 'ssn' field, if present 
  
 String 
  
 code 
  
 = 
  
 "function redactSSN(message, metadata) {" 
  
 + 
  
 "  const data = JSON.parse(message.data);" 
  
 + 
  
 "  delete data['ssn'];" 
  
 + 
  
 "  message.data = JSON.stringify(data);" 
  
 + 
  
 "  return message;" 
  
 + 
  
 "}" 
 ; 
  
 String 
  
 functionName 
  
 = 
  
 "redactSSN" 
 ; 
  
  JavaScriptUDF 
 
  
 udf 
  
 = 
  
  JavaScriptUDF 
 
 . 
 newBuilder 
 (). 
  setCode 
 
 ( 
 code 
 ). 
  setFunctionName 
 
 ( 
 functionName 
 ). 
 build 
 (); 
  
  MessageTransform 
 
  
 transform 
  
 = 
  
  MessageTransform 
 
 . 
 newBuilder 
 (). 
  setJavascriptUdf 
 
 ( 
 udf 
 ). 
 build 
 (); 
  
 try 
  
 ( 
  TopicAdminClient 
 
  
 topicAdminClient 
  
 = 
  
  TopicAdminClient 
 
 . 
 create 
 ()) 
  
 { 
  
  Topic 
 
  
 topic 
  
 = 
  
 topicAdminClient 
 . 
 createTopic 
 ( 
  
  Topic 
 
 . 
 newBuilder 
 () 
  
 . 
 setName 
 ( 
 topicName 
 . 
  toString 
 
 ()) 
  
 // Add the UDF message transform 
  
 . 
 addMessageTransforms 
 ( 
 transform 
 ) 
  
 . 
 build 
 ()); 
  
 System 
 . 
 out 
 . 
 println 
 ( 
 "Created topic with SMT: " 
  
 + 
  
 topic 
 . 
  getName 
 
 ()); 
  
 } 
  
 catch 
  
 ( 
  AlreadyExistsException 
 
  
 e 
 ) 
  
 { 
  
 System 
 . 
 out 
 . 
 println 
 ( 
 topicName 
  
 + 
  
 "already exists." 
 ); 
  
 } 
  
 } 
 }

Python

Before trying this sample, follow the Python setup instructions in the Pub/Sub quickstart using client libraries . For more information, see the Pub/Sub Python API reference documentation .

To authenticate to Pub/Sub, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .

  from 
  
 google.cloud 
  
 import 
 pubsub_v1 
 from 
  
 google.pubsub_v1.types 
  
 import 
  JavaScriptUDF 
 
 , 
  MessageTransform 
 
 , 
  Topic 
 
 # TODO(developer) 
 # project_id = "your-project-id" 
 # topic_id = "your-topic-id" 
 code 
 = 
 """function redactSSN(message, metadata) { 
 const data = JSON.parse(message.data); 
 delete data['ssn']; 
 message.data = JSON.stringify(data); 
 return message; 
 }""" 
 udf 
 = 
 JavaScriptUDF 
 ( 
 code 
 = 
 code 
 , 
 function_name 
 = 
 "redactSSN" 
 ) 
 transforms 
 = 
 [ 
 MessageTransform 
 ( 
 javascript_udf 
 = 
 udf 
 )] 
 publisher 
 = 
 pubsub_v1 
 . 
  PublisherClient 
 
 () 
 topic_path 
 = 
 publisher 
 . 
 topic_path 
 ( 
 project_id 
 , 
 topic_id 
 ) 
 request 
 = 
 Topic 
 ( 
 name 
 = 
 topic_path 
 , 
 message_transforms 
 = 
 transforms 
 ) 
 topic 
 = 
 publisher 
 . 
 create_topic 
 ( 
 request 
 = 
 request 
 ) 
 print 
 ( 
 f 
 "Created topic: 
 { 
 topic 
 . 
 name 
 } 
 with SMT" 
 )

Go

Before trying this sample, follow the Go setup instructions in the Pub/Sub quickstart using client libraries . For more information, see the Pub/Sub Go API reference documentation .

To authenticate to Pub/Sub, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .

  // Copyright 2025 Google LLC 
 // 
 // Licensed under the Apache License, Version 2.0 (the "License"); 
 // you may not use this file except in compliance with the License. 
 // You may obtain a copy of the License at 
 // 
 //     https://www.apache.org/licenses/LICENSE-2.0 
 // 
 // Unless required by applicable law or agreed to in writing, software 
 // distributed under the License is distributed on an "AS IS" BASIS, 
 // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 
 // See the License for the specific language governing permissions and 
 // limitations under the License. 
 package 
  
 topics 
 import 
  
 ( 
  
 "context" 
  
 "fmt" 
  
 "io" 
  
 "cloud.google.com/go/pubsub" 
 ) 
 // createTopicWithSMT creates a topic with a single message transform function applied. 
 func 
  
 createTopicWithSMT 
 ( 
 w 
  
 io 
 . 
 Writer 
 , 
  
 projectID 
 , 
  
 topicID 
  
 string 
 ) 
  
 error 
  
 { 
  
 // projectID := "my-project-id" 
  
 // topicID := "my-topic" 
  
 ctx 
  
 := 
  
 context 
 . 
 Background 
 () 
  
 client 
 , 
  
 err 
  
 := 
  
 pubsub 
 . 
  NewClient 
 
 ( 
 ctx 
 , 
  
 projectID 
 ) 
  
 if 
  
 err 
  
 != 
  
 nil 
  
 { 
  
 return 
  
 fmt 
 . 
 Errorf 
 ( 
 "pubsub.NewClient: %w" 
 , 
  
 err 
 ) 
  
 } 
  
 defer 
  
 client 
 . 
 Close 
 () 
  
 code 
  
 := 
  
 `function redactSSN(message, metadata) { 
 const data = JSON.parse(message.data); 
 delete data['ssn']; 
 message.data = JSON.stringify(data); 
 return message; 
 }` 
  
 transform 
  
 := 
  
 pubsub 
 . 
 MessageTransform 
 { 
  
 Transform 
 : 
  
 pubsub 
 . 
 JavaScriptUDF 
 { 
  
 FunctionName 
 : 
  
 "redactSSN" 
 , 
  
 Code 
 : 
  
 code 
 , 
  
 }, 
  
 } 
  
 cfg 
  
 := 
  
& pubsub 
 . 
 TopicConfig 
 { 
  
 MessageTransforms 
 : 
  
 [] 
 pubsub 
 . 
 MessageTransform 
 { 
 transform 
 }, 
  
 } 
  
 t 
 , 
  
 err 
  
 := 
  
 client 
 . 
  CreateTopicWithConfig 
 
 ( 
 ctx 
 , 
  
 topicID 
 , 
  
 cfg 
 ) 
  
 if 
  
 err 
  
 != 
  
 nil 
  
 { 
  
 return 
  
 fmt 
 . 
 Errorf 
 ( 
 "CreateTopic: %w" 
 , 
  
 err 
 ) 
  
 } 
  
 fmt 
 . 
 Fprintf 
 ( 
 w 
 , 
  
 "Created topic with message transform: %v\n" 
 , 
  
 t 
 ) 
  
 return 
  
 nil 
 }

Create a topic with SMTs Stay organized with collections Save and categorize content based on your preferences.

Before you begin

Required roles and permissions

Required permissions

Create a topic with SMTs

Console

gcloud

Java

Python

Go

What's next

Create a topic with SMTs