Console
    Contact Us Start free

    User Defined Functions (UDFs) overview

    A JavaScript User-Defined Function (UDF)is a type of Single Message Transform (SMT). UDFs provide a flexible way to implement custom transformation logic within Pub/Sub, similar to BigQuery JavaScript UDFs .

    UDFs accept a single message as input, perform the defined actions on the input, and return the result of the process.

    UDFs have the following key properties:

    • Function name:The name of the JavaScript function within the provided code that Pub/Sub applies to messages.

    • Code:The JavaScript code that defines the transformation logic. This code must contain a function with the following signature:

        /** 
 * Transforms a Pub/Sub message. 
 * @return {(Object<string, (string | Object<string, string>)>|* null)} - To 
 * filter a message, return `null`. To transform a message, return a map with 
 * the following keys: 
 *   - (required) 'data' : {string} 
 *   - (optional) 'attributes' : {Object<string, string>} 
 * Returning empty `attributes` will remove all attributes from the message. 
 * 
 * @param  {(Object<string, (string | Object<string, string>)>} - Pub/Sub 
 * message. Keys: 
 *   - (required) 'data' : {string} 
 *   - (required) 'attributes' : {Object<string, string>} 
 * 
 * @param  {Object<string, any>} metadata - Pub/Sub message metadata. 
 * Keys: 
 *   - (optional) 'message_id'  : {string} 
 *   - (optional) 'publish_time': {string} YYYY-MM-DDTHH:MM:SSZ format 
 *   - (optional) 'ordering_key': {string} 
 */ 
 function 
  
< function_name 
> ( 
 message 
 , 
  
 metadata 
 ) 
  
 { 
  
 // Perform custom transformation logic 
  
 return 
  
 message 
 ; 
  
 // to filter a message instead, return `null` 
 }

    Inputs

    • message argument:A JavaScript object representing the Pub/Sub message. It contains the following properties:

      • data : ( String , required) The message payload.

      • attributes : ( Object<String, String> , optional) A map of key-value pairs representing message attributes.

    • metadata argument:A JavaScript object containing immutable metadata about the Pub/Sub message:

      • message_id : ( String , optional) The unique ID of the message.

      • publish_time : ( String , optional) The message's publish time in RFC 3339 format (YYYY-MM-DDTHH:mm:ssZ).

      • ordering_key : ( String , optional) The message's ordering key , if applicable.

    Outputs

    • To transform a message, edit the contents of message.data and message.attributes , and return the altered message object.

    • To filter a message, return null .

    How UDFs transform a message

    The result of running a UDF on a message can be one of the following:

    • The UDF transforms a message.

    • The UDF returns null .

      • Topic SMTs: Pub/Sub returns success to the publisher and includes a message ID in the response for the filtered messages. Pub/Sub does not store the message or send it to any subscribers.

      • Subscription SMTs: Pub/Sub acknowledges the message delivery without sending the message to a subscriber.

    • The UDF throws an error.

      • Topic SMTs: Pub/Sub returns the error to the publisher and does not publish any of the messages.

      • Subscription SMTs: Pub/Sub negatively acknowledges the message.

    Limitations

    Pub/Sub enforces resource limits on UDFs to ensure efficient transformation operations. The limitations include:

    • A maximum of 20 KB of code per UDF
    • A maximum of 500 ms of execution time per message
    • Support for only ECMAScript standard built-ins
    • No calls to external APIs
    • No imports of external libraries

    Sample UDFs

    Here are some sample UDFs for publishing and subscribing.

    Function: Convert a day of the week integer to the corresponding string

    When you add the following UDF to a topic or a subscription, the following changes take place during message publish or delivery:

    1. Pub/Sub applies the function to the message. If the message does not have a JSON payload, the UDF throws an error.

    2. The UDF looks for a field called dayOfWeek and if the value of this field is a number between 0 and 6, converts it to a corresponding day of the week such as Monday . If the field does not exist or the number is not in the range of 0 to 6, the code sets the dayOfWeek field to Unknown .

    3. The UDF serializes the modified payload back into the message.

    4. Pub/Sub passes the updated message to the next step in your pipeline.

      function 
  
 intToString 
 ( 
 message 
 , 
  
 metadata 
 ) 
  
 { 
  
 const 
  
 data 
  
 = 
  
 JSON 
 . 
 parse 
 ( 
 message 
 . 
 data 
 ); 
  
 switch 
 ( 
 `data["dayOfWeek"]` 
 ) 
  
 { 
  
 case 
  
 0 
 : 
  
 data 
 [ 
 "dayOfWeek" 
 ] 
  
 = 
  
 "Sunday" 
 ; 
  
 break 
 ; 
  
 case 
  
 1 
 : 
  
 data 
 [ 
 "dayOfWeek" 
 ] 
  
 = 
  
 "Monday" 
 ; 
  
 break 
 ; 
  
 case 
  
 2 
 : 
  
 data 
 [ 
 "dayOfWeek" 
 ] 
  
 = 
  
 "Tuesday" 
 ; 
  
 break 
 ; 
  
 case 
  
 3 
 : 
  
 data 
 [ 
 "dayOfWeek" 
 ] 
  
 = 
  
 "Wednesday" 
 ; 
  
 break 
 ; 
  
 case 
  
 4 
 : 
  
 data 
 [ 
 "dayOfWeek" 
 ] 
  
 = 
  
 "Thursday" 
 ; 
  
 break 
 ; 
  
 case 
  
 5 
 : 
  
 data 
 [ 
 "dayOfWeek" 
 ] 
  
 = 
  
 "Friday" 
 ; 
  
 break 
 ; 
  
 case 
  
 6 
 : 
  
 data 
 [ 
 "dayOfWeek" 
 ] 
  
 = 
  
 "Saturday" 
 ; 
  
 break 
 ; 
  
 default 
 : 
  
 data 
 [ 
 "dayOfWeek" 
 ] 
  
 = 
  
 "Unknown" 
 ; 
  
 } 
  
 message 
 . 
 data 
  
 = 
  
 JSON 
 . 
 stringify 
 ( 
 data 
 ); 
  
 return 
  
 message 
 ; 
 }

    Function: Redact a social security number

    When you add the following UDF to a topic or a subscription, the following changes take place during message publish or delivery:

    1. Pub/Sub applies the function to the message. If the message does not have a JSON payload, the UDF throws an error.

    2. The UDF removes the field ssn from the message payload (if it exists).

    3. The UDF serializes the modified payload back into the message.

    4. Pub/Sub passes the updated message to the next step in your pipeline.

      function 
  
 redactSSN 
 ( 
 message 
 , 
  
 metadata 
 ) 
  
 { 
  
 const 
  
 data 
  
 = 
  
 JSON 
 . 
 parse 
 ( 
 message 
 . 
 data 
 ); 
  
 delete 
  
 data 
 [ 
 'ssn' 
 ]; 
  
 message 
 . 
 data 
  
 = 
  
 JSON 
 . 
 stringify 
 ( 
 data 
 ); 
  
 return 
  
 message 
 ; 
 }

    Function: Filter out and auto-ack specific messages

    When you add the following UDF to a topic or a subscription, the following changes take place during message publish or delivery:

    1. Pub/Sub applies the function to the message. If the message does not have a JSON payload, the UDF throws an error.

    2. The UDF checks if the payload contains a field called region .

    3. If the value of the region field is not US , the function returns null, causing Pub/Sub to filter the message.

    4. If the value of the region field is US , Pub/Sub passes the original message to the next step in your pipeline.

      function 
  
 filterForUSRegion 
 ( 
 message 
 , 
  
 metadata 
 ) 
  
 { 
  
 const 
  
 data 
  
 = 
  
 JSON 
 . 
 parse 
 ( 
 message 
 . 
 data 
 ); 
  
 if 
  
 ( 
 data 
 [ 
 "region" 
 ] 
  
 !== 
  
 "US" 
 ) 
  
 { 
  
 return 
  
 null 
 ; 
  
 } 
  
 return 
  
 message 
 ; 
 }

    Function: Validate message content to ensure the amount is not greater than 100

    When you add the following UDF to a topic or a subscription, the following changes take place during message publish or delivery:

    1. Pub/Sub applies the function to the message. If the message does not have a JSON payload, the UDF throws an error.

    2. The UDF checks if the message contains a field called amount .

    3. If the value of the amount field is greater than 100 , the function throws an error.

    4. If the value of the amount field is not greater than 100 , the function returns the original message.

    5. Pub/Sub then either marks the message as failed, or passes the original message to the next step in your pipeline.

      function 
  
 validateAmount 
 ( 
 message 
 , 
  
 metadata 
 ) 
  
 { 
  
 const 
  
 data 
  
 = 
  
 JSON 
 . 
 parse 
 ( 
 message 
 . 
 data 
 ); 
  
 if 
  
 ( 
 data 
 [ 
 "amount" 
 ] 
 > 
 100 
 ) 
  
 { 
  
 throw 
  
 new 
  
 Error 
 ( 
 "Amount is invalid" 
 ); 
  
 } 
  
 return 
  
 message 
 ; 
 }

    What's next
    Create a Mobile Website
    View Site in Mobile | Classic
    Share by: