Publishing failures are typically caused by client-side bottlenecks, such as insufficient service CPUs, bad thread health, or network congestion. The publisher retry policy defines the number of times Pub/Sub tries to deliver a message and the length of time between each attempt.
This document provides information about using retry requests with messages published to a topic.
Before you begin
Before configuring the publish workflow, ensure you have completed the following tasks:
- Learn about topics and the publishing workflow .
- Create a topic .
Required roles
To get the permissions that
you need to retry message requests to a topic,
ask your administrator to grant you the Pub/Sub Publisher
( roles/pubsub.publisher
)
IAM role on the topic.
For more information about granting roles, see Manage access to projects, folders, and organizations
.
You might also be able to get the required permissions through custom roles or other predefined roles .
You need additional permissions to create or update topics and subscriptions.
About retry requests
Retry settings control how the Pub/Sub client libraries retry publish requests. The client libraries have any of the following retry settings:
- Initial request timeout: the amount of time before a client library stops waiting for the initial publish request to complete.
- Retry delay: the amount of time after a request times out that a client library waits to retry the request.
- Total timeout: the amount of time after a client library stops retrying publish requests.
To retry publish requests, the initial request timeout must be shorter than the total timeout. For example, if you're using exponential backoff, the client libraries compute the request timeout and retry delay as follows:
- After each publish request, the request timeout increases by the request timeout multiplier, up to the maximum request timeout.
- After each retry, the retry delay increases by the retry delay multiplier, up to the maximum retry delay.
Retry a message request
During the publishing process, you might see transient or permanent publishing failures. For transient errors, you typically don't need to take any special action as Pub/Sub automatically retries the messages.
An error can also occur when a publish operation succeeds but the publish response is not received in time by the publisher client. In this case too, the publish operation is retried. As a result, you can have two identical messages with different message IDs.
In case of persistent errors, consider implementing appropriate actions outside of the publishing process to avoid overwhelming Pub/Sub.
Publishing failures are automatically retried, except for errors that don't warrant retries. This sample code demonstrates creating a publisher with custom retry settings (note that not all client libraries support custom retry settings; see the API Reference documentation for your chosen language):
C++
Before trying this sample, follow the C++ setup instructions in Quickstart: Using Client Libraries . For more information, see the Pub/Sub C++ API reference documentation .
C#
Before trying this sample, follow the C# setup instructions in Quickstart: Using Client Libraries . For more information, see the Pub/Sub C# API reference documentation .
Go
The following sample uses the major version of the Go Pub/Sub client library (v2). If you are still using the v1 library, see the migration guide to v2 . To see a list of v1 code samples, see the deprecated code samples .
Before trying this sample, follow the Go setup instructions in Quickstart: Using Client Libraries . For more information, see the Pub/Sub Go API reference documentation .
Java
Before trying this sample, follow the Java setup instructions in Quickstart: Using Client Libraries . For more information, see the Pub/Sub Java API reference documentation .
Node.js
Before trying this sample, follow the Node.js setup instructions in Quickstart: Using Client Libraries . For more information, see the Pub/Sub Node.js API reference documentation .
Node.js
Before trying this sample, follow the Node.js setup instructions in Quickstart: Using Client Libraries . For more information, see the Pub/Sub Node.js API reference documentation .
Python
Before trying this sample, follow the Python setup instructions in Quickstart: Using Client Libraries . For more information, see the Pub/Sub Python API reference documentation .
Retry requests with ordering keys
Assume you have a single publisher client. You are using the Pub/Sub client libraries to publish messages 1, 2, and 3 for the same ordering key A. Now, assume that the published response for message 1 is not received by the publisher client before the RPC deadline expires. Message 1 must be republished. The sequence of messages received by the subscriber client then becomes 1, 1, 2, and 3, if you assume message 2 is published only after message 1 gets completed successfully. Each published message has its own message ID. From the subscriber client's perspective, four messages were published, with the first two having identical content.
Retrying publish requests with ordering keys can also be complicated by batch settings. The client library batches messages together for more efficient publishing. Continue with the previous example and assume that messages 1 and 2 are batched together. This batch is sent to the server as a single request. If the server fails to return a response in time, the publisher client retries this batch of two messages. Therefore, it is possible that the subscriber client receives messages 1, 2, 1, 2, and 3. If you use a Pub/Sub client library for publishing messages in order and a publish operation fails, the service fails the publish operations for all remaining messages on the same ordering key. A publisher client can then decide to follow any one of the following operations:
-
Republish all the failed messages in order
-
Republish a subset of the failed messages in order
-
Publish a new set of messages
If a non-retryable error occurs, the client library doesn't publish the message and stops publishing other messages with the same ordering key. For example, when a publisher sends a message to a topic that doesn't exist, a non-retryable error occurs. To continue publishing messages with the same ordering key, call a method to resume publishing and then start publishing again.
The following sample shows you how to resume publishing messages with the same ordering key.
C++
Before trying this sample, follow the C++ setup instructions in Quickstart: Using Client Libraries . For more information, see the Pub/Sub C++ API reference documentation .
C#
Before trying this sample, follow the C# setup instructions in Quickstart: Using Client Libraries . For more information, see the Pub/Sub C# API reference documentation .
Go
The following sample uses the major version of the Go Pub/Sub client library (v2). If you are still using the v1 library, see the migration guide to v2 . To see a list of v1 code samples, see the deprecated code samples .
Before trying this sample, follow the Go setup instructions in Quickstart: Using Client Libraries . For more information, see the Pub/Sub Go API reference documentation .
Java
Before trying this sample, follow the Java setup instructions in Quickstart: Using Client Libraries . For more information, see the Pub/Sub Java API reference documentation .
Node.js
Before trying this sample, follow the Node.js setup instructions in Quickstart: Using Client Libraries . For more information, see the Pub/Sub Node.js API reference documentation .
Python
Before trying this sample, follow the Python setup instructions in Quickstart: Using Client Libraries . For more information, see the Pub/Sub Python API reference documentation .
Ruby
The following sample uses Ruby Pub/Sub client library v3. If you are still using the v2 library, see the migration guide to v3 . To see a list of Ruby v2 code samples, see the deprecated code samples .
Before trying this sample, follow the Ruby setup instructions in Quickstart: Using Client Libraries . For more information, see the Pub/Sub Ruby API reference documentation .
What's next
To learn how to configure advanced publishing options, see the following:
