Testing apps locally with the emulator

To develop and test your application locally, you can use the Pub/Sub emulator, which provides local emulation of the production Pub/Sub service. You run the Pub/Sub emulator using the Google Cloud CLI.

To run your application against the emulator, first start the emulator and set the environment variables. Your application must communicate with the emulator instead of the production Pub/Sub service. The resources created and messages published to the emulator are maintained for the lifetime of the emulator session.

Note: To test push subscriptions in the emulator, you can use unencrypted push endpoints (that is, http ), as in the following examples. In production, only secure push endpoints ( https ) are allowed.

Before you begin

Complete the following prerequisites before you use the Pub/Sub emulator:

Set up a Python development environment .
Install a JDK .
Install Google Cloud CLI .
Build an application by using the Cloud Client Libraries .

Install the emulator

Install the emulator from a command prompt:

gcloud components install pubsub-emulator
gcloud components update

Install the emulator as a container image

To install and run the emulator as a container, download and install the gCloud Docker image .

Starting the emulator

Start the emulator by invoking pubsub start from a command prompt. Before running the command, replace PUBSUB_PROJECT_ID with a valid Google Cloud project ID . string. The string doesn't need to represent a real Google Cloud project because the Pub/Sub emulator runs locally.

gcloud beta emulators pubsub start --project= PUBSUB_PROJECT_ID 
[options]

See gcloud beta emulators pubsub start for a complete list of flags.

After you start the emulator, you see a message that resembles the following:

...
[pubsub] This is the Pub/Sub fake.
[pubsub] Implementation may be incomplete or differ from the real system.
...
[pubsub] INFO: Server started, listening on 8085

This message indicates that the Pub/Sub server runs at the emulator endpoint on your local machine instead of the Google Cloud endpoint. All operations happen locally, including the following:

Creating a topic or subscription
Publishing
Subscribing

Setting environment variables

After you start the emulator, you must set the environment variables so that your application connects to the emulator instead of Pub/Sub. Set these environment variables on the same machine that you use to run your application.

You must set the environment variables each time you start the emulator. The environment variables depend on dynamically assigned port numbers that could change when you restart the emulator.

Automatically setting the variables

If your application and the emulator run on the same machine, you can set the environment variables automatically:

Linux / macOS

Run env-init using command substitution:

$(gcloud beta emulators pubsub env-init)

Windows

Create and run a batch file using output from env-init :

gcloud beta emulators pubsub env-init > set_vars.cmd && set_vars.cmd

Your application will now connect to the Pub/Sub emulator.

Manually setting the variables

If your application and the emulator run on different machines, set the environment variables manually:

Run the env-init command:
```
gcloud beta emulators pubsub env-init
```
On the machine that runs your application, set the PUBSUB_EMULATOR_HOST environment variable and value as directed by the output of the env-init command. This configuration connects you application to the emulator. You can optionally set the PUBSUB_PROJECT_ID environment variable for the project that you want to use for the emulator.
Linux / macOS
```
export PUBSUB_EMULATOR_HOST=[::1]:8432
export PUBSUB_PROJECT_ID=my-project-id
```
Windows
```
set PUBSUB_EMULATOR_HOST=[::1]:8432
set PUBSUB_PROJECT_ID=my-project-id
```

Your application will now connect to the Pub/Sub emulator.

Note:If you're using the Python App Engine Standard local development server , you must pass this environment variable on the command line as follows:

 dev_appserver.py  
app.yaml  
--env_var  
 PUBSUB_EMULATOR_HOST 
 = 
 ${ 
 PUBSUB_EMULATOR_HOST 
 }

dev_appserver.py is included in your [PATH_TO_CLOUD_SDK]/google-cloud-sdk/bin/dev_appserver.py .

Using the emulator

To use the emulator, you must have an application built using the Cloud Client Libraries . The emulator does not support Google Cloud console or gcloud pubsub commands.

The following example demonstrates using the emulator and an application that uses the Python Cloud Client Library to perform various operations. Examples of these operations include how to create a topic, publish messages, and read messages.

Complete the following steps on the machine where you set the emulator environment variables :

Get the Pub/Sub Python samples from GitHub by cloning the full Python repository .
In your cloned repository, navigate to the samples/snippets directory. You complete the rest of these steps in this directory.
From within the samples/snippets directory, install the dependencies you need to run the example:
```
pip install -r requirements.txt
```

Create a topic:

python publisher.py PUBSUB_PROJECT_ID 
create TOPIC_ID

(Optional) If you don't have a local push endpoint for testing push subscriptions in the emulator, complete the following steps to create one on http://[::1]:3000/messages .
1. Install JSON Server .
```
npm install -g json-server
```
2. Start JSON Server.
```
json-server --port 3000 --watch db.json
```
  where db.json contains the following starter code:
```
{
   "messages": []
}
```
3. Note down http://[::1]:3000/messages for PUSH_ENDPOINT in the next step.

Create a subscription to the topic:

Create a pull subscription:

python subscriber.py PUBSUB_PROJECT_ID 
create TOPIC_ID 
 SUBSCRIPTION_ID

Create a push subscription:

python subscriber.py PUBSUB_PROJECT_ID 
create-push TOPIC_ID 
 SUBSCRIPTION_ID 
\ PUSH_ENDPOINT

Publish messages to the topic:

python publisher.py PUBSUB_PROJECT_ID 
publish TOPIC_ID

Read the messages published to the topic:

Retrieve messages from your pull subscription:

python subscriber.py PUBSUB_PROJECT_ID 
receive SUBSCRIPTION_ID

Observe the messages delivered to your local push endpoint. For example, messages look like the following:

{
  "messages": [
      {
          "subscription": "projects/ PUBSUB_PROJECT_ID 
/subscriptions/ SUBSCRIPTION_ID 
",
          "message": {
              "data": "TWVzc2FnZSBudW1iZXIgMQ==",
              "messageId": "10",
              "attributes": {}
          },
          "id": 1
      },
      ...
  ]
}

Accessing environment variables

In all languages except for Java and C#, if you have set PUBSUB_EMULATOR_HOST as described in Setting environment variables , the Pub/Sub client libraries automatically call the API running in the local instance rather than Pub/Sub.

However, C# and Java client libraries require you to modify your code to use the emulator:

C#

Before trying this sample, follow the C# setup instructions in the Pub/Sub quickstart using client libraries . For more information, see the Pub/Sub C# API reference documentation .

To authenticate to Pub/Sub, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .

  using 
  
  Google.Api.Gax 
 
 ; 
 using 
  
  Google.Cloud.PubSub.V1 
 
 ; 
 using 
  
 System 
 ; 
 using 
  
 System.Collections.Generic 
 ; 
 using 
  
 System.Threading.Tasks 
 ; 
 public 
  
 class 
  
 EmulatorSupportSample 
 { 
  
 public 
  
 async 
  
 Task 
  
 WithEmulatorAsync 
 ( 
 string 
  
 projectId 
 , 
  
 string 
  
 topicId 
 , 
  
 string 
  
 subscriptionId 
 ) 
  
 { 
  
 // Use EmulatorDetection.EmulatorOrProduction to create service clients that will 
  
 // that will connect to the PubSub emulator if the PUBSUB_EMULATOR_HOST environment 
  
 // variable is set, but will otherwise connect to the production environment. 
  
 // Create the PublisherServiceApiClient using the PublisherServiceApiClientBuilder 
  
 // and setting the EmulatorDection property. 
  
  PublisherServiceApiClient 
 
  
 publisherService 
  
 = 
  
 await 
  
 new 
  
  PublisherServiceApiClientBuilder 
 
  
 { 
  
 EmulatorDetection 
  
 = 
  
  EmulatorDetection 
 
 . 
  EmulatorOrProduction 
 
  
 }. 
 BuildAsync 
 (); 
  
 // Use the client as you'd normally do, to create a topic in this example. 
  
  TopicName 
 
  
 topicName 
  
 = 
  
 new 
  
  TopicName 
 
 ( 
 projectId 
 , 
  
 topicId 
 ); 
  
 publisherService 
 . 
  CreateTopic 
 
 ( 
 topicName 
 ); 
  
 // Create the SubscriberServiceApiClient using the SubscriberServiceApiClientBuilder 
  
 // and setting the EmulatorDection property. 
  
  SubscriberServiceApiClient 
 
  
 subscriberService 
  
 = 
  
 await 
  
 new 
  
  SubscriberServiceApiClientBuilder 
 
  
 { 
  
 EmulatorDetection 
  
 = 
  
  EmulatorDetection 
 
 . 
  EmulatorOrProduction 
 
  
 }. 
 BuildAsync 
 (); 
  
 // Use the client as you'd normally do, to create a subscription in this example. 
  
  SubscriptionName 
 
  
 subscriptionName 
  
 = 
  
 new 
  
  SubscriptionName 
 
 ( 
 projectId 
 , 
  
 subscriptionId 
 ); 
  
 subscriberService 
 . 
  CreateSubscription 
 
 ( 
 subscriptionName 
 , 
  
 topicName 
 , 
  
 pushConfig 
 : 
  
 null 
 , 
  
 ackDeadlineSeconds 
 : 
  
 60 
 ); 
  
 // Create the PublisherClient using PublisherClientBuilder to set the EmulatorDetection property. 
  
  PublisherClient 
 
  
 publisher 
  
 = 
  
 await 
  
 new 
  
  PublisherClientBuilder 
 
  
 { 
  
 TopicName 
  
 = 
  
 topicName 
 , 
  
 EmulatorDetection 
  
 = 
  
  EmulatorDetection 
 
 . 
  EmulatorOrProduction 
 
  
 }. 
 BuildAsync 
 (); 
  
 // Use the client as you'd normally do, to send a message in this example. 
  
 await 
  
 publisher 
 . 
  PublishAsync 
 
 ( 
 "Hello, Pubsub" 
 ); 
  
 await 
  
 publisher 
 . 
  ShutdownAsync 
 
 ( 
 TimeSpan 
 . 
 FromSeconds 
 ( 
 15 
 )); 
  
 // Create the SubscriberClient using SubscriberClientBuild to set the EmulatorDetection property. 
  
  SubscriberClient 
 
  
 subscriber 
  
 = 
  
 await 
  
 new 
  
  SubscriberClientBuilder 
 
  
 { 
  
 SubscriptionName 
  
 = 
  
 subscriptionName 
 , 
  
 EmulatorDetection 
  
 = 
  
  EmulatorDetection 
 
 . 
  EmulatorOrProduction 
 
  
 }. 
 BuildAsync 
 (); 
  
 List<PubsubMessage> 
  
 receivedMessages 
  
 = 
  
 new 
  
 List<PubsubMessage> 
 (); 
  
 // Use the client as you'd normally do, to listen for messages in this example. 
  
 await 
  
 subscriber 
 . 
  StartAsync 
 
 (( 
 msg 
 , 
  
 cancellationToken 
 ) 
  
 = 
>  
 { 
  
 receivedMessages 
 . 
 Add 
 ( 
 msg 
 ); 
  
 Console 
 . 
 WriteLine 
 ( 
 $"Received message {msg.MessageId} published at {msg.PublishTime.ToDateTime()}" 
 ); 
  
 Console 
 . 
 WriteLine 
 ( 
 $"Text: '{msg.Data.ToStringUtf8()}'" 
 ); 
  
 // In this example we stop the subscriber when the message is received. 
  
 // You may leave the subscriber running, and it will continue to received published messages 
  
 // if any. 
  
 // This is non-blocking, and the returned Task may be awaited. 
  
 subscriber 
 . 
  StopAsync 
 
 ( 
 TimeSpan 
 . 
 FromSeconds 
 ( 
 15 
 )); 
  
 // Return Reply.Ack to indicate this message has been handled. 
  
 return 
  
 Task 
 . 
 FromResult 
 ( 
  SubscriberClient 
 
 . 
  Reply 
 
 . 
  Ack 
 
 ); 
  
 }); 
  
 } 
 }

Java

Before trying this sample, follow the Java setup instructions in the Pub/Sub quickstart using client libraries . For more information, see the Pub/Sub Java API reference documentation .

To authenticate to Pub/Sub, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .

  import 
  
 com.google.api.core. ApiFuture 
 
 ; 
 import 
  
 com.google.api.gax.core. CredentialsProvider 
 
 ; 
 import 
  
 com.google.api.gax.core. NoCredentialsProvider 
 
 ; 
 import 
  
 com.google.api.gax.grpc. GrpcTransportChannel 
 
 ; 
 import 
  
 com.google.api.gax.rpc. FixedTransportChannelProvider 
 
 ; 
 import 
  
 com.google.api.gax.rpc. TransportChannelProvider 
 
 ; 
 import 
  
 com.google.cloud.pubsub.v1. Publisher 
 
 ; 
 import 
  
 com.google.cloud.pubsub.v1. TopicAdminClient 
 
 ; 
 import 
  
 com.google.cloud.pubsub.v1. TopicAdminSettings 
 
 ; 
 import 
  
 com.google.protobuf. ByteString 
 
 ; 
 import 
  
 com.google.pubsub.v1. PubsubMessage 
 
 ; 
 import 
  
 com.google.pubsub.v1. Topic 
 
 ; 
 import 
  
 com.google.pubsub.v1. TopicName 
 
 ; 
 import 
  
 io.grpc.ManagedChannel 
 ; 
 import 
  
 io.grpc.ManagedChannelBuilder 
 ; 
 public 
  
 class 
 UsePubSubEmulatorExample 
  
 { 
  
 public 
  
 static 
  
 void 
  
 main 
 ( 
 String 
 ... 
  
 args 
 ) 
  
 throws 
  
 Exception 
  
 { 
  
 String 
  
 hostport 
  
 = 
  
 System 
 . 
  getenv 
 
 ( 
 "PUBSUB_EMULATOR_HOST" 
 ); 
  
 ManagedChannel 
  
 channel 
  
 = 
  
 ManagedChannelBuilder 
 . 
 forTarget 
 ( 
 hostport 
 ). 
 usePlaintext 
 (). 
 build 
 (); 
  
 try 
  
 { 
  
  TransportChannelProvider 
 
  
 channelProvider 
  
 = 
  
  FixedTransportChannelProvider 
 
 . 
 create 
 ( 
  GrpcTransportChannel 
 
 . 
 create 
 ( 
 channel 
 )); 
  
  CredentialsProvider 
 
  
 credentialsProvider 
  
 = 
  
  NoCredentialsProvider 
 
 . 
 create 
 (); 
  
 // Set the channel and credentials provider when creating a `TopicAdminClient`. 
  
 // Can be done similarly for a `SubscriptionAdminClient`. 
  
  TopicAdminClient 
 
  
 topicAdminClient 
  
 = 
  
  TopicAdminClient 
 
 . 
 create 
 ( 
  
  TopicAdminSettings 
 
 . 
 newBuilder 
 () 
  
 . 
 setTransportChannelProvider 
 ( 
 channelProvider 
 ) 
  
 . 
 setCredentialsProvider 
 ( 
 credentialsProvider 
 ) 
  
 . 
 build 
 ()); 
  
  TopicName 
 
  
 topicName 
  
 = 
  
  TopicName 
 
 . 
 of 
 ( 
 "my-project-id" 
 , 
  
 "my-topic-id" 
 ); 
  
  Topic 
 
  
 topic 
  
 = 
  
 topicAdminClient 
 . 
  createTopic 
 
 ( 
 topicName 
 ); 
  
 System 
 . 
 out 
 . 
 println 
 ( 
 "Created topic: " 
  
 + 
  
 topic 
 . 
  getName 
 
 ()); 
  
 // Set the channel and credentials provider when creating a `Publisher`. 
  
 // Can be done similarly for a `Subscriber`. 
  
  Publisher 
 
  
 publisher 
  
 = 
  
  Publisher 
 
 . 
 newBuilder 
 ( 
 topicName 
 ) 
  
 . 
 setChannelProvider 
 ( 
 channelProvider 
 ) 
  
 . 
 setCredentialsProvider 
 ( 
 credentialsProvider 
 ) 
  
 . 
 build 
 (); 
  
 String 
  
 message 
  
 = 
  
 "Hello World!" 
 ; 
  
  ByteString 
 
  
 data 
  
 = 
  
  ByteString 
 
 . 
  copyFromUtf8 
 
 ( 
 message 
 ); 
  
  PubsubMessage 
 
  
 pubsubMessage 
  
 = 
  
  PubsubMessage 
 
 . 
 newBuilder 
 (). 
  setData 
 
 ( 
 data 
 ). 
 build 
 (); 
  
 ApiFuture<String> 
  
 messageIdFuture 
  
 = 
  
  publish 
 
er . 
  publish 
 
 ( 
 pubsubMessage 
 ); 
  
 String 
  
 messageId 
  
 = 
  
 messageIdFuture 
 . 
 get 
 (); 
  
 System 
 . 
 out 
 . 
 println 
 ( 
 "Published message ID: " 
  
 + 
  
 messageId 
 ); 
  
 } 
  
 finally 
  
 { 
  
 channel 
 . 
 shutdown 
 (); 
  
 } 
  
 } 
 }

Stopping the emulator

To stop the emulator, press Control+C .

After you stop the emulator, run the following command to remove the PUBSUB_EMULATOR_HOST environment variable so your application will connect to Pub/Sub:

Linux / macOS

unset PUBSUB_EMULATOR_HOST

Windows

set PUBSUB_EMULATOR_HOST=

Emulator command-line arguments

For details on command-line arguments for the Pub/Sub emulator, see gcloud beta emulators pubsub .

Supported features

The emulator supports the following Pub/Sub features:

Publishing messages
Receiving messages from push and pull subscriptions
Ordering messages
Replaying messages
Forwarding messages to dead-letter topics
Retry policies on message delivery
Schema support for Avro

Known limitations

UpdateTopic and UpdateSnapshot RPCs are not supported.
IAM operations are not supported.
Configurable message retention is not supported; all messages are retained indefinitely.
Subscription expiration is not supported. Subscriptions don't expire.
Filtering is not supported.
Schema support for protocol buffers.
BigQuery subscriptions can be created, but don't send messages to BigQuery.
Seek to a timestamp for ordered subscriptions is not supported.
Topics and subscriptions can be created with Single Message Transforms (SMTs) but messages won't be transformed.

To file issues, submit a Public issue tracker .

What's next

To learn how to use the Pub/Sub emulator with minikube, see this blog post .

Testing apps locally with the emulator Stay organized with collections Save and categorize content based on your preferences.

Before you begin

Install the emulator

Install the emulator as a container image

Starting the emulator

Setting environment variables

Automatically setting the variables

Linux / macOS

Windows

Manually setting the variables

Using the emulator

Accessing environment variables

C#

Java

Stopping the emulator

Emulator command-line arguments

Supported features

Known limitations

What's next

Testing apps locally with the emulator