Configure a MongoDB database for CDC

Preview

This feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms . Pre-GA features are available "as is" and might have limited support. For more information, see the launch stage descriptions .

This page describes how to configure change data capture (CDC) to stream data from a MongoDB database to a supported destination , such as BigQuery or Cloud Storage. You can configure Datastream to replicate data from a fully managed cloud database service called Atlas, or, if you're replicating from a self-managed MongoDB instance, using the MongoDB Shell.

Configure a MongoDB Atlas instance

To use Datastream with a MongoDB Atlas instance, you first need to create a Datastream user and grant them database access:

1. In the MongoDB Atlas dashboard , under Security, click Database access.
2. Click New database userand select the password authentication method for your user.
3. Enter the username and password for your Datastream user.
4. Select Grant specific user privilegesunder Database user privileges.
5. Add these roles/privileges under Specific privileges:
   • readAnyDatabase
6. Click Add user.

Configure a MongoDB self-hosted instance

This page describes how to configure a MongoDB self-hosted database for use with Datastream for both replica set and sharded cluster.

Replica set

For information about deploying a MongoDB replica set, see Deploy a self-managed replica set in MongoDB documentation.

To configure Datastream for use with a self-managed MongoDB replica set, follow these steps:

1. Install the MongoDB Shell. For information about installing the MongoDB Shell, see the MongoDB documentation .
2. Open the terminal and run the mongosh command to connect to your replica set or primary node.
3. Create a user for Datastream in the admin database:

   
use  
admin  
db.createUser ({ 
  
user:  
 " USERNAME 
" 
,  
pwd:  
 " PASSWORD 
" 
,  
roles:  
 [ 
  
 "readAnyDatabase" 
,  
 { 
role:  
 "read" 
,  
db:  
 "admin" 
 } 
  
 ] 
  
 }) 
 

Replace USERNAME and PASSWORD with your username and password.

Sharded cluster

For information about deploying a sharded cluster, see Deploy a sharded cluster in MongoDB documentation.

To configure Datastream for use with a self-managed MongoDB sharded cluster, follow these steps:

1. Install the MongoDB Shell. For information about installing the MongoDB Shell, see the MongoDB documentation .
2. Open the terminal and run the mongosh command to connect to your MongoDB sharded cluster.
3. Create an identical Datastream user in every primary shard node and the mongos query router:

 use  
admin
db.createUser ({ 
  
user:  
 " USERNAME 
" 
,  
pwd:  
 " PASSWORD 
" 
,  
roles:  
 [ 
  
 "readAnyDatabase" 
  
 ] 
 }) 
 

Replace USERNAME and PASSWORD with your username and password. You can pick any username and password, but it must be consistent across all primary shard nodes and the mongos query router.

To find out which shards are connected to the mongos query router, run the following command:

db.adminCommand({ listShards: 1 })