Configure a Cloud SQL for PostgreSQL database for CDC

This page describes how to configure change data capture (CDC) to stream data from a Cloud SQL for PostgreSQL database to a supported destination , such as BigQuery or Cloud Storage.

Enable logical replication

1. Navigate to Cloud SQL in the Google Cloud console.

2. Open the Cloud SQL instance and click EDIT.

3. Go to the Flagssection.

4. Click ADD FLAG.

5. Choose the cloudsql.logical_decoding flag from the drop-down menu.

6. Set the flag value to on.

7. Click SAVEto save your changes. You'll need to restart your instance to update your instance with the changes.

8. Once your instance has been restarted, confirm your changes under Database flagson the Overviewpage.

Create a publication and a replication slot

1. Connect to the database as a user with sufficient privileges to create a replication slot; otherwise, run the following command:

   ALTER USER USER_NAME 
   WITH REPLICATION;

   Replace the following:

   • USER_NAME : The name of the user to whom you want to grant replication privileges.

2. Create a publication. We recommend that you create a publication only for the tables that you want to replicate. This allows Datastream to read-only the relevant data, and lowers the load on the database and Datastream:

   CREATE PUBLICATION PUBLICATION_NAME 
   FOR TABLE SCHEMA1 
   . TABLE1 
   , SCHEMA2 
   . TABLE2 
   ;

   Replace the following:

   • PUBLICATION_NAME : The name of your publication. You'll need to provide this name when you create a stream in the Datastream stream creation wizard.
   • SCHEMA : The name of the schema that contains the table.
   • TABLE : The name of the table that you want to replicate.

   You can also create a publication for all tables in your database. Note that this approach increases the load on both the source database and Datastream:

   CREATE PUBLICATION PUBLICATION_NAME 
   FOR ALL TABLES;

3. Create a replication slot by entering the following SQL command:

   SELECT PG_CREATE_LOGICAL_REPLICATION_SLOT(' REPLICATION_SLOT_NAME 
   ', 'pgoutput');

   Replace the following:

   • REPLICATION_SLOT_NAME : The name of your replication slot. You'll need to provide this name when you create a stream in the Datastream stream creation wizard.

Create a Datastream user

1. Connect to the database using a PostgreSQL client.

2. Enter the following PostgreSQL command:

   CREATE USER USER_NAME 
   WITH REPLICATION LOGIN PASSWORD ' USER_PASSWORD 
   ';

   Replace the following:

   • USER_NAME : The name of the Datastream user that you want to create.
   • USER_PASSWORD : The login password for the Datastream user that you want to create.

3. Grant the following privileges to the user you created:

   GRANT SELECT ON ALL TABLES IN SCHEMA SCHEMA_NAME 
   TO USER_NAME 
   ;
   GRANT USAGE ON SCHEMA SCHEMA_NAME 
   TO USER_NAME 
   ;
   ALTER DEFAULT PRIVILEGES IN SCHEMA SCHEMA_NAME 
   GRANT SELECT ON TABLES TO USER_NAME 
   ;

   Replace the following:

   • SCHEMA_NAME : The name of the schema to which you want to grant the privileges.
   • USER_NAME : The user to whom you want to grant the privileges.

What's next

• Learn more about how Datastream works with PostgreSQL sources .