Configure an Amazon Aurora MySQL database for CDC

This page describes how to configure change data capture (CDC) to stream data from an Amazon Aurora MySQL database to a supported destination , such as BigQuery or Cloud Storage.

Create a parameter group

1. Launch your Amazon RDS Dashboard.

2. In the Navigation Drawer, click Parameter Groups.

3. Click Create Parameter Group. The Create Parameter Grouppage appears.

4. Use the following table to populate the fields of this page:

   Field	Description
   Parameter group family	Select the family that matches your database.
   Type	Select DB Cluster Parameter Group .
   Group name	Provide a name for the parameter group.
   Description	Provide a description for the parameter group.

5. Click Create.

6. Select the checkbox to the left of your newly created parameter group.

7. Under Parameter group actions, click Edit.

8. Set the following parameter values:

   • For GTID-basedreplication:

     Parameter

     Value

     binlog_format

     ROW

     log_bin_use_v1_row_events

     1

     read_only

     0

     net_read_timeout

     3600

     net_write_timeout

     3600

     wait_timeout

     86400

     expire-logs-days

     7

     binlog_row_image

     FULL

     max_allowed_packet

     1G (recommended value)

     log-replica-updates

     1

     enforce-gtid-consistency

     ON

     gtid_mode

     ON

     You can only set this parameter to ON after you set enforce-gtid-consistency to ON . In general, gtid_mode can have the following values:

     • OFF
     • OFF_PERMISSIVE
     • ON_PERMISSIVE
     • ON

     You can only change the values of the gtid_mode parameter one step at a time. For example, if the current mode is set to OFF , do the following:

     1. Change the setting to OFF_PERMISSIVE .
     2. Change the setting to ON_PERMISSIVE .
     3. Change the setting to ON .

     For more information, see MySQL documentation .

   • For binlog-basedreplication:

     Parameter	Value
     binlog_format	ROW

9. Click Save Changes.

Assign the parameter group to the database instance

1. Launch your Amazon RDS Dashboard.

2. In the Navigation Drawer, click Databases, and then select your database instance.

3. From the Instance Actionsmenu, select Modify. The Modify DB Instancedialog appears.

4. In the Additional configurationsection, select the database cluster parameter group that you created .

5. Set the Backup retention periodto 7 days.

6. Click Continue.

7. In the Scheduling of Modificationspane, select the Apply immediatelyoption.

Set the binary log retention period

1. Enter the following MySQL command:

    call 
     
    mysql 
    . 
    rds_set_configuration 
    ( 
    'binlog retention hours' 
    , 
     
    168 
    ); 
   

2. Restart your MySQL server so that the changes you made can take effect.

Create a Datastream user

1. Create a Datastream user:

    CREATE 
     
    USER 
     
    'datastream' 
    @ 
    '%' 
     
    IDENTIFIED 
     
    BY 
     
    ' YOUR_PASSWORD 
   ' 
    ; 
   

2. Grant the following privileges to them:

    GRANT 
     
    REPLICATION 
     
    SLAVE 
    , 
     
    SELECT 
    , 
     
    REPLICATION 
     
    CLIENT 
     
    ON 
     
    * 
    . 
    * 
     
    TO 
     
    'datastream' 
    @ 
    '%' 
    ; 
    GRANT 
     
    EXECUTE 
     
    ON 
     
    PROCEDURE 
     
    mysql 
    . 
    rds_show_configuration 
     
    TO 
     
    'datastream' 
    @ 
    '%' 
    ; 
    FLUSH 
     
    PRIVILEGES 
    ; 
   

What's next

• Learn more about how Datastream works with MySQL sources .