Create a change stream-enabled table and capture changes

Create a table with a change stream enabled

1. In the Google Cloud console, go to the Bigtable Instancespage.

   Go to Instances

2. Click the ID of the instance that you are using for this quickstart.

   If you don't have an instance available, create an instance with the default configurations in a region near you.

3. In the left navigation pane, click Tables.

4. Click Create a table.

5. Name the table change-streams-quickstart .

6. Add a column family named cf .

7. Select Enable change stream.

8. Click Create.

Initialize a data pipeline to capture the change stream

1. In the Cloud Shell, run the following commands to download the code and run it.

    git  
   clone  
   https://github.com/GoogleCloudPlatform/java-docs-samples.git cd 
     
   java-docs-samples/bigtable/beam/change-streams
   mvn  
   compile  
   exec:java  
   -Dexec.mainClass = 
   ChangeStreamsHelloWorld  
    \ 
    "-Dexec.args=--project= PROJECT_ID 
   --bigtableProjectId= PROJECT_ID 
   \ 
    --bigtableInstanceId= BIGTABLE_INSTANCE_ID 
   --bigtableTableId=change-streams-quickstart \ 
    --runner=dataflow --region= BIGTABLE_REGION 
   --experiments=use_runner_v2" 
    
   

   Replace the following:

   • PROJECT_ID : the ID of the project that you are using
   • BIGTABLE_INSTANCE_ID : the ID of the instance to contain the new table
   • BIGTABLE_REGION : the region that your Bigtable instance is in, such as us-east5

2. In the Google Cloud console, go to the Dataflowpage.

   Go to Dataflow

3. Click the job with a name that begins with changestreamquickstart.

4. At the bottom of the screen, click Showto open the logs panel.

5. Click Worker logsto monitor the output of the change stream.

6. In the Cloud Shell, write some data to Bigtable to see the change stream process.

    cbt  
   -instance = 
    BIGTABLE_INSTANCE_ID 
     
   -project = 
    PROJECT_ID 
     
    \ 
   import  
   change-streams-quickstart  
   quickstart-data.csv  
   column-family = 
   cf 
   

7. In the Google Cloud console, make sure that Severityis set to at least Info .

8. The worker log output logs contain this:

    Change captured: user123#2023,USER,SetCell,cf,col1,abc
   Change captured: user546#2023,USER,SetCell,cf,col1,def
   Change captured: user789#2023,USER,SetCell,cf,col1,ghi