Console
    Contact Us Start free

    Manage private connectivity configurations

    In this page, you learn how to use the Datastream API to create, retrieve information about, and delete private connectivity configurations.

    Private connectivity configurations contain information that Datastream uses to communicate with a data source over a private network (internally within Google Cloud, or with external sources connected over VPN or Interconnect). This communication happens through a Private Service Connect interface or a Virtual Private Cloud (VPC) peering connection.

    Create a private connectivity configuration using VPC peering

    Before creating a private connectivity configuration, make sure that you:

    • Have a VPC network to peer to Datastream's private network. For more information about creating a VPC network, see Using VPC Network Peering .
    • Have an available IP range (with a CIDR block of /29 at minimum) on the VPC network. Datastream uses this IP range to create VPC peering with the provided VPC, so the range can't overlap with subnets within that VPC. As a result, Datastream can communicate with the source.

    There are two parameters associated with creating a private connectivity configuration:

    • vpcName: Specify the name and path of the VPC network.
    • subnet: Provide an available IP range on the VPC network.

    The following code shows a request to create a private connectivity configuration, and specifies the connectivity method as using VPC peering ( vpcPeeringConfig ).

    REST

     POST 
  
 h 
 tt 
 ps 
 : 
 //datastream.googleapis.com/v1/projects/ PROJECT_ID 
/locations/ 
  LOCATION 
 
/priva te 
 Co 
 nne 
 c 
 t 
 io 
 ns 
 ?priva 
 te 
 Co 
 nne 
 c 
 t 
 io 
 n 
 Id= PRIVATE_CONNECTIVITY_CONFIGURATION_ID 
 
 { 
  
 "displayName" 
 : 
  
 " DISPLAY_NAME 
" 
 , 
  
 "vpcPeeringConfig" 
 : 
  
 { 
  
 "vpcName" 
 : 
  
 " VPC_NAME 
" 
 , 
  
 "subnet" 
 : 
  
 " SUBNET 
" 
  
 } 
 }

    For example:

    POST https://datastream.googleapis.com/v1/projects/myProjectId/locations/
us-central1/privateConnections?privateConnectionId=myPrivateConnection
{
  "displayName": "my-private-connection",
  "vpcPeeringConfig": {
    "vpcName": "projects/my-project/global/networks/my-vpc",
    "subnet": "10.11.12.13/29"
  }
}

    After the private connectivity configuration is created, you can view information about it by calling the privateConnections/get method. Your output appears, similar to the following:

    {
  "name": "projects/projectId/location/us-central1/privateConnections/myPrivateConnection",
  "createTime": "2019-12-22T16:17:37.159786963Z",
  "updateTime": "2019-12-22T16:17:37.159786963Z",
  "displayName": "my-private-connection",
  "state": "CREATED",
  "vpcPeeringConfig": {
    "vpcName": "projects/my-project/global/networks/my-vpc",
    "subnet": "10.11.12.13/29"
  }  
}

    gcloud

     gcloud 
  
 da 
 tastrea 
 m 
  
 priva 
 te 
 - 
 co 
 nne 
 c 
 t 
 io 
 ns 
  
 crea 
 te 
  
  PRIVATE_CONNECTION_NAME 
 
  
 -- 
 loca 
 t 
 io 
 n 
 = LOCATION 
 
  
  
 -- 
 display 
 - 
 na 
 me= DISPLAY_NAME 
 
  
 -- 
 vpc= VPC_NAME 
 
  
 -- 
 sub 
 net 
 = SUBNET

    For example:

    gcloud datastream private-connections create my-privateConnection --location=us-central1  
--display-name=my-private-connection --vpc=my-vpc --subnet=10.11.12.13/29

    Create a private connectivity configuration using a Private Service Connect interface

    Before creating a private connectivity configuration, make sure that you:

    • Have a VPC network to connect to the Datastream private network. For more information about creating a VPC network, see Create and manage VPC networks .

    • Create a network attachment in your VPC project.

    • Verify that Google Cloud and the on-premises firewall allow traffic from the network attachment IP address range to the source database from which you want to stream data.

    The following code shows a request to create a private connectivity configuration using Private Service Connect interfaces. You first need to run a command to create a private connectivity configuration with the validateOnly flag ( --validate-only for Google Cloud CLI):

    REST

     POST 
  
 "https://datastream.googleapis.com/v1/projects/ PROJECT_ID 
/locations/ 
  LOCATION 
 
/privateConnections?privateConnectionId= PRIVATE_CONNECTIVITY_CONFIGURATION_ID 
  
&validateOnly=true"

    gcloud

     gcloud 
  
 da 
 tastrea 
 m 
  
 priva 
 te 
 - 
 co 
 nne 
 c 
 t 
 io 
 ns 
  
 crea 
 te 
  
  PRIVATE_CONNECTION_NAME 
 
  
 -- 
 loca 
 t 
 io 
 n 
 = LOCATION 
 
  
  
 -- 
 display 
 - 
 na 
 me= DISPLAY_NAME 
 
  
 -- 
 net 
 work 
 - 
 a 
 tta 
 chme 
 nt 
 = NETWORK_ATTACHMENT_NAME 
 
  
  
 -- 
 valida 
 te 
 - 
 o 
 nl 
 y

    In the response, you get the name of the producer project to approve. You need to add it to your network attachment allowed projects list:

    1. Use the describe method to get the current producer-accept-list for your network attachment. For more information, see the Compute Engine Google Cloud SDK documentation .
    2. Add the producer project to this list using the update method. For more information, see the Compute Engine Google Cloud SDK documentation .

    Once you add the producer project to the allowed projects list, run the command to create the private connectivity configuration again, this time without the validateOnly flag:

    REST

     POST 
  
 h 
 tt 
 ps 
 : 
 //datastream.googleapis.com/v1/projects/ PROJECT_ID 
/locations/ 
  LOCATION 
 
/priva te 
 Co 
 nne 
 c 
 t 
 io 
 ns 
 ?priva 
 te 
 Co 
 nne 
 c 
 t 
 io 
 n 
 Id= PRIVATE_CONNECTIVITY_CONFIGURATION_ID 
 
 { 
  
 "displayName" 
 : 
  
 " DISPLAY_NAME 
" 
 , 
  
 "pscInterfaceConfig" 
 : 
  
 { 
  
 "networkAttachment" 
 : 
  
 " NETWORK_ATTACHMENT_NAME 
" 
  
 } 
 }

    For example:

    POST https://datastream.googleapis.com/v1/projects/myProjectId/locations/
us-central1/privateConnections?privateConnectionId=myPrivateConnection
{
  "displayName": "my-private-connection",
  "pscInterfaceConfig": {
    "networkAttachment": "projects/my-project/regions/us-central1/networkAttachments/my-na"
  }
}

    gcloud

     gcloud 
  
 da 
 tastrea 
 m 
  
 priva 
 te 
 - 
 co 
 nne 
 c 
 t 
 io 
 ns 
  
 crea 
 te 
  
  PRIVATE_CONNECTION_NAME 
 
  
 -- 
 loca 
 t 
 io 
 n 
 = LOCATION 
 
  
  
 -- 
 display 
 - 
 na 
 me= DISPLAY_NAME 
 
  
 -- 
 net 
 work 
 - 
 a 
 tta 
 chme 
 nt 
 = NETWORK_ATTACHMENT_NAME

    For example:

    gcloud datastream private-connections create my-private-connection --location=us-central1  
--display-name=my-private-connection --network-attachment=my-network-attachment

    Get information about a private connectivity configuration

    The following code shows a request to retrieve information about a private connectivity configuration that has been created.

    REST

     GET 
  
 h 
 tt 
 ps 
 : 
 //datastream.googleapis.com/v1/projects/ PROJECT_ID 
/locations/ 
  LOCATION 
 
/priva te 
 Co 
 nne 
 c 
 t 
 io 
 ns 
 / PRIVATE_CONNECTIVITY_CONFIGURATION_ID

    For example:

    GET https://datastream.googleapis.com/v1/projects/myProjectId/locations/
us-central1/privateConnections/myPrivateConnection

    gcloud

    For more information about using gcloud to retrieve information about your private connectivity configuration, see the Google Cloud SDK documentation .

    List private connectivity configurations

    The following code shows a request to retrieve information about all of your private connectivity configurations.

    REST

     GET 
  
 h 
 tt 
 ps 
 : 
 //datastream.googleapis.com/v1/projects/ PROJECT_ID 
/locations/ 
  LOCATION 
 
/priva te 
 Co 
 nne 
 c 
 t 
 io 
 ns

    gcloud

    For more information about using gcloud to retrieve information about all of your private connectivity configurations, see the Google Cloud SDK documentation .

    Delete a private connectivity configuration

    The following code shows a request to delete a private connectivity configuration. After the configuration is deleted, any connection profiles using the configuration won't work. In addition, any streams using these connection profiles won't be operable.

    To resolve this issue, assign a different private connectivity configuration to each connection profile. For more information, see Update a connection profile .

    REST

     DELETE 
  
 h 
 tt 
 ps 
 : 
 //datastream.googleapis.com/v1/projects/ PROJECT_ID 
/locations/ 
  LOCATION 
 
/priva te 
 Co 
 nne 
 c 
 t 
 io 
 ns 
 / 
  [ 
 priva 
 te 
 - 
 co 
 nne 
 c 
 t 
 ivi 
 t 
 y 
 - 
 co 
 nf 
 igura 
 t 
 io 
 n 
 - 
 id 
 ] 
 
  
 { 
 "force" 
 : 
  
 true 
 }

    For example:

    DELETE https://datastream.googleapis.com/v1/projects/myProjectId/locations/
us-central1/privateConnections/myPrivateConnection {"force": true}

    gcloud

    For more information about using gcloud to delete your private connectivity configuration, see the Google Cloud SDK documentation .

    What's next
    Design a Mobile Site
    View Site in Mobile | Classic
    Share by: