Regional endpoints

Dataproc supports regional endpoints based on Compute Engine regions . You must specify a region, such as "us-east1" or "europe-west1", when you create a Dataproc cluster. Dataproc will isolate cluster resources, such as VM instances and Cloud Storage and metadata storage, within a zone within the specified region.

You can optionally specify a zone within the specified cluster region, such as "us-east1-a" or "europe-west1-b", when you create a cluster. If you do not specify the zone, Dataproc Auto Zone Placement will choose a zone within your specified cluster region to locate clusters resources.

The regional namespace corresponds to the /regions/ REGION segment of Dataproc resource URIs (see, for example, the cluster networkUri ).

Regional endpoint semantics

Regional endpoint names follow a standard naming convention based on Compute Engine regions . For example, the name for the Central US region is us-central1 , and the name of the Western Europe region is europe-west1 . Run the gcloud compute regions list command to see a listing of available regions.

Create a cluster

gcloud

When you create a cluster, specify a region using the required --region flag.

gcloud dataproc clusters create CLUSTER_NAME 
\
    --region= REGION 
\
     other args ...

REST API

Use the REGION URL parameter in a clusters.create request to specify the cluster region.

gRPC

Set the client transport address to the regional endpoint using the following pattern:

REGION -dataproc.googleapis.com

Python ( google-cloud-python ) example:

 from 
  
 google.cloud 
  
 import 
 dataproc_v1 
 from 
  
 google.cloud.dataproc_v1.gapic.transports 
  
 import 
 cluster_controller_grpc_transport 
 transport 
 = 
 cluster_controller_grpc_transport 
 . 
 ClusterControllerGrpcTransport 
 ( 
 address 
 = 
 ' us-central1-dataproc.googleapis.com:443' 
 ) 
 client 
 = 
 dataproc_v1 
 . 
 ClusterControllerClient 
 ( 
 transport 
 ) 
 project_id 
 = 
 'my-project' 
 region 
 = 
 'us-central1' 
 cluster 
 = 
 { 
 ... 
 }

Java ( google-cloud-java ) example:

 ClusterControllerSettings 
  
 settings 
  
 = 
  
 ClusterControllerSettings 
 . 
 newBuilder 
 () 
  
 . 
 setEndpoint 
 ( 
 " us-central1-dataproc.googleapis.com:443" 
 ) 
  
 . 
 build 
 (); 
  
 try 
  
 ( 
 ClusterControllerClient 
  
 clusterControllerClient 
  
 = 
  
 ClusterControllerClient 
 . 
 create 
 ( 
 settings 
 )) 
  
 { 
  
 String 
  
 projectId 
  
 = 
  
 "my-project" 
 ; 
  
 String 
  
 region 
  
 = 
  
 "us-central1" 
 ; 
  
 Cluster 
  
 cluster 
  
 = 
  
 Cluster 
 . 
 newBuilder 
 (). 
 build 
 (); 
  
 Cluster 
  
 response 
  
 = 
  
 clusterControllerClient 
 . 
 createClusterAsync 
 ( 
 projectId 
 , 
  
 region 
 , 
  
 cluster 
 ). 
 get 
 (); 
  
 }

Console

Specify a Dataproc region in the Location section of the Set up clusterpanel on the Dataproc Create a cluster page in the Google Cloud console.