Python Client for Google Cloud Dataproc API

Google Cloud Dataproc API : Manages Hadoop-based clusters and jobs on Google Cloud Platform.

• Client Library Documentation

• Product Documentation

Quick Start

In order to use this library, you first need to go through the following steps:

1. Select or create a Cloud Platform project.

2. Enable billing for your project.

3. Enable the Google Cloud Dataproc API.

4. Setup Authentication.

Installation

Install this library in a virtualenv using pip. virtualenv is a tool to create isolated Python environments. The basic problem it addresses is one of dependencies and versions, and indirectly permissions.

With virtualenv , it’s possible to install this library without needing system install permissions, and without clashing with the installed system dependencies.

Supported Python Versions

Python >= 3.5

Deprecated Python Versions

Python == 2.7. Python 2.7 support will be removed on January 1, 2020.

Mac/Linux

 pip install virtualenv
virtualenv <your-env>
source <your-env>/bin/activate
<your-env>/bin/pip install google-cloud-dataproc 

Windows

 pip install virtualenv
virtualenv <your-env>
<your-env>\Scripts\activate
<your-env>\Scripts\pip.exe install google-cloud-dataproc 

Example Usage

 from google.cloud import dataproc_v1 
client = dataproc_v1 
. ClusterControllerClient 
()

project_id = ''
region = ''


# Iterate over all results
for element in client. list_clusters 
(project_id, region):
    # process element
    pass

# Or iterate over results one page at a time
for page in client. list_clusters 
(project_id, region).pages:
    for element in page:
        # process element
        pass 

Next Steps

• Read the Client Library Documentation for Google Cloud Dataproc API API to see other available methods on the client.

• Read the Product documentation to learn more about the product and see How-to Guides.