Console
    Contact Us Start free

    Google Cloud Dataproc V1 Client - Class PySparkBatch (3.2.2)

    Reference documentation and code samples for the Google Cloud Dataproc V1 Client class PySparkBatch.

    A configuration for running an Apache PySpark batch workload.

    Generated from protobuf message google.cloud.dataproc.v1.PySparkBatch

    Methods

    __construct

    Constructor.

    Parameters
    Name
    Description
    data
    array

    Optional. Data for populating the Message object.

    ↳ main_python_file_uri
    string

    Required. The HCFS URI of the main Python file to use as the Spark driver. Must be a .py file.

    ↳ args
    array

    Optional. The arguments to pass to the driver. Do not include arguments that can be set as batch properties, such as --conf , since a collision can occur that causes an incorrect batch submission.

    ↳ python_file_uris
    array

    Optional. HCFS file URIs of Python files to pass to the PySpark framework. Supported file types: .py , .egg , and .zip .

    ↳ jar_file_uris
    array

    Optional. HCFS URIs of jar files to add to the classpath of the Spark driver and tasks.

    ↳ file_uris
    array

    Optional. HCFS URIs of files to be placed in the working directory of each executor.

    ↳ archive_uris
    array

    Optional. HCFS URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar , .tar , .tar.gz , .tgz , and .zip .

    getMainPythonFileUri

    Required. The HCFS URI of the main Python file to use as the Spark driver. Must be a .py file.

    Generated from protobuf field string main_python_file_uri = 1 [(.google.api.field_behavior) = REQUIRED];

    Returns
    Type
    Description
    string

    setMainPythonFileUri

    Required. The HCFS URI of the main Python file to use as the Spark driver. Must be a .py file.

    Generated from protobuf field string main_python_file_uri = 1 [(.google.api.field_behavior) = REQUIRED];

    Parameter
    Name
    Description
    var
    string
    Returns
    Type
    Description
    $this

    getArgs

    Optional. The arguments to pass to the driver. Do not include arguments that can be set as batch properties, such as --conf , since a collision can occur that causes an incorrect batch submission.

    Generated from protobuf field repeated string args = 2 [(.google.api.field_behavior) = OPTIONAL];

    Returns
    Type
    Description
    Google\Protobuf\Internal\RepeatedField

    setArgs

    Optional. The arguments to pass to the driver. Do not include arguments that can be set as batch properties, such as --conf , since a collision can occur that causes an incorrect batch submission.

    Generated from protobuf field repeated string args = 2 [(.google.api.field_behavior) = OPTIONAL];

    Parameter
    Name
    Description
    var
    string[]
    Returns
    Type
    Description
    $this

    getPythonFileUris

    Optional. HCFS file URIs of Python files to pass to the PySpark framework. Supported file types: .py , .egg , and .zip .

    Generated from protobuf field repeated string python_file_uris = 3 [(.google.api.field_behavior) = OPTIONAL];

    Returns
    Type
    Description
    Google\Protobuf\Internal\RepeatedField

    setPythonFileUris

    Optional. HCFS file URIs of Python files to pass to the PySpark framework. Supported file types: .py , .egg , and .zip .

    Generated from protobuf field repeated string python_file_uris = 3 [(.google.api.field_behavior) = OPTIONAL];

    Parameter
    Name
    Description
    var
    string[]
    Returns
    Type
    Description
    $this

    getJarFileUris

    Optional. HCFS URIs of jar files to add to the classpath of the Spark driver and tasks.

    Generated from protobuf field repeated string jar_file_uris = 4 [(.google.api.field_behavior) = OPTIONAL];

    Returns
    Type
    Description
    Google\Protobuf\Internal\RepeatedField

    setJarFileUris

    Optional. HCFS URIs of jar files to add to the classpath of the Spark driver and tasks.

    Generated from protobuf field repeated string jar_file_uris = 4 [(.google.api.field_behavior) = OPTIONAL];

    Parameter
    Name
    Description
    var
    string[]
    Returns
    Type
    Description
    $this

    getFileUris

    Optional. HCFS URIs of files to be placed in the working directory of each executor.

    Generated from protobuf field repeated string file_uris = 5 [(.google.api.field_behavior) = OPTIONAL];

    Returns
    Type
    Description
    Google\Protobuf\Internal\RepeatedField

    setFileUris

    Optional. HCFS URIs of files to be placed in the working directory of each executor.

    Generated from protobuf field repeated string file_uris = 5 [(.google.api.field_behavior) = OPTIONAL];

    Parameter
    Name
    Description
    var
    string[]
    Returns
    Type
    Description
    $this

    getArchiveUris

    Optional. HCFS URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar , .tar , .tar.gz , .tgz , and .zip .

    Generated from protobuf field repeated string archive_uris = 6 [(.google.api.field_behavior) = OPTIONAL];

    Returns
    Type
    Description
    Google\Protobuf\Internal\RepeatedField

    setArchiveUris

    Optional. HCFS URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar , .tar , .tar.gz , .tgz , and .zip .

    Generated from protobuf field repeated string archive_uris = 6 [(.google.api.field_behavior) = OPTIONAL];

    Parameter
    Name
    Description
    var
    string[]
    Returns
    Type
    Description
    $this
    Create a Mobile Website
    View Site in Mobile | Classic
    Share by: