Reference documentation and code samples for the Google Cloud Dataproc V1 Client class PySparkBatch.
A configuration for running anApache
PySparkbatch workload.
Generated from protobuf messagegoogle.cloud.dataproc.v1.PySparkBatch
Namespace
Google \ Cloud \ Dataproc \ V1
Methods
__construct
Constructor.
Parameters
Name
Description
data
array
Optional. Data for populating the Message object.
↳ main_python_file_uri
string
Required. The HCFS URI of the main Python file to use as the Spark driver. Must be a .py file.
↳ args
array
Optional. The arguments to pass to the driver. Do not include arguments that can be set as batch properties, such as--conf, since a collision can occur that causes an incorrect batch submission.
↳ python_file_uris
array
Optional. HCFS file URIs of Python files to pass to the PySpark framework. Supported file types:.py,.egg, and.zip.
↳ jar_file_uris
array
Optional. HCFS URIs of jar files to add to the classpath of the Spark driver and tasks.
↳ file_uris
array
Optional. HCFS URIs of files to be placed in the working directory of each executor.
↳ archive_uris
array
Optional. HCFS URIs of archives to be extracted into the working directory of each executor. Supported file types:.jar,.tar,.tar.gz,.tgz, and.zip.
getMainPythonFileUri
Required. The HCFS URI of the main Python file to use as the Spark driver.
Must be a .py file.
Returns
Type
Description
string
setMainPythonFileUri
Required. The HCFS URI of the main Python file to use as the Spark driver.
Must be a .py file.
Parameter
Name
Description
var
string
Returns
Type
Description
$this
getArgs
Optional. The arguments to pass to the driver. Do not include arguments
that can be set as batch properties, such as--conf, since a collision
can occur that causes an incorrect batch submission.
Optional. The arguments to pass to the driver. Do not include arguments
that can be set as batch properties, such as--conf, since a collision
can occur that causes an incorrect batch submission.
Parameter
Name
Description
var
string[]
Returns
Type
Description
$this
getPythonFileUris
Optional. HCFS file URIs of Python files to pass to the PySpark
framework. Supported file types:.py,.egg, and.zip.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-09-04 UTC."],[],[],null,["# Google Cloud Dataproc V1 Client - Class PySparkBatch (3.14.0)\n\nVersion latestkeyboard_arrow_down\n\n- [3.14.0 (latest)](/php/docs/reference/cloud-dataproc/latest/V1.PySparkBatch)\n- [3.13.4](/php/docs/reference/cloud-dataproc/3.13.4/V1.PySparkBatch)\n- [3.12.0](/php/docs/reference/cloud-dataproc/3.12.0/V1.PySparkBatch)\n- [3.11.0](/php/docs/reference/cloud-dataproc/3.11.0/V1.PySparkBatch)\n- [3.10.1](/php/docs/reference/cloud-dataproc/3.10.1/V1.PySparkBatch)\n- [3.9.0](/php/docs/reference/cloud-dataproc/3.9.0/V1.PySparkBatch)\n- [3.8.1](/php/docs/reference/cloud-dataproc/3.8.1/V1.PySparkBatch)\n- [3.7.1](/php/docs/reference/cloud-dataproc/3.7.1/V1.PySparkBatch)\n- [3.6.1](/php/docs/reference/cloud-dataproc/3.6.1/V1.PySparkBatch)\n- [3.5.1](/php/docs/reference/cloud-dataproc/3.5.1/V1.PySparkBatch)\n- [3.4.0](/php/docs/reference/cloud-dataproc/3.4.0/V1.PySparkBatch)\n- [3.3.0](/php/docs/reference/cloud-dataproc/3.3.0/V1.PySparkBatch)\n- [3.2.2](/php/docs/reference/cloud-dataproc/3.2.2/V1.PySparkBatch)\n- [2.6.1](/php/docs/reference/cloud-dataproc/2.6.1/V1.PySparkBatch)\n- [2.5.0](/php/docs/reference/cloud-dataproc/2.5.0/V1.PySparkBatch)\n- [2.3.0](/php/docs/reference/cloud-dataproc/2.3.0/V1.PySparkBatch)\n- [2.2.3](/php/docs/reference/cloud-dataproc/2.2.3/V1.PySparkBatch)\n- [2.1.0](/php/docs/reference/cloud-dataproc/2.1.0/V1.PySparkBatch)\n- [2.0.0](/php/docs/reference/cloud-dataproc/2.0.0/V1.PySparkBatch) \nReference documentation and code samples for the Google Cloud Dataproc V1 Client class PySparkBatch.\n\nA configuration for running an\n[Apache\nPySpark](https://spark.apache.org/docs/latest/api/python/getting_started/quickstart.html)\nbatch workload.\n\nGenerated from protobuf message `google.cloud.dataproc.v1.PySparkBatch`\n\nNamespace\n---------\n\nGoogle \\\\ Cloud \\\\ Dataproc \\\\ V1\n\nMethods\n-------\n\n### __construct\n\nConstructor.\n\n### getMainPythonFileUri\n\nRequired. The HCFS URI of the main Python file to use as the Spark driver.\n\nMust be a .py file.\n\n### setMainPythonFileUri\n\nRequired. The HCFS URI of the main Python file to use as the Spark driver.\n\nMust be a .py file.\n\n### getArgs\n\nOptional. The arguments to pass to the driver. Do not include arguments\nthat can be set as batch properties, such as `--conf`, since a collision\ncan occur that causes an incorrect batch submission.\n\n### setArgs\n\nOptional. The arguments to pass to the driver. Do not include arguments\nthat can be set as batch properties, such as `--conf`, since a collision\ncan occur that causes an incorrect batch submission.\n\n### getPythonFileUris\n\nOptional. HCFS file URIs of Python files to pass to the PySpark\nframework. Supported file types: `.py`, `.egg`, and `.zip`.\n\n### setPythonFileUris\n\nOptional. HCFS file URIs of Python files to pass to the PySpark\nframework. Supported file types: `.py`, `.egg`, and `.zip`.\n\n### getJarFileUris\n\nOptional. HCFS URIs of jar files to add to the classpath of the\nSpark driver and tasks.\n\n### setJarFileUris\n\nOptional. HCFS URIs of jar files to add to the classpath of the\nSpark driver and tasks.\n\n### getFileUris\n\nOptional. HCFS URIs of files to be placed in the working directory of\neach executor.\n\n### setFileUris\n\nOptional. HCFS URIs of files to be placed in the working directory of\neach executor.\n\n### getArchiveUris\n\nOptional. HCFS URIs of archives to be extracted into the working directory\nof each executor. Supported file types:\n`.jar`, `.tar`, `.tar.gz`, `.tgz`, and `.zip`.\n\n### setArchiveUris\n\nOptional. HCFS URIs of archives to be extracted into the working directory\nof each executor. Supported file types:\n`.jar`, `.tar`, `.tar.gz`, `.tgz`, and `.zip`."]]
