Reference documentation and code samples for the Google Cloud Dataproc V1 Client class SparkBatch.
A configuration for running an Apache Spark batch workload.
Generated from protobuf message google.cloud.dataproc.v1.SparkBatch
Methods
__construct
Constructor.
data
array
Optional. Data for populating the Message object.
↳ main_jar_file_uri
string
Optional. The HCFS URI of the jar file that contains the main class.
↳ main_class
string
Optional. The name of the driver main class. The jar file that contains the class must be in the classpath or specified in jar_file_uris
.
↳ args
array
Optional. The arguments to pass to the driver. Do not include arguments that can be set as batch properties, such as --conf
, since a collision can occur that causes an incorrect batch submission.
↳ jar_file_uris
array
Optional. HCFS URIs of jar files to add to the classpath of the Spark driver and tasks.
↳ file_uris
array
Optional. HCFS URIs of files to be placed in the working directory of each executor.
↳ archive_uris
array
Optional. HCFS URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar
, .tar
, .tar.gz
, .tgz
, and .zip
.
getMainJarFileUri
Optional. The HCFS URI of the jar file that contains the main class.
string
hasMainJarFileUri
setMainJarFileUri
Optional. The HCFS URI of the jar file that contains the main class.
var
string
$this
getMainClass
Optional. The name of the driver main class. The jar file that contains
the class must be in the classpath or specified in jar_file_uris
.
string
hasMainClass
setMainClass
Optional. The name of the driver main class. The jar file that contains
the class must be in the classpath or specified in jar_file_uris
.
var
string
$this
getArgs
Optional. The arguments to pass to the driver. Do not include arguments
that can be set as batch properties, such as --conf
, since a collision
can occur that causes an incorrect batch submission.
setArgs
Optional. The arguments to pass to the driver. Do not include arguments
that can be set as batch properties, such as --conf
, since a collision
can occur that causes an incorrect batch submission.
var
string[]
$this
getJarFileUris
Optional. HCFS URIs of jar files to add to the classpath of the Spark driver and tasks.
setJarFileUris
Optional. HCFS URIs of jar files to add to the classpath of the Spark driver and tasks.
var
string[]
$this
getFileUris
Optional. HCFS URIs of files to be placed in the working directory of each executor.
setFileUris
Optional. HCFS URIs of files to be placed in the working directory of each executor.
var
string[]
$this
getArchiveUris
Optional. HCFS URIs of archives to be extracted into the working directory
of each executor. Supported file types: .jar
, .tar
, .tar.gz
, .tgz
, and .zip
.
setArchiveUris
Optional. HCFS URIs of archives to be extracted into the working directory
of each executor. Supported file types: .jar
, .tar
, .tar.gz
, .tgz
, and .zip
.
var
string[]
$this
getDriver
string
