Reference documentation and code samples for the Google Cloud Dataplex V1 Client class SparkTaskConfig.
User-specified config for running a Spark task.
Generated from protobuf message google.cloud.dataplex.v1.Task.SparkTaskConfig
Namespace
Google \ Cloud \ Dataplex \ V1 \ TaskMethods
__construct
Constructor.
data
array
Optional. Data for populating the Message object.
↳ main_jar_file_uri
string
The Cloud Storage URI of the jar file that contains the main class. The execution args are passed in as a sequence of named process arguments ( --key=value
).
↳ main_class
string
The name of the driver's main class. The jar file that contains the class must be in the default CLASSPATH or specified in jar_file_uris
. The execution args are passed in as a sequence of named process arguments ( --key=value
).
↳ python_script_file
string
The Gcloud Storage URI of the main Python file to use as the driver. Must be a .py file. The execution args are passed in as a sequence of named process arguments ( --key=value
).
↳ sql_script_file
string
A reference to a query file. This should be the Cloud Storage URI of the query file. The execution args are used to declare a set of script variables ( set key="value";
).
↳ sql_script
string
The query text. The execution args are used to declare a set of script variables ( set key="value";
).
↳ file_uris
array
Optional. Cloud Storage URIs of files to be placed in the working directory of each executor.
↳ archive_uris
array
Optional. Cloud Storage URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
↳ infrastructure_spec
getMainJarFileUri
The Cloud Storage URI of the jar file that contains the main class.
The execution args are passed in as a sequence of named process
arguments ( --key=value
).
string
hasMainJarFileUri
setMainJarFileUri
The Cloud Storage URI of the jar file that contains the main class.
The execution args are passed in as a sequence of named process
arguments ( --key=value
).
var
string
$this
getMainClass
The name of the driver's main class. The jar file that contains the
class must be in the default CLASSPATH or specified in jar_file_uris
.
The execution args are passed in as a sequence of named process
arguments ( --key=value
).
string
hasMainClass
setMainClass
The name of the driver's main class. The jar file that contains the
class must be in the default CLASSPATH or specified in jar_file_uris
.
The execution args are passed in as a sequence of named process
arguments ( --key=value
).
var
string
$this
getPythonScriptFile
The Gcloud Storage URI of the main Python file to use as the driver.
Must be a .py file. The execution args are passed in as a sequence of
named process arguments ( --key=value
).
string
hasPythonScriptFile
setPythonScriptFile
The Gcloud Storage URI of the main Python file to use as the driver.
Must be a .py file. The execution args are passed in as a sequence of
named process arguments ( --key=value
).
var
string
$this
getSqlScriptFile
A reference to a query file. This should be the Cloud Storage URI of
the query file. The execution args are used to declare a set of script
variables ( set key="value";
).
string
hasSqlScriptFile
setSqlScriptFile
A reference to a query file. This should be the Cloud Storage URI of
the query file. The execution args are used to declare a set of script
variables ( set key="value";
).
var
string
$this
getSqlScript
The query text.
The execution args are used to declare a set of script variables
( set key="value";
).
string
hasSqlScript
setSqlScript
The query text.
The execution args are used to declare a set of script variables
( set key="value";
).
var
string
$this
getFileUris
Optional. Cloud Storage URIs of files to be placed in the working directory of each executor.
setFileUris
Optional. Cloud Storage URIs of files to be placed in the working directory of each executor.
var
string[]
$this
getArchiveUris
Optional. Cloud Storage URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
setArchiveUris
Optional. Cloud Storage URIs of archives to be extracted into the working directory of each executor. Supported file types: .jar, .tar, .tar.gz, .tgz, and .zip.
var
string[]
$this
getInfrastructureSpec
Optional. Infrastructure specification for the execution.
hasInfrastructureSpec
clearInfrastructureSpec
setInfrastructureSpec
Optional. Infrastructure specification for the execution.
$this
getDriver
string
