A Dataproc job for running Apache Pig queries on YARN.
| JSON representation |
|---|
{ "continueOnFailure" : boolean , "scriptVariables" : { string : string , ... } , "properties" : { string : string , ... } , "jarFileUris" : [ string ] , "loggingConfig" : { object ( |
continueOnFailure
boolean
Optional. Whether to continue executing queries if a query fails. The default value is false
. Setting to true
can be useful when executing independent parallel queries.
scriptVariables
map (key: string, value: string)
Optional. Mapping of query variable names to values (equivalent to the Pig command: name=[value]
).
An object containing a list of "key": value
pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }
.
properties
map (key: string, value: string)
Optional. A mapping of property names to values, used to configure Pig. Properties that conflict with values set by the Dataproc API might be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml
, /etc/pig/conf/pig.properties, and classes in user code.
An object containing a list of "key": value
pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }
.
jarFileUris[]
string
Optional. HCFS URIs of jar files to add to the CLASSPATH of the Pig Client and Hadoop MapReduce (MR) tasks. Can contain Pig UDFs.
loggingConfig
object (
LoggingConfig
)
Optional. The runtime log config for job execution.
queries
. Required. The sequence of Pig queries to execute, specified as an HCFS file URI or a list of queries. queries
can be only one of the following:queryFileUri
string
The HCFS URI of the script that contains the Pig queries.
queryList
object (
QueryList
)
A list of queries.
