PigJob
(
mapping
=
None
,
*
,
ignore_unknown_fields
=
False
,
**
kwargs
)
A Dataproc job for running Apache Pig <https://pig.apache.org/>
__
queries on YARN.
Attributes
Name
Description
query_file_uri
str
The HCFS URI of the script that contains the Pig queries.
query_list
.gcd_jobs.QueryList
A list of queries.
continue_on_failure
bool
Optional. Whether to continue executing queries if a query fails. The default value is
false
. Setting to true
can be useful when executing independent parallel queries.script_variables
Sequence[ .gcd_jobs.PigJob.ScriptVariablesEntry
]
Optional. Mapping of query variable names to values (equivalent to the Pig command:
name=[value]
).properties
Sequence[ .gcd_jobs.PigJob.PropertiesEntry
]
Optional. A mapping of property names to values, used to configure Pig. Properties that conflict with values set by the Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site.xml, /etc/pig/conf/pig.properties, and classes in user code.
jar_file_uris
Sequence[str]
Optional. HCFS URIs of jar files to add to the CLASSPATH of the Pig Client and Hadoop MapReduce (MR) tasks. Can contain Pig UDFs.
logging_config
.gcd_jobs.LoggingConfig
Optional. The runtime log config for job execution.
Classes
PropertiesEntry
PropertiesEntry
(
mapping
=
None
,
*
,
ignore_unknown_fields
=
False
,
**
kwargs
)
The abstract base class for a message.
Parameters
Name
Description
kwargs
dict
Keys and values corresponding to the fields of the message.
mapping
Union[dict, .Message
]
A dictionary or message to be used to determine the values for this message.
ignore_unknown_fields
Optional(bool)
If True, do not raise errors for unknown fields. Only applied if mapping
is a mapping type or there are keyword parameters.
ScriptVariablesEntry
ScriptVariablesEntry
(
mapping
=
None
,
*
,
ignore_unknown_fields
=
False
,
**
kwargs
)
The abstract base class for a message.
Parameters
Name
Description
kwargs
dict
Keys and values corresponding to the fields of the message.
mapping
Union[dict, .Message
]
A dictionary or message to be used to determine the values for this message.
ignore_unknown_fields
Optional(bool)
If True, do not raise errors for unknown fields. Only applied if mapping
is a mapping type or there are keyword parameters.
