Reference documentation and code samples for the Google Cloud Dataflow V1beta3 Client class Environment.
Describes the environment in which a Dataflow Job runs.
Generated from protobuf message google.dataflow.v1beta3.Environment
Methods
__construct
Constructor.
data
array
Optional. Data for populating the Message object.
↳ temp_storage_prefix
string
The prefix of the resources the system should use for temporary storage. The system will append the suffix "/temp-{JOBNAME} to this resource prefix, where {JOBNAME} is the value of the job_name field. The resulting bucket and object prefix is used as the prefix of the resources used to store temporary data needed during the job execution. NOTE: This will override the value in taskrunner_settings. The supported resource type is: Google Cloud Storage: storage.googleapis.com/{bucket}/{object} bucket.storage.googleapis.com/{object}
↳ cluster_manager_api_service
string
The type of cluster manager API to use. If unknown or unspecified, the service will attempt to choose a reasonable default. This should be in the form of the API service name, e.g. "compute.googleapis.com".
↳ experiments
array
The list of experiments to enable. This field should be used for SDK related experiments and not for service related experiments. The proper field for service related experiments is service_options.
↳ service_options
array
The list of service options to enable. This field should be used for service related experiments only. These experiments, when graduating to GA, should be replaced by dedicated fields or become default (i.e. always on).
↳ service_kms_key_name
string
If set, contains the Cloud KMS key identifier used to encrypt data at rest, AKA a Customer Managed Encryption Key (CMEK). Format: projects/PROJECT_ID/locations/LOCATION/keyRings/KEY_RING/cryptoKeys/KEY
↳ worker_pools
array< Google\Cloud\Dataflow\V1beta3\WorkerPool
>
The worker pools. At least one "harness" worker pool must be specified in order for the job to have workers.
↳ user_agent
↳ version
Google\Protobuf\Struct
A structure describing which components and their versions of the service are required in order to run the job.
↳ dataset
string
The dataset for the current project where various workflow related tables are stored. The supported resource type is: Google BigQuery: bigquery.googleapis.com/{dataset}
↳ sdk_pipeline_options
Google\Protobuf\Struct
The Cloud Dataflow SDK pipeline options specified by the user. These options are passed through the service and are used to recreate the SDK pipeline options on the worker in a language agnostic and platform independent way.
↳ internal_experiments
↳ service_account_email
string
Identity to run virtual machines as. Defaults to the default account.
↳ flex_resource_scheduling_goal
int
Which Flexible Resource Scheduling mode to run in.
↳ worker_region
string
The Compute Engine region ( https://cloud.google.com/compute/docs/regions-zones/regions-zones ) in which worker processing should occur, e.g. "us-west1". Mutually exclusive with worker_zone. If neither worker_region nor worker_zone is specified, default to the control plane's region.
↳ worker_zone
string
The Compute Engine zone ( https://cloud.google.com/compute/docs/regions-zones/regions-zones ) in which worker processing should occur, e.g. "us-west1-a". Mutually exclusive with worker_region. If neither worker_region nor worker_zone is specified, a zone in the control plane's region is chosen based on available capacity.
↳ shuffle_mode
int
Output only. The shuffle mode used for the job.
↳ debug_options
getTempStoragePrefix
The prefix of the resources the system should use for temporary storage. The system will append the suffix "/temp-{JOBNAME} to this resource prefix, where {JOBNAME} is the value of the job_name field. The resulting bucket and object prefix is used as the prefix of the resources used to store temporary data needed during the job execution. NOTE: This will override the value in taskrunner_settings.
The supported resource type is: Google Cloud Storage: storage.googleapis.com/{bucket}/{object} bucket.storage.googleapis.com/{object}
string
setTempStoragePrefix
The prefix of the resources the system should use for temporary storage. The system will append the suffix "/temp-{JOBNAME} to this resource prefix, where {JOBNAME} is the value of the job_name field. The resulting bucket and object prefix is used as the prefix of the resources used to store temporary data needed during the job execution. NOTE: This will override the value in taskrunner_settings.
The supported resource type is: Google Cloud Storage: storage.googleapis.com/{bucket}/{object} bucket.storage.googleapis.com/{object}
var
string
$this
getClusterManagerApiService
The type of cluster manager API to use. If unknown or unspecified, the service will attempt to choose a reasonable default. This should be in the form of the API service name, e.g. "compute.googleapis.com".
string
setClusterManagerApiService
The type of cluster manager API to use. If unknown or unspecified, the service will attempt to choose a reasonable default. This should be in the form of the API service name, e.g. "compute.googleapis.com".
var
string
$this
getExperiments
The list of experiments to enable. This field should be used for SDK related experiments and not for service related experiments. The proper field for service related experiments is service_options.
setExperiments
The list of experiments to enable. This field should be used for SDK related experiments and not for service related experiments. The proper field for service related experiments is service_options.
var
string[]
$this
getServiceOptions
The list of service options to enable. This field should be used for service related experiments only. These experiments, when graduating to GA, should be replaced by dedicated fields or become default (i.e. always on).
setServiceOptions
The list of service options to enable. This field should be used for service related experiments only. These experiments, when graduating to GA, should be replaced by dedicated fields or become default (i.e. always on).
var
string[]
$this
getServiceKmsKeyName
If set, contains the Cloud KMS key identifier used to encrypt data at rest, AKA a Customer Managed Encryption Key (CMEK).
Format: projects/PROJECT_ID/locations/LOCATION/keyRings/KEY_RING/cryptoKeys/KEY
string
setServiceKmsKeyName
If set, contains the Cloud KMS key identifier used to encrypt data at rest, AKA a Customer Managed Encryption Key (CMEK).
Format: projects/PROJECT_ID/locations/LOCATION/keyRings/KEY_RING/cryptoKeys/KEY
var
string
$this
getWorkerPools
The worker pools. At least one "harness" worker pool must be specified in order for the job to have workers.
setWorkerPools
The worker pools. At least one "harness" worker pool must be specified in order for the job to have workers.
$this
getUserAgent
A description of the process that generated the request.
hasUserAgent
clearUserAgent
setUserAgent
A description of the process that generated the request.
$this
getVersion
A structure describing which components and their versions of the service are required in order to run the job.
hasVersion
clearVersion
setVersion
A structure describing which components and their versions of the service are required in order to run the job.
$this
getDataset
The dataset for the current project where various workflow related tables are stored.
The supported resource type is: Google BigQuery: bigquery.googleapis.com/{dataset}
string
setDataset
The dataset for the current project where various workflow related tables are stored.
The supported resource type is: Google BigQuery: bigquery.googleapis.com/{dataset}
var
string
$this
getSdkPipelineOptions
The Cloud Dataflow SDK pipeline options specified by the user. These options are passed through the service and are used to recreate the SDK pipeline options on the worker in a language agnostic and platform independent way.
hasSdkPipelineOptions
clearSdkPipelineOptions
setSdkPipelineOptions
The Cloud Dataflow SDK pipeline options specified by the user. These options are passed through the service and are used to recreate the SDK pipeline options on the worker in a language agnostic and platform independent way.
$this
getInternalExperiments
Experimental settings.
hasInternalExperiments
clearInternalExperiments
setInternalExperiments
Experimental settings.
$this
getServiceAccountEmail
Identity to run virtual machines as. Defaults to the default account.
string
setServiceAccountEmail
Identity to run virtual machines as. Defaults to the default account.
var
string
$this
getFlexResourceSchedulingGoal
Which Flexible Resource Scheduling mode to run in.
int
setFlexResourceSchedulingGoal
Which Flexible Resource Scheduling mode to run in.
var
int
$this
getWorkerRegion
The Compute Engine region ( https://cloud.google.com/compute/docs/regions-zones/regions-zones ) in which worker processing should occur, e.g. "us-west1". Mutually exclusive with worker_zone. If neither worker_region nor worker_zone is specified, default to the control plane's region.
string
setWorkerRegion
The Compute Engine region ( https://cloud.google.com/compute/docs/regions-zones/regions-zones ) in which worker processing should occur, e.g. "us-west1". Mutually exclusive with worker_zone. If neither worker_region nor worker_zone is specified, default to the control plane's region.
var
string
$this
getWorkerZone
The Compute Engine zone ( https://cloud.google.com/compute/docs/regions-zones/regions-zones ) in which worker processing should occur, e.g. "us-west1-a". Mutually exclusive with worker_region. If neither worker_region nor worker_zone is specified, a zone in the control plane's region is chosen based on available capacity.
string
setWorkerZone
The Compute Engine zone ( https://cloud.google.com/compute/docs/regions-zones/regions-zones ) in which worker processing should occur, e.g. "us-west1-a". Mutually exclusive with worker_region. If neither worker_region nor worker_zone is specified, a zone in the control plane's region is chosen based on available capacity.
var
string
$this
getShuffleMode
Output only. The shuffle mode used for the job.
int
setShuffleMode
Output only. The shuffle mode used for the job.
var
int
$this
getDebugOptions
Any debugging options to be supplied to the job.
hasDebugOptions
clearDebugOptions
setDebugOptions
Any debugging options to be supplied to the job.
$this
