ExecutionTemplate

The description a notebook execution workload.

JSON representation

JSON representation
{ "scaleTier" : enum ( `ScaleTier` ) , "masterType" : string , "acceleratorConfig" : { object ( `SchedulerAcceleratorConfig` ) } , "labels" : { string : string , ... } , "inputNotebookFile" : string , "containerImageUri" : string , "outputNotebookFolder" : string , "paramsYamlFile" : string , "parameters" : string , "serviceAccount" : string , "jobType" : enum ( `JobType` ) , "kernelSpec" : string , "tensorboard" : string , // Union field `job_parameters` can be only one of the following: "dataprocParameters" : { object ( `DataprocParameters` ) } , "vertexAiParameters" : { object ( `VertexAIParameters` ) } // End of list of possible types for union field `job_parameters` . }

 { 
 "scaleTier" 
 : 
 enum (  ScaleTier 
 
) 
 , 
 "masterType" 
 : 
 string 
 , 
 "acceleratorConfig" 
 : 
 { 
 object (  SchedulerAcceleratorConfig 
 
) 
 } 
 , 
 "labels" 
 : 
 { 
 string 
 : 
 string 
 , 
 ... 
 } 
 , 
 "inputNotebookFile" 
 : 
 string 
 , 
 "containerImageUri" 
 : 
 string 
 , 
 "outputNotebookFolder" 
 : 
 string 
 , 
 "paramsYamlFile" 
 : 
 string 
 , 
 "parameters" 
 : 
 string 
 , 
 "serviceAccount" 
 : 
 string 
 , 
 "jobType" 
 : 
 enum (  JobType 
 
) 
 , 
 "kernelSpec" 
 : 
 string 
 , 
 "tensorboard" 
 : 
 string 
 , 
 // Union field job_parameters 
can be only one of the following: 
 "dataprocParameters" 
 : 
 { 
 object (  DataprocParameters 
 
) 
 } 
 , 
 "vertexAiParameters" 
 : 
 { 
 object (  VertexAIParameters 
 
) 
 } 
 // End of list of possible types for union field job_parameters 
. 
 }

Fields

scaleTier
 (deprecated)

enum ( ScaleTier )

Required. Scale tier of the hardware used for notebook execution. DEPRECATED Will be discontinued. As right now only CUSTOM is supported.

masterType

string

Specifies the type of virtual machine to use for your training job's master worker. You must specify this field when scaleTier is set to CUSTOM .

You can use certain Compute Engine machine types directly in this field. The following types are supported:

n1-standard-4
n1-standard-8
n1-standard-16
n1-standard-32
n1-standard-64
n1-standard-96
n1-highmem-2
n1-highmem-4
n1-highmem-8
n1-highmem-16
n1-highmem-32
n1-highmem-64
n1-highmem-96
n1-highcpu-16
n1-highcpu-32
n1-highcpu-64
n1-highcpu-96

Alternatively, you can use the following legacy machine types:

standard
large_model
complex_model_s
complex_model_m
complex_model_l
standard_gpu
complex_model_m_gpu
complex_model_l_gpu
standard_p100
complex_model_m_p100
standard_v100
large_model_v100
complex_model_m_v100
complex_model_l_v100

Finally, if you want to use a TPU for training, specify cloud_tpu in this field. Learn more about the special configuration options for training with TPU .

acceleratorConfig

object ( SchedulerAcceleratorConfig )

Configuration (count and accelerator type) for hardware running notebook execution.

labels

map (key: string, value: string)

Labels for execution. If execution is scheduled, a field included will be 'nbs-scheduled'. Otherwise, it is an immediate execution, and an included field will be 'nbs-immediate'. Use fields to efficiently index between various types of executions.

An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" } .

inputNotebookFile

string

Path to the notebook file to execute. Must be in a Google Cloud Storage bucket. Format: gs://{bucket_name}/{folder}/{notebook_file_name} Ex: gs://notebook_user/scheduled_notebooks/sentiment_notebook.ipynb

containerImageUri

string

Container Image URI to a DLVM Example: 'gcr.io/deeplearning-platform-release/base-cu100' More examples can be found at: https://cloud.google.com/ai-platform/deep-learning-containers/docs/choosing-container

outputNotebookFolder

string

Path to the notebook folder to write to. Must be in a Google Cloud Storage bucket path. Format: gs://{bucket_name}/{folder} Ex: gs://notebook_user/scheduled_notebooks

paramsYamlFile

string

Parameters to be overridden in the notebook during execution. Ref https://papermill.readthedocs.io/en/latest/usage-parameterize.html on how to specifying parameters in the input notebook and pass them here in an YAML file. Ex: gs://notebook_user/scheduled_notebooks/sentiment_notebook_params.yaml

parameters

string

Parameters used within the 'inputNotebookFile' notebook.

serviceAccount

string

The email address of a service account to use when running the execution. You must have the iam.serviceAccounts.actAs permission for the specified service account.

jobType

enum ( JobType )

The type of Job to be used on this execution.

kernelSpec

string

Name of the kernel spec to use. This must be specified if the kernel spec name on the execution target does not match the name in the input notebook file.

tensorboard

string

The name of a Vertex AI [Tensorboard] resource to which this execution will upload Tensorboard logs. Format: projects/{project}/locations/{location}/tensorboards/{tensorboard}

Union field job_parameters . Parameters for an execution type. NOTE: There are currently no extra parameters for VertexAI jobs. job_parameters can be only one of the following:

dataprocParameters

object ( DataprocParameters )

Parameters used in Dataproc JobType executions.

vertexAiParameters

object ( VertexAIParameters )

Parameters used in Vertex AI JobType executions.

ScaleTier

Required. Specifies the machine types, the number of replicas for workers and parameter servers.

Enums

SCALE_TIER_UNSPECIFIED

Unspecified Scale Tier.

BASIC

A single worker instance. This tier is suitable for learning how to use Cloud ML, and for experimenting with new models using small datasets.

STANDARD_1

Many workers and a few parameter servers.

PREMIUM_1

A large number of workers with many parameter servers.

BASIC_GPU

A single worker instance with a K80 GPU.

BASIC_TPU

A single worker instance with a Cloud TPU.

CUSTOM

The CUSTOM tier is not a set tier, but rather enables you to use your own cluster specification. When you use this tier, set values to configure your processing cluster according to these guidelines:

You must set ExecutionTemplate.masterType to specify the type of machine to use for your master node. This is the only required setting.

SchedulerAcceleratorConfig

Definition of a hardware accelerator. Note that not all combinations of type and coreCount are valid. See GPUs on Compute Engine to find a valid combination. TPUs are not supported.

JSON representation
{ "type" : enum ( `SchedulerAcceleratorType` ) , "coreCount" : string }

Fields

Fields
`type`	`enum ( SchedulerAcceleratorType )` Type of this accelerator.
`coreCount`	`string ( int64 format)` Count of cores of this accelerator.

type

enum ( SchedulerAcceleratorType )

Type of this accelerator.

coreCount

string ( int64 format)

Count of cores of this accelerator.

SchedulerAcceleratorType

Hardware accelerator types for AI Platform Training jobs.

Enums
`SCHEDULER_ACCELERATOR_TYPE_UNSPECIFIED`	Unspecified accelerator type. Default to no GPU.
`NVIDIA_TESLA_K80`	Nvidia Tesla K80 GPU.
`NVIDIA_TESLA_P100`	Nvidia Tesla P100 GPU.
`NVIDIA_TESLA_V100`	Nvidia Tesla V100 GPU.
`NVIDIA_TESLA_P4`	Nvidia Tesla P4 GPU.
`NVIDIA_TESLA_T4`	Nvidia Tesla T4 GPU.
`NVIDIA_TESLA_A100`	Nvidia Tesla A100 GPU.
`TPU_V2`	TPU v2.
`TPU_V3`	TPU v3.

JobType

The backend used for this execution.

Enums
`JOB_TYPE_UNSPECIFIED`	No type specified.
`VERTEX_AI`	Custom Job in `aiplatform.googleapis.com` . Default value for an execution.
`DATAPROC`	Run execution on a cluster with Dataproc as a job. https://cloud.google.com/dataproc/docs/reference/rest/v1/projects.regions.jobs

DataprocParameters

Parameters used in Dataproc JobType executions.

JSON representation
{ "cluster" : string }

Fields

Fields
`cluster`	`string` URI for cluster used to run Dataproc execution. Format: `projects/{PROJECT_ID}/regions/{REGION}/clusters/{CLUSTER_NAME}`

cluster

string

URI for cluster used to run Dataproc execution. Format: projects/{PROJECT_ID}/regions/{REGION}/clusters/{CLUSTER_NAME}

VertexAIParameters

Parameters used in Vertex AI JobType executions.

JSON representation
{ "network" : string , "env" : { string : string , ... } }

Fields

Fields
`network`	`string` The full name of the Compute Engine network to which the Job should be peered. For example, `projects/12345/global/networks/myVPC` . Format is of the form `projects/{project}/global/networks/{network}` . Where `{project}` is a project number, as in `12345` , and `{network}` is a network name. Private services access must already be configured for the network. If left unspecified, the job is not peered with any network.
`env`	`map (key: string, value: string)` Environment variables. At most 100 environment variables can be specified and unique. Example: `GCP_BUCKET=gs://my-bucket/samples/` An object containing a list of `"key": value` pairs. Example: `{ "name": "wrench", "mass": "1.3kg", "count": "3" }` .

network

string

The full name of the Compute Engine network to which the Job should be peered. For example, projects/12345/global/networks/myVPC . Format is of the form projects/{project}/global/networks/{network} . Where {project} is a project number, as in 12345 , and {network} is a network name.

Private services access must already be configured for the network. If left unspecified, the job is not peered with any network.

env

map (key: string, value: string)

Environment variables. At most 100 environment variables can be specified and unique. Example: GCP_BUCKET=gs://my-bucket/samples/

An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" } .