Console
    Contact Us Start free

    REST Resource: projects.locations.pipelineJobs

    Resource: PipelineJob

    An instance of a machine learning PipelineJob.

    Fields
    name string

    Output only. The resource name of the PipelineJob.

    displayName string

    The display name of the Pipeline. The name can be up to 128 characters long and can consist of any UTF-8 characters.

    createTime string ( Timestamp format)

    Output only. Pipeline creation time.

    Uses RFC 3339, where generated output will always be Z-normalized and use 0, 3, 6 or 9 fractional digits. Offsets other than "Z" are also accepted. Examples: "2014-10-02T15:01:23Z" , "2014-10-02T15:01:23.045123456Z" or "2014-10-02T15:01:23+05:30" .

    startTime string ( Timestamp format)

    Output only. Pipeline start time.

    Uses RFC 3339, where generated output will always be Z-normalized and use 0, 3, 6 or 9 fractional digits. Offsets other than "Z" are also accepted. Examples: "2014-10-02T15:01:23Z" , "2014-10-02T15:01:23.045123456Z" or "2014-10-02T15:01:23+05:30" .

    endTime string ( Timestamp format)

    Output only. Pipeline end time.

    Uses RFC 3339, where generated output will always be Z-normalized and use 0, 3, 6 or 9 fractional digits. Offsets other than "Z" are also accepted. Examples: "2014-10-02T15:01:23Z" , "2014-10-02T15:01:23.045123456Z" or "2014-10-02T15:01:23+05:30" .

    updateTime string ( Timestamp format)

    Output only. timestamp when this PipelineJob was most recently updated.

    Uses RFC 3339, where generated output will always be Z-normalized and use 0, 3, 6 or 9 fractional digits. Offsets other than "Z" are also accepted. Examples: "2014-10-02T15:01:23Z" , "2014-10-02T15:01:23.045123456Z" or "2014-10-02T15:01:23+05:30" .

    pipelineSpec object ( Struct format)

    The spec of the pipeline.

    state enum ( PipelineState )

    Output only. The detailed state of the job.

    jobDetail object ( PipelineJobDetail )

    Output only. The details of pipeline run. Not available in the list view.

    error object ( Status )

    Output only. The error that occurred during pipeline execution. Only populated when the pipeline's state is FAILED or CANCELLED.

    labels map (key: string, value: string)

    The labels with user-defined metadata to organize PipelineJob.

    label keys and values can be no longer than 64 characters (Unicode codepoints), can only contain lowercase letters, numeric characters, underscores and dashes. International characters are allowed.

    See https://goo.gl/xmQnxf for more information and examples of labels.

    Note there is some reserved label key for Vertex AI Pipelines. - vertex-ai-pipelines-run-billing-id , user set value will get overrided.

    runtimeConfig object ( RuntimeConfig )

    Runtime config of the pipeline.

    encryptionSpec object ( EncryptionSpec )

    Customer-managed encryption key spec for a pipelineJob. If set, this PipelineJob and all of its sub-resources will be secured by this key.

    network string

    The full name of the Compute Engine network to which the Pipeline Job's workload should be peered. For example, projects/12345/global/networks/myVPC . Format is of the form projects/{project}/global/networks/{network} . Where {project} is a project number, as in 12345 , and {network} is a network name.

    Private services access must already be configured for the network. Pipeline job will apply the network configuration to the Google Cloud resources being launched, if applied, such as Vertex AI Training or Dataflow job. If left unspecified, the workload is not peered with any network.

    reservedIpRanges[] string

    A list of names for the reserved ip ranges under the VPC network that can be used for this Pipeline Job's workload.

    If set, we will deploy the Pipeline Job's workload within the provided ip ranges. Otherwise, the job will be deployed to any ip ranges under the provided VPC network.

    Example: ['vertex-ai-ip-range'].

    pscInterfaceConfig object ( PscInterfaceConfig )

    Optional. Configuration for PSC-I for PipelineJob.

    templateUri string

    A template uri from where the PipelineJob.pipeline_spec , if empty, will be downloaded. Currently, only uri from Vertex Template Registry & Gallery is supported. Reference to https://cloud.google.com/vertex-ai/docs/pipelines/create-pipeline-template .

    templateMetadata object ( PipelineTemplateMetadata )

    Output only. Pipeline template metadata. Will fill up fields if PipelineJob.template_uri is from supported template registry.

    scheduleName string

    Output only. The schedule resource name. Only returned if the Pipeline is created by Schedule API.

    preflightValidations boolean

    Optional. Whether to do component level validations before job creation.

    JSON representation 
     { 
 "name" 
 : 
 string 
 , 
 "displayName" 
 : 
 string 
 , 
 "createTime" 
 : 
 string 
 , 
 "startTime" 
 : 
 string 
 , 
 "endTime" 
 : 
 string 
 , 
 "updateTime" 
 : 
 string 
 , 
 "pipelineSpec" 
 : 
 { 
 object 
 } 
 , 
 "state" 
 : 
 enum (  PipelineState 
 
) 
 , 
 "jobDetail" 
 : 
 { 
 object (  PipelineJobDetail 
 
) 
 } 
 , 
 "error" 
 : 
 { 
 object (  Status 
 
) 
 } 
 , 
 "labels" 
 : 
 { 
 string 
 : 
 string 
 , 
 ... 
 } 
 , 
 "runtimeConfig" 
 : 
 { 
 object (  RuntimeConfig 
 
) 
 } 
 , 
 "encryptionSpec" 
 : 
 { 
 object (  EncryptionSpec 
 
) 
 } 
 , 
 "serviceAccount" 
 : 
 string 
 , 
 "network" 
 : 
 string 
 , 
 "reservedIpRanges" 
 : 
 [ 
 string 
 ] 
 , 
 "pscInterfaceConfig" 
 : 
 { 
 object (  PscInterfaceConfig 
 
) 
 } 
 , 
 "templateUri" 
 : 
 string 
 , 
 "templateMetadata" 
 : 
 { 
 object (  PipelineTemplateMetadata 
 
) 
 } 
 , 
 "scheduleName" 
 : 
 string 
 , 
 "preflightValidations" 
 : 
 boolean 
 }

    PipelineJobDetail

    The runtime detail of PipelineJob.

    Fields
    pipelineContext object ( Context )

    Output only. The context of the pipeline.

    pipelineRunContext object ( Context )

    Output only. The context of the current pipeline run.

    taskDetails[] object ( PipelineTaskDetail )

    Output only. The runtime details of the tasks under the pipeline.

    JSON representation 
     { 
 "pipelineContext" 
 : 
 { 
 object (  Context 
 
) 
 } 
 , 
 "pipelineRunContext" 
 : 
 { 
 object (  Context 
 
) 
 } 
 , 
 "taskDetails" 
 : 
 [ 
 { 
 object (  PipelineTaskDetail 
 
) 
 } 
 ] 
 }

    PipelineTaskDetail

    The runtime detail of a task execution.

    Fields
    taskId string ( int64 format)

    Output only. The system generated id of the task.

    parentTaskId string ( int64 format)

    Output only. The id of the parent task if the task is within a component scope. Empty if the task is at the root level.

    taskName string

    Output only. The user specified name of the task that is defined in pipelineSpec .

    createTime string ( Timestamp format)

    Output only. Task create time.

    Uses RFC 3339, where generated output will always be Z-normalized and use 0, 3, 6 or 9 fractional digits. Offsets other than "Z" are also accepted. Examples: "2014-10-02T15:01:23Z" , "2014-10-02T15:01:23.045123456Z" or "2014-10-02T15:01:23+05:30" .

    startTime string ( Timestamp format)

    Output only. Task start time.

    Uses RFC 3339, where generated output will always be Z-normalized and use 0, 3, 6 or 9 fractional digits. Offsets other than "Z" are also accepted. Examples: "2014-10-02T15:01:23Z" , "2014-10-02T15:01:23.045123456Z" or "2014-10-02T15:01:23+05:30" .

    endTime string ( Timestamp format)

    Output only. Task end time.

    Uses RFC 3339, where generated output will always be Z-normalized and use 0, 3, 6 or 9 fractional digits. Offsets other than "Z" are also accepted. Examples: "2014-10-02T15:01:23Z" , "2014-10-02T15:01:23.045123456Z" or "2014-10-02T15:01:23+05:30" .

    executorDetail object ( PipelineTaskExecutorDetail )

    Output only. The detailed execution info.

    state enum ( State )

    Output only. state of the task.

    execution object ( Execution )

    Output only. The execution metadata of the task.

    error object ( Status )

    Output only. The error that occurred during task execution. Only populated when the task's state is FAILED or CANCELLED.

    pipelineTaskStatus[] object ( PipelineTaskStatus )

    Output only. A list of task status. This field keeps a record of task status evolving over time.

    inputs map (key: string, value: object ( ArtifactList ))

    Output only. The runtime input artifacts of the task.

    outputs map (key: string, value: object ( ArtifactList ))

    Output only. The runtime output artifacts of the task.

    taskUniqueName string

    Output only. The unique name of a task. This field is used by rerun pipeline job. Console UI and Vertex AI SDK will support triggering pipeline job reruns. The name is constructed by concatenating all the parent tasks name with the task name. For example, if a task named "child_task" has a parent task named "parent_task_1" and parent task 1 has a parent task named "parent_task_2", the task unique name will be "parent_task_2.parent_task_1.child_task".

    JSON representation 
     { 
 "taskId" 
 : 
 string 
 , 
 "parentTaskId" 
 : 
 string 
 , 
 "taskName" 
 : 
 string 
 , 
 "createTime" 
 : 
 string 
 , 
 "startTime" 
 : 
 string 
 , 
 "endTime" 
 : 
 string 
 , 
 "executorDetail" 
 : 
 { 
 object (  PipelineTaskExecutorDetail 
 
) 
 } 
 , 
 "state" 
 : 
 enum (  State 
 
) 
 , 
 "execution" 
 : 
 { 
 object (  Execution 
 
) 
 } 
 , 
 "error" 
 : 
 { 
 object (  Status 
 
) 
 } 
 , 
 "pipelineTaskStatus" 
 : 
 [ 
 { 
 object (  PipelineTaskStatus 
 
) 
 } 
 ] 
 , 
 "inputs" 
 : 
 { 
 string 
 : 
 { 
 object (  ArtifactList 
 
) 
 } 
 , 
 ... 
 } 
 , 
 "outputs" 
 : 
 { 
 string 
 : 
 { 
 object (  ArtifactList 
 
) 
 } 
 , 
 ... 
 } 
 , 
 "taskUniqueName" 
 : 
 string 
 }

    PipelineTaskExecutorDetail

    The runtime detail of a pipeline executor.

    Fields
    details Union type
    details can be only one of the following:
    containerDetail object ( ContainerDetail )

    Output only. The detailed info for a container executor.

    customJobDetail object ( CustomJobDetail )

    Output only. The detailed info for a custom job executor.

    JSON representation 
     { 
 // details 
 "containerDetail" 
 : 
 { 
 object (  ContainerDetail 
 
) 
 } 
 , 
 "customJobDetail" 
 : 
 { 
 object (  CustomJobDetail 
 
) 
 } 
 // Union type 
 }

    ContainerDetail

    The detail of a container execution. It contains the job names of the lifecycle of a container execution.

    Fields
    mainJob string

    Output only. The name of the CustomJob for the main container execution.

    preCachingCheckJob string

    Output only. The name of the CustomJob for the pre-caching-check container execution. This job will be available if the PipelineJob.pipeline_spec specifies the pre_caching_check hook in the lifecycle events.

    failedMainJobs[] string

    Output only. The names of the previously failed CustomJob for the main container executions. The list includes the all attempts in chronological order.

    failedPreCachingCheckJobs[] string

    Output only. The names of the previously failed CustomJob for the pre-caching-check container executions. This job will be available if the PipelineJob.pipeline_spec specifies the pre_caching_check hook in the lifecycle events. The list includes the all attempts in chronological order.

    JSON representation 
     { 
 "mainJob" 
 : 
 string 
 , 
 "preCachingCheckJob" 
 : 
 string 
 , 
 "failedMainJobs" 
 : 
 [ 
 string 
 ] 
 , 
 "failedPreCachingCheckJobs" 
 : 
 [ 
 string 
 ] 
 }

    CustomJobDetail

    The detailed info for a custom job executor.

    Fields
    job string

    Output only. The name of the CustomJob .

    failedJobs[] string

    Output only. The names of the previously failed CustomJob . The list includes the all attempts in chronological order.

    JSON representation 
     { 
 "job" 
 : 
 string 
 , 
 "failedJobs" 
 : 
 [ 
 string 
 ] 
 }

    State

    Specifies state of TaskExecution

    Enums
    STATE_UNSPECIFIED Unspecified.
    PENDING Specifies pending state for the task.
    RUNNING Specifies task is being executed.
    SUCCEEDED Specifies task completed successfully.
    CANCEL_PENDING Specifies Task cancel is in pending state.
    CANCELLING Specifies task is being cancelled.
    CANCELLED Specifies task was cancelled.
    FAILED Specifies task failed.
    SKIPPED Specifies task was skipped due to cache hit.
    NOT_TRIGGERED Specifies that the task was not triggered because the task's trigger policy is not satisfied. The trigger policy is specified in the condition field of PipelineJob.pipeline_spec .

    PipelineTaskStatus

    A single record of the task status.

    Fields
    updateTime string ( Timestamp format)

    Output only. Update time of this status.

    Uses RFC 3339, where generated output will always be Z-normalized and use 0, 3, 6 or 9 fractional digits. Offsets other than "Z" are also accepted. Examples: "2014-10-02T15:01:23Z" , "2014-10-02T15:01:23.045123456Z" or "2014-10-02T15:01:23+05:30" .

    state enum ( State )

    Output only. The state of the task.

    error object ( Status )

    Output only. The error that occurred during the state. May be set when the state is any of the non-final state (PENDING/RUNNING/CANCELLING) or FAILED state. If the state is FAILED, the error here is final and not going to be retried. If the state is a non-final state, the error indicates a system-error being retried.

    JSON representation 
     { 
 "updateTime" 
 : 
 string 
 , 
 "state" 
 : 
 enum (  State 
 
) 
 , 
 "error" 
 : 
 { 
 object (  Status 
 
) 
 } 
 }

    ArtifactList

    A list of artifact metadata.

    Fields
    artifacts[] object ( Artifact )

    Output only. A list of artifact metadata.

    JSON representation 
     { 
 "artifacts" 
 : 
 [ 
 { 
 object (  Artifact 
 
) 
 } 
 ] 
 }

    RuntimeConfig

    The runtime config of a PipelineJob.

    Fields
    parameters
    (deprecated)     map (key: string, value: object ( Value ))

    Deprecated. Use RuntimeConfig.parameter_values instead. The runtime parameters of the PipelineJob. The parameters will be passed into PipelineJob.pipeline_spec to replace the placeholders at runtime. This field is used by pipelines built using PipelineJob.pipeline_spec.schema_version 2.0.0 or lower, such as pipelines built using Kubeflow Pipelines SDK 1.8 or lower.

    gcsOutputDirectory string

    Required. A path in a Cloud Storage bucket, which will be treated as the root output directory of the pipeline. It is used by the system to generate the paths of output artifacts. The artifact paths are generated with a sub-path pattern {job_id}/{taskId}/{outputKey} under the specified output directory. The service account specified in this pipeline must have the storage.objects.get and storage.objects.create permissions for this bucket.

    parameterValues map (key: string, value: value ( Value format))

    The runtime parameters of the PipelineJob. The parameters will be passed into PipelineJob.pipeline_spec to replace the placeholders at runtime. This field is used by pipelines built using PipelineJob.pipeline_spec.schema_version 2.1.0, such as pipelines built using Kubeflow Pipelines SDK 1.9 or higher and the v2 DSL.

    failurePolicy enum ( PipelineFailurePolicy )

    Represents the failure policy of a pipeline. Currently, the default of a pipeline is that the pipeline will continue to run until no more tasks can be executed, also known as PIPELINE_FAILURE_POLICY_FAIL_SLOW. However, if a pipeline is set to PIPELINE_FAILURE_POLICY_FAIL_FAST, it will stop scheduling any new tasks when a task has failed. Any scheduled tasks will continue to completion.

    inputArtifacts map (key: string, value: object ( InputArtifact ))

    The runtime artifacts of the PipelineJob. The key will be the input artifact name and the value would be one of the InputArtifact.

    JSON representation 
     { 
 "parameters" 
 : 
 { 
 string 
 : 
 { 
 object (  Value 
 
) 
 } 
 , 
 ... 
 } 
 , 
 "gcsOutputDirectory" 
 : 
 string 
 , 
 "parameterValues" 
 : 
 { 
 string 
 : 
 value 
 , 
 ... 
 } 
 , 
 "failurePolicy" 
 : 
 enum (  PipelineFailurePolicy 
 
) 
 , 
 "inputArtifacts" 
 : 
 { 
 string 
 : 
 { 
 object (  InputArtifact 
 
) 
 } 
 , 
 ... 
 } 
 }

    Value

    value is the value of the field.

    Fields
    value Union type
    value can be only one of the following:
    intValue string ( int64 format)

    An integer value.

    doubleValue number

    A double value.

    stringValue string

    A string value.

    JSON representation 
     { 
 // value 
 "intValue" 
 : 
 string 
 , 
 "doubleValue" 
 : 
 number 
 , 
 "stringValue" 
 : 
 string 
 // Union type 
 }

    PipelineFailurePolicy

    Represents the failure policy of a pipeline. Currently, the default of a pipeline is that the pipeline will continue to run until no more tasks can be executed, also known as PIPELINE_FAILURE_POLICY_FAIL_SLOW. However, if a pipeline is set to PIPELINE_FAILURE_POLICY_FAIL_FAST, it will stop scheduling any new tasks when a task has failed. Any scheduled tasks will continue to completion.

    Enums
    PIPELINE_FAILURE_POLICY_UNSPECIFIED Default value, and follows fail slow behavior.
    PIPELINE_FAILURE_POLICY_FAIL_SLOW Indicates that the pipeline should continue to run until all possible tasks have been scheduled and completed.
    PIPELINE_FAILURE_POLICY_FAIL_FAST Indicates that the pipeline should stop scheduling new tasks after a task has failed.

    InputArtifact

    The type of an input artifact.

    Fields
    kind Union type
    kind can be only one of the following:
    artifactId string

    Artifact resource id from MLMD. Which is the last portion of an artifact resource name: projects/{project}/locations/{location}/metadataStores/default/artifacts/{artifactId} . The artifact must stay within the same project, location and default metadatastore as the pipeline.

    JSON representation 
     { 
 // kind 
 "artifactId" 
 : 
 string 
 // Union type 
 }

    PipelineTemplateMetadata

    Pipeline template metadata if PipelineJob.template_uri is from supported template registry. Currently, the only supported registry is Artifact Registry.

    Fields
    version string

    The versionName in artifact registry.

    Will always be presented in output if the PipelineJob.template_uri is from supported template registry.

    Format is "sha256:abcdef123456...".

    JSON representation 
     { 
 "version" 
 : 
 string 
 }

    Methods

    batchCancel

    		Batch cancel PipelineJobs.

    batchDelete

    		Batch deletes PipelineJobs The Operation is atomic.

    cancel

    		Cancels a PipelineJob.

    create

    		Creates a PipelineJob.

    delete

    		Deletes a PipelineJob.

    get

    		Gets a PipelineJob.

    list

    		Lists PipelineJobs in a Location.
    Design a Mobile Site
    View Site in Mobile | Classic
    Share by: