This document explains how to configure custom status events, which describe a job's runnables, when you create and run a Batch job. To learn about status events, see View a job's history through status events .
Custom status events let you provide additional details in a task's history about the progress of its runnables, which can help make a job easier to analyze and troubleshoot. For example, you can configure custom status events that describe when a runnable starts, a runnable ends, a barrier runnable is reached, or an important event happens during the progression of your code.
Before you begin
- If you haven't used Batch before, review Get started with Batch and enable Batch by completing the prerequisites for projects and users .
-
To get the permissions that you need to create a job, ask your administrator to grant you the following IAM roles:
- Batch Job Editor
(
roles/batch.jobsEditor) on the project - Service Account User
(
roles/iam.serviceAccountUser) on the job's service account , which by default is the default Compute Engine service account
For more information about granting roles, see Manage access to projects, folders, and organizations .
You might also be able to get the required permissions through custom roles or other predefined roles .
- Batch Job Editor
(
Configure custom status events
Configure custom status events by using one or more of following options when you are creating a job :
-
Describe a runnable's state by defining its display name. You can do this when you create a job using the gcloud CLI or Batch API.
-
Indicate important runtime events by writing a structured task log with the
batch/custom/eventfield for each event. You can do this when using any method to create a job as part of the definitions of script and container runnables.
Describe a runnable's state
You can configure custom status events that describe a runnable's state by
defining a runnable's display name
( displayName
field
).
The resulting custom status events vary slightly for different types of
runnables:
-
If you define a display name for a container runnable or script runnable , then Batch automatically adds two types of custom status events. The first custom status event indicates whenever a task starts this runnable. The second custom status event indicates whenever a tasks finishes this runnable and the corresponding exit code .
-
If you define a display name for a barrier runnable , then Batch automatically adds a custom status event that indicates whenever a task reaches this barrier.
To create and run a job with custom status events that describes a
runnable's state, define the displayName
field for one or more
runnables using the gcloud CLI, Batch API or
library.
gcloud
Use the Google Cloud CLI to create a job
that
includes the displayName
field in one or more runnables
definitions
in the JSON file:
...
"runnables"
:
[
{
"displayName"
:
DISPLAY_NAME
,
...
}
]
...
For example, a job with custom status events that describes each runnable's state can have a JSON configuration file similar to the following:
{
"taskGroups"
:
[
{
"taskSpec"
:
{
"runnables"
:
[
{
"displayName"
:
" DISPLAY_NAME1
"
,
"script"
:
{
"text"
:
"echo Hello world from script 1 for task ${BATCH_TASK_INDEX}"
}
},
{
"displayName"
:
" DISPLAY_NAME2
"
,
"barrier"
:
{}
},
{
"displayName"
:
" DISPLAY_NAME3
"
,
"script"
:
{
"text"
:
"echo Hello world from script 2 for task ${BATCH_TASK_INDEX}"
}
}
]
},
"taskCount"
:
3
}
],
"logsPolicy"
:
{
"destination"
:
"CLOUD_LOGGING"
}
}
Replace DISPLAY_NAME1
, DISPLAY_NAME2
, and DISPLAY_NAME3
with the name of the runnable, which
must be unique within the job—for example, script 1
, barrier 1
, and script 2
.
API
Use the REST API to create a job
that
includes the displayName
field in one or more runnables
definitions
in the JSON file:
...
"runnables"
:
[
{
"displayName"
:
DISPLAY_NAME
,
...
}
]
...
For example, a job with custom status events that describes each runnable's state can have a JSON configuration file similar to the following:
{
"taskGroups"
:
[
{
"taskSpec"
:
{
"runnables"
:
[
{
"displayName"
:
" DISPLAY_NAME1
"
,
"script"
:
{
"text"
:
"echo Hello world from script 1 for task ${BATCH_TASK_INDEX}"
}
},
{
"displayName"
:
" DISPLAY_NAME2
"
,
"barrier"
:
{}
},
{
"displayName"
:
" DISPLAY_NAME3
"
,
"script"
:
{
"text"
:
"echo Hello world from script 2 for task ${BATCH_TASK_INDEX}"
}
}
]
},
"taskCount"
:
3
}
],
"logsPolicy"
:
{
"destination"
:
"CLOUD_LOGGING"
}
}
Replace DISPLAY_NAME1
, DISPLAY_NAME2
, and DISPLAY_NAME3
with the name of the runnable, which
must be unique within the job—for example, script 1
, barrier 1
, and script 2
.
Go
Java
Node.js
Python
After the example job has finished running, the resulting custom status events for each task are similar to the following:
statusEvents:
...
- description: 'script at index #0 with display name [ DISPLAY_NAME1
] started.'
eventTime: '...'
type: RUNNABLE_EVENT
- description: 'script at index #0 with display name [ DISPLAY_NAME1
] finished with exit
code 0.'
eventTime: '...'
type: RUNNABLE_EVENT
- description: 'barrier at index #2 with display name [ DISPLAY_NAME2
] reached.'
eventTime: '...'
type: BARRIER_REACHED_EVENT
- description: 'script at index #2 with display name [ DISPLAY_NAME3
] started.'
eventTime: '...'
type: RUNNABLE_EVENT
- description: 'script at index #2 with display name [ DISPLAY_NAME3
] finished with exit
code 0.'
eventTime: '...'
type: RUNNABLE_EVENT
...
Indicate important runtime events
You can configure custom status events that indicate when an important event
happens while a runnable is running by configuring that runnable to write a
structured task log that defines a string for the Batch
custom status event ( batch/custom/event
) field.
If a container runnable or script runnable writes a structured task log
that defines the batch/custom/event
JSON field, it produces a
custom status event at that time. Although you might configure the structured
task log to include additional fields, the custom status event only includes
the string for the batch/custom/event
field.
To create and run a job with custom status events that indicate when an
important event happens, configure one or more runnables to write a structured log by printing JSON
and define the batch/custom/event
field as part of the log.
...
"runnables"
:
[
{
...
"echo '{\"batch/custom/event\":\" EVENT_DESCRIPTION
\"}'"
...
}
]
...
"logsPolicy"
:
{
"destination"
:
"CLOUD_LOGGING"
}
...
For example, a job with custom status events that indicate when an important event happens can have a JSON configuration file similar to the following:
{
"taskGroups"
:
[
{
"taskSpec"
:
{
"runnables"
:
[
{
"script"
:
{
"text"
:
"sleep 30; echo '{\"batch/custom/event\": \" EVENT_DESCRIPTION
\"}'; sleep 30"
}
}
]
},
"taskCount"
:
3
}
],
"logsPolicy"
:
{
"destination"
:
"CLOUD_LOGGING"
}
}
Replace EVENT_DESCRIPTION
with a description for the
custom status event—for example, halfway done
.
After the example job has finished running, the resulting custom status event for each task is similar to the following:
statusEvents:
...
- description: EVENT_DESCRIPTION
eventTime: '...'
type: RUNNABLE_CUSTOM_EVENT
...
What's next
- If you have issues creating or running a job, see Troubleshooting .
- Learn how to view status events .
- Learn how to write task logs .
- Learn about more job creation options .
