Imports a list of externally generated EvaluatedAnnotations.
Endpoint
posthttps://{service-endpoint}/v1/{parent}:batchImport
Where {service-endpoint}
is one of the supported service endpoints
.
Path parameters
parent
string
Required. The name of the parent ModelEvaluationSlice resource. Format: projects/{project}/locations/{location}/models/{model}/evaluations/{evaluation}/slices/{slice}
Request body
The request body contains data with the following structure:
evaluatedAnnotations[]
object ( EvaluatedAnnotation
)
Required. Evaluated annotations resource to be imported.
Response body
Response message for ModelService.BatchImportEvaluatedAnnotations
If successful, the response body contains data with the following structure:
importedEvaluatedAnnotationsCount
integer
Output only. Number of EvaluatedAnnotations imported.
JSON representation |
---|
{ "importedEvaluatedAnnotationsCount" : integer } |
EvaluatedAnnotation
True positive, false positive, or false negative.
EvaluatedAnnotation is only available under ModelEvaluationSlice with slice of annotationSpec
dimension.
type
enum ( EvaluatedAnnotationType
)
Output only. type of the EvaluatedAnnotation.
predictions[]
value ( Value
format)
Output only. The model predicted annotations.
For true positive, there is one and only one prediction, which matches the only one ground truth annotation in groundTruths
.
For false positive, there is one and only one prediction, which doesn't match any ground truth annotation of the corresponding [data_item_view_id][EvaluatedAnnotation.data_item_view_id].
For false negative, there are zero or more predictions which are similar to the only ground truth annotation in groundTruths
but not enough for a match.
The schema of the prediction is stored in ModelEvaluation.annotation_schema_uri
groundTruths[]
value ( Value
format)
Output only. The ground truth Annotations, i.e. the Annotations that exist in the test data the Model is evaluated on.
For true positive, there is one and only one ground truth annotation, which matches the only prediction in predictions
.
For false positive, there are zero or more ground truth annotations that are similar to the only prediction in predictions
, but not enough for a match.
For false negative, there is one and only one ground truth annotation, which doesn't match any predictions created by the model.
The schema of the ground truth is stored in ModelEvaluation.annotation_schema_uri
dataItemPayload
value ( Value
format)
Output only. The data item payload that the Model predicted this EvaluatedAnnotation on.
evaluatedDataItemViewId
string
Output only. id of the EvaluatedDataItemView under the same ancestor ModelEvaluation. The EvaluatedDataItemView consists of all ground truths and predictions on dataItemPayload
.
explanations[]
object ( EvaluatedAnnotationExplanation
)
Explanations of predictions
. Each element of the explanations indicates the explanation for one explanation method.
The attributions list in the EvaluatedAnnotationExplanation.explanation
object corresponds to the predictions
list. For example, the second element in the attributions list explains the second element in the predictions list.
errorAnalysisAnnotations[]
object ( ErrorAnalysisAnnotation
)
Annotations of model error analysis results.
JSON representation |
---|
{ "type" : enum ( |
EvaluatedAnnotationType
Describes the type of the EvaluatedAnnotation. The type is determined
Enums | |
---|---|
EVALUATED_ANNOTATION_TYPE_UNSPECIFIED
|
Invalid value. |
TRUE_POSITIVE
|
The EvaluatedAnnotation is a true positive. It has a prediction created by the Model and a ground truth Annotation which the prediction matches. |
FALSE_POSITIVE
|
The EvaluatedAnnotation is false positive. It has a prediction created by the Model which does not match any ground truth annotation. |
FALSE_NEGATIVE
|
The EvaluatedAnnotation is false negative. It has a ground truth annotation which is not matched by any of the model created predictions. |
EvaluatedAnnotationExplanation
Explanation result of the prediction produced by the Model.
explanationType
string
Explanation type.
For AutoML Image Classification models, possible values are:
-
image-integrated-gradients
-
image-xrai
explanation
object ( Explanation
)
Explanation attribution response details.
JSON representation |
---|
{
"explanationType"
:
string
,
"explanation"
:
{
object (
|
ErrorAnalysisAnnotation
Model error analysis for each annotation.
attributedItems[]
object ( AttributedItem
)
Attributed items for a given annotation, typically representing neighbors from the training sets constrained by the query type.
queryType
enum ( QueryType
)
The query type used for finding the attributed items.
outlierScore
number
The outlier score of this annotated item. Usually defined as the min of all distances from attributed items.
outlierThreshold
number
The threshold used to determine if this annotation is an outlier or not.
JSON representation |
---|
{ "attributedItems" : [ { object ( |
AttributedItem
Attributed items for a given annotation, typically representing neighbors from the training sets constrained by the query type.
annotationResourceName
string
The unique id for each annotation. Used by FE to allocate the annotation in DB.
distance
number
The distance of this item to the annotation.
JSON representation |
---|
{ "annotationResourceName" : string , "distance" : number } |
QueryType
The query type used for finding the attributed items.
Enums | |
---|---|
QUERY_TYPE_UNSPECIFIED
|
Unspecified query type for model error analysis. |
ALL_SIMILAR
|
Query similar samples across all classes in the dataset. |
SAME_CLASS_SIMILAR
|
Query similar samples from the same class of the input sample. |
SAME_CLASS_DISSIMILAR
|
Query dissimilar samples from the same class of the input sample. |