CustomMetric
(
name
:
str
,
metric_function
:
typing
.
Callable
[
[
typing
.
Dict
[
str
,
typing
.
Any
]],
typing
.
Dict
[
str
,
typing
.
Any
]
],
)
The custom evaluation metric.
A fully-customized CustomMetric that can be used to evaluate a single model by defining a metric function for a computation-based metric. The CustomMetric is computed on the client-side using the user-defined metric function in SDK only, not by the Vertex Gen AI Evaluation Service.
Attributes: name: The name of the metric. metric_function: The user-defined evaluation function to compute a metric score. Must use the dataset row dictionary as the metric function input and return per-instance metric result as a dictionary output. The metric score must mapped to the name of the CustomMetric as key.
Methods
CustomMetric
CustomMetric
(
name
:
str
,
metric_function
:
typing
.
Callable
[
[
typing
.
Dict
[
str
,
typing
.
Any
]],
typing
.
Dict
[
str
,
typing
.
Any
]
],
)
Initializes the evaluation metric.
