Details for managed rubric-based metrics

This page provides a full list of managed rubric-based metrics offered by the Gen AI evaluation service, which you can use in the GenAI Client in Vertex AI SDK.

For more information about test-driven evaluation, see Define your evaluation metrics .

Overview

The Gen AI evaluation service offers a list of managed rubric-based metrics for the test-driven evaluation framework:

• For metrics with adaptive rubrics, most of them include both the workflow for rubric generation for each prompt and rubric validation. You can run them separately if needed. See Run an evaluation for details.

• For metrics with static rubrics, no per-prompt rubrics are generated. For details regarding the intended outputs, see Metric details .

Each managed rubric-based metric has a versioning number. The metric uses the latest version by default, but you can pin to a specific version if needed:

  from 
  
 vertexai 
  
 import 
 types 
 text_quality_metric 
 = 
 types 
 . 
 RubricMetric 
 . 
 TEXT_QUALITY 
 general_quality_v1 
 = 
 types 
 . 
 RubricMetric 
 . 
 GENERAL_QUALITY 
 ( 
 version 
 = 
 'v1' 
 ) 
 

Backward compatibility

For metrics offered as a Metric prompt templates , you can still access the pointwise metrics through the GenAI Client in Vertex AI SDK through the same approach. Pairwise metrics are not supported by the GenAI Client in Vertex AI SDK, but see Run an evaluation to compare two models in the same evaluation.

  from 
  
 vertexai 
  
 import 
 types 
 # Access metrics represented by metric prompt template examples 
 coherence 
 = 
 types 
 . 
 RubricMetric 
 . 
 COHERENCE 
 fluency 
 = 
 types 
 . 
 RubricMetric 
 . 
 FLUENCY 
 

Managed metrics details

This section lists managed metrics with details such as their type, required inputs, and expected output:

• General quality
• Text quality
• Instruction following
• Grounding
• Safety
• Multi-turn general quality
• Multi-turn text quality
• Agent final response match
• Agent final response reference free

General quality

Text quality

Instruction following

Grounding

Safety

Multi-turn general quality

Multi-turn text quality

Agent final response match

Agent final response reference free

What's next

• Prepare your evaluation dataset .