Settings for LLM models.
| JSON representation |
|---|
{
"model"
:
string
,
"promptText"
:
string
,
"parameters"
:
{
object (
|
| Fields | |
|---|---|
model
|
The selected LLM model. |
promptText
|
The custom prompt to use. |
parameters
|
Generative model parameters. |
Parameters
Generative model parameters to control the model behavior.
| JSON representation |
|---|
{ "temperature" : number , "inputTokenLimit" : enum ( |
| Fields | |
|---|---|
temperature
|
The temperature used for sampling during response generation. Value ranges from 0 to 1. Temperature controls the degree of randomness in token selection. Lower temperature means less randomness, while higher temperature means more randomness. Valid range: [0.0, 1.0] |
inputTokenLimit
|
The input token limit. This setting is currently only supported by playbooks. |
outputTokenLimit
|
The output token limit. This setting is currently only supported by playbooks. Only one of outputTokenLimit and max_output_tokens is allowed to be set. |
InputTokenLimit
The input token limits for 1 LLM call. For the limit of each model, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models for more information.
| Enums | |
|---|---|
INPUT_TOKEN_LIMIT_UNSPECIFIED
|
Limit not specified. Treated as 'INPUT_TOKEN_LIMIT_SHORT'. |
INPUT_TOKEN_LIMIT_SHORT
|
Input token limit up to 8k. |
INPUT_TOKEN_LIMIT_MEDIUM
|
Input token limit up to 32k. |
INPUT_TOKEN_LIMIT_LONG
|
Input token limit up to 100k. |
OutputTokenLimit
The output token limits for 1 LLM call. The limits are subject to change. For the limit of each model, see https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models for more information.
| Enums | |
|---|---|
OUTPUT_TOKEN_LIMIT_UNSPECIFIED
|
Limit not specified. |
OUTPUT_TOKEN_LIMIT_SHORT
|
Input token limit up to 512 tokens. |
OUTPUT_TOKEN_LIMIT_MEDIUM
|
Input token limit up to 1k. |
OUTPUT_TOKEN_LIMIT_LONG
|
Input token limit up to 2k. |
