About supervised fine-tuning for Translation LLM models

Supervised fine-tuning is a good option when you have a translation task with available labeled text data. It's particularly effective for domain-specific applications where the translation significantly differs from the general data the large model was originally trained on.

Supervised fine-tuning adapts model behavior with a labeled dataset. This process adjusts the model's weights to minimize the difference between its predictions and the actual labels.

Supported models

The following Translation LLM models support supervised tuning:

translation-llm-002 (In Public Preview, supports text only)

Limitations

Maximum input and output tokens:
- Serving: 1,000 (~4000 characters)
Validation dataset size: 1024 examples
Training dataset file size: Up to 1GB for JSONL
Training example length: 1,000 (~4000 characters)
Adapter size:
- Translation LLM V2 : Supported value is only 4. Using any other values (e.g., 1 or 8) will result in failure.

Use cases for using supervised fine-tuning

General pretrained translation model works well when the text to be translated is based on general commonplace text structures that the model learned from. If you want a model to learn something niche or domain-specific that deviates from general translation, then you might want to consider tuning that model. For example, you can use model tuning to teach the model the following:

Specific content of an industry domain with jargon or style
Specific structures or formats for generating output.
Specific behaviors such as when to provide a terse or verbose output.
Specific customized outputs for specific types of inputs.

Configure a tuning job region

User data, such as the transformed dataset and the tuned model, is stored in the tuning job region. The only supported region is us-central1 .

If you use the Vertex AI SDK, you can specify the region at initialization. For example:

  import 
  
  vertexai 
 
  vertexai 
 
 . 
 init 
 ( 
 project 
 = 
 'myproject' 
 , 
 location 
 = 
 'us-central1' 
 )

If you create a supervised fine-tuning job by sending a POST request using the tuningJobs.create method, then you use the URL to specify the region where the tuning job runs. For example, in the following URL, you specify a region by replacing both instances of TUNING_JOB_REGION with the region where the job runs.
```
   
https:// TUNING_JOB_REGION 
-aiplatform.googleapis.com/v1/projects/ PROJECT_ID 
/locations/ TUNING_JOB_REGION 
/tuningJobs 
```
If you use the Google Cloud console , you can select the region name in the Regiondrop down field on the Model detailspage. This is the same page where you select the base model and a tuned model name.

Quota

Quota is enforced on the number of concurrent tuning jobs. Every project comes with a default quota to run at least one tuning job. This is a global quota, shared across all available regions and supported models. If you want to run more jobs concurrently, you need to request additional quota for Global concurrent tuning jobs .

Pricing

Supervised fine-tuning for translation-llm-002 is in Preview . While tuning is in Preview, there is no charge to tune a model or to use it for inference.

Training tokens are calculated by the total number of tokens in your training dataset, multiplied by your number of epochs.

What's next

Prepare a supervised fine-tuning dataset .