Generate multimodal embeddings

This page describes how to generate multimodal embeddings using the supported Vertex AI multimodal model, multimodalembedding@001 .

You can use the Vertex AI multimodal embedding models referred to in Supported models .

This page assumes that you're familiar with AlloyDB for PostgreSQL and generative AI concepts . For more information about embeddings, see What are embeddings .

Before you begin

Before you use multimodal embeddings, do the following:

Integrate with Vertex AI and install the extension

Integrate with Vertex AI .

Ensure that the latest version of google_ml_integration is installed.

To check the installed version, run the following command:

  
SELECT  
extversion  
FROM  
pg_extension  
WHERE  
 extname 
  
 = 
  
 'google_ml_integration' 
 ; 
  
extversion  
  
 1 
.4.3  
 ( 
 1 
  
row )

If the extension isn't installed or if the installed version is earlier than 1.4.3, update the extension by running the following commands:
```
  
CREATE  
EXTENSION  
IF  
NOT  
EXISTS  
google_ml_integration ; 
  
ALTER  
EXTENSION  
google_ml_integration  
UPDATE ; 
  
```
If you experience issues when you run the preceding commands, or if the extension isn't updated to version 1.4.3 after you run the preceding commands, contact AlloyDB support.

After you ensure that the version is current, install the preview functionality by running the upgrade_to_preview_version procedure:

  
CALL  
google_ml.upgrade_to_preview_version () 
 ; 
  
SELECT  
extversion  
FROM  
pg_extension  
WHERE  
 extname 
  
 = 
  
 'google_ml_integration' 
 ; 
  
extversion  
  
 1 
.4.4  
 ( 
 1 
  
row )

Access data in Cloud Storage to generate multimodal embeddings

To generate multimodal embeddings, refer to content in Cloud Storage using a gs:// URI.
Access Cloud Storage content through your current project's Vertex AI service agent. By default, the Vertex AI service agent already has permission to access the bucket in the same project. For more information, see IAM roles and permissions index .
To access data in a Cloud Storage bucket in another Google Cloud project, run the following gcloud CLI command to grant the Storage Object Viewer role ( roles/storage.objectViewer ) to the Vertex AI service agent of your AlloyDB project.
```
 gcloud  
projects  
add-iam-policy-binding  
<ANOTHER_PROJECT_ID>  
 \ 
--member = 
 "serviceAccount:service-<PROJECT_ID>@gcp-sa-aiplatform.iam.gserviceaccount.com" 
  
 \ 
--role = 
 "roles/storage.objectViewer" 
 
```
For more information, see Set and manage IAM policies on buckets .

To generate multimodal embeddings, select one of the following schemas.