This page documents AlloyDB Omni version 16-next using the containers deployment option. Choose a different deployment option .

Invoke predictions with model endpoint management

Select a documentation version:

This page describes how to invoke predictions using the model endpoints registered in the Model endpoint management.

Before you begin

Make sure that you have registered your model endpoint with Model endpoint management. For more information, see Register and call remote AI models in AlloyDB Omni .

Invoke predictions for generic models

Use the google_ml.predict_row() SQL function to call a registered generic model endpoint to invoke predictions.

  SELECT 
  
 google_ml 
 . 
 predict_row 
 ( 
  
 model_id 
  
 = 
>  
 & #39;MODE 
L_ID' 
 , 
  
 reques>t_ body 
 
  
 = 
  
 'REQU 
 
EST_BODY' );

Replace the following:

MODEL_ID : the model ID you defined when registering the model endpoint.
REQUEST_BODY : the parameters to the prediction function, in JSON format.

Examples

This section includes some examples for invoking predictions using registered model endpoints.

To generate predictions for a registered gemini-1.5-pro:streamGenerateContent model endpoint, run the following statement:

   
 SELECT 
  
 json_array_elements 
 ( 
  
 google_ml 
 . 
 predict_row 
 ( 
  
 model_id 
  
 = 
>  
 'gemini-1.5-pro:streamGenerateContent' 
 , 
  
 reques>t_body 
  
 = 
  
 '{ "contents": [ { "role": "user", "parts": [ { "text": "For TPCH database schema as mentioned here https://www.tpc.org/TPC_Documents_Current_Versions/pdf/TPC-H_v3.0.1.pdf , generate a SQL query to find all s>upplier names w>hich> are located> in the In>dia >nation.&q 
 
uot; } ] } ] }' )) 
 - 
  
 'candidates' 
  
 - 
  
 0 
  
 - 
  
 'content' 
  
 - 
  
 'parts' 
  
 - 
  
 0 
  
 - 
  
 'text' 
 ;

To generate predictions for a registered facebook/bart-large-mnli model endpoint on Hugging Face, run the following statement:

   
 SELECT 
  
 google_ml 
 . 
 predict_row 
 ( 
  
 model_id 
  
 = 
>  
 'facebook/bart-large-mnli' 
 , 
  
 reques>t_body 
  
 = 
  
 '{ 
 "inputs": "Hi, I recently bought a device from your company but it is not working as advertised and I would like to get reimbursed!", 
 "parameters": {"candidate_la 
 
bels": ["refund", "legal", "faq"]} }' 
  
 );

To generate predictions for a registered Anthropic claude-3-opus-20240229 model endpoint, run the following statement:

   
 SELECT 
  
 google_ml 
 . 
 predict_row 
 ( 
 'anthropic-opus' 
 , 
  
 '{ 
 "model": "claude-3-opus-20240229", 
 "max_tokens": 1024, 
 "messages": [ 
 {&quo 
 
t;role": "user", "content": "Hello, world"} ] 
 }' 
 );

Invoke predictions with model endpoint management Stay organized with collections Save and categorize content based on your preferences.

Before you begin

Invoke predictions for generic models

Examples

Invoke predictions with model endpoint management