Method: projects.locations.publishers.models.predict

Perform an online prediction.

Endpoint

post https://{service-endpoint}/v1/{endpoint}:predict

Where {service-endpoint} is one of the supported service endpoints .

Path parameters

endpoint string

Required. The name of the Endpoint requested to serve the prediction. Format: projects/{project}/locations/{location}/endpoints/{endpoint}

Request body

The request body contains data with the following structure:

Fields
instances[] value ( Value format)

Required. The instances that are the input to the prediction call. A DeployedModel may have an upper limit on the number of instances it supports per request, and when it is exceeded the prediction call errors in case of AutoML Models, or, in case of customer created Models, the behaviour is as documented by that Model. The schema of any single instance may be specified via Endpoint's DeployedModels' Model's PredictSchemata's instanceSchemaUri .

parameters value ( Value format)

The parameters that govern the prediction. The schema of the parameters may be specified via Endpoint's DeployedModels' Model's PredictSchemata's parametersSchemaUri .

Example request

Image

Python

  import 
 vertexai 
 from 
 vertexai.vision_models 
 import 
 Image 
 , 
 MultiModalEmbeddingModel 
 # TODO(developer): Update project_id and location 
 vertexai 
 . 
 init 
 ( 
 project 
 = 
 PROJECT_ID 
 , 
 location 
 = 
" us 
 - 
 central1 
" ) 
 model 
 = 
 MultiModalEmbeddingModel 
 . 
 from_pretrained 
 ( 
" multimodalembedding 
" ) 
 image 
 = 
 Image 
 . 
 load_from_file 
 ( 
" gs 
 : 
 // 
 cloud 
 - 
 samples 
 - 
 data 
 / 
 vertex 
 - 
 ai 
 / 
 llm 
 / 
 prompts 
 / 
 landmark1 
 . 
 png 
" ) 
 embeddings 
 = 
 model 
 . 
 get_embeddings 
 ( 
 image 
 = 
 image 
 , 
 contextual_text 
 = 
" Colosseum 
" , 
 dimension 
 = 
 1408 
 , 
 ) 
 print 
 ( 
 f"Image 
 Embedding 
 : 
 { 
 embeddings 
 . 
 image_embedding 
 }") 
 print 
 ( 
 f"Text 
 Embedding 
 : 
 { 
 embeddings 
 . 
 text_embedding 
 }") 
 
  multimodal_embedding_image 
 . 
 py

Video

Python

  import 
 vertexai 
 from 
 vertexai.vision_models 
 import 
 MultiModalEmbeddingModel 
 , 
 Video 
 from 
 vertexai.vision_models 
 import 
 VideoSegmentConfig 
 # TODO(developer): Update project_id and location 
 vertexai 
 . 
 init 
 ( 
 project 
 = 
 PROJECT_ID 
 , 
 location 
 = 
" us 
 - 
 central1 
" ) 
 model 
 = 
 MultiModalEmbeddingModel 
 . 
 from_pretrained 
 ( 
" multimodalembedding 
" ) 
 embeddings 
 = 
 model 
 . 
 get_embeddings 
 ( 
 video 
 = 
 Video 
 . 
 load_from_file 
 ( 
" gs 
 : 
 // 
 cloud 
 - 
 samples 
 - 
 data 
 / 
 vertex 
 - 
 ai 
 - 
 vision 
 / 
 highway_vehicles 
 . 
 mp4 
" ), 
 video_segment_config 
 = 
 VideoSegmentConfig 
 ( 
 end_offset_sec 
 = 
 1 
 ), 
 contextual_text 
 = 
" Cars 
 on 
 Highway 
" , 
 ) 
 # Video Embeddings are segmented based on the video_segment_config. 
 print 
 ( 
" Video 
 Embeddings 
 :") 
 for 
 video_embedding 
 in 
 embeddings 
 . 
 video_embeddings 
 : 
 print 
 ( 
 f"Video 
 Segment 
 : 
 { 
 video_embedding 
 . 
 start_offset_sec 
 } 
 - 
 { 
 video_embedding 
 . 
 end_offset_sec 
 } 
" ) 
 print 
 ( 
 f"Embedding 
 : 
 { 
 video_embedding 
 . 
 embedding 
 }") 
 print 
 ( 
 f"Text 
 Embedding 
 : 
 { 
 embeddings 
 . 
 text_embedding 
 }") 
 
  multimodal_embedding_video 
 . 
 py

Response body

If successful, the response body contains an instance of PredictResponse .