Send requests to the Vertex AI API for Gemini

Send requests to the Gemini API in Vertex AI by using a programming language SDK or the REST API to begin building your generative AI application on Google Cloud.

Create a Google Cloud account to get started

In order to try this quickstart, you'll need to create a Google Cloud account. With this account, you'll get $300 in free credit, plus free usage of over 20 products— not charged against your $300 credit.

Get started for free

For more information on getting set up on Google Cloud after your account is created, see Get set up on Google Cloud .

Set up your environment

Click to learn how to set up your environment

Learn how to set up your environment by selecting one of the following tabs:

Python

Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Go to project selector

Enable the Vertex AI and Cloud Storage APIs.

Enable the APIs

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Go to project selector

Enable the Vertex AI and Cloud Storage APIs.

Enable the APIs

In the Google Cloud console, activate Cloud Shell.

Activate Cloud Shell

At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

Note: If you are familiar with Gemini API in Google AI Studio, note that Gemini API for Vertex AI uses Identity and Access Management instead of API keys to manage access.
If you're using a local shell, then create local authentication credentials for your user account:
```
gcloud  
auth  
application-default  
login
```
You don't need to do this if you're using Cloud Shell.
Install or update the Vertex AI SDK for Python by running the following command:
```
pip3  
install  
--upgrade  
 "google-cloud-aiplatform>=1.38" 
  
```

Node.js

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Go to project selector

Enable the Vertex AI and Cloud Storage APIs.

Enable the APIs

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Go to project selector

Enable the Vertex AI and Cloud Storage APIs.

Enable the APIs

In the Google Cloud console, activate Cloud Shell.

Activate Cloud Shell

At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

Note: If you are familiar with Gemini API in Google AI Studio, note that Gemini API for Vertex AI uses Identity and Access Management instead of API keys to manage access.
If you're using a local shell, then create local authentication credentials for your user account:
```
gcloud  
auth  
application-default  
login
```
You don't need to do this if you're using Cloud Shell.
Install or update the Vertex AI SDK for Node.js by running the following command:
```
npm  
install  
@google-cloud/vertexai  
```

Java

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Go to project selector

Enable the Vertex AI and Cloud Storage APIs.

Enable the APIs

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Go to project selector

Enable the Vertex AI and Cloud Storage APIs.

Enable the APIs

In the Google Cloud console, activate Cloud Shell.

Activate Cloud Shell

At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

Note: If you are familiar with Gemini API in Google AI Studio, note that Gemini API for Vertex AI uses Identity and Access Management instead of API keys to manage access.
If you're using a local shell, then create local authentication credentials for your user account:
```
gcloud  
auth  
application-default  
login
```
You don't need to do this if you're using Cloud Shell.

To add google-cloud-vertexai as a dependency, add the appropriate code for your environment:

Maven with BOM

Add the following HTML to your pom.xml :

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>com.google.cloud</groupId>
      <artifactId>libraries-bom</artifactId>
      <version>26.32.0</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>
<dependencies>
  <dependency>
    <groupId>com.google.cloud</groupId>
    <artifactId>google-cloud-vertexai</artifactId>
  </dependency>
</dependencies>

Maven without BOM

Add the following HTML to your pom.xml :

<dependency>
  <groupId>com.google.cloud</groupId>
  <artifactId>google-cloud-vertexai</artifactId>
  <version>0.4.0</version>
</dependency>

Gradle without BOM

Add the following to your build.gradle

implementation  
 'com.google.cloud:google-cloud-vertexai:0.4.0'

Go

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Go to project selector

Enable the Vertex AI and Cloud Storage APIs.

Enable the APIs

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Go to project selector

Enable the Vertex AI and Cloud Storage APIs.

Enable the APIs

In the Google Cloud console, activate Cloud Shell.

Activate Cloud Shell

At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

Note: If you are familiar with Gemini API in Google AI Studio, note that Gemini API for Vertex AI uses Identity and Access Management instead of API keys to manage access.
If you're using a local shell, then create local authentication credentials for your user account:
```
gcloud  
auth  
application-default  
login
```
You don't need to do this if you're using Cloud Shell.
Review the available Vertex AI API Go packages to determine which package best meets your project's needs:
- Package cloud.google.com/go/vertexai ( recommended)
  
  vertexai is a human authored package that provides access to common capabilities and features.
  
  This package is recommended as the starting point for most developers building with the Vertex AI API. To access capabilities and features not yet covered by this package, use the auto-generated aiplatform instead.
- Package cloud.google.com/go/aiplatform
  
  aiplatform is an auto-generated package.
  
  This package is intended for projects that require access to Vertex AI API capabilities and features not yet provided by the human authored vertexai package.

Install the desired Go package based on your project's needs by running one of the following commands:

#  
Human  
authored  
package.  
Recommended  
 for 
  
most  
developers.
go  
get  
cloud.google.com/go/vertexai  

#  
Auto-generated  
package.
go  
get  
cloud.google.com/go/aiplatform

C#

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Go to project selector

Enable the Vertex AI and Cloud Storage APIs.

Enable the APIs

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Go to project selector

Enable the Vertex AI and Cloud Storage APIs.

Enable the APIs

In the Google Cloud console, activate Cloud Shell.

Activate Cloud Shell

At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

Note: If you are familiar with Gemini API in Google AI Studio, note that Gemini API for Vertex AI uses Identity and Access Management instead of API keys to manage access.
If you're using a local shell, then create local authentication credentials for your user account:
```
gcloud  
auth  
application-default  
login
```
You don't need to do this if you're using Cloud Shell.

REST

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Go to project selector

Enable the Vertex AI and Cloud Storage APIs.

Enable the APIs

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Go to project selector

Enable the Vertex AI and Cloud Storage APIs.

Enable the APIs

In the Google Cloud console, activate Cloud Shell.

Activate Cloud Shell

At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.

Note: If you are familiar with Gemini API in Google AI Studio, note that Gemini API for Vertex AI uses Identity and Access Management instead of API keys to manage access.
Configure environment variables by entering the following. Replace PROJECT_ID with the ID of your Google Cloud project.
```
 MODEL_ID 
 = 
 "gemini-1.5-flash-001" 
 PROJECT_ID 
 = 
 " PROJECT_ID 
" 
  
```

Provision the endpoint:

gcloud  
beta  
services  
identity  
create  
--service = 
aiplatform.googleapis.com  
--project = 
 ${ 
 PROJECT_ID 
 }

Optional: If you are using Cloud Shell and you are asked to authorize Cloud Shell, click Authorize .

Send a text-only request

After you set up your local environment , send a text-only request to the Vertex AI Gemini API. The following sample returns a list of possible names for a specialty flower store.

Python

To send a prompt request, create a Python file ( .py ) and copy the following code into the file. Set the value of PROJECT_ID to the ID of your Google Cloud project. Then, run the Python file from the command line or in an IDE, or embed the code into your application where appropriate.

  import 
 vertexai 
 from 
 vertexai.generative_models 
 import 
 GenerativeModel 
 # TODO(developer): Update & uncomment line below 
 # PROJECT_ID = "your-project-id 
" vertexai 
 . 
 init 
 ( 
 project 
 = 
 PROJECT_ID 
 , 
 location 
 = 
" us 
 - 
 central1 
" ) 
 model 
 = 
 GenerativeModel 
 ( 
" gemini 
 - 
 1.5 
 - 
 flash 
 - 
 001 
" ) 
 response 
 = 
 model 
 . 
 generate_content 
 ( 
" What's 
 a 
 good 
 name 
 for 
 a 
 flower 
 shop 
 that 
 specializes 
 in 
 selling 
 bouquets 
 of 
 dried 
 flowers 
 ? 
" ) 
 print 
 ( 
 response 
 . 
 text 
 )

Node.js

To send a prompt request, create a Node.js file ( .js ) and copy the following code into the file. Replace PROJECT_ID with the ID of your Google Cloud project. After, run the Javascript file from the command line or in an IDE, or embed the code into your application where appropriate.

  const 
  
 { 
 VertexAI 
 } 
  
 = 
  
 require 
 ( 
' @ 
 google 
 - 
 cloud 
 / 
 vertexai 
' ); 
 /** 
 * TODO(developer): Update these variables before running the sample. 
 */ 
 async 
  
 function 
  
 generate_from_text_input 
 ( 
 projectId 
  
 = 
  
'  PROJECT_ID 
 
' ) 
  
 { 
  
 const 
  
 vertexAI 
  
 = 
  
 new 
  
 VertexAI 
 ({ 
 project 
 : 
  
 projectId 
 , 
  
 location 
 : 
  
' us 
 - 
 central1 
' }); 
  
 const 
  
 generativeModel 
  
 = 
  
 vertexAI 
 . 
 getGenerativeModel 
 ({ 
  
 model 
 : 
  
' gemini 
 - 
 1.5 
 - 
 flash 
 - 
 001 
' , 
  
 }); 
  
 const 
  
 prompt 
  
 = 
  
" What's 
  
 a 
  
 good 
  
 name 
  
 for 
  
 a 
  
 flower 
  
 shop 
  
 that 
  
 specializes 
  
 in 
  
 selling 
  
 bouquets 
  
 of 
  
 dried 
  
 flowers 
 ? 
" ; 
  
 const 
  
 resp 
  
 = 
  
 await 
  
 generativeModel 
 . 
 generateContent 
 ( 
 prompt 
 ); 
  
 const 
  
 contentResponse 
  
 = 
  
 await 
  
 resp 
 . 
 response 
 ; 
  
 console 
 . 
 log 
 ( 
 JSON 
 . 
 stringify 
 ( 
 contentResponse 
 )); 
 }

Java

To send a prompt request, create a Java file ( .java ) and copy the following code into the file. Set your-google-cloud-project-id to your Google Cloud project ID. After, run the Java file from the command line or in an IDE, or embed the code into your application where appropriate.

  import 
  
 com.google.cloud.vertexai.VertexAI 
 ; 
 import 
  
 com.google.cloud.vertexai.api.GenerateContentResponse 
 ; 
 import 
  
 com.google.cloud.vertexai.generativeai.GenerativeModel 
 ; 
 import 
  
 com.google.cloud.vertexai.generativeai.ResponseHandler 
 ; 
 import 
  
 java.io.IOException 
 ; 
 public 
  
 class 
 TextInput 
  
 { 
  
 public 
  
 static 
  
 void 
  
 main 
 ( 
 String 
 [] 
  
 args 
 ) 
  
 throws 
  
 IOException 
  
 { 
  
 // TODO(developer): Replace these variables before running the sample. 
  
 String 
  
 projectId 
  
 = 
  
" your 
 - 
 google 
 - 
 cloud 
 - 
 project 
 - 
 id 
" ; 
  
 String 
  
 location 
  
 = 
  
" us 
 - 
 central1 
" ; 
  
 String 
  
 modelName 
  
 = 
  
" gemini 
 - 
 1.5 
 - 
 flash 
 - 
 001 
" ; 
  
 String 
  
 textPrompt 
  
 = 
  
" What's 
  
 a 
  
 good 
  
 name 
  
 for 
  
 a 
  
 flower 
  
 shop 
  
 that 
  
 specializes 
  
 in 
  
 selling 
  
 bouquets 
  
 of 
"  
 + 
 " 
 dried 
  
 flowers 
 ? 
" ; 
  
 String 
  
 output 
  
 = 
  
 textInput 
 ( 
 projectId 
 , 
  
 location 
 , 
  
 modelName 
 , 
  
 textPrompt 
 ); 
  
 System 
 . 
 out 
 . 
 println 
 ( 
 output 
 ); 
  
 } 
  
 // Passes the provided text input to the Gemini model and returns the text-only response. 
  
 // For the specified textPrompt, the model returns a list of possible store names. 
  
 public 
  
 static 
  
 String 
  
 textInput 
 ( 
  
 String 
  
 projectId 
 , 
  
 String 
  
 location 
 , 
  
 String 
  
 modelName 
 , 
  
 String 
  
 textPrompt 
 ) 
  
 throws 
  
 IOException 
  
 { 
  
 // Initialize client that will be used to send requests. This client only needs 
  
 // to be created once, and can be reused for multiple requests. 
  
 try 
  
 ( 
 VertexAI 
  
 vertexAI 
  
 = 
  
 new 
  
 VertexAI 
 ( 
 projectId 
 , 
  
 location 
 )) 
  
 { 
  
 GenerativeModel 
  
 model 
  
 = 
  
 new 
  
 GenerativeModel 
 ( 
 modelName 
 , 
  
 vertexAI 
 ); 
  
 GenerateContentResponse 
  
 response 
  
 = 
  
 model 
 . 
 generateContent 
 ( 
 textPrompt 
 ); 
  
 String 
  
 output 
  
 = 
  
 ResponseHandler 
 . 
 getText 
 ( 
 response 
 ); 
  
 return 
  
 output 
 ; 
  
 } 
  
 } 
 }

Go

To send a prompt request, create a Go file ( .go ) and copy the following code into the file. Replace projectID with the ID of your Google Cloud project. After, run the Go file from the command line or in an IDE, or embed the code into your application where appropriate.

  import 
  
 ( 
  
" context 
"  
" encoding 
 / 
 json 
"  
" fmt 
"  
" io 
"  
" cloud 
 . 
 google 
 . 
 com 
 / 
 go 
 / 
 vertexai 
 / 
 genai 
" ) 
 func 
  
 generateContentFromText 
 ( 
 w 
  
 io 
 . 
 Writer 
 , 
  
 projectID 
  
 string 
 ) 
  
 error 
  
 { 
  
 location 
  
 := 
  
" us 
 - 
 central1 
"  
 modelName 
  
 := 
  
" gemini 
 - 
 1.5 
 - 
 flash 
 - 
 001 
"  
 ctx 
  
 := 
  
 context 
 . 
 Background 
 () 
  
 client 
 , 
  
 err 
  
 := 
  
 genai 
 . 
 NewClient 
 ( 
 ctx 
 , 
  
 projectID 
 , 
  
 location 
 ) 
  
 if 
  
 err 
  
 != 
  
 nil 
  
 { 
  
 return 
  
 fmt 
 . 
 Errorf 
 ( 
" error 
  
 creating 
  
 client 
 : 
  
 % 
 w 
" , 
  
 err 
 ) 
  
 } 
  
 gemini 
  
 := 
  
 client 
 . 
 GenerativeModel 
 ( 
 modelName 
 ) 
  
 prompt 
  
 := 
  
 genai 
 . 
 Text 
 ( 
  
" What's 
  
 a 
  
 good 
  
 name 
  
 for 
  
 a 
  
 flower 
  
 shop 
  
 that 
  
 specializes 
  
 in 
  
 selling 
  
 bouquets 
  
 of 
  
 dried 
  
 flowers 
 ? 
" ) 
  
 resp 
 , 
  
 err 
  
 := 
  
 gemini 
 . 
 GenerateContent 
 ( 
 ctx 
 , 
  
 prompt 
 ) 
  
 if 
  
 err 
  
 != 
  
 nil 
  
 { 
  
 return 
  
 fmt 
 . 
 Errorf 
 ( 
" error 
  
 generating 
  
 content 
 : 
  
 % 
 w 
" , 
  
 err 
 ) 
  
 } 
  
 // See the JSON response in 
  
 // https://pkg.go.dev/cloud.google.com/go/vertexai/genai#GenerateContentResponse. 
  
 rb 
 , 
  
 err 
  
 := 
  
 json 
 . 
 MarshalIndent 
 ( 
 resp 
 , 
  
"" , 
 " 
" ) 
  
 if 
  
 err 
  
 != 
  
 nil 
  
 { 
  
 return 
  
 fmt 
 . 
 Errorf 
 ( 
" json 
 . 
 MarshalIndent 
 : 
  
 % 
 w 
" , 
  
 err 
 ) 
  
 } 
  
 fmt 
 . 
 Fprintln 
 ( 
 w 
 , 
  
 string 
 ( 
 rb 
 )) 
  
 return 
  
 nil 
 }

C#

To send a prompt request, create a C# file ( .cs ) and copy the following code into the file. Set your-project-id to your Google Cloud project ID. After, run the C# file from the command line or in an IDE, or embed the code into your application where appropriate.

  using 
  
 Google.Cloud.AIPlatform.V1 
 ; 
 using 
  
 System 
 ; 
 using 
  
 System.Threading.Tasks 
 ; 
 public 
  
 class 
  
 TextInputSample 
 { 
  
 public 
  
 async 
  
 Task<string> 
  
 TextInput 
 ( 
  
 string 
  
 projectId 
  
 = 
  
" your 
 - 
 project 
 - 
 id 
" , 
  
 string 
  
 location 
  
 = 
  
" us 
 - 
 central1 
" , 
  
 string 
  
 publisher 
  
 = 
  
" google 
" , 
  
 string 
  
 model 
  
 = 
  
" gemini 
 - 
 1.5 
 - 
 flash 
 - 
 001 
" ) 
  
 { 
  
 var 
  
 predictionServiceClient 
  
 = 
  
 new 
  
 PredictionServiceClientBuilder 
  
 { 
  
 Endpoint 
  
 = 
  
 $ 
" { 
 location 
 } 
 - 
 aiplatform 
 . 
 googleapis 
 . 
 com 
"  
 }. 
 Build 
 (); 
  
 string 
  
 prompt 
  
 = 
  
 @"What's 
  
 a 
  
 good 
  
 name 
  
 for 
  
 a 
  
 flower 
  
 shop 
  
 that 
  
 specializes 
  
 in 
  
 selling 
  
 bouquets 
  
 of 
  
 dried 
  
 flowers 
 ? 
" ; 
  
 var 
  
 generateContentRequest 
  
 = 
  
 new 
  
 GenerateContentRequest 
  
 { 
  
 Model 
  
 = 
  
 $ 
" projects 
 / 
 { 
 projectId 
 } 
 / 
 locations 
 / 
 { 
 location 
 } 
 / 
 publishers 
 / 
 { 
 publisher 
 } 
 / 
 models 
 / 
 { 
 model 
 }", 
  
 Contents 
  
 = 
  
 { 
  
 new 
  
 Content 
  
 { 
  
 Role 
  
 = 
  
" USER 
" , 
  
 Parts 
  
 = 
  
 { 
  
 new 
  
 Part 
  
 { 
  
 Text 
  
 = 
  
 prompt 
  
 } 
  
 } 
  
 } 
  
 } 
  
 }; 
  
 GenerateContentResponse 
  
 response 
  
 = 
  
 await 
  
 predictionServiceClient 
 . 
 GenerateContentAsync 
 ( 
 generateContentRequest 
 ); 
  
 string 
  
 responseText 
  
 = 
  
 response 
 . 
 Candidates 
 [ 
 0 
 ]. 
 Content 
 . 
 Parts 
 [ 
 0 
 ]. 
 Text 
 ; 
  
 Console 
 . 
 WriteLine 
 ( 
 responseText 
 ); 
  
 return 
  
 responseText 
 ; 
  
 } 
 }

REST

You can send this prompt request from Cloud Shell, the command line, or in an IDE, or you can embed the REST call into your application where appropriate.

curl  
-X  
POST  
 \ 
-H  
"Authorization:  
Bearer  
 $( 
gcloud  
auth  
print-access-token ) 
"  
 \ 
-H  
"Content-Type:  
application/json"  
 \ 
https://us-central1-aiplatform.googleapis.com/v1/projects/ ${ 
 PROJECT_ID 
 } 
/locations/us-central1/publishers/google/models/ ${ 
 MODEL_ID 
 } 
:streamGenerateContent  
-d  
 \ 
 $'{ 
 "contents": { 
 "role": "user", 
 "parts": [ 
 { 
 "text": "What 
 \'s 
 a good name for a flower shop that specializes in selling bouquets of dried flowers? 
"  
 } 
  
 ] 
  
 } 
 } 
'

The model returns a response. Note that the response is generated in sections with each section separately evaluated for safety.

Send a request that includes an image

After you set up your local environment , send a request that includes text and an image to the Vertex AI Gemini API. The following sample returns a description of the provided image ( image for Java sample ).

Python

  import 
 vertexai 
 from 
 vertexai.generative_models 
 import 
 GenerativeModel 
 , 
 Part 
 # TODO(developer): Update & uncomment line below 
 # PROJECT_ID = "your-project-id 
" vertexai 
 . 
 init 
 ( 
 project 
 = 
 PROJECT_ID 
 , 
 location 
 = 
" us 
 - 
 central1 
" ) 
 model 
 = 
 GenerativeModel 
 ( 
" gemini 
 - 
 1.5 
 - 
 flash 
 - 
 001 
" ) 
 response 
 = 
 model 
 . 
 generate_content 
 ( 
 [ 
 Part 
 . 
 from_uri 
 ( 
" gs 
 : 
 // 
 cloud 
 - 
 samples 
 - 
 data 
 / 
 generative 
 - 
 ai 
 / 
 image 
 / 
 scones 
 . 
 jpg 
" , 
 mime_type 
 = 
" image 
 / 
 jpeg 
" , 
 ), 
" What 
 is 
 shown 
 in 
 this 
 image 
 ? 
" , 
 ] 
 ) 
 print 
 ( 
 response 
 . 
 text 
 )

Node.js

  const 
  
 { 
 VertexAI 
 } 
  
 = 
  
 require 
 ( 
' @ 
 google 
 - 
 cloud 
 / 
 vertexai 
' ); 
 /** 
 * TODO(developer): Update these variables before running the sample. 
 */ 
 async 
  
 function 
  
 createNonStreamingMultipartContent 
 ( 
  
 projectId 
  
 = 
  
'  PROJECT_ID 
 
' , 
  
 location 
  
 = 
  
' us 
 - 
 central1 
' , 
  
 model 
  
 = 
  
' gemini 
 - 
 1.5 
 - 
 flash 
 - 
 001 
' , 
  
 image 
  
 = 
  
' gs 
 : 
 //generativeai-downloads/images/scones.jpg', 
  
 mimeType 
  
 = 
  
' image 
 / 
 jpeg 
' ) 
  
 { 
  
 // Initialize Vertex with your Cloud project and location 
  
 const 
  
 vertexAI 
  
 = 
  
 new 
  
 VertexAI 
 ({ 
 project 
 : 
  
 projectId 
 , 
  
 location 
 : 
  
 location 
 }); 
  
 // Instantiate the model 
  
 const 
  
 generativeVisionModel 
  
 = 
  
 vertexAI 
 . 
 getGenerativeModel 
 ({ 
  
 model 
 : 
  
 model 
 , 
  
 }); 
  
 // For images, the SDK supports both Google Cloud Storage URI and base64 strings 
  
 const 
  
 filePart 
  
 = 
  
 { 
  
 fileData 
 : 
  
 { 
  
 fileUri 
 : 
  
 image 
 , 
  
 mimeType 
 : 
  
 mimeType 
 , 
  
 }, 
  
 }; 
  
 const 
  
 textPart 
  
 = 
  
 { 
  
 text 
 : 
  
' what 
  
 is 
  
 shown 
  
 in 
  
 this 
  
 image 
 ? 
' , 
  
 }; 
  
 const 
  
 request 
  
 = 
  
 { 
  
 contents 
 : 
  
 [{ 
 role 
 : 
  
' user 
' , 
  
 parts 
 : 
  
 [ 
 filePart 
 , 
  
 textPart 
 ]}], 
  
 }; 
  
 console 
 . 
 log 
 ( 
' Prompt 
  
 Text 
 : 
' ); 
  
 console 
 . 
 log 
 ( 
 request 
 . 
 contents 
 [ 
 0 
 ]. 
 parts 
 [ 
 1 
 ]. 
 text 
 ); 
  
 console 
 . 
 log 
 ( 
' Non 
 - 
 Streaming 
  
 Response 
  
 Text 
 : 
' ); 
  
 // Generate a response 
  
 const 
  
 response 
  
 = 
  
 await 
  
 generativeVisionModel 
 . 
 generateContent 
 ( 
 request 
 ); 
  
 // Select the text from the response 
  
 const 
  
 fullTextResponse 
  
 = 
  
 response 
 . 
 response 
 . 
 candidates 
 [ 
 0 
 ]. 
 content 
 . 
 parts 
 [ 
 0 
 ]. 
 text 
 ; 
  
 console 
 . 
 log 
 ( 
 fullTextResponse 
 ); 
 }

Java

  import 
  
 com.google.cloud.vertexai.VertexAI 
 ; 
 import 
  
 com.google.cloud.vertexai.api.GenerateContentResponse 
 ; 
 import 
  
 com.google.cloud.vertexai.generativeai.ContentMaker 
 ; 
 import 
  
 com.google.cloud.vertexai.generativeai.GenerativeModel 
 ; 
 import 
  
 com.google.cloud.vertexai.generativeai.PartMaker 
 ; 
 import 
  
 java.io.IOException 
 ; 
 public 
  
 class 
 Quickstart 
  
 { 
  
 public 
  
 static 
  
 void 
  
 main 
 ( 
 String 
 [] 
  
 args 
 ) 
  
 throws 
  
 IOException 
  
 { 
  
 // TODO(developer): Replace these variables before running the sample. 
  
 String 
  
 projectId 
  
 = 
  
" your 
 - 
 google 
 - 
 cloud 
 - 
 project 
 - 
 id 
" ; 
  
 String 
  
 location 
  
 = 
  
" us 
 - 
 central1 
" ; 
  
 String 
  
 modelName 
  
 = 
  
" gemini 
 - 
 1.5 
 - 
 flash 
 - 
 001 
" ; 
  
 String 
  
 output 
  
 = 
  
 quickstart 
 ( 
 projectId 
 , 
  
 location 
 , 
  
 modelName 
 ); 
  
 System 
 . 
 out 
 . 
 println 
 ( 
 output 
 ); 
  
 } 
  
 // Analyzes the provided Multimodal input. 
  
 public 
  
 static 
  
 String 
  
 quickstart 
 ( 
 String 
  
 projectId 
 , 
  
 String 
  
 location 
 , 
  
 String 
  
 modelName 
 ) 
  
 throws 
  
 IOException 
  
 { 
  
 // Initialize client that will be used to send requests. This client only needs 
  
 // to be created once, and can be reused for multiple requests. 
  
 try 
  
 ( 
 VertexAI 
  
 vertexAI 
  
 = 
  
 new 
  
 VertexAI 
 ( 
 projectId 
 , 
  
 location 
 )) 
  
 { 
  
 String 
  
 imageUri 
  
 = 
  
" gs 
 : 
 //generativeai-downloads/images/scones.jpg"; 
  
 GenerativeModel 
  
 model 
  
 = 
  
 new 
  
 GenerativeModel 
 ( 
 modelName 
 , 
  
 vertexAI 
 ); 
  
 GenerateContentResponse 
  
 response 
  
 = 
  
 model 
 . 
 generateContent 
 ( 
 ContentMaker 
 . 
 fromMultiModalData 
 ( 
  
 PartMaker 
 . 
 fromMimeTypeAndData 
 ( 
" image 
 / 
 png 
" , 
  
 imageUri 
 ), 
  
" What's 
  
 in 
  
 this 
  
 photo 
"  
 )); 
  
 return 
  
 response 
 . 
 toString 
 (); 
  
 } 
  
 } 
 }

Go

  import 
  
 ( 
  
" context 
"  
" encoding 
 / 
 json 
"  
" fmt 
"  
" io 
"  
" cloud 
 . 
 google 
 . 
 com 
 / 
 go 
 / 
 vertexai 
 / 
 genai 
" ) 
 func 
  
 tryGemini 
 ( 
 w 
  
 io 
 . 
 Writer 
 , 
  
 projectID 
  
 string 
 , 
  
 location 
  
 string 
 , 
  
 modelName 
  
 string 
 ) 
  
 error 
  
 { 
  
 // location := "us-central1 
"  
 // modelName := "gemini-1.5-flash-001 
"  
 ctx 
  
 := 
  
 context 
 . 
 Background 
 () 
  
 client 
 , 
  
 err 
  
 := 
  
 genai 
 . 
 NewClient 
 ( 
 ctx 
 , 
  
 projectID 
 , 
  
 location 
 ) 
  
 if 
  
 err 
  
 != 
  
 nil 
  
 { 
  
 return 
  
 fmt 
 . 
 Errorf 
 ( 
" error 
  
 creating 
  
 client 
 : 
  
 % 
 w 
" , 
  
 err 
 ) 
  
 } 
  
 gemini 
  
 := 
  
 client 
 . 
 GenerativeModel 
 ( 
 modelName 
 ) 
  
 img 
  
 := 
  
 genai 
 . 
 FileData 
 { 
  
 MIMEType 
 : 
  
" image 
 / 
 jpeg 
" , 
  
 FileURI 
 : 
  
" gs 
 : 
 //generativeai-downloads/images/scones.jpg", 
  
 } 
  
 prompt 
  
 := 
  
 genai 
 . 
 Text 
 ( 
" What 
  
 is 
  
 in 
  
 this 
  
 image 
 ? 
" ) 
  
 resp 
 , 
  
 err 
  
 := 
  
 gemini 
 . 
 GenerateContent 
 ( 
 ctx 
 , 
  
 img 
 , 
  
 prompt 
 ) 
  
 if 
  
 err 
  
 != 
  
 nil 
  
 { 
  
 return 
  
 fmt 
 . 
 Errorf 
 ( 
" error 
  
 generating 
  
 content 
 : 
  
 % 
 w 
" , 
  
 err 
 ) 
  
 } 
  
 rb 
 , 
  
 err 
  
 := 
  
 json 
 . 
 MarshalIndent 
 ( 
 resp 
 , 
  
"" , 
 " 
" ) 
  
 if 
  
 err 
  
 != 
  
 nil 
  
 { 
  
 return 
  
 fmt 
 . 
 Errorf 
 ( 
" json 
 . 
 MarshalIndent 
 : 
  
 % 
 w 
" , 
  
 err 
 ) 
  
 } 
  
 fmt 
 . 
 Fprintln 
 ( 
 w 
 , 
  
 string 
 ( 
 rb 
 )) 
  
 return 
  
 nil 
 }

C#

  using 
  
 Google.Api.Gax.Grpc 
 ; 
 using 
  
 Google.Cloud.AIPlatform.V1 
 ; 
 using 
  
 System.Text 
 ; 
 using 
  
 System.Threading.Tasks 
 ; 
 public 
  
 class 
  
 GeminiQuickstart 
 { 
  
 public 
  
 async 
  
 Task<string> 
  
 GenerateContent 
 ( 
  
 string 
  
 projectId 
  
 = 
  
" your 
 - 
 project 
 - 
 id 
" , 
  
 string 
  
 location 
  
 = 
  
" us 
 - 
 central1 
" , 
  
 string 
  
 publisher 
  
 = 
  
" google 
" , 
  
 string 
  
 model 
  
 = 
  
" gemini 
 - 
 1.5 
 - 
 flash 
 - 
 001 
"  
 ) 
  
 { 
  
 // Create client 
  
 var 
  
 predictionServiceClient 
  
 = 
  
 new 
  
 PredictionServiceClientBuilder 
  
 { 
  
 Endpoint 
  
 = 
  
 $ 
" { 
 location 
 } 
 - 
 aiplatform 
 . 
 googleapis 
 . 
 com 
"  
 }. 
 Build 
 (); 
  
 // Initialize content request 
  
 var 
  
 generateContentRequest 
  
 = 
  
 new 
  
 GenerateContentRequest 
  
 { 
  
 Model 
  
 = 
  
 $ 
" projects 
 / 
 { 
 projectId 
 } 
 / 
 locations 
 / 
 { 
 location 
 } 
 / 
 publishers 
 / 
 { 
 publisher 
 } 
 / 
 models 
 / 
 { 
 model 
 }", 
  
 GenerationConfig 
  
 = 
  
 new 
  
 GenerationConfig 
  
 { 
  
 Temperature 
  
 = 
  
 0.4f 
 , 
  
 TopP 
  
 = 
  
 1 
 , 
  
 TopK 
  
 = 
  
 32 
 , 
  
 MaxOutputTokens 
  
 = 
  
 2048 
  
 }, 
  
 Contents 
  
 = 
  
 { 
  
 new 
  
 Content 
  
 { 
  
 Role 
  
 = 
  
" USER 
" , 
  
 Parts 
  
 = 
  
 { 
  
 new 
  
 Part 
  
 { 
  
 Text 
  
 = 
  
" What's 
  
 in 
  
 this 
  
 photo 
 ? 
"  
 }, 
  
 new 
  
 Part 
  
 { 
  
 FileData 
  
 = 
  
 new 
 () 
  
 { 
  
 MimeType 
  
 = 
  
" image 
 / 
 png 
" , 
  
 FileUri 
  
 = 
  
" gs 
 : 
 //generativeai-downloads/images/scones.jpg" } } 
  
 } 
  
 } 
  
 } 
  
 }; 
  
 // Make the request, returning a streaming response 
  
 using 
  
 PredictionServiceClient.StreamGenerateContentStream 
  
 response 
  
 = 
  
 predictionServiceClient 
 . 
 StreamGenerateContent 
 ( 
 generateContentRequest 
 ); 
  
 StringBuilder 
  
 fullText 
  
 = 
  
 new 
 (); 
  
 // Read streaming responses from server until complete 
  
 AsyncResponseStream<GenerateContentResponse> 
  
 responseStream 
  
 = 
  
 response 
 . 
 GetResponseStream 
 (); 
  
 await 
  
 foreach 
  
 ( 
 GenerateContentResponse 
  
 responseItem 
  
 in 
  
 responseStream 
 ) 
  
 { 
  
 fullText 
 . 
 Append 
 ( 
 responseItem 
 . 
 Candidates 
 [ 
 0 
 ]. 
 Content 
 . 
 Parts 
 [ 
 0 
 ]. 
 Text 
 ); 
  
 } 
  
 return 
  
 fullText 
 . 
 ToString 
 (); 
  
 } 
 }

REST

You can send this prompt request from Cloud Shell, the command line, or in an IDE, or you can embed the REST call into your application where appropriate.

curl  
-X  
POST  
 \ 
-H  
"Authorization:  
Bearer  
 $( 
gcloud  
auth  
print-access-token ) 
"  
 \ 
-H  
"Content-Type:  
application/json"  
 \ 
https://us-central1-aiplatform.googleapis.com/v1/projects/ ${ 
 PROJECT_ID 
 } 
/locations/us-central1/publishers/google/models/ ${ 
 MODEL_ID 
 } 
:streamGenerateContent  
-d  
 \ 
 $'{ 
 "contents": { 
 "role": "user", 
 "parts": [ 
 { 
 "fileData": { 
 "mimeType": "image/jpeg", 
 "fileUri": "gs://generativeai-downloads/images/scones.jpg 
"  
 } 
  
 } 
,  
 { 
  
"text":  
"Describe  
this  
picture."  
 } 
  
 ] 
  
 } 
 } 
'

The model returns a response. Note that the response is generated in sections with each section separately evaluated for safety.

Send a request that includes audio and video

After you set up your local environment , send a request that includes text, audio, and video to the Vertex AI Gemini API. The following sample returns a description of the provided video , including anything important from the audio track.

Python

  import 
 vertexai 
 from 
 vertexai.generative_models 
 import 
 GenerativeModel 
 , 
 Part 
 # TODO(developer): Update & uncomment line below 
 # PROJECT_ID = "your-project-id 
" vertexai 
 . 
 init 
 ( 
 project 
 = 
 PROJECT_ID 
 , 
 location 
 = 
" us 
 - 
 central1 
" ) 
 model 
 = 
 GenerativeModel 
 ( 
" gemini 
 - 
 1.5 
 - 
 flash 
 - 
 001 
" ) 
 prompt 
 = 
""" Provide 
 a 
 description 
 of 
 the 
 video 
 . 
 The 
 description 
 should 
 also 
 contain 
 anything 
 important 
 which 
 people 
 say 
 in 
 the 
 video 
 . 
""" video_file 
 = 
 Part 
 . 
 from_uri 
 ( 
 uri 
 = 
" gs 
 : 
 // 
 cloud 
 - 
 samples 
 - 
 data 
 / 
 generative 
 - 
 ai 
 / 
 video 
 / 
 pixel8 
 . 
 mp4 
" , 
 mime_type 
 = 
" video 
 / 
 mp4 
" , 
 ) 
 contents 
 = 
 [ 
 video_file 
 , 
 prompt 
 ] 
 response 
 = 
 model 
 . 
 generate_content 
 ( 
 contents 
 ) 
 print 
 ( 
 response 
 . 
 text 
 )

Node.js

  const 
  
 { 
 VertexAI 
 } 
  
 = 
  
 require 
 ( 
' @ 
 google 
 - 
 cloud 
 / 
 vertexai 
' ); 
 /** 
 * TODO(developer): Update these variables before running the sample. 
 */ 
 async 
  
 function 
  
 analyze_video_with_audio 
 ( 
 projectId 
  
 = 
  
'  PROJECT_ID 
 
' ) 
  
 { 
  
 const 
  
 vertexAI 
  
 = 
  
 new 
  
 VertexAI 
 ({ 
 project 
 : 
  
 projectId 
 , 
  
 location 
 : 
  
' us 
 - 
 central1 
' }); 
  
 const 
  
 generativeModel 
  
 = 
  
 vertexAI 
 . 
 getGenerativeModel 
 ({ 
  
 model 
 : 
  
' gemini 
 - 
 1.5 
 - 
 flash 
 - 
 001 
' , 
  
 }); 
  
 const 
  
 filePart 
  
 = 
  
 { 
  
 file_data 
 : 
  
 { 
  
 file_uri 
 : 
  
' gs 
 : 
 //cloud-samples-data/generative-ai/video/pixel8.mp4', 
  
 mime_type 
 : 
  
' video 
 / 
 mp4 
' , 
  
 }, 
  
 }; 
  
 const 
  
 textPart 
  
 = 
  
 { 
  
 text 
 : 
  
 ` 
 Provide a description of the video. 
 The description should also contain anything important which people say in the video.` 
 , 
  
 }; 
  
 const 
  
 request 
  
 = 
  
 { 
  
 contents 
 : 
  
 [{ 
 role 
 : 
  
' user 
' , 
  
 parts 
 : 
  
 [ 
 filePart 
 , 
  
 textPart 
 ]}], 
  
 }; 
  
 const 
  
 resp 
  
 = 
  
 await 
  
 generativeModel 
 . 
 generateContent 
 ( 
 request 
 ); 
  
 const 
  
 contentResponse 
  
 = 
  
 await 
  
 resp 
 . 
 response 
 ; 
  
 console 
 . 
 log 
 ( 
 JSON 
 . 
 stringify 
 ( 
 contentResponse 
 )); 
 }

Java

  import 
  
 com.google.cloud.vertexai.VertexAI 
 ; 
 import 
  
 com.google.cloud.vertexai.api.GenerateContentResponse 
 ; 
 import 
  
 com.google.cloud.vertexai.generativeai.ContentMaker 
 ; 
 import 
  
 com.google.cloud.vertexai.generativeai.GenerativeModel 
 ; 
 import 
  
 com.google.cloud.vertexai.generativeai.PartMaker 
 ; 
 import 
  
 com.google.cloud.vertexai.generativeai.ResponseHandler 
 ; 
 import 
  
 java.io.IOException 
 ; 
 public 
  
 class 
 VideoInputWithAudio 
  
 { 
  
 public 
  
 static 
  
 void 
  
 main 
 ( 
 String 
 [] 
  
 args 
 ) 
  
 throws 
  
 IOException 
  
 { 
  
 // TODO(developer): Replace these variables before running the sample. 
  
 String 
  
 projectId 
  
 = 
  
" your 
 - 
 google 
 - 
 cloud 
 - 
 project 
 - 
 id 
" ; 
  
 String 
  
 location 
  
 = 
  
" us 
 - 
 central1 
" ; 
  
 String 
  
 modelName 
  
 = 
  
" gemini 
 - 
 1.5 
 - 
 flash 
 - 
 001 
" ; 
  
 videoAudioInput 
 ( 
 projectId 
 , 
  
 location 
 , 
  
 modelName 
 ); 
  
 } 
  
 // Analyzes the given video input, including its audio track. 
  
 public 
  
 static 
  
 String 
  
 videoAudioInput 
 ( 
 String 
  
 projectId 
 , 
  
 String 
  
 location 
 , 
  
 String 
  
 modelName 
 ) 
  
 throws 
  
 IOException 
  
 { 
  
 // Initialize client that will be used to send requests. This client only needs 
  
 // to be created once, and can be reused for multiple requests. 
  
 try 
  
 ( 
 VertexAI 
  
 vertexAI 
  
 = 
  
 new 
  
 VertexAI 
 ( 
 projectId 
 , 
  
 location 
 )) 
  
 { 
  
 String 
  
 videoUri 
  
 = 
  
" gs 
 : 
 //cloud-samples-data/generative-ai/video/pixel8.mp4"; 
  
 GenerativeModel 
  
 model 
  
 = 
  
 new 
  
 GenerativeModel 
 ( 
 modelName 
 , 
  
 vertexAI 
 ); 
  
 GenerateContentResponse 
  
 response 
  
 = 
  
 model 
 . 
 generateContent 
 ( 
  
 ContentMaker 
 . 
 fromMultiModalData 
 ( 
  
" Provide 
  
 a 
  
 description 
  
 of 
  
 the 
  
 video 
 . 
 \ 
 n 
  
 The 
  
 description 
  
 should 
  
 also 
  
"  
 + 
  
" contain 
  
 anything 
  
 important 
  
 which 
  
 people 
  
 say 
  
 in 
  
 the 
  
 video 
 .", 
  
 PartMaker 
 . 
 fromMimeTypeAndData 
 ( 
" video 
 / 
 mp4 
" , 
  
 videoUri 
 ) 
  
 )); 
  
 String 
  
 output 
  
 = 
  
 ResponseHandler 
 . 
 getText 
 ( 
 response 
 ); 
  
 System 
 . 
 out 
 . 
 println 
 ( 
 output 
 ); 
  
 return 
  
 output 
 ; 
  
 } 
  
 } 
 }

Go

  import 
  
 ( 
  
" context 
"  
" errors 
"  
" fmt 
"  
" io 
"  
" mime 
"  
" path 
 / 
 filepath 
"  
" cloud 
 . 
 google 
 . 
 com 
 / 
 go 
 / 
 vertexai 
 / 
 genai 
" ) 
 // generateMultimodalContent shows how to send video and text prompts to a model, writing the response to 
 // the provided io.Writer. 
 func 
  
 generateMultimodalContent 
 ( 
 w 
  
 io 
 . 
 Writer 
 , 
  
 projectID 
 , 
  
 location 
 , 
  
 modelName 
  
 string 
 ) 
  
 error 
  
 { 
  
 // location := "us-central1 
"  
 // modelName := "gemini-1.5-flash-001 
"  
 ctx 
  
 := 
  
 context 
 . 
 Background 
 () 
  
 client 
 , 
  
 err 
  
 := 
  
 genai 
 . 
 NewClient 
 ( 
 ctx 
 , 
  
 projectID 
 , 
  
 location 
 ) 
  
 if 
  
 err 
  
 != 
  
 nil 
  
 { 
  
 return 
  
 fmt 
 . 
 Errorf 
 ( 
" unable 
  
 to 
  
 create 
  
 client 
 : 
  
 % 
 w 
" , 
  
 err 
 ) 
  
 } 
  
 defer 
  
 client 
 . 
 Close 
 () 
  
 model 
  
 := 
  
 client 
 . 
 GenerativeModel 
 ( 
 modelName 
 ) 
  
 // Given a video file URL, prepare video file as genai.Part 
  
 part 
  
 := 
  
 genai 
 . 
 FileData 
 { 
  
 MIMEType 
 : 
  
 mime 
 . 
 TypeByExtension 
 ( 
 filepath 
 . 
 Ext 
 ( 
" pixel8 
 . 
 mp4 
" )), 
  
 FileURI 
 : 
  
" gs 
 : 
 //cloud-samples-data/generative-ai/video/pixel8.mp4", 
  
 } 
  
 res 
 , 
  
 err 
  
 := 
  
 model 
 . 
 GenerateContent 
 ( 
 ctx 
 , 
  
 part 
 , 
  
 genai 
 . 
 Text 
 ( 
 ` 
 Provide a description of the video. 
 The description should also contain anything important which people say in the video. 
 ` 
 )) 
  
 if 
  
 err 
  
 != 
  
 nil 
  
 { 
  
 return 
  
 fmt 
 . 
 Errorf 
 ( 
" unable 
  
 to 
  
 generate 
  
 contents 
 : 
  
 % 
 w 
" , 
  
 err 
 ) 
  
 } 
  
 if 
  
 len 
 ( 
 res 
 . 
 Candidates 
 ) 
  
 == 
  
 0 
  
 || 
  
 len 
 ( 
 res 
 . 
 Candidates 
 [ 
 0 
 ]. 
 Content 
 . 
 Parts 
 ) 
  
 == 
  
 0 
  
 { 
  
 return 
  
 errors 
 . 
 New 
 ( 
" empty 
  
 response 
  
 from 
  
 model 
" ) 
  
 } 
  
 fmt 
 . 
 Fprintf 
 ( 
 w 
 , 
  
" generated 
  
 response 
 : 
  
 % 
 s 
 \ 
 n 
" , 
  
 res 
 . 
 Candidates 
 [ 
 0 
 ]. 
 Content 
 . 
 Parts 
 [ 
 0 
 ]) 
  
 return 
  
 nil 
 }

C#

  using 
  
 Google.Cloud.AIPlatform.V1 
 ; 
 using 
  
 System 
 ; 
 using 
  
 System.Threading.Tasks 
 ; 
 public 
  
 class 
  
 VideoInputWithAudio 
 { 
  
 public 
  
 async 
  
 Task<string> 
  
 DescribeVideo 
 ( 
  
 string 
  
 projectId 
  
 = 
  
" your 
 - 
 project 
 - 
 id 
" , 
  
 string 
  
 location 
  
 = 
  
" us 
 - 
 central1 
" , 
  
 string 
  
 publisher 
  
 = 
  
" google 
" , 
  
 string 
  
 model 
  
 = 
  
" gemini 
 - 
 1.5 
 - 
 flash 
 - 
 001 
" ) 
  
 { 
  
 var 
  
 predictionServiceClient 
  
 = 
  
 new 
  
 PredictionServiceClientBuilder 
  
 { 
  
 Endpoint 
  
 = 
  
 $ 
" { 
 location 
 } 
 - 
 aiplatform 
 . 
 googleapis 
 . 
 com 
"  
 }. 
 Build 
 (); 
  
 string 
  
 prompt 
  
 = 
  
 @"Provide 
  
 a 
  
 description 
  
 of 
  
 the 
  
 video 
 . 
 The 
  
 description 
  
 should 
  
 also 
  
 contain 
  
 anything 
  
 important 
  
 which 
  
 people 
  
 say 
  
 in 
  
 the 
  
 video 
 ."; 
  
 var 
  
 generateContentRequest 
  
 = 
  
 new 
  
 GenerateContentRequest 
  
 { 
  
 Model 
  
 = 
  
 $ 
" projects 
 / 
 { 
 projectId 
 } 
 / 
 locations 
 / 
 { 
 location 
 } 
 / 
 publishers 
 / 
 { 
 publisher 
 } 
 / 
 models 
 / 
 { 
 model 
 }", 
  
 Contents 
  
 = 
  
 { 
  
 new 
  
 Content 
  
 { 
  
 Role 
  
 = 
  
" USER 
" , 
  
 Parts 
  
 = 
  
 { 
  
 new 
  
 Part 
  
 { 
  
 Text 
  
 = 
  
 prompt 
  
 }, 
  
 new 
  
 Part 
  
 { 
  
 FileData 
  
 = 
  
 new 
 () 
  
 { 
  
 MimeType 
  
 = 
  
" video 
 / 
 mp4 
" , 
  
 FileUri 
  
 = 
  
" gs 
 : 
 //cloud-samples-data/generative-ai/video/pixel8.mp4" }} 
  
 } 
  
 } 
  
 } 
  
 }; 
  
 GenerateContentResponse 
  
 response 
  
 = 
  
 await 
  
 predictionServiceClient 
 . 
 GenerateContentAsync 
 ( 
 generateContentRequest 
 ); 
  
 string 
  
 responseText 
  
 = 
  
 response 
 . 
 Candidates 
 [ 
 0 
 ]. 
 Content 
 . 
 Parts 
 [ 
 0 
 ]. 
 Text 
 ; 
  
 Console 
 . 
 WriteLine 
 ( 
 responseText 
 ); 
  
 return 
  
 responseText 
 ; 
  
 } 
 }

REST

You can send this prompt request from Cloud Shell, the command line, or in an IDE, or you can embed the REST call into your application where appropriate.

curl  
-X  
POST  
 \ 
-H  
"Authorization:  
Bearer  
 $( 
gcloud  
auth  
print-access-token ) 
"  
 \ 
-H  
"Content-Type:  
application/json"  
 \ 
https://us-central1-aiplatform.googleapis.com/v1/projects/ ${ 
 PROJECT_ID 
 } 
/locations/us-central1/publishers/google/models/ ${ 
 MODEL_ID 
 } 
:streamGenerateContent  
-d  
 \ 
 $'{ 
 "contents": { 
 "role": "user", 
 "parts": [ 
 { 
 "fileData": { 
 "mimeType": "video/mp4", 
 "fileUri": "gs://cloud-samples-data/generative-ai/video/pixel8.mp4 
"  
 } 
  
 } 
,  
 { 
  
"text":  
"Provide  
a  
description  
of  
the  
video.  
The  
description  
should  
also  
contain  
anything  
important  
which  
people  
say  
 in 
  
the  
video."  
 } 
  
 ] 
  
 } 
 } 
'

The model returns a response. Note that the response is generated in sections with each section separately evaluated for safety.

What's next

Learn more about the Gemini API in Vertex AI .
Explore the Vertex AI Gemini API SDK reference for Python , Node.js , Java , Go , or C# .
See the Model API for Gemini in Vertex AI .
Learn about calling Vertex AI models by using the OpenAI library .