Starting April 29, 2025, Gemini 1.5 Pro and Gemini 1.5 Flash models are not available in projects that have no prior usage of these models, including new projects. For details, see Model versions and lifecycle .

Remove objects from an image using inpaint

This page describes removing objects from an image. Imagen on Vertex AI lets you specify a mask area, called inpainting, to remove objects from an image. You can bring your own mask, or you can let Imagen on Vertex AI generate a mask for you.

Content removal example

The following example uses inpainting to remove content from an existing image using an image mask:

Inputs

Base image ^* to edit	Mask area specified using tools in the Google Cloud console	Text prompt
		Prompt: (no prompt provided)

^{* Image credit: Inside Weather
on Unsplash
.}

Output after specifying a mask area in the Google Cloud console

A sample edited image depicts the couch and rug in the input
example without lemons

View Imagen for Editing and Customization model card

Before you begin

Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Go to project selector

Verify that billing is enabled for your Google Cloud project .

Enable the Vertex AI API.

Enable the API

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Go to project selector

Verify that billing is enabled for your Google Cloud project .

Enable the Vertex AI API.

Enable the API

Set up authentication for your environment.

Select the tab for how you plan to use the samples on this page:
Console

When you use the Google Cloud console to access Google Cloud services and APIs, you don't need to set up authentication.
Java

To use the Java samples on this page in a local development environment, install and initialize the gcloud CLI, and then set up Application Default Credentials with your user credentials.
For more information, see Set up ADC for a local development environment in the Google Cloud authentication documentation.
Node.js

To use the Node.js samples on this page in a local development environment, install and initialize the gcloud CLI, and then set up Application Default Credentials with your user credentials.
For more information, see Set up ADC for a local development environment in the Google Cloud authentication documentation.
Python

To use the Python samples on this page in a local development environment, install and initialize the gcloud CLI, and then set up Application Default Credentials with your user credentials.
For more information, see Set up ADC for a local development environment in the Google Cloud authentication documentation.
REST

To use the REST API samples on this page in a local development environment, you use the credentials you provide to the gcloud CLI.
For more information, see Authenticate for using REST in the Google Cloud authentication documentation.

Remove with a defined mask area

Use the following samples to specify inpainting to remove content. In these samples you specify a base image, a text prompt, and a mask area to modify the base image.

Imagen 3

Use the following samples to send an inpainting request using the Imagen 3 model.

Console

In the Google Cloud console, go to the Vertex AI > Media Studio page.

Go to Media Studio
Click Upload . In the displayed file dialog, select a file to upload.
Click Inpaint .
In the Parameters panel, click Inpaint (Remove) .
Do one of the following:
- Upload your own mask:
  1. Create a mask on your computer.
  2. Click Upload mask . In the displayed dialog, select a mask to upload.
- Define your own mask: in the editing toolbar, use the mask tools ( box, brush, or masked_transitions invert tool) to specify the area or areas to add content to.
Optional: In the Parameters panel, adjust the following options:
- Model : the Imagen model to use
- Number of results : the number of result to generate
- Negative prompt : items to avoid generating
In the prompt field, enter a prompt to modify the image.
Click Generate .

Python

Install

pip install --upgrade google-genai

To learn more, see the SDK reference documentation .

Set environment variables to use the Gen AI SDK with Vertex AI:

 # Replace the `GOOGLE_CLOUD_PROJECT` and `GOOGLE_CLOUD_LOCATION` values 
 # with appropriate values for your project. 
 export 
  
 GOOGLE_CLOUD_PROJECT 
 = 
 GOOGLE_CLOUD_PROJECT 
 export 
  
 GOOGLE_CLOUD_LOCATION 
 = 
 us-central1 
 export 
  
 GOOGLE_GENAI_USE_VERTEXAI 
 = 
True

  from 
  
 google 
  
 import 
 genai 
 from 
  
 google.genai.types 
  
 import 
 ( 
 RawReferenceImage 
 , 
 MaskReferenceImage 
 , 
 MaskReferenceConfig 
 , 
 EditImageConfig 
 , 
 ) 
 client 
 = 
 genai 
 . 
 Client 
 () 
 # TODO(developer): Update and un-comment below line 
 # output_file = "output-image.png" 
 raw_ref 
 = 
 RawReferenceImage 
 ( 
 reference_image 
 = 
 Image 
 . 
 from_file 
 ( 
 location 
 = 
 "test_resources/fruit.png" 
 ), 
 reference_id 
 = 
 0 
 , 
 ) 
 mask_ref 
 = 
 MaskReferenceImage 
 ( 
 reference_id 
 = 
 1 
 , 
 reference_image 
 = 
 Image 
 . 
 from_file 
 ( 
 location 
 = 
 "test_resources/fruit_mask.png" 
 ), 
 config 
 = 
 MaskReferenceConfig 
 ( 
 mask_mode 
 = 
 "MASK_MODE_USER_PROVIDED" 
 , 
 mask_dilation 
 = 
 0.01 
 , 
 ), 
 ) 
 image 
 = 
 client 
 . 
 models 
 . 
 edit_image 
 ( 
 model 
 = 
 "imagen-3.0-capability-001" 
 , 
 prompt 
 = 
 "" 
 , 
 reference_images 
 = 
 [ 
 raw_ref 
 , 
 mask_ref 
 ], 
 config 
 = 
 EditImageConfig 
 ( 
 edit_mode 
 = 
 "EDIT_MODE_INPAINT_REMOVAL" 
 , 
 ), 
 ) 
 image 
 . 
 generated_images 
 [ 
 0 
 ] 
 . 
 image 
 . 
 save 
 ( 
 output_file 
 ) 
 print 
 ( 
 f 
 "Created output image using 
 { 
 len 
 ( 
 image 
 . 
 generated_images 
 [ 
 0 
 ] 
 . 
 image 
 . 
 image_bytes 
 ) 
 } 
 bytes" 
 ) 
 # Example response: 
 # Created output image using 1234567 bytes

REST

For more information, see the Edit images API reference.

Before using any of the request data, make the following replacements:

PROJECT_ID : Your Google Cloud project ID .
LOCATION : Your project's region. For example, us-central1 , europe-west2 , or asia-northeast3 . For a list of available regions, see Generative AI on Vertex AI locations .
prompt : For best results, omit a prompt and negativePrompt when you use inpainting for removal.
B64_BASE_IMAGE : The base image to edit or upscale. The image must be specified as a base64-encoded byte string. Size limit: 10 MB.
B64_MASK_IMAGE : The black and white image you want to use as a mask layer to edit the original image. The image must be specified as a base64-encoded byte string. Size limit: 10 MB.
MASK_DILATION - float. The percentage of image width to dilate this mask by. A value of 0.01 is recommended to compensate for imperfect input masks.
EDIT_STEPS - integer. The number of sampling steps for the base model. For inpainting removal, start at 12 steps. Increase steps to upper limit of 75 if the quality doesn't meet your requirements. Increasing steps also increases request latency.
EDIT_IMAGE_COUNT - The number of edited images. Accepted integer values: 1-4. Default value: 4.

HTTP method and URL:

POST https:// LOCATION 
-aiplatform.googleapis.com/v1/projects/ PROJECT_ID 
/locations/ LOCATION 
/publishers/google/models/imagen-3.0-capability-001:predict

Request JSON body:

{
  "instances": [
    {
      "prompt": "",
      "referenceImages": [
        {
          "referenceType": "REFERENCE_TYPE_RAW",
          "referenceId": 1,
          "referenceImage": {
            "bytesBase64Encoded": " B64_BASE_IMAGE 
"
          }
        },
        {
          "referenceType": "REFERENCE_TYPE_MASK",
          "referenceId": 2,
          "referenceImage": {
            "bytesBase64Encoded": " B64_MASK_IMAGE 
"
          },
          "maskImageConfig": {
            "maskMode": "MASK_MODE_USER_PROVIDED",
            "dilation": MASK_DILATION 
}
        }
      ]
    }
  ],
  "parameters": {
    "editConfig": {
      "baseSteps": EDIT_STEPS 
}, "editMode": "EDIT_MODE_INPAINT_REMOVAL","sampleCount": EDIT_IMAGE_COUNT 
}
}

To send your request, choose one of these options:

curl

Note: The following command assumes that you have logged in to the gcloud CLI with your user account by running gcloud init or gcloud auth login , or by using Cloud Shell , which automatically logs you into the gcloud CLI . You can check the currently active account by running gcloud auth list .

Save the request body in a file named request.json , and execute the following command:

curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https:// LOCATION 
-aiplatform.googleapis.com/v1/projects/ PROJECT_ID 
/locations/ LOCATION 
/publishers/google/models/imagen-3.0-capability-001:predict"

PowerShell

Note: The following command assumes that you have logged in to the gcloud CLI with your user account by running gcloud init or gcloud auth login . You can check the currently active account by running gcloud auth list .

Save the request body in a file named request.json , and execute the following command:

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https:// LOCATION 
-aiplatform.googleapis.com/v1/projects/ PROJECT_ID 
/locations/ LOCATION 
/publishers/google/models/imagen-3.0-capability-001:predict" | Select-Object -Expand Content

The following sample response is for a request with "sampleCount": 2 . The response returns two prediction objects, with the generated image bytes base64-encoded.

{
  "predictions": [
    {
      "bytesBase64Encoded": " BASE64_IMG_BYTES 
",
      "mimeType": "image/png"
    },
    {
      "mimeType": "image/png",
      "bytesBase64Encoded": " BASE64_IMG_BYTES 
"
    }
  ]
}

Imagen 2

Caution: Starting on June 24, 2025, Imagen versions 1 and 2 are deprecated. Imagen models imagegeneration@002 , imagegeneration@005 , and imagegeneration@006 will be removed on September 24, 2025. For more information about migrating to Imagen 3, see Migrate to Imagen 3 .

Use the following samples to send an inpainting request using the Imagen 2 model.

Console

In the Google Cloud console, go to the Vertex AI > Media Studio page.

Go to Media Studio
In the lower task panel, click Edit image .
Click Upload to select your locally stored product image to edit.
In the editing toolbar, use the mask tools ( box, brush, or masked_transitions invert tool) to specify the area or areas to remove content from.
Optional. In the Parameters panel, adjust the Number of results , Negative prompt (optional for removal), Text prompt guidance , or other parameters.
Leave the prompt field empty.
Click Generate .

REST

Before using any of the request data, make the following replacements:

PROJECT_ID : Your Google Cloud project ID .
LOCATION : Your project's region. For example, us-central1 , europe-west2 , or asia-northeast3 . For a list of available regions, see Generative AI on Vertex AI locations .
B64_BASE_IMAGE : The base image to edit or upscale. The image must be specified as a base64-encoded byte string. Size limit: 10 MB.
B64_MASK_IMAGE : The black and white image you want to use as a mask layer to edit the original image. The image must be specified as a base64-encoded byte string. Size limit: 10 MB.
EDIT_IMAGE_COUNT : The number of edited images. Default value: 4.

HTTP method and URL:

POST https:// LOCATION 
-aiplatform.googleapis.com/v1/projects/ PROJECT_ID 
/locations/ LOCATION 
/publishers/google/models/imagegeneration@006:predict

Request JSON body:

{
  "instances": [
    {
      "prompt": "",
      "image": {
          "bytesBase64Encoded": " B64_BASE_IMAGE 
"
      },
      "mask": {
        "image": {
          "bytesBase64Encoded": " B64_MASK_IMAGE 
"
        }
      }
    }
  ],
  "parameters": {
    "sampleCount": EDIT_IMAGE_COUNT 
,
    "editConfig": { "editMode": "inpainting-remove"}
  }
}

To send your request, choose one of these options:

curl

Save the request body in a file named request.json , and execute the following command:

curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https:// LOCATION 
-aiplatform.googleapis.com/v1/projects/ PROJECT_ID 
/locations/ LOCATION 
/publishers/google/models/imagegeneration@006:predict"

PowerShell

Save the request body in a file named request.json , and execute the following command:

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https:// LOCATION 
-aiplatform.googleapis.com/v1/projects/ PROJECT_ID 
/locations/ LOCATION 
/publishers/google/models/imagegeneration@006:predict" | Select-Object -Expand Content

The following sample response is for a request with "sampleCount": 2 . The response returns two prediction objects, with the generated image bytes base64-encoded.

{
  "predictions": [
    {
      "bytesBase64Encoded": " BASE64_IMG_BYTES 
",
      "mimeType": "image/png"
    },
    {
      "mimeType": "image/png",
      "bytesBase64Encoded": " BASE64_IMG_BYTES 
"
    }
  ]
}

Python

To learn how to install or update the Vertex AI SDK for Python, see Install the Vertex AI SDK for Python . For more information, see the Python API reference documentation .

  import 
  
 vertexai 
 from 
  
 vertexai.preview.vision_models 
  
 import 
 Image 
 , 
 ImageGenerationModel 
 # TODO(developer): Update and un-comment below lines 
 # PROJECT_ID = "your-project-id" 
 # input_file = "input-image.png" 
 # mask_file = "mask-image.png" 
 # output_file = "outpur-image.png" 
 # prompt = "" # The text prompt describing the entire image. 
 vertexai 
 . 
 init 
 ( 
 project 
 = 
 PROJECT_ID 
 , 
 location 
 = 
 "us-central1" 
 ) 
 model 
 = 
 ImageGenerationModel 
 . 
 from_pretrained 
 ( 
 "imagegeneration@006" 
 ) 
 base_img 
 = 
 Image 
 . 
 load_from_file 
 ( 
 location 
 = 
 input_file 
 ) 
 mask_img 
 = 
 Image 
 . 
 load_from_file 
 ( 
 location 
 = 
 mask_file 
 ) 
 images 
 = 
 model 
 . 
 edit_image 
 ( 
 base_image 
 = 
 base_img 
 , 
 mask 
 = 
 mask_img 
 , 
 prompt 
 = 
 prompt 
 , 
 edit_mode 
 = 
 "inpainting-remove" 
 , 
 # Optional parameters 
 # negative_prompt="", # Describes the object being removed (i.e., "person") 
 ) 
 images 
 [ 
 0 
 ] 
 . 
 save 
 ( 
 location 
 = 
 output_file 
 , 
 include_generation_parameters 
 = 
 False 
 ) 
 # Optional. View the edited image in a notebook. 
 # images[0].show() 
 print 
 ( 
 f 
 "Created output image using 
 { 
 len 
 ( 
 images 
 [ 
 0 
 ] 
 . 
 _image_bytes 
 ) 
 } 
 bytes" 
 ) 
 # Example response: 
 # Created output image using 12345678 bytes

Java

Before trying this sample, follow the Java setup instructions in the Vertex AI quickstart using client libraries . For more information, see the Vertex AI Java API reference documentation .

To authenticate to Vertex AI, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .

In this sample, you specify the model as part of an EndpointName . The EndpointName is passed to the predict method which is called on a PredictionServiceClient . The service returns an edited version of the image, which is then saved locally.

For more information about model versions and features, see Imagen models .

  import 
  
 com.google.api.gax.rpc. ApiException 
 
 ; 
 import 
  
 com.google.cloud.aiplatform.v1. EndpointName 
 
 ; 
 import 
  
 com.google.cloud.aiplatform.v1. PredictResponse 
 
 ; 
 import 
  
 com.google.cloud.aiplatform.v1. PredictionServiceClient 
 
 ; 
 import 
  
 com.google.cloud.aiplatform.v1. PredictionServiceSettings 
 
 ; 
 import 
  
 com.google.gson.Gson 
 ; 
 import 
  
 com.google.protobuf. InvalidProtocolBufferException 
 
 ; 
 import 
  
 com.google.protobuf. Value 
 
 ; 
 import 
  
 com.google.protobuf.util. JsonFormat 
 
 ; 
 import 
  
 java.io.IOException 
 ; 
 import 
  
 java.nio.file.Files 
 ; 
 import 
  
 java.nio.file.Path 
 ; 
 import 
  
 java.nio.file.Paths 
 ; 
 import 
  
 java.util.Base64 
 ; 
 import 
  
 java.util.Collections 
 ; 
 import 
  
 java.util.HashMap 
 ; 
 import 
  
 java.util.Map 
 ; 
 public 
  
 class 
 EditImageInpaintingRemoveMaskSample 
  
 { 
  
 public 
  
 static 
  
 void 
  
 main 
 ( 
 String 
 [] 
  
 args 
 ) 
  
 throws 
  
 IOException 
  
 { 
  
 // TODO(developer): Replace these variables before running the sample. 
  
 String 
  
 projectId 
  
 = 
  
 "my-project-id" 
 ; 
  
 String 
  
 location 
  
 = 
  
 "us-central1" 
 ; 
  
 String 
  
 inputPath 
  
 = 
  
 "/path/to/my-input.png" 
 ; 
  
 String 
  
 maskPath 
  
 = 
  
 "/path/to/my-mask.png" 
 ; 
  
 String 
  
 prompt 
  
 = 
  
 "" 
 ; 
  
 // The text prompt describing the entire image. 
  
 editImageInpaintingRemoveMask 
 ( 
 projectId 
 , 
  
 location 
 , 
  
 inputPath 
 , 
  
 maskPath 
 , 
  
 prompt 
 ); 
  
 } 
  
 // Edit an image using a mask file. Inpainting can remove an object from the masked area. 
  
 public 
  
 static 
  
  PredictResponse 
 
  
 editImageInpaintingRemoveMask 
 ( 
  
 String 
  
 projectId 
 , 
  
 String 
  
 location 
 , 
  
 String 
  
 inputPath 
 , 
  
 String 
  
 maskPath 
 , 
  
 String 
  
 prompt 
 ) 
  
 throws 
  
  ApiException 
 
 , 
  
 IOException 
  
 { 
  
 final 
  
 String 
  
 endpoint 
  
 = 
  
 String 
 . 
 format 
 ( 
 "%s-aiplatform.googleapis.com:443" 
 , 
  
 location 
 ); 
  
  PredictionServiceSettings 
 
  
 predictionServiceSettings 
  
 = 
  
  PredictionServiceSettings 
 
 . 
 newBuilder 
 (). 
 setEndpoint 
 ( 
 endpoint 
 ). 
 build 
 (); 
  
 // Initialize client that will be used to send requests. This client only needs to be created 
  
 // once, and can be reused for multiple requests. 
  
 try 
  
 ( 
  PredictionServiceClient 
 
  
 predictionServiceClient 
  
 = 
  
  PredictionServiceClient 
 
 . 
 create 
 ( 
 predictionServiceSettings 
 )) 
  
 { 
  
 final 
  
  EndpointName 
 
  
 endpointName 
  
 = 
  
  EndpointName 
 
 . 
  ofProjectLocationPublisherModelName 
 
 ( 
  
 projectId 
 , 
  
 location 
 , 
  
 "google" 
 , 
  
 "imagegeneration@006" 
 ); 
  
 // Encode image and mask to Base64 
  
 String 
  
 imageBase64 
  
 = 
  
 Base64 
 . 
 getEncoder 
 (). 
 encodeToString 
 ( 
 Files 
 . 
 readAllBytes 
 ( 
 Paths 
 . 
 get 
 ( 
 inputPath 
 ))); 
  
 String 
  
 maskBase64 
  
 = 
  
 Base64 
 . 
 getEncoder 
 (). 
 encodeToString 
 ( 
 Files 
 . 
 readAllBytes 
 ( 
 Paths 
 . 
 get 
 ( 
 maskPath 
 ))); 
  
 // Create the image and image mask maps 
  
 Map<String 
 , 
  
 String 
>  
 imageMap 
  
 = 
  
 new 
  
 HashMap 
<> (); 
  
 imageMap 
 . 
 put 
 ( 
 "bytesBase64Encoded" 
 , 
  
 imageBase64 
 ); 
  
 Map<String 
 , 
  
 String 
>  
 maskMap 
  
 = 
  
 new 
  
 HashMap 
<> (); 
  
 maskMap 
 . 
 put 
 ( 
 "bytesBase64Encoded" 
 , 
  
 maskBase64 
 ); 
  
 Map<String 
 , 
  
 Map 
>  
 imageMaskMap 
  
 = 
  
 new 
  
 HashMap 
<> (); 
  
 imageMaskMap 
 . 
 put 
 ( 
 "image" 
 , 
  
 maskMap 
 ); 
  
 Map<String 
 , 
  
 Object 
>  
 instancesMap 
  
 = 
  
 new 
  
 HashMap 
<> (); 
  
 instancesMap 
 . 
 put 
 ( 
 "prompt" 
 , 
  
 prompt 
 ); 
  
 // [ "prompt", "<my-prompt>" ] 
  
 instancesMap 
 . 
 put 
 ( 
  
 "image" 
 , 
  
 imageMap 
 ); 
  
 // [ "image", [ "bytesBase64Encoded", "iVBORw0KGgo...==" ] ] 
  
 instancesMap 
 . 
 put 
 ( 
  
 "mask" 
 , 
  
 imageMaskMap 
 ); 
  
 // [ "mask", [ "image", [ "bytesBase64Encoded", "iJKDF0KGpl...==" ] ] ] 
  
 instancesMap 
 . 
 put 
 ( 
 "editMode" 
 , 
  
 "inpainting-remove" 
 ); 
  
 // [ "editMode", "inpainting-remove" ] 
  
  Value 
 
  
 instances 
  
 = 
  
 mapToValue 
 ( 
 instancesMap 
 ); 
  
 // Optional parameters 
  
 Map<String 
 , 
  
 Object 
>  
 paramsMap 
  
 = 
  
 new 
  
 HashMap 
<> (); 
  
 paramsMap 
 . 
 put 
 ( 
 "sampleCount" 
 , 
  
 1 
 ); 
  
  Value 
 
  
 parameters 
  
 = 
  
 mapToValue 
 ( 
 paramsMap 
 ); 
  
  PredictResponse 
 
  
 predictResponse 
  
 = 
  
 predictionServiceClient 
 . 
 predict 
 ( 
  
 endpointName 
 , 
  
 Collections 
 . 
 singletonList 
 ( 
 instances 
 ), 
  
 parameters 
 ); 
  
 for 
  
 ( 
  Value 
 
  
 prediction 
  
 : 
  
 predictResponse 
 . 
  getPredictionsList 
 
 ()) 
  
 { 
  
 Map<String 
 , 
  
 Value 
>  
 fieldsMap 
  
 = 
  
 prediction 
 . 
 getStructValue 
 (). 
 getFieldsMap 
 (); 
  
 if 
  
 ( 
 fieldsMap 
 . 
 containsKey 
 ( 
 "bytesBase64Encoded" 
 )) 
  
 { 
  
 String 
  
 bytesBase64Encoded 
  
 = 
  
 fieldsMap 
 . 
 get 
 ( 
 "bytesBase64Encoded" 
 ). 
 getStringValue 
 (); 
  
 Path 
  
 tmpPath 
  
 = 
  
 Files 
 . 
 createTempFile 
 ( 
 "imagen-" 
 , 
  
 ".png" 
 ); 
  
 Files 
 . 
 write 
 ( 
 tmpPath 
 , 
  
 Base64 
 . 
 getDecoder 
 (). 
 decode 
 ( 
 bytesBase64Encoded 
 )); 
  
 System 
 . 
 out 
 . 
 format 
 ( 
 "Image file written to: %s\n" 
 , 
  
 tmpPath 
 . 
 toUri 
 ()); 
  
 } 
  
 } 
  
 return 
  
 predictResponse 
 ; 
  
 } 
  
 } 
  
 private 
  
 static 
  
  Value 
 
  
 mapToValue 
 ( 
 Map<String 
 , 
  
 Object 
>  
 map 
 ) 
  
 throws 
  
  InvalidProtocolBufferException 
 
  
 { 
  
 Gson 
  
 gson 
  
 = 
  
 new 
  
 Gson 
 (); 
  
 String 
  
 json 
  
 = 
  
 gson 
 . 
 toJson 
 ( 
 map 
 ); 
  
  Value 
 
 . 
 Builder 
  
 builder 
  
 = 
  
  Value 
 
 . 
 newBuilder 
 (); 
  
  JsonFormat 
 
 . 
 parser 
 (). 
 merge 
 ( 
 json 
 , 
  
 builder 
 ); 
  
 return 
  
 builder 
 . 
 build 
 (); 
  
 } 
 }

Node.js

Before trying this sample, follow the Node.js setup instructions in the Vertex AI quickstart using client libraries . For more information, see the Vertex AI Node.js API reference documentation .

To authenticate to Vertex AI, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .

In this sample, you call the predict method on a PredictionServiceClient . The service generates images which are then saved locally. For more information about model versions and features, see Imagen models .

  /** 
 * TODO(developer): Update these variables before running the sample. 
 */ 
 const 
  
 projectId 
  
 = 
  
 process 
 . 
 env 
 . 
 CAIP_PROJECT_ID 
 ; 
 const 
  
 location 
  
 = 
  
 'us-central1' 
 ; 
 const 
  
 inputFile 
  
 = 
  
 'resources/volleyball_game.png' 
 ; 
 const 
  
 maskFile 
  
 = 
  
 'resources/volleyball_game_inpainting_remove_mask.png' 
 ; 
 const 
  
 prompt 
  
 = 
  
 'volleyball game' 
 ; 
 const 
  
 aiplatform 
  
 = 
  
 require 
 ( 
 ' @google-cloud/aiplatform 
' 
 ); 
 // Imports the Google Cloud Prediction Service Client library 
 const 
  
 { 
 PredictionServiceClient 
 } 
  
 = 
  
 aiplatform 
 . 
 v1 
 ; 
 // Import the helper module for converting arbitrary protobuf.Value objects 
 const 
  
 { 
 helpers 
 } 
  
 = 
  
 aiplatform 
 ; 
 // Specifies the location of the api endpoint 
 const 
  
 clientOptions 
  
 = 
  
 { 
  
 apiEndpoint 
 : 
  
 ` 
 ${ 
 location 
 } 
 -aiplatform.googleapis.com` 
 , 
 }; 
 // Instantiates a client 
 const 
  
 predictionServiceClient 
  
 = 
  
 new 
  
  PredictionServiceClient 
 
 ( 
 clientOptions 
 ); 
 async 
  
 function 
  
 editImageInpaintingRemoveMask 
 () 
  
 { 
  
 const 
  
 fs 
  
 = 
  
 require 
 ( 
 'fs' 
 ); 
  
 const 
  
 util 
  
 = 
  
 require 
 ( 
 'util' 
 ); 
  
 // Configure the parent resource 
  
 const 
  
 endpoint 
  
 = 
  
 `projects/ 
 ${ 
 projectId 
 } 
 /locations/ 
 ${ 
 location 
 } 
 /publishers/google/models/imagegeneration@006` 
 ; 
  
 const 
  
 imageFile 
  
 = 
  
 fs 
 . 
 readFileSync 
 ( 
 inputFile 
 ); 
  
 // Convert the image data to a Buffer and base64 encode it. 
  
 const 
  
 encodedImage 
  
 = 
  
 Buffer 
 . 
 from 
 ( 
 imageFile 
 ). 
 toString 
 ( 
 'base64' 
 ); 
  
 const 
  
 maskImageFile 
  
 = 
  
 fs 
 . 
 readFileSync 
 ( 
 maskFile 
 ); 
  
 // Convert the image mask data to a Buffer and base64 encode it. 
  
 const 
  
 encodedMask 
  
 = 
  
 Buffer 
 . 
 from 
 ( 
 maskImageFile 
 ). 
 toString 
 ( 
 'base64' 
 ); 
  
 const 
  
 promptObj 
  
 = 
  
 { 
  
 prompt 
 : 
  
 prompt 
 , 
  
 // The text prompt describing the entire image 
  
 editMode 
 : 
  
 'inpainting-remove' 
 , 
  
 image 
 : 
  
 { 
  
 bytesBase64Encoded 
 : 
  
 encodedImage 
 , 
  
 }, 
  
 mask 
 : 
  
 { 
  
 image 
 : 
  
 { 
  
 bytesBase64Encoded 
 : 
  
 encodedMask 
 , 
  
 }, 
  
 }, 
  
 }; 
  
 const 
  
 instanceValue 
  
 = 
  
  helpers 
 
 . 
 toValue 
 ( 
 promptObj 
 ); 
  
 const 
  
 instances 
  
 = 
  
 [ 
 instanceValue 
 ]; 
  
 const 
  
 parameter 
  
 = 
  
 { 
  
 // Optional parameters 
  
 seed 
 : 
  
 100 
 , 
  
 // Controls the strength of the prompt 
  
 // 0-9 (low strength), 10-20 (medium strength), 21+ (high strength) 
  
 guidanceScale 
 : 
  
 21 
 , 
  
 sampleCount 
 : 
  
 1 
 , 
  
 }; 
  
 const 
  
 parameters 
  
 = 
  
  helpers 
 
 . 
 toValue 
 ( 
 parameter 
 ); 
  
 const 
  
 request 
  
 = 
  
 { 
  
 endpoint 
 , 
  
 instances 
 , 
  
 parameters 
 , 
  
 }; 
  
 // Predict request 
  
 const 
  
 [ 
 response 
 ] 
  
 = 
  
 await 
  
 predictionServiceClient 
 . 
 predict 
 ( 
 request 
 ); 
  
 const 
  
 predictions 
  
 = 
  
 response 
 . 
 predictions 
 ; 
  
 if 
  
 ( 
 predictions 
 . 
 length 
  
 === 
  
 0 
 ) 
  
 { 
  
 console 
 . 
 log 
 ( 
  
 'No image was generated. Check the request parameters and prompt.' 
  
 ); 
  
 } 
  
 else 
  
 { 
  
 let 
  
 i 
  
 = 
  
 1 
 ; 
  
 for 
  
 ( 
 const 
  
 prediction 
  
 of 
  
 predictions 
 ) 
  
 { 
  
 const 
  
 buff 
  
 = 
  
 Buffer 
 . 
 from 
 ( 
  
 prediction 
 . 
 structValue 
 . 
 fields 
 . 
 bytesBase64Encoded 
 . 
 stringValue 
 , 
  
 'base64' 
  
 ); 
  
 // Write image content to the output file 
  
 const 
  
 writeFile 
  
 = 
  
 util 
 . 
 promisify 
 ( 
 fs 
 . 
 writeFile 
 ); 
  
 const 
  
 filename 
  
 = 
  
 `output 
 ${ 
 i 
 } 
 .png` 
 ; 
  
 await 
  
 writeFile 
 ( 
 filename 
 , 
  
 buff 
 ); 
  
 console 
 . 
 log 
 ( 
 `Saved image 
 ${ 
 filename 
 } 
 ` 
 ); 
  
 i 
 ++ 
 ; 
  
 } 
  
 } 
 } 
 await 
  
 editImageInpaintingRemoveMask 
 ();

Remove with automatic mask detection

Use the following samples to specify inpainting to remove content. In these samples you specify a base image and a text prompt. Imagen automatically detects and creates a mask area to modify the base image.

Imagen 3

Use the following samples to send an inpainting request using the Imagen 3 model.

Console

In the Google Cloud console, go to the Vertex AI > Media Studio page.

Go to Media Studio
Click Upload . In the displayed file dialog, select a file to upload.
Click Inpaint .
In the Parameters panel, select Inpaint (Remove)
In the editing toolbar, click background_replace Extract .
Select one of the mask extraction options:
- Background elements : detects the background elements and creates a mask around them.
- Foreground elements : detects the foreground objects and creates a mask around them.
- background_replace People : detects people and creates a mask around them.
Optional: In the Parameters panel, adjust the following options:
- Model : the Imagen model to use
- Number of results : the number of result to generate
- Negative prompt : items to avoid generating
In the prompt field, enter a new prompt to modify the image.
Click send Generate .

Python

Install

pip install --upgrade google-genai

To learn more, see the SDK reference documentation .

Set environment variables to use the Gen AI SDK with Vertex AI:

 # Replace the `GOOGLE_CLOUD_PROJECT` and `GOOGLE_CLOUD_LOCATION` values 
 # with appropriate values for your project. 
 export 
  
 GOOGLE_CLOUD_PROJECT 
 = 
 GOOGLE_CLOUD_PROJECT 
 export 
  
 GOOGLE_CLOUD_LOCATION 
 = 
 us-central1 
 export 
  
 GOOGLE_GENAI_USE_VERTEXAI 
 = 
True

  from 
  
 google 
  
 import 
 genai 
 from 
  
 google.genai.types 
  
 import 
 ( 
 RawReferenceImage 
 , 
 MaskReferenceImage 
 , 
 MaskReferenceConfig 
 , 
 EditImageConfig 
 , 
 ) 
 client 
 = 
 genai 
 . 
 Client 
 () 
 # TODO(developer): Update and un-comment below line 
 # output_file = "output-image.png" 
 raw_ref 
 = 
 RawReferenceImage 
 ( 
 reference_image 
 = 
 Image 
 . 
 from_file 
 ( 
 location 
 = 
 "test_resources/fruit.png" 
 ), 
 reference_id 
 = 
 0 
 , 
 ) 
 mask_ref 
 = 
 MaskReferenceImage 
 ( 
 reference_id 
 = 
 1 
 , 
 reference_image 
 = 
 None 
 , 
 config 
 = 
 MaskReferenceConfig 
 ( 
 mask_mode 
 = 
 "MASK_MODE_FOREGROUND" 
 , 
 ), 
 ) 
 image 
 = 
 client 
 . 
 models 
 . 
 edit_image 
 ( 
 model 
 = 
 "imagen-3.0-capability-001" 
 , 
 prompt 
 = 
 "" 
 , 
 reference_images 
 = 
 [ 
 raw_ref 
 , 
 mask_ref 
 ], 
 config 
 = 
 EditImageConfig 
 ( 
 edit_mode 
 = 
 "EDIT_MODE_INPAINT_REMOVAL" 
 , 
 ), 
 ) 
 image 
 . 
 generated_images 
 [ 
 0 
 ] 
 . 
 image 
 . 
 save 
 ( 
 output_file 
 ) 
 print 
 ( 
 f 
 "Created output image using 
 { 
 len 
 ( 
 image 
 . 
 generated_images 
 [ 
 0 
 ] 
 . 
 image 
 . 
 image_bytes 
 ) 
 } 
 bytes" 
 ) 
 # Example response: 
 # Created output image using 1234567 bytes

REST

For more information, see the Edit images API reference.

Before using any of the request data, make the following replacements:

PROJECT_ID : Your Google Cloud project ID .
LOCATION : Your project's region. For example, us-central1 , europe-west2 , or asia-northeast3 . For a list of available regions, see Generative AI on Vertex AI locations .
prompt : For best results, omit a prompt and negativePrompt when you use inpainting for removal.
B64_BASE_IMAGE : The base image to edit or upscale. The image must be specified as a base64-encoded byte string. Size limit: 10 MB.
MASK_MODE - A string that sets the type of automatic mask creation the model uses. Available values:
- MASK_MODE_BACKGROUND : Automatically generates a mask using background segmentation. Use this setting for modifying background content.
- MASK_MODE_FOREGROUND : Automatically generates a mask using foreground segmentation. Use this setting to modify foreground content, such as removing these foreground objects (removal using inpainting).
- MASK_MODE_SEMANTIC : Automatically generates a mask using semantic segmentation based on the segmentation classes you specify in the maskImageConfig.maskClasses array. For example:
```
"maskImageConfig": {
            "maskMode": "MASK_MODE_SEMANTIC", "maskClasses": [175, 176], // bicycle, car"dilation": 0.01
          }
```
MASK_DILATION - float. The percentage of image width to dilate this mask by. A value of 0.01 is recommended to compensate for imperfect input masks.
EDIT_STEPS - integer. The number of sampling steps for the base model. For inpainting removal, start at 12 steps. Increase steps to upper limit of 75 if the quality doesn't meet your requirements. Increasing steps also increases request latency.
EDIT_IMAGE_COUNT - The number of edited images. Accepted integer values: 1-4. Default value: 4.

HTTP method and URL:

POST https:// LOCATION 
-aiplatform.googleapis.com/v1/projects/ PROJECT_ID 
/locations/ LOCATION 
/publishers/google/models/imagen-3.0-capability-001:predict

Request JSON body:

{
  "instances": [
    {
      "prompt": "",
      "referenceImages": [
        {
          "referenceType": "REFERENCE_TYPE_RAW",
          "referenceId": 1,
          "referenceImage": {
            "bytesBase64Encoded": " B64_BASE_IMAGE 
"
          }
        },
        {
          "referenceType": "REFERENCE_TYPE_MASK",
          "referenceId": 2,
          "maskImageConfig": { "maskMode": " MASK_MODE 
","dilation": MASK_DILATION 
}
        }
      ]
    }
  ],
  "parameters": {
    "editConfig": {
      "baseSteps": EDIT_STEPS 
}, "editMode": "EDIT_MODE_INPAINT_REMOVAL","sampleCount": EDIT_IMAGE_COUNT 
}
}

To send your request, choose one of these options:

curl

Save the request body in a file named request.json , and execute the following command:

curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https:// LOCATION 
-aiplatform.googleapis.com/v1/projects/ PROJECT_ID 
/locations/ LOCATION 
/publishers/google/models/imagen-3.0-capability-001:predict"

PowerShell

Save the request body in a file named request.json , and execute the following command:

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https:// LOCATION 
-aiplatform.googleapis.com/v1/projects/ PROJECT_ID 
/locations/ LOCATION 
/publishers/google/models/imagen-3.0-capability-001:predict" | Select-Object -Expand Content

The following sample response is for a request with "sampleCount": 2 . The response returns two prediction objects, with the generated image bytes base64-encoded.

{
  "predictions": [
    {
      "bytesBase64Encoded": " BASE64_IMG_BYTES 
",
      "mimeType": "image/png"
    },
    {
      "mimeType": "image/png",
      "bytesBase64Encoded": " BASE64_IMG_BYTES 
"
    }
  ]
}

Imagen 2

Use the following samples to send an inpainting request using the Imagen 2 model.

Console

In the Google Cloud console, go to the Vertex AI > Media Studio page.

Go to Media Studio
In the lower task panel, click Edit image .
Click Upload to select your locally stored product image to edit.
In the editing toolbar, click background_replace Extract .
Select one of the mask extraction options:
- Background elements - Detects the background elements and creates a mask around them.
- Foreground elements - Detects the foreground objects and creates a mask around them.
- background_replace People - Detects people and creates a mask around them.
Optional. In the Parameters panel, adjust the Number of results , Negative prompt , Text prompt guidance , or other parameters.
Leave the prompt field empty.
Click Generate .

REST

Before using any of the request data, make the following replacements:

PROJECT_ID : Your Google Cloud project ID .
LOCATION : Your project's region. For example, us-central1 , europe-west2 , or asia-northeast3 . For a list of available regions, see Generative AI on Vertex AI locations .
B64_BASE_IMAGE : The base image to edit or upscale. The image must be specified as a base64-encoded byte string. Size limit: 10 MB.
EDIT_IMAGE_COUNT : The number of edited images. Default value: 4.
MASK_TYPE : Prompts the model to generate a mask instead of you needing to provide one. Consequently, when you provide this parameter, you should omit a mask object. Available values:
- background : Automatically generates a mask to all regions except primary object, person, or subject in the image.
- foreground : Automatically generates a mask to the primary object, person, or subject in the image.
- semantic : Use automatic segmentation to create a mask area for one or more of the segmentation classes . Set the segmentation classes using the classes parameter and the corresponding class_id values. You can specify up to 5 classes. When you use the semantic mask type, the maskMode object should look like the following:
```
"maskMode": {
  "maskType": "semantic",
  "classes": [ class_id1, class_id2 
]
}
```

HTTP method and URL:

POST https:// LOCATION 
-aiplatform.googleapis.com/v1/projects/ PROJECT_ID 
/locations/ LOCATION 
/publishers/google/models/imagegeneration@006:predict

Request JSON body:

{
  "instances": [
    {
      "prompt": "",
      "image": {
        "bytesBase64Encoded": " B64_BASE_IMAGE 
"
      }
    }
  ],
  "parameters": {
    "sampleCount": EDIT_IMAGE_COUNT 
,
    "editConfig": { "editMode": "inpainting-remove",
      "maskMode": {
        "maskType": " MASK_TYPE 
"
      }}
  }
}

To send your request, choose one of these options:

curl

Save the request body in a file named request.json , and execute the following command:

curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https:// LOCATION 
-aiplatform.googleapis.com/v1/projects/ PROJECT_ID 
/locations/ LOCATION 
/publishers/google/models/imagegeneration@006:predict"

PowerShell

Save the request body in a file named request.json , and execute the following command:

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https:// LOCATION 
-aiplatform.googleapis.com/v1/projects/ PROJECT_ID 
/locations/ LOCATION 
/publishers/google/models/imagegeneration@006:predict" | Select-Object -Expand Content

The following sample response is for a request with "sampleCount": 2 . The response returns two prediction objects, with the generated image bytes base64-encoded.

{
  "predictions": [
    {
      "bytesBase64Encoded": " BASE64_IMG_BYTES 
",
      "mimeType": "image/png"
    },
    {
      "mimeType": "image/png",
      "bytesBase64Encoded": " BASE64_IMG_BYTES 
"
    }
  ]
}

Python

To learn how to install or update the Vertex AI SDK for Python, see Install the Vertex AI SDK for Python . For more information, see the Python API reference documentation .

  import 
  
 vertexai 
 from 
  
 vertexai.preview.vision_models 
  
 import 
 Image 
 , 
 ImageGenerationModel 
 # TODO(developer): Update and un-comment below lines 
 # PROJECT_ID = "your-project-id" 
 # input_file = "input-image.png" 
 # mask_mode = "foreground" # 'background', 'foreground', or 'semantic' 
 # output_file = "output-image.png" 
 # prompt = "sports car" # The text prompt describing what you want to see in the edited image. 
 vertexai 
 . 
 init 
 ( 
 project 
 = 
 PROJECT_ID 
 , 
 location 
 = 
 "us-central1" 
 ) 
 model 
 = 
 ImageGenerationModel 
 . 
 from_pretrained 
 ( 
 "imagegeneration@006" 
 ) 
 base_img 
 = 
 Image 
 . 
 load_from_file 
 ( 
 location 
 = 
 input_file 
 ) 
 images 
 = 
 model 
 . 
 edit_image 
 ( 
 base_image 
 = 
 base_img 
 , 
 mask_mode 
 = 
 mask_mode 
 , 
 prompt 
 = 
 prompt 
 , 
 edit_mode 
 = 
 "inpainting-remove" 
 , 
 ) 
 images 
 [ 
 0 
 ] 
 . 
 save 
 ( 
 location 
 = 
 output_file 
 , 
 include_generation_parameters 
 = 
 False 
 ) 
 # Optional. View the edited image in a notebook. 
 # images[0].show() 
 print 
 ( 
 f 
 "Created output image using 
 { 
 len 
 ( 
 images 
 [ 
 0 
 ] 
 . 
 _image_bytes 
 ) 
 } 
 bytes" 
 ) 
 # Example response: 
 # Created output image using 1279948 bytes

Limitations

The following sections explain limitations of Imagen's remove objects feature.

Modified pixels

Pixels generated by the model that aren't in the mask aren't guaranteed to be identical to the input and are generated at the model's resolution (such as 1024x1024). Very minute changes may exist in the generated image.

If you want perfect preservation of the image, then we recommend that you blend the generated image with the input image, using the mask. Typically, if the input image resolution is 2K or higher, blending the generated image and input image is required.

Removal limitation

Some small objects adjacent to masks may also be removed. As a best practice, we recommend that you make the mask as precise as possible.

Removing large areas in outdoor images sky regions may result in unwanted artifacts. As a best practice, we recommend that you provide a prompt.

What's next

Read articles about Imagen and other Generative AI on Vertex AI products:

Remove objects from an image using inpaint Stay organized with collections Save and categorize content based on your preferences.

Content removal example

Before you begin

Console

Java

Node.js

Python

REST

Remove with a defined mask area

Imagen 3

Console

Python

Install

REST

curl

PowerShell

Imagen 2

Console

REST

curl

PowerShell

Python

Java

Node.js

Remove with automatic mask detection

Imagen 3

Console

Python

Install

REST

curl

PowerShell

Imagen 2

Console

REST

curl

PowerShell

Python

Limitations

Modified pixels

Removal limitation

What's next

Remove objects from an image using inpaint