Update a schema

You can update the schema for any data containing data that supports a schema, such as structured data , website data with structured data , or other unstructured data with metadata .

You can update the schema in the Google Cloud console or by using the schemas.patch API method. Updating the schema for a website is supported only over the REST API.

To update the schema, you can add new fields, change indexable, searchable, and retrievable annotations for a field, or mark a field as a key property, such as title , uri and description .

Update your schema

You can update your schema in the Google Cloud console or using the API.

Console

To update a schema in the Google Cloud console, follow these steps:

Review the Requirements and limitations section to check that your schema update is valid.
If you are updating field annotations (setting fields as indexable, retrievable, dynamic facetable, searchable, or completable), review Configure field settings for the limitations and requirements of each annotation type.
Check that you have completed data ingestion. Otherwise, the schema might not be available to edit yet.
In the Google Cloud console, go to the AI Applicationspage.

AI Applications
In the navigation menu, click Data Stores.
In the Namecolumn, click the data store with the schema that you want to update.
Click the Schematab to view the schema for your data.

This tab might be empty if this is the first time you're editing the fields.
Click the Editbutton.
Update your schema:
- Map key properties:In the Key propertiescolumn of your schema, select a key property to map a field to. For example, if a field called details always contains the description of a document, map that field to the key property Description.
- Update number of dimensions (Advanced): You can update this setting if you are using custom vector embeddings with Vertex AI Search. See Advanced: Use custom embeddings .
- Update field annotations:To update annotations for a field, select or deselect a field's annotation setting. Available annotations are Retrievable, Indexable, Dynamic Facetable, Searchable, and Completable. Some field settings have limitations. See Configure field settings for descriptions and requirements for each annotation type.
- Add a new field:Adding new fields to your schema before importing new documents with those fields can shorten the time it takes AI Applications to reindex your data after import.
  1. Click Add new fieldsto expand that section.
  2. Click add_box Add nodeand specify settings for the new field.
    
    To indicate an array, set Arrayto Yes. For example, to add an array of strings, set typeto string and Arrayto Yes .
    
    For a website data store index, all fields that you add are arrays by default.
Click Saveto apply your schema changes.

Changing the schema triggers reindexing. For large data stores, reindexing can take hours.

REST

To use the API to update your schema, follow these steps:

Review the Requirements and limitations and the Limitation examples (REST only) sections to check that your schema changes are valid.

To update the schema for data stores with websites or unstructured data with metadata, skip to Step 5 to call the schema.patch method.
If you are updating field annotations (setting fields as indexable, retrievable, dynamic facetable, or searchable), review Configure field settings for the limitations and requirements of each annotation type.
If you are editing an auto-detected schema, make sure that you have completed data ingestion. Otherwise, the schema might not be available to edit yet.
Find your data store ID. If you already have your data store ID, skip to the next step.
1. In the Google Cloud console, go to the AI Applicationspage and in the navigation menu, click Data Stores.
  
  Go to the Data Stores page
2. Click the name of your data store.
3. On the Datapage for your data store, get the data store ID.

Use the schemas.patch API method to provide your new JSON schema as a JSON object.

 curl  
-X  
PATCH  
 \ 
-H  
 "Authorization: Bearer 
 $( 
gcloud  
auth  
print-access-token ) 
 " 
  
 \ 
-H  
 "Content-Type: application/json" 
  
 \ 
 "https://discoveryengine.googleapis.com/v1beta/projects/ PROJECT_ID 
/locations/global/collections/default_collection/dataStores/ DATA_STORE_ID 
/schemas/default_schema" 
  
 \ 
-d  
 '{ 
 "structSchema": JSON_SCHEMA_OBJECT 
 
 }'

Replace the following:

PROJECT_ID : the ID of your Google Cloud project.
DATA_STORE_ID : the ID of the Vertex AI Search data store.

JSON_SCHEMA_OBJECT : your new JSON schema as a JSON object. For example:

 { 
  
 "$schema" 
 : 
  
 "https://json-schema.org/draft/2020-12/schema" 
 , 
  
 "type" 
 : 
  
 "object" 
 , 
  
 "properties" 
 : 
  
 { 
  
 "title" 
 : 
  
 { 
  
 "type" 
 : 
  
 "string" 
 , 
  
 "keyPropertyMapping" 
 : 
  
 "title" 
  
 }, 
  
 "categories" 
 : 
  
 { 
  
 "type" 
 : 
  
 "array" 
 , 
  
 "items" 
 : 
  
 { 
  
 "type" 
 : 
  
 "string" 
 , 
  
 "keyPropertyMapping" 
 : 
  
 "category" 
  
 } 
  
 }, 
  
 "uri" 
 : 
  
 { 
  
 "type" 
 : 
  
 "string" 
 , 
  
 "keyPropertyMapping" 
 : 
  
 "uri" 
  
 } 
  
 } 
 }

Example command and result

curl -X PATCH -H "Authorization: Bearer $(gcloud auth print-access-token)" -H "Content-Type: application/json" "https://discoveryengine.googleapis.com/v1/projects/my-project-123/locations/global/collections/default_collection/dataStores/my-data-store/schemas/default_schema" -d '{
"structSchema": {
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"title": {
"type": "string",
"keyPropertyMapping": "title"
},
"categories": {
"type": "array",
"items": {
"type": "string",
"keyPropertyMapping": "category"
}
},
"uri": {
"type": "string",
"keyPropertyMapping": "uri"
}
}
}
}'
{
"name": "projects/123456/locations/global/collections/default_collection/dataStores/my-data-store/schemas/default_schema/operations/update-schema-10569824819404198922",
"metadata": {
"@type": "type.googleapis.com/google.cloud.discoveryengine.v1.UpdateSchemaMetadata"
}
}

Optional: Review the schema by following the procedure View a schema definition .

C#

For more information, see the AI Applications C# API reference documentation .

To authenticate to AI Applications, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .

  using 
  
  Google.Cloud.DiscoveryEngine.V1 
 
 ; 
 using 
  
  Google.LongRunning 
 
 ; 
 public 
  
 sealed 
  
 partial 
  
 class 
  
 GeneratedSchemaServiceClientSnippets 
 { 
  
 /// <summary>Snippet for UpdateSchema</summary> 
  
 /// <remarks> 
  
 /// This snippet has been automatically generated and should be regarded as a code template only. 
  
 /// It will require modifications to work: 
  
 /// - It may require correct/in-range values for request initialization. 
  
 /// - It may require specifying regional endpoints when creating the service client as shown in 
  
 ///   https://cloud.google.com/dotnet/docs/reference/help/client-configuration#endpoint. 
  
 /// </remarks> 
  
 public 
  
 void 
  
 UpdateSchemaRequestObject 
 () 
  
 { 
  
 // Create client 
  
  SchemaServiceClient 
 
  
 schemaServiceClient 
  
 = 
  
  SchemaServiceClient 
 
 . 
  Create 
 
 (); 
  
 // Initialize request argument(s) 
  
  UpdateSchemaRequest 
 
  
 request 
  
 = 
  
 new 
  
  UpdateSchemaRequest 
 
  
 { 
  
 Schema 
  
 = 
  
 new 
  
  Schema 
 
 (), 
  
 AllowMissing 
  
 = 
  
 false 
 , 
  
 }; 
  
 // Make the request 
  
 Operation<Schema 
 , 
  
 UpdateSchemaMetadata 
>  
 response 
  
 = 
  
 schemaServiceClient 
 . 
  UpdateSchema 
 
 ( 
 request 
 ); 
  
 // Poll until the returned long-running operation is complete 
  
 Operation<Schema 
 , 
  
 UpdateSchemaMetadata 
>  
 completedResponse 
  
 = 
  
 response 
 . 
 PollUntilCompleted 
 (); 
  
 // Retrieve the operation result 
  
  Schema 
 
  
 result 
  
 = 
  
 completedResponse 
 . 
 Result 
 ; 
  
 // Or get the name of the operation 
  
 string 
  
 operationName 
  
 = 
  
 response 
 . 
 Name 
 ; 
  
 // This name can be stored, then the long-running operation retrieved later by name 
  
 Operation<Schema 
 , 
  
 UpdateSchemaMetadata 
>  
 retrievedResponse 
  
 = 
  
 schemaServiceClient 
 . 
  PollOnceUpdateSchema 
 
 ( 
 operationName 
 ); 
  
 // Check if the retrieved long-running operation has completed 
  
 if 
  
 ( 
 retrievedResponse 
 . 
 IsCompleted 
 ) 
  
 { 
  
 // If it has completed, then access the result 
  
  Schema 
 
  
 retrievedResult 
  
 = 
  
 retrievedResponse 
 . 
 Result 
 ; 
  
 } 
  
 } 
 }

Go

For more information, see the AI Applications Go API reference documentation .

To authenticate to AI Applications, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .

  package 
  
 main 
 import 
  
 ( 
  
 "context" 
  
 discoveryengine 
  
 "cloud.google.com/go/discoveryengine/apiv1" 
  
 discoveryenginepb 
  
 "cloud.google.com/go/discoveryengine/apiv1/discoveryenginepb" 
 ) 
 func 
  
 main 
 () 
  
 { 
  
 ctx 
  
 := 
  
 context 
 . 
 Background 
 () 
  
 // This snippet has been automatically generated and should be regarded as a code template only. 
  
 // It will require modifications to work: 
  
 // - It may require correct/in-range values for request initialization. 
  
 // - It may require specifying regional endpoints when creating the service client as shown in: 
  
 //   https://pkg.go.dev/cloud.google.com/go#hdr-Client_Options 
  
 c 
 , 
  
 err 
  
 := 
  
 discoveryengine 
 . 
  NewSchemaClient 
 
 ( 
 ctx 
 ) 
  
 if 
  
 err 
  
 != 
  
 nil 
  
 { 
  
 // TODO: Handle error. 
  
 } 
  
 defer 
  
 c 
 . 
 Close 
 () 
  
 req 
  
 := 
  
& discoveryenginepb 
 . 
 UpdateSchemaRequest 
 { 
  
 // TODO: Fill request struct fields. 
  
 // See https://pkg.go.dev/cloud.google.com/go/discoveryengine/apiv1/discoveryenginepb#UpdateSchemaRequest. 
  
 } 
  
 op 
 , 
  
 err 
  
 := 
  
 c 
 . 
 UpdateSchema 
 ( 
 ctx 
 , 
  
 req 
 ) 
  
 if 
  
 err 
  
 != 
  
 nil 
  
 { 
  
 // TODO: Handle error. 
  
 } 
  
 resp 
 , 
  
 err 
  
 := 
  
 op 
 . 
 Wait 
 ( 
 ctx 
 ) 
  
 if 
  
 err 
  
 != 
  
 nil 
  
 { 
  
 // TODO: Handle error. 
  
 } 
  
 // TODO: Use resp. 
  
 _ 
  
 = 
  
 resp 
 }

Java

For more information, see the AI Applications Java API reference documentation .

To authenticate to AI Applications, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .

  import 
  
 com.google.cloud.discoveryengine.v1. Schema 
 
 ; 
 import 
  
 com.google.cloud.discoveryengine.v1. SchemaServiceClient 
 
 ; 
 import 
  
 com.google.cloud.discoveryengine.v1. UpdateSchemaRequest 
 
 ; 
 public 
  
 class 
 SyncUpdateSchema 
  
 { 
  
 public 
  
 static 
  
 void 
  
 main 
 ( 
 String 
 [] 
  
 args 
 ) 
  
 throws 
  
 Exception 
  
 { 
  
 syncUpdateSchema 
 (); 
  
 } 
  
 public 
  
 static 
  
 void 
  
 syncUpdateSchema 
 () 
  
 throws 
  
 Exception 
  
 { 
  
 // This snippet has been automatically generated and should be regarded as a code template only. 
  
 // It will require modifications to work: 
  
 // - It may require correct/in-range values for request initialization. 
  
 // - It may require specifying regional endpoints when creating the service client as shown in 
  
 // https://cloud.google.com/java/docs/setup#configure_endpoints_for_the_client_library 
  
 try 
  
 ( 
  SchemaServiceClient 
 
  
 schemaServiceClient 
  
 = 
  
  SchemaServiceClient 
 
 . 
 create 
 ()) 
  
 { 
  
  UpdateSchemaRequest 
 
  
 request 
  
 = 
  
  UpdateSchemaRequest 
 
 . 
 newBuilder 
 () 
  
 . 
 setSchema 
 ( 
  Schema 
 
 . 
 newBuilder 
 (). 
 build 
 ()) 
  
 . 
 setAllowMissing 
 ( 
 true 
 ) 
  
 . 
 build 
 (); 
  
  Schema 
 
  
 response 
  
 = 
  
 schemaServiceClient 
 . 
  updateSchemaAsync 
 
 ( 
 request 
 ). 
 get 
 (); 
  
 } 
  
 } 
 }

Python

For more information, see the AI Applications Python API reference documentation .

To authenticate to AI Applications, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .

  # This snippet has been automatically generated and should be regarded as a 
 # code template only. 
 # It will require modifications to work: 
 # - It may require correct/in-range values for request initialization. 
 # - It may require specifying regional endpoints when creating the service 
 #   client as shown in: 
 #   https://googleapis.dev/python/google-api-core/latest/client_options.html 
 from 
  
 google.cloud 
  
 import 
 discoveryengine_v1 
 def 
  
 sample_update_schema 
 (): 
 # Create a client 
 client 
 = 
 discoveryengine_v1 
 . 
  SchemaServiceClient 
 
 () 
 # Initialize request argument(s) 
 request 
 = 
 discoveryengine_v1 
 . 
  UpdateSchemaRequest 
 
 ( 
 ) 
 # Make the request 
 operation 
 = 
 client 
 . 
  update_schema 
 
 ( 
 request 
 = 
 request 
 ) 
 print 
 ( 
 "Waiting for operation to complete..." 
 ) 
 response 
 = 
 operation 
 . 
 result 
 () 
 # Handle the response 
 print 
 ( 
 response 
 )

Ruby

For more information, see the AI Applications Ruby API reference documentation .

To authenticate to AI Applications, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .

  require 
  
 "google/cloud/discovery_engine/v1" 
 ## 
 # Snippet for the update_schema call in the SchemaService service 
 # 
 # This snippet has been automatically generated and should be regarded as a code 
 # template only. It will require modifications to work: 
 # - It may require correct/in-range values for request initialization. 
 # - It may require specifying regional endpoints when creating the service 
 # client as shown in https://cloud.google.com/ruby/docs/reference. 
 # 
 # This is an auto-generated example demonstrating basic usage of 
 # Google::Cloud::DiscoveryEngine::V1::SchemaService::Client#update_schema. 
 # 
 def 
  
 update_schema 
  
 # Create a client object. The client can be reused for multiple calls. 
  
 client 
  
 = 
  
 Google 
 :: 
 Cloud 
 :: 
 DiscoveryEngine 
 :: 
 V1 
 :: 
 SchemaService 
 :: 
 Client 
 . 
 new 
  
 # Create a request. To set request fields, pass in keyword arguments. 
  
 request 
  
 = 
  
 Google 
 :: 
 Cloud 
 :: 
 DiscoveryEngine 
 :: 
 V1 
 :: 
 UpdateSchemaRequest 
 . 
 new 
  
 # Call the update_schema method. 
  
 result 
  
 = 
  
 client 
 . 
 update_schema 
  
 request 
  
 # The returned object is of type Gapic::Operation. You can use it to 
  
 # check the status of an operation, cancel it, or wait for results. 
  
 # Here is how to wait for a response. 
  
 result 
 . 
 wait_until_done! 
  
 timeout 
 : 
  
 60 
  
 if 
  
 result 
 . 
 response? 
  
 p 
  
 result 
 . 
 response 
  
 else 
  
 puts 
  
 "No response received." 
  
 end 
 end

Requirements and limitations

When updating a schema, be sure that the new schema is backward compatible with the schema you are updating. To update a schema with a new schema that is not backward compatible, you need to delete all the documents in the data store, delete the schema, and create a new schema.

Updating a schema triggers re-indexing of all documents. This can take time and incur additional costs:

Time.Reindexing a large data store can take hours or days.
Expense.Reindexing can incur costs, depending on the parser. For example, reindexing data stores that use the OCR parser or the layout parser both incur costs. For more information, see Document AI feature pricing .

Schema updates don't support the following:

Changing a field type.A schema update doesn't support changing the type of the field. For example, a field mapped to integer can't be changed to string .
Removing a field.After it's defined, a field can't be removed. You can continue adding new fields but you can't remove an existing field.

Limitation examples (REST only)

This section shows examples of valid and invalid types of schema updates. These examples use the following example JSON schema:

  { 
  
 "$schema" 
 : 
  
 "https://json-schema.org/draft/2020-12/schema" 
 , 
  
 "type" 
 : 
  
 "object" 
 , 
  
 "properties" 
 : 
  
 { 
  
 "title" 
 : 
  
 { 
  
 "type" 
 : 
  
 "string" 
  
 }, 
  
 "description" 
 : 
  
 { 
  
 "type" 
 : 
  
 "string" 
 , 
  
 "keyPropertyMapping" 
 : 
  
 "description" 
  
 }, 
  
 "categories" 
 : 
  
 { 
  
 "type" 
 : 
  
 "array" 
 , 
  
 "items" 
 : 
  
 { 
  
 "type" 
 : 
  
 "string" 
 , 
  
 "keyPropertyMapping" 
 : 
  
 "category" 
  
 } 
  
 } 
  
 } 
 }

Examples of supported updates

The following updates to the example schema are supported.

Adding a field.In this example, the field properties.uri has been added to the schema.

  { 
  
 "$schema" 
 : 
  
 "https://json-schema.org/draft/2020-12/schema" 
 , 
  
 "type" 
 : 
  
 "object" 
 , 
  
 "properties" 
 : 
  
 { 
  
 "title" 
 : 
  
 { 
  
 "type" 
 : 
  
 "string" 
  
 }, 
  
 "description" 
 : 
  
 { 
  
 "type" 
 : 
  
 "string" 
 , 
  
 "keyPropertyMapping" 
 : 
  
 "description" 
  
 }, 
  
 "uri" 
 : 
  
 { 
  
 // Added field. This is supported. 
  
 "type" 
 : 
  
 "string" 
 , 
  
 "keyPropertyMapping" 
 : 
  
 "uri" 
  
 }, 
  
 "categories" 
 : 
  
 { 
  
 "type" 
 : 
  
 "array" 
 , 
  
 "items" 
 : 
  
 { 
  
 "type" 
 : 
  
 "string" 
 , 
  
 "keyPropertyMapping" 
 : 
  
 "category" 
  
 } 
  
 } 
  
 } 
 }

Adding or removing key property annotations for title , description or uri .In this example, keyPropertyMapping has been added to the title field.

  { 
  
 "$schema" 
 : 
  
 "https://json-schema.org/draft/2020-12/schema" 
 , 
  
 "type" 
 : 
  
 "object" 
 , 
  
 "properties" 
 : 
  
 { 
  
 "title" 
 : 
  
 { 
  
 "type" 
 : 
  
 "string" 
 , 
  
 "keyPropertyMapping" 
 : 
  
 "title" 
  
 // Added "keyPropertyMapping". This is supported. 
  
 }, 
  
 "description" 
 : 
  
 { 
  
 "type" 
 : 
  
 "string" 
 , 
  
 "keyPropertyMapping" 
 : 
  
 "description" 
  
 }, 
  
 "categories" 
 : 
  
 { 
  
 "type" 
 : 
  
 "array" 
 , 
  
 "items" 
 : 
  
 { 
  
 "type" 
 : 
  
 "string" 
 , 
  
 "keyPropertyMapping" 
 : 
  
 "category" 
  
 } 
  
 } 
  
 } 
 }

Examples of invalid schema updates

The following updates to the example schema aren't supported.

Changing a field type.In this example, the title field's type has been changed from string to number. This is not supported.

   
 { 
  
 "$schema" 
 : 
  
 "https://json-schema.org/draft/2020-12/schema" 
 , 
  
 "type" 
 : 
  
 "object" 
 , 
  
 "properties" 
 : 
  
 { 
  
 "title" 
 : 
  
 { 
  
 "type" 
 : 
  
 "number" 
  
 // Changed from string. Not allowed. 
  
 }, 
  
 "description" 
 : 
  
 { 
  
 "type" 
 : 
  
 "string" 
 , 
  
 "keyPropertyMapping" 
 : 
  
 "description" 
  
 }, 
  
 "categories" 
 : 
  
 { 
  
 "type" 
 : 
  
 "array" 
 , 
  
 "items" 
 : 
  
 { 
  
 "type" 
 : 
  
 "string" 
 , 
  
 "keyPropertyMapping" 
 : 
  
 "category" 
  
 } 
  
 } 
  
 } 
  
 }

Removing a field.In this example, the title field has been removed. This is not supported.

   
 { 
  
 "$schema" 
 : 
  
 "https://json-schema.org/draft/2020-12/schema" 
 , 
  
 "type" 
 : 
  
 "object" 
 , 
  
 "properties" 
 : 
  
 { 
  
 // "title" is removed. Not allowed. 
  
 "description" 
 : 
  
 { 
  
 "type" 
 : 
  
 "string" 
 , 
  
 "keyPropertyMapping" 
 : 
  
 "description" 
  
 }, 
  
 "uri" 
 : 
  
 { 
  
 "type" 
 : 
  
 "string" 
 , 
  
 "keyPropertyMapping" 
 : 
  
 "uri" 
  
 }, 
  
 "categories" 
 : 
  
 { 
  
 "type" 
 : 
  
 "array" 
 , 
  
 "items" 
 : 
  
 { 
  
 "type" 
 : 
  
 "string" 
 , 
  
 "keyPropertyMapping" 
 : 
  
 "category" 
  
 } 
  
 } 
  
 } 
  
 }

Update a schema Stay organized with collections Save and categorize content based on your preferences.

Update your schema

Console

REST

Example command and result

C#

Go

Java

Python

Ruby

Requirements and limitations

Limitation examples (REST only)

Examples of supported updates

Examples of invalid schema updates

What's next

Update a schema