Process documents with the ML.PROCESS_DOCUMENT function

This document describes how to use the ML.PROCESS_DOCUMENT function with a remote model to extract useful insights from documents in an object table .

Supported locations

You must create the remote model used in this procedure in either the US or EU multi-region . You must run the ML.PROCESS_DOCUMENT function in the same region as the remote model.

Required roles

To create a remote model and process documents, you need the following Identity and Access Management (IAM) roles at the project level:

Create a document processor: Document AI Editor ( roles/documentai.editor )
Create and use BigQuery datasets, tables, and models: BigQuery Data Editor ( roles/bigquery.dataEditor )
Create, delegate, and use BigQuery connections: BigQuery Connections Admin ( roles/bigquery.connectionsAdmin )

If you don't have a default connection configured, you can create and set one as part of running the CREATE MODEL statement. To do so, you must have BigQuery Admin ( roles/bigquery.admin ) on your project. For more information, see Configure the default connection .
Grant permissions to the connection's service account: Project IAM Admin ( roles/resourcemanager.projectIamAdmin )
Create BigQuery jobs: BigQuery Job User ( roles/bigquery.jobUser )

These predefined roles contain the permissions required to perform the tasks in this document. To see the exact permissions that are required, expand the Required permissionssection:

Required permissions

Create a dataset: bigquery.datasets.create
Create, delegate, and use a connection: bigquery.connections.*
Set service account permissions: resourcemanager.projects.getIamPolicy and resourcemanager.projects.setIamPolicy
Create a model and run inference:
- bigquery.jobs.create
- bigquery.models.create
- bigquery.models.getData
- bigquery.models.updateData
- bigquery.models.updateMetadata
Create an object table: bigquery.tables.create and bigquery.tables.update
Create a document processor:
- documentai.processors.create
- documentai.processors.update
- documentai.processors.delete

You might also be able to get these permissions with custom roles or other predefined roles .

Before you begin

Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Go to project selector

Verify that billing is enabled for your Google Cloud project .

Enable the BigQuery, BigQuery Connection API, and Document AI APIs.

Enable the APIs

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Go to project selector

Verify that billing is enabled for your Google Cloud project .

Enable the BigQuery, BigQuery Connection API, and Document AI APIs.

Enable the APIs

Create a processor

Create a processor in Document AI to process the documents. The processor must be of a supported type .

Create a dataset

You must create the dataset, the connection and the document processor in the same region.

Create a BigQuery dataset to contain your resources:

Console

In the Google Cloud console, go to the BigQuerypage.

Go to the BigQuery page
In the Explorerpane, click your project name.
Click View actions > Create dataset.
On the Create datasetpage, do the following:
- For Dataset ID, type a name for the dataset.
- For Location type, select a location for the dataset.
- Click Create dataset.

bq

To create a new dataset, use the bq mk command with the --location flag:
```
bq --location= LOCATION 
mk -d DATASET_ID 
```
Replace the following:
- LOCATION : the dataset's location .
- DATASET_ID is the ID of the dataset that you're creating.
Confirm that the dataset was created:
```
bq  
ls
```

Create a connection

You can skip this step if you either have a default connection configured, or you have the BigQuery Admin role.

Create a Cloud resource connection for the remote model to use, and get the connection's service account. Create the connection in the same location as the dataset that you created in the previous step.

Select one of the following options:

Console

Go to the BigQuerypage.

Go to BigQuery
In the Explorerpane, click Add data:

The Add datadialog opens.
In the Filter Bypane, in the Data Source Typesection, select Business Applications.

Alternatively, in the Search for data sourcesfield, you can enter Vertex AI .
In the Featured data sourcessection, click Vertex AI.
Click the Vertex AI Models: BigQuery Federationsolution card.
In the Connection typelist, select Vertex AI remote models, remote functions and BigLake (Cloud Resource).
In the Connection IDfield, enter a name for your connection.
Click Create connection.
Click Go to connection.
In the Connection infopane, copy the service account ID for use in a later step.

bq

In a command-line environment, create a connection:
```
bq  
mk  
--connection  
--location = 
 REGION 
  
--project_id = 
 PROJECT_ID 
  
 \ 
  
--connection_type = 
CLOUD_RESOURCE  
 CONNECTION_ID 
```
The --project_id parameter overrides the default project.

Replace the following:
- REGION : your connection region
- PROJECT_ID : your Google Cloud project ID
- CONNECTION_ID : an ID for your connection
When you create a connection resource, BigQuery creates a unique system service account and associates it with the connection.

Troubleshooting: If you get the following connection error, update the Google Cloud SDK :
```
Flags parsing error: flag --connection_type=CLOUD_RESOURCE: value should be one of...
```

Retrieve and copy the service account ID for use in a later step:

bq  
show  
--connection  
 PROJECT_ID 
. REGION 
. CONNECTION_ID

The output is similar to the following:

name                          properties
1234. REGION 
. CONNECTION_ID 
{"serviceAccountId": "connection-1234-9u56h9@gcp-sa-bigquery-condel.iam.gserviceaccount.com"}

Terraform

Use the google_bigquery_connection resource.

To authenticate to BigQuery, set up Application Default Credentials. For more information, see Set up authentication for client libraries .

The following example creates a Cloud resource connection named my_cloud_resource_connection in the US region:

 # This queries the provider for project information.
data "google_project" "default" {}

# This creates a cloud resource connection in the US region named my_cloud_resource_connection.
# Note: The cloud resource nested object has only one output field - serviceAccountId.
resource "google_bigquery_connection" "default" {
  connection_id = "my_cloud_resource_connection"
  project       = data.google_project.default.project_id
  location      = "US"
  cloud_resource {}
}

To apply your Terraform configuration in a Google Cloud project, complete the steps in the following sections.

Prepare Cloud Shell

Launch Cloud Shell .
Set the default Google Cloud project where you want to apply your Terraform configurations.

You only need to run this command once per project, and you can run it in any directory.
```
export GOOGLE_CLOUD_PROJECT= PROJECT_ID 
```
Environment variables are overridden if you set explicit values in the Terraform configuration file.

Prepare the directory

Each Terraform configuration file must have its own directory (also called a root module ).

In Cloud Shell , create a directory and a new file within that directory. The filename must have the .tf extension—for example main.tf . In this tutorial, the file is referred to as main.tf .
```
mkdir DIRECTORY 
&& cd DIRECTORY 
&& touch main.tf
```
If you are following a tutorial, you can copy the sample code in each section or step.

Copy the sample code into the newly created main.tf .

Optionally, copy the code from GitHub. This is recommended when the Terraform snippet is part of an end-to-end solution.
Review and modify the sample parameters to apply to your environment.
Save your changes.
Initialize Terraform. You only need to do this once per directory.
```
terraform init
```
Optionally, to use the latest Google provider version, include the -upgrade option:
```
terraform init -upgrade
```

Apply the changes

Review the configuration and verify that the resources that Terraform is going to create or update match your expectations:
```
terraform plan
```
Make corrections to the configuration as necessary.
Apply the Terraform configuration by running the following command and entering yes at the prompt:
```
terraform apply
```
Wait until Terraform displays the "Apply complete!" message.
Open your Google Cloud project to view the results. In the Google Cloud console, navigate to your resources in the UI to make sure that Terraform has created or updated them.

Grant access to the service account

Select one of the following options:

Console

Go to the IAM & Adminpage.

Go to IAM & Admin
Click Grant Access.

The Add principalsdialog opens.
In the New principalsfield, enter the service account ID that you copied earlier.
In the Select a rolefield, select Document AI, and then select Document AI Viewer.
Click Add another role.
In the Select a rolefield, select Cloud Storage, and then select Storage Object Viewer.
Click Save.

gcloud

Use the gcloud projects add-iam-policy-binding command :

gcloud projects add-iam-policy-binding ' PROJECT_NUMBER 
' --member='serviceAccount: MEMBER 
' --role='roles/documentai.viewer' --condition=None
gcloud projects add-iam-policy-binding ' PROJECT_NUMBER 
' --member='serviceAccount: MEMBER 
' --role='roles/storage.objectViewer' --condition=None

Replace the following:

PROJECT_NUMBER : your project number.
MEMBER : the service account ID that you copied earlier.

Failure to grant the permission results in a Permission denied error.

Create a model

Create a remote model with a REMOTE_SERVICE_TYPE of CLOUD_AI_DOCUMENT_V1 :

 CREATE 
  
 OR 
  
 REPLACE 
  
 MODEL 
 ` PROJECT_ID 
. DATASET_ID 
. MODEL_NAME 
` 
 REMOTE 
  
 WITH 
  
 CONNECTION 
  
 { 
 DEFAULT 
  
 | 
  
 ` PROJECT_ID 
. REGION 
. CONNECTION_ID 
` 
 } 
 OPTIONS 
  
 ( 
  
 REMOTE_SERVICE_TYPE 
  
 = 
  
 'CLOUD_AI_DOCUMENT_V1' 
 , 
  
 DOCUMENT_PROCESSOR 
  
 = 
  
 ' PROCESSOR_ID 
' 
 );

Replace the following:

PROJECT_ID : your project ID.
DATASET_ID : the ID of the dataset to contain the model.
MODEL_NAME : the name of the model.
REGION : the region used by the connection.
CONNECTION_ID : the connection ID—for example, myconnection .
When you view the connection details in the Google Cloud console, the connection ID is the value in the last section of the fully qualified connection ID that is shown in Connection ID—for example projects/myproject/locations/connection_location/connections/ myconnection .
PROCESSOR_ID : the document processor ID. To find this value, view the processor details , and then look at the IDrow in the Basic Informationsection.

To see the model output columns, click Go to modelin the query result after the model is created. The output columns are shown in the Labelssection of the Schematab.

Create an object table

Create an object table over a set of documents in Cloud Storage. The documents in the object table must be of a supported type .

Process documents

Process all the documents with the ML.PROCESS_DOCUMENT :

 SELECT 
  
 * 
 FROM 
  
 ML 
 . 
 PROCESS_DOCUMENT 
 ( 
  
 MODEL 
  
 ` PROJECT_ID 
. DATASET_ID 
. MODEL_NAME 
` 
 , 
  
 TABLE 
  
 ` PROJECT_ID 
. DATASET_ID 
. OBJECT_TABLE_NAME 
` 
  
 [ 
 , 
  
 PROCESS_OPTIONS 
  
 => 
  
 ( 
  
 JSON 
  
 ' PROCESS_OPTIONS 
' 
 ) 
 ] 
 );

Replace the following:

PROJECT_ID : your project ID.
DATASET_ID : the ID of the dataset that contains the model.
MODEL_NAME : the name of the model.
OBJECT_TABLE_NAME : the name of the object table that contains the URIs of the documents to process.
PROCESS_OPTIONS : the json configuration that specifies how to process documents. For example, you use this to specify document chunking for the layout parser

Alternatively, process some of the documents with the ML.PROCESS_DOCUMENT :

 SELECT 
  
 * 
 FROM 
  
 ML 
 . 
 PROCESS_DOCUMENT 
 ( 
  
 MODEL 
  
 ` PROJECT_ID 
. DATASET_ID 
. MODEL_NAME 
` 
 , 
  
 ( 
 SELECT 
  
 * 
  
 FROM 
  
 ` PROJECT_ID 
. DATASET_ID 
. OBJECT_TABLE_NAME 
` 
  
 WHERE 
  
  FILTERS 
 
  
 LIMIT 
  
  NUM_DOCUMENTS 
 
  
 ) 
  
 [ 
 , 
  
 PROCESS_OPTIONS 
  
 => 
  
 ( 
  
 JSON 
  
 ' PROCESS_OPTIONS 
' 
 ) 
 ] 
 );

Replace the following:

PROJECT_ID : your project ID.
DATASET_ID : the ID of the dataset that contains the model.
MODEL_NAME : the name of the model.
OBJECT_TABLE_NAME : the name of the object table that contains the URIs of the documents to process.
FILTERS : conditions to filter out the documents you want to process on the object table columns.
NUM_DOCUMENTS : the max number of documents you want to process.
PROCESS_OPTIONS : the json configuration that defines the configuration, such as chunking config for layout parser

Examples

Example 1

The following example uses the expense parser to process the documents represented by the documents table:

 SELECT 
  
 * 
 FROM 
  
 ML 
 . 
 PROCESS_DOCUMENT 
 ( 
  
 MODEL 
  
 `myproject.mydataset.expense_parser` 
 , 
  
 TABLE 
  
 `myproject.mydataset.documents` 
 );

This query returns the parsed expense reports, including the currency, total amount, receipt date, and line items on the expense reports. The ml_process_document_result column contains the raw output of the expense parser, and the ml_process_document_status column contains any errors returned by the document processing.

Example 2

The following example shows how to filter the object table to choose which documents to process, and then write the results to a new table:

 CREATE 
  
 TABLE 
  
 `myproject.mydataset.expense_details` 
 AS 
 SELECT 
  
 uri 
 , 
  
 content_type 
 , 
  
 receipt_date 
 , 
  
 purchase_time 
 , 
  
 total_amount 
 , 
  
 currency 
 FROM 
  
 ML 
 . 
 PROCESS_DOCUMENT 
 ( 
  
 MODEL 
  
 `myproject.mydataset.expense_parser` 
 , 
  
 ( 
 SELECT 
  
 * 
  
 FROM 
  
 `myproject.mydataset.expense_reports` 
  
 WHERE 
  
 uri 
  
 LIKE 
  
 '%restaurant%' 
 ));

What's next

For more information about model inference in BigQuery ML, see Model inference overview .
For more information about using Cloud AI APIs to perform AI tasks, see AI application overview .
For more information about supported SQL statements and functions for generative AI models, see End-to-end user journeys for generative AI models .