Create data agents

This document describes how to create, edit, manage, and delete data agents in BigQuery.

In BigQuery, you can have conversations with data agents to ask questions about BigQuery data using natural language. Data agents contain table metadata and use-case-specific query processing instructions that define the best way to answer user questions about a set of knowledge sources, such as tables, views, or user-defined functions (UDFs) that you select.

Before you begin

Verify that billing is enabled for your Google Cloud project .
Enable the BigQuery, Gemini Data Analytics, and Gemini for Google Cloud APIs.

Roles required to enable APIs

To enable APIs, you need the Service Usage Admin IAM role ( roles/serviceusage.serviceUsageAdmin ), which contains the serviceusage.services.enable permission. Learn how to grant roles .

Enable the APIs

Required roles

To work with data agents, you must have one of the following Conversational Analytics API Identity and Access Management roles :

Create, edit, share, and delete all data agents in the project: Gemini Data Analytics Data Agent Owner ( roles/geminidataanalytics.dataAgentOwner ) on the project.
Create, edit, share, and delete your own data agents in the project: Gemini Data Analytics Data Agent Creator ( roles/geminidataanalytics.dataAgentCreator ) on the project. This role automatically grants you the Gemini Data Analytics Data Agent Owner role on the data agents that you create.
View and edit all data agents in the project: Gemini Data Analytics Data Agent Editor ( roles/geminidataanalytics.dataAgentEditor ) at the project level.
View all data agents in the project: Gemini Data Analytics Data Agent Viewer ( roles/geminidataanalytics.dataAgentViewer ).

Additionally, you must have the following roles to create or edit a data agent:

Gemini Data Analytics Stateless Chat User ( roles/geminidataanalytics.dataAgentStatelessUser ).
BigQuery Data Viewer ( roles/bigquery.dataViewer ) on any table that the data agent uses as a knowledge source.
Data Catalog Viewer ( roles/datacatalog.catalogViewer ) on the project
If a data table uses column-level access control , Fine-Grained Reader ( roles/datacatalog.categoryFineGrainedReader ) on the appropriate policy tag. For more information, see Roles used with column-level access control .
If a data table uses row-level access control , you must have the row-level access policy on that table. For more information, see Create or update row-level access policies .
If a data table uses data masking , Masked Reader ( roles/bigquerydatapolicy.maskedReader ) on the appropriate data policy. For more information, see Roles for querying masked data .

To work with BigQuery resources, such as viewing tables or running queries, see BigQuery roles .

Best practices

Conversational analytics automatically runs queries on your behalf to answer your questions. Consider the following factors that might increase query cost:

Large table sizes
Use of data joins in queries
Frequent calls to AI functions within queries

Generate insights

You can optionally generate data insights in Knowledge Catalog for any table that you want to use as a knowledge source.

Generated insights provide table metadata that the data agent can use to help generate responses to your questions.

If you don't generate insights beforehand, the system automatically generates them when you select a table as a knowledge source while creating a data agent.

Work with the sample data agent

If you're unfamiliar with configuring agents for conversational analytics, you can optionally view the predefined sample agent generated for every Google Cloud project. You can chat with it and view its parameters to see how it was created, but you can't modify it.

To view the sample agent, do the following:

In the Google Cloud console, go to the BigQuery Agentspage.

Go to Agents
Select the Agent catalogtab.
Under the section Sample agents by Google, click the sample agent card.

Create a data agent

The following sections describe how to create a data agent.

After you create an agent, you can edit its settings .

Initial steps

In the Google Cloud console, go to the BigQuery Agentspage.

Go to Agents
Select the Agent catalogtab.
Click New agent. The New agentpage opens.
In the Editorsection, in the Agent namefield, type a descriptive name for the data agent—for example, Q4 sales data or User activity logs .
In the Agent descriptionfield, type a description of the data agent. A good description explains what the agent does, what data it uses, and helps you know when this is the right data agent to chat with—for example, Ask questions about customer orders and revenue .
In the Knowledge sourcessection, click Add source. The Add knowledge sourcepage opens.
In the Recentssection, select any tables, views, or UDFs that you want to use as knowledge sources. UDFs are prefixed with an 'fx' indicator in the Google Cloud console.
To view additional knowledge sources, select Show more.
Optional: Add a knowledge source that isn't listed in the Recentssection:
1. In the Searchsection, type the source name into the Search for tablesfield, and then press Enter. The source name doesn't need to be exact.
2. In the Search resultssection, select one or more sources.
Click Add. The new agent page reopens.

Customize table and field descriptions

To improve data agent accuracy, you can optionally provide additional table metadata. Only the data agent uses this metadata, and it doesn't affect the source table.

Follow these best practices when you create a table and field descriptions:

Use these descriptions as a guide to understand how the data agent understands the schema. If the descriptions suggested by the agent are correct, you can accept them.
If the data agent doesn't show an understanding of the schema after you configure these descriptions, then manually adjust the descriptions to provide the correct information.

Follow these steps to configure table and field descriptions:

In the Knowledge sourcessection, click the Customizelink for a table.
Create a table description. You can type a description in the Table Descriptionfield or accept the suggestion from Gemini.
In the Fieldssection, review the Gemini-suggested field descriptions.
Select any field descriptions that you want to accept and click Accept suggestions. Select any descriptions that you want to reject and click Reject suggestions.
Manually edit any field description by clicking Editnext to the field. The Edit fieldpane opens.
1. In the Descriptionfield, type a field description.
2. To save the field description, click Update.
To save the description and field updates, click Update. The new agent page reopens.
Repeat these steps for each table that needs customization.

Create agent instructions

The agent should understand context for user questions without needing any custom instructions. Create custom instructions for the agent only if you need to change the agent's behavior or improve the context in ways that aren't already supported by other context features—for example, custom table and field metadata, or verified queries.

In the Instructionssection, type instructions for the data agent in the Agent instructionsfield. Because the data agent uses these instructions to understand the context for user questions and to provide answers, make the instructions as clear as possible.

If you don't get a satisfactory answer from the agent, then add structured context such as descriptions, examples or glossary terms. If you still don't get a satisfactory answer, add custom instructions like the examples in the following table.

For even more examples of instructions, click Show examples.

Information type

Description

Examples

Key fields

The most important fields for analysis.

"The most important fields in this table are: Customer ID, Product ID, Order Date."

Filtering and grouping

Fields that the agent should use to filter and group data.

"When a question is about a timeline or 'over time,' always use the order_created_date column." "When someone says 'by product,' group by the product_category column."

Default filtering

Fields to filter on by default.

"Unless stated otherwise, always filter the data on order_status = 'Complete'."

Synonyms and business terms

Alternative terms for key fields.

"If someone asks about 'Revenue' or 'Sales', use the total_sale_amount column." "We consider 'loyal' customers to be those with purchase_count > 5."

Excluded fields

Fields that the data agent should avoid using.

"Never use these fields: Transaction Date Derived, City Derived."

Join relationships

How two or more tables are related to each other, and which columns are used to join them. The agent must use standard SQL JOINs on column pairs to combine data. See the example column.

Customer Activity

order_items.user_id = users.id
(to link sales to customers)
events.user_id = users.id
(to link website activity to logged-in customers)

Create verified queries

An agent uses verified queries in two ways:

If an agent can use a verified query to answer a question that you ask it, to ensure a trustworthy answer, the agent invokes the query exactly as written.
If the agent can't use the verified query to answer a question, it still uses the query as a reference to understand the data and the best practices for querying it.

You can select verified queries from a list generated by the system, or create your own.

To create a verified query for the data agent, formerly known as a golden query , do the following:

Select one or more Gemini-suggested verified queries:
1. In the Verified Queriessection, click Review suggestions. The Review suggested verified queriespage opens.
2. Review the suggested verified queries. Select any that apply to your use case.
3. Click Add. The new agent page reopens.
To create your own verified query, click Add query. The Add verified querypage opens.
1. In the Questionfield, type the user question that the verified query answers.
2. Click Generate SQLto have Gemini generate a verified query that corresponds to the user question that you specified.
3. Modify the verified query if you choose.
4. Click Runand verify that the query returns the results that you expect.
5. Click Add. The new agent page reopens.
Repeat these steps as needed to create additional verified queries.

Create parameterized verified queries

Parameterized verified queries extract values from a user's question for the conversational analytics agent and provide tailored results.

Analysts and builders can create reusable SQL templates that contain placeholders for these values. The templates dynamically substitute parameters at runtime to answer a wider range of user questions than do regular verified queries.

When a user asks a question that matches the template's pattern, the conversational analytics agent extracts parameter values from the question— for example, product name, region, and date. It then injects these values into the parameterized SQL query. Matching responses from the query template appear as verified.

Parameterized verified queries significantly enhance the power and flexibility of verified queries. They ensure consistent, trusted answers across various inputs and reduce the number of individual queries to maintain. For more information, see the following:

To create a parameterized verified query, see Create parameterized verified queries .
To learn about parameterized queries in general, see Run parameterized queries .

How it works

An expert, such as a data analyst, defines a verified query by using a template question—for example, "What were the sales for @product in @region ?" Then, the expert creates or modifies the verified query using SQL parameters, as the following example shows:

  SELECT 
  
 * 
  
 FROM 
  
 sales 
  
 WHERE 
  
 region 
  
 = 
  
 @ 
 region 
  
 AND 
  
 product 
  
 = 
  
 @ 
 product

After the verified query is saved, a user can ask the conversational analytics agent a question in natural language; for example, "What were the sales for laptops in North America?"

To answer the user's question, the conversational agent performs the following steps:

Matches the question to the pattern associated with the parameterized verified query. The agent uses natural language understanding (NLU) to identify and extract the values for @region (North America) and @product (laptops) from the user's question.
Substitutes the extracted values into the @region and @product placeholders in the SQL template.
Runs the complete SQL query; for example, SELECT * FROM sales WHERE region = 'North America' AND product = 'Laptops' .
Returns the results to the user. Matches are always marked as verified.

Tips for creating effective parameterized queries

Use clear parameter names. Use descriptive names for parameters—for example, @start_date instead of @d1 .
Create detailed parameter descriptions. The large language model (LLM) for conversational analytics uses parameter descriptions to identify the parameters and their values from user questions. For example, num_enrollments is an effective parameter name, but number of student enrollments from ages 5-14 is a parameter description that gives more context about the query.
Ensure consistent data typing. Ensure that the data types expected by the SQL query match the data types likely to be extracted from the user's question.
Provide a well-defined scope. Create templates for common and important question patterns where the query construction is complex, or the logic is unintuitive. Doing so helps the LLM return optimal results.
Test thoroughly. Test with various natural language phrasings to ensure that parameters are extracted correctly.

Create a parameterized verified query

You can select verified queries from a list generated by the system, or you can create your own.

Before you create or modify a query, draft the query, considering the natural language pattern and your question. For example, if you ask "Do we know the total stock for organic bananas in the US-EAST warehouse?," you can rewrite the question as the parameterized verified query as "What is the total stock for @product in the @region warehouse?" The conversational analytics agent turns this question into a SQL query that you update with default values.

To create a parameterized verified query for a data agent, you can either create a new query when you create a new agent, or you can edit an existing verified query for a new or existing agent.

The following instructions use a sample verified query to configure with parameters.

Select an existing Gemini-suggested verified query

In the Verified Queriessection of a new or existing agent, click Review suggestions. The Review suggested verified queriespage opens.
Select the checkbox next to a suggested verified query.
In the query window, click Show moreto expand the query description.
To open the existing query, click Edit.
To finish configuring the query, see Configure the parameters for the verified query .

Create an agent and then create a verified query

See Initial steps and continue through the rest of the configuration steps to Verified queries.
In the Google Cloud console, in the Verified queriessection of the new agent, click Add query. The Add verified querypage opens.
To finish configuring the query, see Configure the parameters for the verified query .

Configure the parameters for the verified query

In the Questionfield, type the user question answered by the verified query.
To specify parameters, use the @ symbol followed by a parameter name. This syntax identifies a placeholder that ingests a value from the user question. Use a natural language question that shows how the parameters will be used in user questions. For example: "What is the total stock for @product in the @region warehouse?"

Click Generate SQL. The SQL looks like the following example:

   
 SELECT 
  
 SUM 
 ( 
 stock 
 ) 
  
 AS 
  
 total_stock 
  
 FROM 
  
 inventory 
  
 WHERE 
  
 product_id 
  
 = 
  
 @ 
 product 
  
 AND 
  
 region 
  
 = 
  
 @ 
 region 
 ;

To add default values to the placeholders in the query, click Manage query parameters, and then click Add query parameter.
For the first parameter, four fields appear for Name, Type, Value, and Description.
- For Name, copy @product from your question and paste it into this field.
- For Type, select STRING.
- For Value, enter organic bananas .
- For Description, enter as specific a description as possible. For example, a product located in a regional warehouse.
For the second parameter, click Add query parameter.
- For Name, copy @region from your question and paste it into this field.
- For Type, select STRING.
- For Value, enter US-EAST .
- For Description, enter as specific a description as possible—for example, a regional warehouse where products are located.
When you have filled out fields for both parameters, click Save.

Test the parameterized verified query

Click Runand verify that the query returns the results that you expect.
To test the query for users in a later screen, copy the entire question field.
Click Saveto exit the Add queryscreen and return to the agent Editpage.
On the agent Editpage, paste the question field, that you copied previously, into the Previewwindow.
1. Replace the @product variable with organic bananas .
2. Replace the @region variable with US-EAST .
Press enter. Check the result. In this case, a valid answer is the total stock number of the bananas in the US-EAST region—for example, 1,000.
To create or edit additional verified queries, repeat these steps as needed.

Now that you have saved the query, a user can ask the question "Do we know the total stock for organic bananas in the US-EAST warehouse?" Conversational analytics then does the following:

Matches this question to the pattern.
Extracts the @product parameter as @product = "organic bananas" and the @region parameter = "US-EAST" from the question.
Executes the query: SELECT SUM(stock) AS total_stock FROM inventory WHERE product_id = 'organic bananas' AND region = 'US-EAST';
Returns the calculated total_stock .

Create or review glossary terms

You can create BigQuery custom glossary terms local to an agent, or review business glossary terms imported from Knowledge Catalog that apply to the knowledge sources that you selected for an agent.

Because business glossary terms from Knowledge Catalog apply globally to BigQuery resources, if you use Knowledge Catalog, create and manage business glossary terms in Knowledge Catalog instead of for individual agents.
If you need to modify business glossary terms imported from Knowledge Catalog, you must edit them in Knowledge Catalog and return to conversational analytics to see them.
BigQuery custom glossary terms stay in BigQuery. They don't appear in Knowledge Catalog.
If you're not using Knowledge Catalog, you can create BigQuery custom glossary terms for terms that you need to define for a specific agent.

Follow these steps to create custom glossary terms for an agent:

In the Glossarysection of the agent Editorpage, click Add term.
In the Custom termssection, you can edit or delete any existing custom terms.
To create one or more new terms, click Create term.
1. Enter a Term, a Definition, and one or more Synonymsseparated by a comma.
2. To create the term, click Add.
3. If you want to delete the new term, click Delete.
To create more custom terms, repeat these steps.

Follow these steps to view business glossary terms imported from Knowledge Catalog:

In the Glossarysection of the agent Editorpage, click Add term.
Navigate to the page section called Imported from Knowledge Catalog.
To modify imported terms in Knowledge Catalog, you must click the link "Go to Knowledge Catalog glossaries."
After you've modified the terms in Knowledge Catalog, you can return to the agent Editorpage to view the modified terms.

Configure settings

In the Settingssection, you can configure the following optional settings:

Create labels to help you organize your Google Cloud resources. Labels are key-value pairs that let you group related objects together or with other Google Cloud resources.
1. In the Settingssection, click Manage labels.
2. Click Add label.
3. In the keyand valuefields, enter your key-value pair for the label.
4. If you want to add more labels, click Add labelagain.
5. To delete a label, click Delete.
6. When you're finished, click Add. The new agent page reopens.
Optional: Set a size limit for the queries processed by the data agent. In the Settingssection, type a value in the Maximum bytes billedfield. You must set this limit to 10485760 or higher, otherwise you receive the following error message:

 Value error. In BigQuery on-demand pricing charges are
rounded up to the nearest MB, with a minimum of 10 MB of data processed
per query. So, max bytes billed must be set to greater or equal to
10485760.

If you don't specify a value, maximum bytes billed defaults to the project's query usage per day quota . The usage per day quota is unlimited unless you have specified a custom quota .

Continue to the next section to place the agent in draft mode or publish the agent.

Preview and publish the agent

In the Previewsection, type an example user question in the Ask a questionfield, and then press Enter. To verify that the data agent returns the data that you expect, review the agent's response. If the response is not what you expect, change the settings in the Editorsection to refine the data agent configuration until you get satisfactory responses. You can continue to test and modify your agent to refine the agent's results.
Click Save.
To place the data agent in draft mode, which you can re-edit later, click Go backto return to the Agent Catalogpage. Because your agent is now in draft mode, it appears in the My draft agentssection on the Agent Catalogtab.

To publish your agent, remain on the agent creation page and proceed to the next step.
Click Publishto publish the data agent and make it available for use in the project. You can create conversations with the data agent by using BigQuery Studio or Data Studio . You can also build your own interface to chat with the data agent by using the Conversational Analytics API.
Optional: In the Your agent has been publisheddialog, click Shareto share the data agent with other users.
1. In the Share permissionspane, click Add principal.
2. In the New principalsfield, enter one or more principals.
3. Click the Select a rolelist.
4. In the Rolelist, select one of the following roles:
  - Gemini Data Analytics Data Agent User ( roles/geminidataanalytics.dataAgentUser ): grants permission to chat with the data agent.
  - Gemini Data Analytics Data Agent Editor ( roles/geminidataanalytics.dataAgentEditor ): grants permission to edit the data agent.
  - Gemini Data Analytics Data Agent Viewer ( roles/geminidataanalytics.dataAgentViewer ): grants permission to view the data agent.
Click Save.
To return to the new agent page, click Close. Immediately after saving or publishing your agent, you can see it in the Agent Catalog.

Manage data agents

You can find existing agents in the Agent Catalogtab, which consists of three sections:

My agents: a list of all agents that you create and publish. You can modify and share published agents with others.
My draft agents: agents that you haven't published yet. You can't share draft agents.
Shared by others in your organization: Agents that others create and share with you. If others grant you permissions, you can edit these shared agents.

Edit a data agent

Follow these steps to edit a data agent:

Go to the BigQuery Agentspage.

Go to Agents
Select the Agent Catalogtab.
Locate the agent card of the data agent that you want to modify.
To open the data agent in the agent editor, click Open actions > click Editon the agent card.
Edit the data agent's configuration as needed.
To save your changes without publishing, click Save.
To publish your changes, click Publish. In the Sharedialog, you can either share the agent with others, or click Cancel.
To return to the Agentspane, click Go back.

Follow these steps to share a published data agent. You can't share draft agents.

Go to the BigQuery Agentspage.

Go to Agents
Select the Agent Catalogtab.
Locate the agent card of the data agent that you want to modify.
To open the data agent in the agent editor, click Open actions > click Editon the agent card.
To share the data agent with other users, click Share.
In the Share permissionspane, click Add principal.
In the New principalsfield, enter one or more principals.
Click the Select a rolelist.
In the Rolelist, select one of the following roles:
- Gemini Data Analytics Data Agent User ( roles/geminidataanalytics.dataAgentUser ): gives permission to chat with the data agent.
- Gemini Data Analytics Data Agent Editor ( roles/geminidataanalytics.dataAgentEditor ): gives permission to edit the data agent.
- Gemini Data Analytics Data Agent Viewer ( roles/geminidataanalytics.dataAgentViewer ): gives permission to view the data agent.
Click Save.
To return to the agent editing page, click Close.
To return to the Agentspane, click Go back.

Delete a data agent

Go to the BigQuery Agentspage.

Go to Agents
Select the Agent Catalogtab.
In either the My agentsor My draft agentssection of the Agent Catalogtab, locate the agent card of the data agent that you want to delete.
Click Open actions > Delete.
In the Delete agent?dialog, click Delete.

Locations

Conversational analytics operates globally; you can't choose which region to use.

What's next

Learn more about conversational analytics in BigQuery .
Learn more about the Conversational Analytics API .
Analyze data with conversations .
Learn more about how the Gemini Data Analytics Data Agent Viewer ( roles/geminidataanalytics.dataAgentViewer ) role gives permission to view the data agent.