About data insights for structured data

The Knowledge Catalog (formerly Dataplex Universal Catalog) data insights automatically generates descriptions, relationship graphs, and SQL queries from your table and dataset metadata. This information helps you quickly understand data structure, content, and relationships without extensive manual setup. To investigate further, you can ask follow-up questions in data canvas .

When exploring a new, unfamiliar table, data analysts often face the cold-start problem of how to get started writing queries. The problem can involve uncertainties about the data structure and key patterns in the data. The Knowledge Catalog data insights feature offers an automated way to explore and understand your data. This helps you uncover patterns, assess data quality, and perform statistical analysis.

How data insights work?

Data insights use Gemini to analyze your metadata and generate the following:

• Descriptions: AI-generated summaries explaining the purpose of the dataset, the structure of the table, and the details of specific columns.

• Sample queries: Tailored SQL queries designed specifically for the schema and content of your dataset or table.

• Relationship graphs: Visualizations that show the connections and dependencies between different tables within your dataset.

Supported resources

Data insights are available for the following structured data types:

• BigQuery datasets, tables, and views
• BigLake tables (including Apache Iceberg)
• External tables
• Iceberg REST Catalog tables

Example of an insights run

Data insights automatically generates natural language queries and their SQL equivalents based on a table's metadata.

Consider a table called telco_churn with the following metadata:

The following are some of the sample queries that data insights generates for this table:

• Identify customers who have subscribed to all premium services and have been a customer for more than 50 months.

    SELECT 
    
   CustomerID 
   , 
    
   Contract 
   , 
    
   Tenure 
   FROM 
    
   agentville_datasets 
   . 
   telco_churn 
   WHERE 
    
   OnlineBackup 
    
   = 
    
   'Yes' 
    
   AND 
    
   TechSupport 
    
   = 
    
   'Yes' 
    
   AND 
    
   StreamingTV 
    
   = 
    
   'Yes' 
    
   AND 
    
   Tenure 
   > 
   50 
   ; 
   
  

• Identify which internet service has the most churned customers.

    SELECT 
    
   InternetService 
   , 
    
   COUNT 
   ( 
   DISTINCT 
    
   CustomerID 
   ) 
    
   AS 
    
   total_customers 
   FROM 
    
   agentville_datasets 
   . 
   telco_churn 
   WHERE 
    
   Churn 
    
   = 
    
   TRUE 
   GROUP 
    
   BY 
    
   InternetService 
   ORDER 
    
   BY 
    
   total_customers 
    
   DESC 
   LIMIT 
    
   1 
   ; 
   
  

• Identify churn rates by segment among high-value customers.

    SELECT 
    
   Contract 
   , 
    
   InternetService 
   , 
    
   Gender 
   , 
    
   PaymentMethod 
   , 
    
   COUNT 
   ( 
   DISTINCT 
    
   CustomerID 
   ) 
    
   AS 
    
   total_customers 
   , 
    
   SUM 
   ( 
   CASE 
    
   WHEN 
    
   Churn 
    
   = 
    
   TRUE 
    
   THEN 
    
   1 
    
   ELSE 
    
   0 
    
   END 
   ) 
    
   AS 
    
   churned_customers 
   , 
    
   ( 
   SUM 
   ( 
   CASE 
    
   WHEN 
    
   Churn 
    
   = 
    
   TRUE 
    
   THEN 
    
   1 
    
   ELSE 
    
   0 
    
   END 
   ) 
    
   / 
    
   COUNT 
   ( 
   DISTINCT 
    
   CustomerID 
   )) 
    
   * 
    
   100 
    
   AS 
    
   churn_rate 
   FROM 
    
   agentville_datasets 
   . 
   telco_churn 
   WHERE 
    
   MonthlyCharges 
   > 
   100 
   GROUP 
    
   BY 
    
   Contract 
   , 
    
   InternetService 
   , 
    
   Gender 
   , 
    
   PaymentMethod 
   ; 
   
  

Modes for generating data insights

When generating data insights, Knowledge Catalog provides two modes:

Pricing

For details about pricing for this feature, see Gemini in BigQuery pricing overview .

Quotas and limits

For information about quotas and limits for this feature, see Quotas for Gemini in BigQuery .

Locations

You can use data insights in all BigQuery locations . To learn about where Gemini in BigQuery processes your data, see Where Gemini in BigQuery processes your data .

Limitations

• For multi-cloud customers, data from other clouds isn't available.
• Data insights doesn't support Geo or JSON column types.
• Insights runs don't guarantee the presentation of queries every time. To increase the likelihood of generating more engaging queries, regenerate insights in BigQuery Studio.

What's next

• Learn how to generate insights in BigQuery .
• Learn how to generate a data profiling scan .
• Learn how to write queries with Gemini assistance in BigQuery .