Query Explain lets you submit Datastore mode queries
to the backend and receive detailed performance statistics on backend query execution
in return. It functions like the EXPLAIN ANALYZE
operation in many
relational database systems.
You can send Query Explain requests using the Datastore mode client libraries .
Query Explain results help you understand how your queries are executed, showing you inefficiencies and the location of likely server-side bottlenecks.
Query Explain:
- Provides insights on planning phase so you can adjust your query indexes and boost efficiency.
- Helps you understand your cost and performance on a per-query basis and lets you quickly iterate through different query patterns in order to optimize their usage.
Understand Query Explain options: default and analyze
Query Explain operations can be performed using the default option or analyze option.
With the default option, Query Explain plans the query, but skips over the execution stage. This will return planner stage information. You can use this to check that a query has the necessary indexes and understand which indexes are used. This will help you verify, for example, that a particular query is using a composite index over having to intersect over many different indexes.
With the analyze option, Query Explain both plans and executes the query. This returns all the previously mentioned planner information along with statistics from the query execution runtime. This will include billing information along with system-level insights into the query execution. You can use this tooling to test various query and index configurations to optimize their cost and latency.
What does Query Explain cost?
When a query is explained with the default option, no index or read operations are performed. Regardless of query complexity, one read operation is charged.
When a query is explained with the analyze option, index and read operations are performed, so you are charged for the query as usual. There is no additional charge for the analysis activity, only the usual charge for the query being executed.
Execute a query with the default option
You can use a client library to submit a default option request.
Note that query explain results are authenticated with Identity and Access Management , using the same permissions for regular query operations.
Java
To learn how to install and use the client library for Datastore mode, see Datastore mode client libraries . For more information, see the Datastore mode Java API reference documentation .
To authenticate to Datastore mode, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .
See the indexes_used
field in the response to learn about the indexes
used in the query plan:
"indexes_used": [
{"query_scope": "Collection Group", "properties": "(__name__ ASC)"},
]
For more information about the report, see the report reference .
Execute a query with the analyze option
You can use a client library to submit a default option request.
Note that query analyze results are authenticated with Identity and Access Management (IAM) , using the same permissions for regular query operations.
Java
To learn how to install and use the client library for Datastore mode, see Datastore mode client libraries . For more information, see the Datastore mode Java API reference documentation .
To authenticate to Datastore mode, set up Application Default Credentials. For more information, see Set up authentication for a local development environment .
See the executionStats
object to find query profiling information such as:
{
"resultsReturned": "5",
"executionDuration": "0.100718s",
"readOperations": "5",
"debugStats": {
"index_entries_scanned": "95000",
"documents_scanned": "5"
"billing_details": {
"documents_billable": "5",
"index_entries_billable": "0",
"small_ops": "0",
"min_query_cost": "0",
}
}
}
For more information about the report, see the report reference .
Interpret results and make adjustments
The following example scenario queries movies by genre and country of production and demonstrates how to optimize the indexes used by the query.
For more information about the report, see the Query Explain report reference .
For illustration, assume the equivalent of this SQL query.
SELECT * FROM movies WHERE category = 'Romantic' AND country = 'USA';
If we use the analyze option, the following report output shows
the query runs on single-field indexes (category ASC, __name__ ASC)
and (country ASC, __name__ ASC)
. It scans 16500 index entries, but returns
only 1200 documents.
// Output query planning info "indexes_used" : [ { "query_scope" : "Collection Group" , "properties" : "(category ASC, __name__ ASC)" }, { "query_scope" : "Collection Group" , "properties" : "(country ASC, __name__ ASC)" }, ] // Output query status { "resultsReturned" : "1200" , "executionDuration" : "0.118882s" , "readOperations" : "1200" , "debugStats" : { "index_entries_scanned" : "16500" , "documents_scanned" : "1200" "billing_details" : { "documents_billable" : "1200" , "index_entries_billable" : "0" , "small_ops" : "0" , "min_query_cost" : "0" , } } }
To optimize the performance of executing the query, you can create a fully-covered composite index (category ASC, country ASC, __name__ ASC).
Running the query in analyze mode again, we can see that the newly-created index is selected for this query, and the query runs much faster and more efficiently.
// Output query planning info "indexes_used" : [ { "query_scope" : "Collection Group" , "properties" : "(category ASC, country ASC, __name__ ASC)" } ] // Output query stats { "resultsReturned" : "1200" , "executionDuration" : "0.026139s" , "readOperations" : "1200" , "debugStats" : { "index_entries_scanned" : "1200" , "documents_scanned" : "1200" "billing_details" : { "documents_billable" : "1200" , "index_entries_billable" : "0" , "small_ops" : "0" , "min_query_cost" : "0" , } } }
