Manage observability buckets

This document describes how to use the Observability API to get information about your observability buckets. It also includes information about how to list datasets, links, and views. To learn more about how Google Cloud Observability stores data, see Storage overview .

Your trace data is stored in observability buckets. This document describes how to manage the storage of your trace data, but it doesn't describe the format of the stored data. For information about that topic, see Trace schema .

This document doesn't apply to the storage of log or metric data. Log and metric data isn't stored in observability buckets.

Before you begin

Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Roles required to select or create a project

Select a project : Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
Create a project : To create a project, you need the Project Creator role ( roles/resourcemanager.projectCreator ), which contains the resourcemanager.projects.create permission. Learn how to grant roles .

Go to project selector

Verify that billing is enabled for your Google Cloud project .

Enable the Observability API.

Roles required to enable APIs

To enable APIs, you need the Service Usage Admin IAM role ( roles/serviceusage.serviceUsageAdmin ), which contains the serviceusage.services.enable permission. Learn how to grant roles .

Enable the API

In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

Roles required to select or create a project

Select a project : Selecting a project doesn't require a specific IAM role—you can select any project that you've been granted a role on.
Create a project : To create a project, you need the Project Creator role ( roles/resourcemanager.projectCreator ), which contains the resourcemanager.projects.create permission. Learn how to grant roles .

Go to project selector

Verify that billing is enabled for your Google Cloud project .

Enable the Observability API.

Roles required to enable APIs

To enable APIs, you need the Service Usage Admin IAM role ( roles/serviceusage.serviceUsageAdmin ), which contains the serviceusage.services.enable permission. Learn how to grant roles .

Enable the API

To get the permissions that you need to list buckets, links, and views, ask your administrator to grant you the Observability Viewer ( roles/observability.viewer ) IAM role on your project. For more information about granting roles, see Manage access to projects, folders, and organizations .

You might also be able to get the required permissions through custom roles or other predefined roles .
Select the tab for how you plan to use the samples on this page:
gcloud

In the Google Cloud console, activate Cloud Shell.

Activate Cloud Shell

At the bottom of the Google Cloud console, a Cloud Shell session starts and displays a command-line prompt. Cloud Shell is a shell environment with the Google Cloud CLI already installed and with values already set for your current project. It can take a few seconds for the session to initialize.
REST

To use the REST API samples on this page in a local development environment, you use the credentials you provide to the gcloud CLI.
For more information, see Authenticate for using REST in the Google Cloud authentication documentation.

List observability buckets

gcloud

Before using any of the command data below, make the following replacements:

LOCATION : The location of the observability buckets. To list all observability buckets, regardless of location, set the location to a hyphen ( - ).
PROJECT_ID : The identifier of the project..

Execute the gcloud beta observability buckets list command:

Linux, macOS, or Cloud Shell

gcloud  
beta  
observability  
buckets  
list  
 \ 
  
--location = 
 LOCATION 
  
--project = 
 PROJECT_ID

Windows (PowerShell)

gcloud  
beta  
observability  
buckets  
list  
 ` 
  
--location = 
 LOCATION 
  
--project = 
 PROJECT_ID

Windows (cmd.exe)

gcloud  
beta  
observability  
buckets  
list  
^  
--location = 
 LOCATION 
  
--project = 
 PROJECT_ID

The response lists the name, description, and create time of each observability buckets. The following is an example of a response when the command is successful:

---
createTime: '2026-01-21T21:39:22.381083860Z'
description: Bucket for storing spans from Cloud Trace.
name: projects/my-project/locations/us/buckets/_Trace

REST

To list the observability buckets that are in your project and in a specific location, send a request to the projects.locations.buckets.list endpoint.

You must specify the parent parameter, which has the following form:

 projects/ PROJECT_ID 
/locations/ LOCATION

The fields in the previous expression have the following meanings:

PROJECT_ID : The identifier of the project.
LOCATION : The location of the observability bucket . If you set LOCATION to a hyphen, (-) , then all observability buckets in your project are listed.

The response is an array of Bucket objects. For each object, the value of the name field has the following format:

 projects/ PROJECT_ID 
/locations/ LOCATION 
/buckets/ BUCKET_ID

For example, when a command was issued to the buckets.list endpoint with the parent parameter set to projects/my-project/locations/us , the response was:

 {
  "buckets": [
    {
      "name": "projects/my-project/locations/us/buckets/_Trace",
      "description": "Trace Bucket",
      "createTime": "2025-01-01T15:42:30.988919645Z",
      "updateTime": "2025-02-04T15:42:30.988919645Z",
      "retentionDays": 30
    }
  ]
}

You can issue commands to other Observability API endpoints to get more information about the bucket whose ID is BUCKET_ID . For example, you can list the datasets on that bucket, and the views and links on each dataset. For a complete list of Observability API endpoints, see the Observability API reference documentation .

List datasets on a observability bucket

gcloud

Before using any of the command data below, make the following replacements:

BUCKET_ID : The ID of the observability bucket. For example, this ID might be _Trace .
LOCATION : The location of the observability buckets.
PROJECT_ID : The identifier of the project..

Execute the gcloud beta observability buckets datasets list command:

Linux, macOS, or Cloud Shell

gcloud  
beta  
observability  
buckets  
datasets  
list  
 \ 
  
--bucket = 
projects/ PROJECT_ID 
/locations/ LOCATION 
/buckets/ BUCKET_ID 
  
 \ 
  
--location = 
 LOCATION 
  
 \ 
  
--project = 
 PROJECT_ID

Windows (PowerShell)

gcloud  
beta  
observability  
buckets  
datasets  
list  
 ` 
  
--bucket = 
projects/ PROJECT_ID 
/locations/ LOCATION 
/buckets/ BUCKET_ID 
  
 ` 
  
--location = 
 LOCATION 
  
 ` 
  
--project = 
 PROJECT_ID

Windows (cmd.exe)

gcloud  
beta  
observability  
buckets  
datasets  
list  
^  
--bucket = 
projects/ PROJECT_ID 
/locations/ LOCATION 
/buckets/ BUCKET_ID 
  
^  
--location = 
 LOCATION 
  
^  
--project = 
 PROJECT_ID

The response lists the name, description, and create time of each dataset. The following is an example of a response when the command is successful:

---
createTime: '2026-01-21T21:39:22.381083860Z'
description: Dataset for storing spans from Cloud Trace.
name: projects/my-project/locations/us/buckets/_Trace/datasets/Spans

REST

To list the datasets for an observability bucket, send a request to the projects.locations.buckets.datasets.list endpoint.

You must specify the parent parameter, which has the following form:

 projects/ PROJECT_ID 
/locations/ LOCATION 
/buckets/ BUCKET_ID

The fields in the previous expression have the following meanings:

PROJECT_ID : The identifier of the project.
LOCATION : The location of the observability bucket .
BUCKET_ID : The ID of the observability bucket. For example, this ID might be _Trace .

The response is an array of Dataset objects. For each object, the value of the name field has the following format:

 projects/ PROJECT_ID 
/locations/ LOCATION 
/buckets/ BUCKET_ID 
/dataset/ DATASET_ID

For example, when a command was issued to the buckets.datasets.list endpoint with the parent parameter set to projects/my-project/locations/us/buckets/_Trace , the response was:

 {
  "datasets": [
    {
      "name": "projects/my-project/locations/us/buckets/_Trace/datasets/Spans",
      "description": "Trace Spans",
      "createTime": "2025-01-01T15:42:30.988919645Z",
      "updateTime": "2025-02-04T15:42:30.988919645Z",
    }
  ]
}

You can issue commands to other Observability API endpoints to get information about the dataset whose ID is DATASET_ID . For example, you can list the views and links on each dataset. For a complete list of Observability API endpoints, see the Observability API reference documentation .

List views on a dataset

gcloud

Before using any of the command data below, make the following replacements:

DATASET_ID : The ID of the dataset. Your trace data is stored in a dataset named Spans .
BUCKET_ID : The ID of the observability bucket. For example, this ID might be _Trace .
LOCATION : The location of the observability buckets.
PROJECT_ID : The identifier of the project..

Execute the gcloud beta observability buckets datasets views list command:

Linux, macOS, or Cloud Shell

gcloud  
beta  
observability  
buckets  
datasets  
views  
list  
 \ 
  
--dataset = 
projects/ PROJECT_ID 
/locations/ LOCATION 
/buckets/ BUCKET_ID 
/datasets/ DATASET_ID 
  
 \ 
  
--bucket = 
 BUCKET_ID 
  
 \ 
  
--location = 
 LOCATION 
  
 \ 
  
--project = 
 PROJECT_ID

Windows (PowerShell)

gcloud  
beta  
observability  
buckets  
datasets  
views  
list  
 ` 
  
--dataset = 
projects/ PROJECT_ID 
/locations/ LOCATION 
/buckets/ BUCKET_ID 
/datasets/ DATASET_ID 
  
 ` 
  
--bucket = 
 BUCKET_ID 
  
 ` 
  
--location = 
 LOCATION 
  
 ` 
  
--project = 
 PROJECT_ID

Windows (cmd.exe)

gcloud  
beta  
observability  
buckets  
datasets  
views  
list  
^  
--dataset = 
projects/ PROJECT_ID 
/locations/ LOCATION 
/buckets/ BUCKET_ID 
/datasets/ DATASET_ID 
  
^  
--bucket = 
 BUCKET_ID 
  
^  
--location = 
 LOCATION 
  
^  
--project = 
 PROJECT_ID

The response lists the name, create time, and update time of each observability views. The following is an example of a response when the command is successful:

---
createTime: '2026-01-21T21:39:22.381083860Z'
displayName: _AllSpans
name: projects/pamstestproject1/locations/us/buckets/_Trace/datasets/Spans/views/_AllSpans
updateTime: '2026-01-21T21:39:22.381083860Z'

REST

To list the views on a dataset, send a request to the projects.locations.buckets.datasets.views.list endpoint.

You must specify the parent parameter, which has the following form:

 projects/ PROJECT_ID 
/locations/ LOCATION 
/buckets/ BUCKET_ID 
/datasets/ DATASET_ID 
/views

The fields in the previous expression have the following meanings:

PROJECT_ID : The identifier of the project.
LOCATION : The location of the observability bucket .
BUCKET_ID : The ID of the observability bucket. For example, this ID might be _Trace .
DATASET_ID : The ID of the dataset being queried. For example, this ID might be Spans .

The response is an array of View objects. For each object, the value of the name field has the following format:

 projects/ PROJECT_ID 
/locations/ LOCATION 
/buckets/ BUCKET_ID 
/views/ OBS_VIEW_ID

In the previous express, the ID of a view is represented by OBS_VIEW_ID . For example, this field might have a value of _AllSpans .

For example, when a command was issued to the buckets.datasets.views.list endpoint with the parent parameter set to projects/my-project/locations/us/buckets/_Trace/datasets/Spans/views , the response was:

 {
  "views": [
    {
      "name": "projects/my-project/locations/us/buckets/_Trace/datasets/Spans/views/_AllSpans",
      "filter": "",
      "createTime": "2025-01-01T15:42:30.988919645Z",
      "updateTime": "2025-02-04T15:42:30.988919645Z",
    }
  ]
}

For a complete list of Observability API endpoints, see the Observability API reference documentation .

List links on a dataset

gcloud

Before using any of the command data below, make the following replacements:

DATASET_ID : The ID of the dataset. Your trace data is stored in a dataset named Spans .
BUCKET_ID : The ID of the observability bucket. For example, this ID might be _Trace .
LOCATION : The location of the observability buckets.
PROJECT_ID : The identifier of the project..

Execute the gcloud beta observability buckets datasets links list command:

Linux, macOS, or Cloud Shell

gcloud  
beta  
observability  
buckets  
datasets  
links  
list  
 \ 
  
--dataset = 
projects/ PROJECT_ID 
/locations/ LOCATION 
/buckets/ BUCKET_ID 
/datasets/ DATASET_ID 
 \ 
  
--bucket = 
 BUCKET_ID 
  
 \ 
  
--location = 
 LOCATION 
  
 \ 
  
--project = 
 PROJECT_ID

Windows (PowerShell)

gcloud  
beta  
observability  
buckets  
datasets  
links  
list  
 ` 
  
--dataset = 
projects/ PROJECT_ID 
/locations/ LOCATION 
/buckets/ BUCKET_ID 
/datasets/ DATASET_ID 
 ` 
  
--bucket = 
 BUCKET_ID 
  
 ` 
  
--location = 
 LOCATION 
  
 ` 
  
--project = 
 PROJECT_ID

Windows (cmd.exe)

gcloud  
beta  
observability  
buckets  
datasets  
links  
list  
^  
--dataset = 
projects/ PROJECT_ID 
/locations/ LOCATION 
/buckets/ BUCKET_ID 
/datasets/ DATASET_ID 
^  
--bucket = 
 BUCKET_ID 
  
^  
--location = 
 LOCATION 
  
^  
--project = 
 PROJECT_ID

The response lists the name and create time of each link. The following is an example of a response when the command is successful:

---
createTime: '2026-04-02T21:23:09.272323714Z'
name: projects/my-project/locations/us/buckets/_Trace/datasets/Spans/links/mydataset

REST

To list the links on a dataset, send a request to the projects.locations.buckets.datasets.links.list endpoint.

You must specify the parent parameter, which has the following form:

 projects/ PROJECT_ID 
/locations/ LOCATION 
/buckets/ BUCKET_ID 
/datasets/ DATASET_ID

The fields in the previous expression have the following meanings:

PROJECT_ID : The identifier of the project.
LOCATION : The location of the observability bucket .
BUCKET_ID : The ID of the observability bucket. For example, this ID might be _Trace .
DATASET_ID : The ID of the dataset being queried. For example, this ID might be Spans .

The response is an array of Link objects. For each object, the value of the name field has the following format:

 projects/ PROJECT_ID 
/locations/ LOCATION 
/buckets/ BUCKET_ID 
/links/ LINK_ID

The LINK_ID is the name of the BigQuery dataset. This field is globally unique for your Google Cloud project.

For example, when a command was issued to the buckets.datasets.links.list endpoint with the parent parameter set to projects/my-project/locations/us/buckets/_Trace/datasets/Spans/links , the response was:

 {
  "links": [
    {
      "name": "projects/my-project/locations/us/buckets/_Trace/datasets/Spans/links/my_link",
      "description": "My link for traces to BigQuery",
      "createTime": "2025-01-12T15:42:30.988919645Z"
    }
  ]
}

For a complete list of Observability API endpoints, see the Observability API reference documentation .

Create a link on a dataset

You can create a link on a dataset, which lets your trace data be queried from BigQuery. You can also delete the Link object which is attached to a dataset.

Before you begin

Complete the required setup to list links .
To get the permissions that you need to create a link on a dataset, ask your administrator to grant you the following IAM roles on your project:
- Observability Editor ( roles/observability.editor )
- BigQuery User ( roles/bigquery.user )
For more information about granting roles, see Manage access to projects, folders, and organizations .

You might also be able to get the required permissions through custom roles or other predefined roles .

Create a linked BigQuery dataset

gcloud

Before using any of the command data below, make the following replacements:

LINK_ID : The name of the BigQuery dataset.
DATASET_ID : The ID of the dataset. Your trace data is stored in a dataset named Spans .
BUCKET_ID : The ID of the observability bucket. For example, this ID might be _Trace .
LOCATION : The location of the observability buckets.
PROJECT_ID : The identifier of the project..

Execute the gcloud beta observability buckets datasets links create command:

Linux, macOS, or Cloud Shell

gcloud  
beta  
observability  
buckets  
datasets  
links  
create  
 \ 
  
projects/ PROJECT_ID 
/locations/ LOCATION 
/buckets/ BUCKET_ID 
/datasets/ DATASET_ID 
/links/ LINK_ID 
  
 \ 
  
--dataset = 
 DATASET_ID 
 \ 
  
--bucket = 
 BUCKET_ID 
  
 \ 
  
--location = 
 LOCATION 
  
 \ 
  
--project = 
 PROJECT_ID

Windows (PowerShell)

gcloud  
beta  
observability  
buckets  
datasets  
links  
create  
 ` 
  
projects/ PROJECT_ID 
/locations/ LOCATION 
/buckets/ BUCKET_ID 
/datasets/ DATASET_ID 
/links/ LINK_ID 
  
 ` 
  
--dataset = 
 DATASET_ID 
 ` 
  
--bucket = 
 BUCKET_ID 
  
 ` 
  
--location = 
 LOCATION 
  
 ` 
  
--project = 
 PROJECT_ID

Windows (cmd.exe)

gcloud  
beta  
observability  
buckets  
datasets  
links  
create  
^  
projects/ PROJECT_ID 
/locations/ LOCATION 
/buckets/ BUCKET_ID 
/datasets/ DATASET_ID 
/links/ LINK_ID 
  
^  
--dataset = 
 DATASET_ID 
^  
--bucket = 
 BUCKET_ID 
  
^  
--location = 
 LOCATION 
  
^  
--project = 
 PROJECT_ID

The create command initiates a long-running operation. The following is an example of a response when the command is successful:

Create request issued for: [mydataset]
Waiting for operation [projects/my-project/locations/us/operations/operation-1775164903749-64e80c9817833-9ff804b6-c3e9cbe7] to complete...done.
Created link [mydataset].

REST

To create a link to a BigQuery dataset, send a request to the projects.locations.buckets.datasets.links.create endpoint.

You must specify the parent parameter, which has the following form:

 projects/ PROJECT_ID 
/locations/ LOCATION 
/buckets/ BUCKET_ID 
/datasets/ DATASET_ID

The fields in the previous expression have the following meaning:

PROJECT_ID : The identifier of the project.
LOCATION : The location of the observability bucket .
BUCKET_ID : The ID of the observability bucket. For example, this ID might be _Trace .
DATASET_ID : The ID of the dataset being queried. For example, this ID might be Spans .

This command requires a query parameter and a request body:

The query parameter, linkId , must be specified and set to the name of the BigQuery dataset. For example, linkId="my_link" . The BigQuery dataset name must be unique for your Google Cloud project, and must be limited to 100 characters and can include only letters, digits, and underscores.
The request body is a Link object. The value of the name field has the following format:
```
 projects/ PROJECT_ID 
/locations/ LOCATION 
/buckets/ BUCKET_ID 
/dataset/ DATASET_ID 
/links/ LINK_ID 
 
```
The value you provide for the name field must match the linked BigQuery dataset referenced by the query parameter.

The LINK_ID field is the name of the BigQuery dataset.

The response is an Operation object. This object contains information about the progress of the method. When the method completes, the Operation object contains status data.

For a complete list of Observability API endpoints, see the Observability API reference documentation .

What's next

To learn about querying your telemetry, see View and analyze telemetry .

Manage observability buckets Stay organized with collections Save and categorize content based on your preferences.

Before you begin

gcloud

REST

List observability buckets

gcloud

Linux, macOS, or Cloud Shell

Windows (PowerShell)

Windows (cmd.exe)

REST

List datasets on a observability bucket

gcloud

Linux, macOS, or Cloud Shell

Windows (PowerShell)

Windows (cmd.exe)

REST

List views on a dataset

gcloud

Linux, macOS, or Cloud Shell

Windows (PowerShell)

Windows (cmd.exe)

REST

List links on a dataset

gcloud

Linux, macOS, or Cloud Shell

Windows (PowerShell)

Windows (cmd.exe)

REST

Create a link on a dataset

Before you begin

Create a linked BigQuery dataset

gcloud

Linux, macOS, or Cloud Shell

Windows (PowerShell)

Windows (cmd.exe)

REST

What's next

Manage observability buckets