DataTap Configuration

Supported in:

Google secops SIEM

This document explains how to use DataTap Configuration to transmit normalized or filtered events to Cloud Pub/Sub. It also describes how to manage the Pub/Sub topics where events are delivered.

Examples (in Python) for making OAuth authenticated requests to the Backstory API are provided for each API call referenced in this document.

Regional Endpoints

Google SecOps provides regional endpoints for each API.

 Canada                      https://northamerica-northeast2-backstory.googleapis.com
Dammam                      https://me-central2-backstory.googleapis.com
Doha                        https://me-central1-backstory.googleapis.com
Europe Multi-Region         https://europe-backstory.googleapis.com
Frankfurt                   https://europe-west3-backstory.googleapis.com
London                      https://europe-west2-backstory.googleapis.com
Mumbai                      https://asia-south1-backstory.googleapis.com
Singapore                   https://asia-southeast1-backstory.googleapis.com
Sydney                      https://australia-southeast1-backstory.googleapis.com
Tel Aviv                    https://me-west1-backstory.googleapis.com
Tokyo                       https://asia-northeast1-backstory.googleapis.com
Turin                       https://europe-west12-backstory.googleapis.com
United States Multi-Region  https://backstory.googleapis.com
Zurich                      https://europe-west6-backstory.googleapis.com

For example:

 https://backstory.googleapis.com/v1/dataTaps
https://europe-backstory.googleapis.com/v1/dataTaps
https://asia-southeast1-backstory.googleapis.com/v1/dataTaps

Before you begin

Give publisher role to publisher@chronicle-data-tap.iam.gserviceaccount.com on your Pub/Sub Topic.

Specifying Topic

When creating or updating DataTap configurations, it's necessary to specify the Pub/Sub topic where the events are sent to. This is done by specifying the Pub/Sub topic using the following format:

projects/<project_id>/topics/<topicId>

Specifying Filter

Filter defines which events are published to the topic specified by the DataTap configuration. Valid values for filter are as follows:

ALL_UDM_EVENTS : All events are sent to the topic.
LABELED_UDM_EVENTS : Only events detected for Google Security Operations Detection Engine rules that filter on a single event and specify a dataTapLabel that is ENABLED . To create a rule with a dataTapLabel , see Use rules in a DataTap configuration .

Specifying serializationFormat

serializationFormat defines the format for sent events. Valid values for serializationFormat include:

JSON: Events are sent in JSON format.
MARSHALLED_PROTO: Events are sent in proto format.

The default value is MARSHALLED_PROTO.

DataTap Configuration API Reference

This section describes the DataTap Configuration API methods.

Create

Creates a DataTap configuration.

Request

 POST https://backstory.googleapis.com/v1/dataTaps

URL parameters

None

Request Body

 {
  "displayName": "<Name of the DataTap>",
  "cloudPubsubSink": {
    "topic": "<topicId>",
  },
  "filter": "<filter>",
  "serializationFormat": "<serializationFormat>"
}

Body Parameters

Parameter Name	Type	Required	Description
displayName	string	Yes	Name for the DataTap configuration being created.
topic	string	Yes	TopicId where events are to be sent. Use the following format: `projects/<project_id>/topics/<topicId>`
filter	enum	Yes	ALL_UDM_EVENTS: Retrieve all normalized events. ALERT_UDM_EVENTS: Retrieve all alert events. LABELED_UDM_EVENTS: Retrieves events detected for Google SecOps Detection Engine rules that filter on a single event and specify a `dataTapLabel` that is `ENABLED` . To create a rule with a `dataTapLabel` , see Use rules in a DataTap configuration .
serializationFormat	enum	No	JSON: Retrieve events in JSON format. MARSHALLED_PROTO: Retrieve events in proto format.

Sample Request

 https://backstory.googleapis.com/v1/dataTaps
{
  "displayName": "tap1",
  "cloudPubsubSink": {
    "topic": "projects/sample-project/topics/sample-topic",
  },
  "filter": "LABELED_UDM_EVENTS",
  "serializationFormat": "JSON"
}

Response

Sample Response

 {
  "customerId": "cccccccc-cccc-cccc-cccc-cccccccccccc",
  "tapId": "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa",
  "displayName": "tap1",
  "cloudPubsubSink": {
    "topic": "projects/sample-project/topics/sample-topic",
  },
  "filter": "LABELED_UDM_EVENTS",
  "serializationFormat": "JSON"
}

Update

Updates a DataTap configuration.

Request

 PATCH https://backstory.googleapis.com/v1/dataTaps/<tapId>

URL parameters

Parameter Name	Type	Required	Description
tapId	string	Yes	tapId given in response when the DataTap configuration was created.

Request Body

 {
  "name": "dataTaps/<tapId>",
  "displayName": "<Name of the DataTap>",
  "cloudPubsubSink": {
    "topic": "<topicId>",
  },
  "filter": "<filter>",
  "serializationFormat": "<serializationFormat>"
}

Body Parameters

Parameter Name	Type	Required	Description
name	string	Yes	Use format: dataTaps/<tapId> tapId given in response when the DataTap configuration was created.
displayName	string	Yes	Name for the DataTap configuration being created.
topic	string	Yes	TopicId where events are to be sent. Use the following format: `projects/<project_id>/topics/<topicId>`
filter	enum	Yes	ALL_UDM_EVENTS: Retrieve all normalized events. ALERT_UDM_EVENTS: Retrieve all alert events.
serializationFormat	enum	No	JSON: Retrieve events in JSON format. MARSHALLED_PROTO: Retrieve events in proto format.

Sample Request

 https://backstory.googleapis.com/v1/dataTaps/aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa

{
  "name": "dataTaps/aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa",
  "displayName": "tap1",
  "cloudPubsubSink": {
    "topic": "projects/sample-project/topics/sample-topic",
  },
  "filter": "ALL_UDM_EVENTS",
  "serializationFormat": "JSON"
}

Response

Sample Response

 {
  "customerId": "cccccccc-cccc-cccc-cccc-cccccccccccc",
  "tapId": "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa",
  "displayName": "tap1",
  "cloudPubsubSink": {
    "topic": "projects/sample-project/topics/sample-topic",
  },
  "filter": "ALL_UDM_EVENTS",
  "serializationFormat": "JSON"
}

Delete

Deletes a DataTap configuration.

Request

 DELETE https://backstory.googleapis.com/v1/dataTaps/<tapId>

URL parameters

Parameter Name	Type	Required	Description
tapId	string	Yes	tapId given in response when the DataTap configuration was created.

Request Body

 {
  "name": "dataTaps/<tapId>",
}

Body Parameters

Parameter Name

Type

Required

Description

name

string

Yes

Use the following format: dataTaps/<tapId>

tapId given in response when the DataTap configuration was created.

Sample Request

 https://backstory.googleapis.com/v1/dataTaps/aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa

{
  "name": "dataTaps/aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa",
}

Response

Sample Response

Returns an empty JSON with 200 OK, indicating the operation has completed successfully.

Get

Get a specific DataTap configuration.

Request

 GET https://backstory.googleapis.com/v1/dataTaps/<tapId>

URL parameters

Parameter Name	Type	Required	Description
tapId	string	Yes	tapId given in response when the DataTap configuration was created.

Request Body

 {
  "name": "dataTaps/<tapId>",
}

Body Parameters

Parameter Name

Type

Required

Description

name

string

Yes

Use format: dataTaps/<tapId>

tapId given in response when the DataTap configuration was created.

Sample Request

 https://backstory.googleapis.com/v1/dataTaps/aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa

{
  "name": "dataTaps/aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa",
}

Response

Sample Response

 {
  "customerId": "cccccccc-cccc-cccc-cccc-cccccccccccc",
  "tapId": "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa",
  "displayName": "tap1",
  "cloudPubsubSink": {
    "topic": "projects/sample-project/topics/sample-topic",
  },
  "filter": "ALL_UDM_EVENTS",
  "serializationFormat": "MARSHALLED_PROTO"
}

List

List all the DataTap configurations of a customer.

Request

 GET https://backstory.googleapis.com/v1/dataTaps

URL parameters

None

Request Body

Empty

Body Parameters

None

Sample Request

 https://backstory.googleapis.com/v1/dataTaps

Response

Sample Response

 [
  {
    "customerId": "cccccccc-cccc-cccc-cccc-cccccccccccc",
    "tapId": "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa",
    "displayName": "tap1",
    "cloudPubsubSink": {
      "topic": "projects/sample-project/topics/sample-topic",
    },
    "filter": "ALL_UDM_EVENTS",
    "serializationFormat": "JSON"
  },
  "filter": "ALERT_UDM_EVENTS",
  "serializationFormat": "MARSHALLED_PROTO"
  }
]