Ingestion methods (new)

This document describes how the new Ingestion methods in the Chronicle API forward logs directly to your Google Security Operations instance. This process eliminates the need for additional hardware or software (for example, forwarders) in your environment.

The Chronicle API service for Google SecOps improves, expands, and adds to the functionality of existing ingestion management API methods and partner API services, for example, the Chronicle Ingestion API .

Explore the Chronicle API

The Chronicle API is a RESTful API with a JSON payload. You can develop systems to call Chronicle API methods directly and forward logs to Google SecOps. Managed Security Service Providers (MSSPs) and your Technology Partners can use these API methods to send logs to Google SecOps.

Forward log data

If you've formatted your log data using Google SecOps Unified Data Model (UDM), you can forward UDM events to your Google SecOps account using the ImportEvents endpoint. Google SecOps can more effectively process this data and improve threat detection because it standardizes UDM events. For information about converting raw logs to UDM, see Format log data as UDM .

You can also forward your data to Google SecOps as unstructured logs using the ImportLogs endpoint. Your unstructured log data is normalized within the Google SecOps infrastructure and made available to you through the Google SecOps console. However, some information may be difficult to extract from the unstructured log data and might only be searchable using the Raw Log Scan.

When an Ingestion method processes a batch of logs, it assigns a batch ID for those logs. If the batch ID matches an existing batch ID, the new batch of logs is not forwarded to the Google SecOps instance. Deduplication is based on the batch ID, which is identical if the batches contain the same logs.

To use Ingestion methods in the Chronicle API, you must grant a user or service account the Chronicle API EditorIAM (Identity and Access Management) role. This role provides the necessary permissions to call the API.

Client libraries

Client libraries provide code samples for using Ingestion methods, offering a quick way to access these methods from a supported language. Although you can use the Ingestion methods by making raw requests to the server, client libraries offer simplifications that significantly reduce the amount of code you need to write. Client libraries for Ingestion methods are available on GitHub, in the public Google secops-wrapper examples folder.

Example: UDM event ingestion

The following client library Python script, ingest_udm_events.py , demonstrates how to ingest UDM events into Google SecOps, using the secops library to interact with the Chronicle API.

Script functions

The script uses a series of functions to demonstrate each scenario.

• Event creation

  The following functions create sample UDM events that include metadata, such as id , event_timestamp , event_type , product_name , and vendor_name , along with specific details for each event type (for example, IP addresses , ports , command lines , MD5 hashes ):

  • create_sample_network_event() generates a network connection event.
  • create_sample_process_event() generates a process launch event.

• Ingestion

  The following functions send (one or more) Chronicle client objects and UDM events to Google SecOps:

  • ingest_single_event() sends a single UDM event (a network event in this example) to the API.
  • ingest_multiple_events() sends a list of UDM events (a network event and a process event) to Google SecOps in a single API call.
  • ingest_event_without_id() shows that if you omit the id field, Google SecOps automatically generates one during ingestion.

Run the script

The main() function parses command-line arguments for customer-id , project-id , and region to configure the Chronicleclient. It then calls the ingestion example functions to demonstrate each scenario. The script includes error handling to catch APIError exceptions during ingestion.

Regional endpoints

Your API endpoint depends on the region where your customer account is provisioned. Google SecOps provides the following list of Regional service endpoints .

API behavior and limitations

• Limitations:

  • Maximum individual log size: 4 MB
  • Maximum batch size (uncompressed): 4 MB
  • Maximum request size: 6 MB
  • Request timeout: 60 seconds

• Best practices:

  • Request timeout: Set to 60 seconds
  • Number of log lines per batch should be less than 100.
  • Batch size should be less than 1 MB.

API reference

Use the Ingestion methods in this section to ingest data into your Google SecOps instance.

ImportEvents

Use this method to import UDM events into Google SecOps.

Reference Documentation: Method: events.import

Full name: projects.locations.instances.events.import

Request

POST https://chronicle.{region}.rep.googleapis.com/v1alpha/{parent}/events:import

Path parameters

Request body

The following example shows how to import log data in UDM format using the ImportEvents API method.

  { 
  
 "parent" 
 : 
  
 "projects/gcp_project_id/locations/us/instances/aaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa" 
 , 
  
 "inlineSource" 
 : 
  
 { 
  
 "events" 
 : 
  
 [{ 
  
 "udm" 
 : 
  
 { 
  
 "metadata" 
 : 
  
 { 
  
 "eventTimestamp" 
 : 
  
 "1970-01-01T03:25:45.000000123Z" 
 , 
  
 "eventType" 
 : 
  
 "USER_LOGIN" 
 , 
  
 "vendorName" 
 : 
  
 "Acme" 
 , 
  
 "productName" 
 : 
  
 "Acme SSO" 
  
 }, 
  
 "principal" 
 : 
  
 { 
  
 "ip" 
 : 
  
 [ 
 "10.1.2.3" 
 ] 
  
 }, 
  
 "target" 
 : 
  
 { 
  
 "user" 
 : 
  
 { 
  
 "userid" 
 : 
  
 "mary@altostrat.com" 
 , 
  
 "userDisplayName" 
 : 
  
 "Mary Jane" 
  
 }, 
  
 "application" 
 : 
  
 "Acme Connect" 
  
 }, 
  
 "extensions" 
 : 
  
 { 
  
 "auth" 
 : 
  
 { 
  
 "type" 
 : 
  
 "MACHINE" 
 , 
  
 "mechanism" 
 : 
  
 [ 
 "NETWORK" 
 ] 
  
 } 
  
 } 
  
 } 
  
 }, 
  
 { 
  
 "udm" 
 : 
  
 { 
  
 "metadata" 
 : 
  
 { 
  
 "eventTimestamp" 
 : 
  
 "1970-01-01T03:25:45.000000123Z" 
 , 
  
 "eventType" 
 : 
  
 "NETWORK_HTTP" 
 , 
  
 "vendorName" 
 : 
  
 "Acme" 
 , 
  
 "productName" 
 : 
  
 "Acme Proxy" 
  
 }, 
  
 "principal" 
 : 
  
 { 
  
 "hostname" 
 : 
  
 "host0" 
 , 
  
 "ip" 
 : 
  
 [ 
 "10.1.2.3" 
 ], 
  
 "port" 
 : 
  
 6000 
  
 }, 
  
 "target" 
 : 
  
 { 
  
 "hostname" 
 : 
  
 "www.altostrat.com" 
 , 
  
 "ip" 
 : 
  
 [ 
 "198.51.100.68" 
 ], 
  
 "port" 
 : 
  
 443 
 , 
  
 "url" 
 : 
  
 "www.altostrat.com/images/logo.png" 
  
 }, 
  
 "network" 
 : 
  
 { 
  
 "http" 
 : 
  
 { 
  
 "method" 
 : 
  
 "GET" 
 , 
  
 "responseCode" 
 : 
  
 200 
  
 } 
  
 } 
  
 } 
  
 }] 
  
 } 
 } 
 

Body parameters

Events.udm object fields

Response

You shouldn't receive a response unless there is an error in the method syntax.

Response error codes

See HTTP response codes for common error codes.

ImportLogs

Import log telemetry. Use this method to send unstructured log entries to Google SecOps in batches. Each batch of log data has a maximum size limit of 4 MB (uncompressed).

Reference Documentation: Method: logs.import

Full name: projects.locations.instances.logTypes.logs.import

Request

POST https://chronicle.{region}.rep.googleapis.com/v1alpha/{parent}/logs:import

Path parameters

Request body

The following example shows how to format AZURE_AD log data from a source file, using the ImportLogs API method.

  { 
  
 "parent" 
 : 
  
 "projects/gcp_project_id/locations/us/instances/aaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa/logTypes/AZURE_AD" 
 , 
  
 "inlineSource" 
 : 
  
 { 
  
 "logs" 
 : 
  
 [{ 
  
 "data" 
 : 
  
 "dGVzdC5sb2c=" 
 , 
  
 "logEntryTime" 
 : 
  
 "2025-06-25T06:07:51.000001234Z" 
 , 
  
 "collectionTime" 
 : 
  
 "2025-06-25T06:07:51.000000123Z" 
 , 
  
 "labels" 
 : 
  
 { 
  
 "google" 
 : 
  
 { 
  
 "value" 
 : 
  
 "dev" 
 , 
  
 "rbacEnabled" 
 : 
  
 true 
  
 } 
  
 } 
  
 }], 
  
 "sourceFilename" 
 : 
  
 "source_filename.txt" 
  
 } 
 } 
 

Body parameters

Log object fields

Response

You shouldn't receive a response unless there is an error in the method syntax.

Response error codes

See HTTP response codes for common error codes.

ImportEntities

Use this method to import entities into Google SecOps.

An entity provides additional context about an entity in a UDM event, like asset or user details. For example, a PROCESS_LAUNCH event describes that user abc@example.corp launched process shady.exe . The event does not include information that user abc@example.com is a recently terminated employee who administers a server storing finance data. Information stored in one or more entities can add this additional context.

You are limited to 4MB of data per request.

Reference Documentation: Method: entities.import

Full name: projects.locations.instances.entities.import

Request

POST https://chronicle.{region}.rep.googleapis.com/v1alpha/{parent}/entities:import

Path parameters

Request body

The following example shows how to import entities data using the ImportEntities API method.

 {
  "parent": "projects/gcp_project_id/locations/us/instances/aaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa",
  "inlineSource": {
    "entities": [{
      "metadata": {
        "collectedTimestamp": "1970-01-01T03:25:45.000000124Z",
        "vendorName": "vendor",
        "productName": "product",
        "entityType": "USER"
      },
      "entity": {
        "user": {
          "userid": "google dev",
          "productObjectId": "dev google"
        }
      }
    }, {
      "metadata": {
        "collectedTimestamp": "1970-01-01T03:25:45.000000124Z",
        "vendorName": "vendor",
        "productName": "product",
        "entityType": "USER"
      },
      "entity": {
        "user": {
          "userid": "google dev",
          "productObjectId": "dev google"
        }
      }
    }],
    "logType": "AZURE_AD_CONTEXT"
  }
} 

Body parameters

Entity object fields

Response

If successful, the response body is empty.

Response error codes

See HTTP response codes for common error codes.

ListLogTypes

Use this method to retrieve a list of all the supported log types.

Reference Documentation: Method: logTypes.list

Full name: projects.locations.instances.logTypes.list

Request

GET https://chronicle.{region}.rep.googleapis.com/v1alpha/{parent}/logTypes

Path parameters

Query parameters

Request body

The request body must be empty.

Response

The following example shows the information returned from the ListLogTypes API method.

 {
  "logTypes": [{
    "name": "projects/gcp_project_id/locations/us/instances/aaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa/logTypes/A10_LOAD_BALANCER",
    "displayName": "A10 Load Balancer"
  }, {
    "name": "projects/gcp_project_id/locations/us/instances/aaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa/logTypes/ABNORMAL_SECURITY",
    "displayName": "Abnormal Security"
  }, {
    "name": "projects/gcp_project_id/locations/us/instances/aaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa/logTypes/ABSOLUTE",
    "displayName": "Absolute Mobile Device Management"
  }, {
    "name": "projects/gcp_project_id/locations/us/instances/aaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa/logTypes/ACALVIO",
    "displayName": "Acalvio"
  }, {
    "name": "projects/gcp_project_id/locations/us/instances/aaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa/logTypes/ACTIVE_IDENTITY_HID",
    "displayName": "Active Identity HID"
  }],
  "nextPageToken": "Qp4b7jgMSY3Byb2plY3RzL21hbGFjaGl0ZS1jYXRmb29kLWJ5b3Atc3RhZ2"
} 

Response fields

LogType object fields

Response error codes

See HTTP response codes for common error codes.

Error handling

Here's what to do if an HTTP request fails, depending on the HTTP response code:

HTTP response codes

• 400 Bad Request: The request is invalid.
  Check the error message for details. Don't retry without modification.
• 401 Unauthorized: Authentication failed.
  Check your credentials.
• 403 Forbidden: Permission denied.
  Make sure the caller has the correct IAM permissions.
• 404 Not Found: The resource specified in the request doesn't exist.
  Check the resource name.
• 429 Resource Exhausted: Quota limit reached or rate limit exceeded.
  Retry using exponential backoff , with a minimum delay of 30 seconds. If the request continues to fail, see What to do when reaching the burst limit .
• 500 Internal Server Error: An error occurred on the server.
  Retry using exponential backoff , with a minimum delay of 1 second.
• 503 Service Unavailable: The service is temporarily unavailable.
  Retry using exponential backoff , with a minimum delay of 1 second.

View Chronicle API HTTP errors

You can view Chronicle API HTTP errors for your Bring Your Own Project (BYOP) projects using the Metrics Explorer in Cloud Monitoring .

To see the HTTP errors for your BYOP project, follow these steps:

1. Go to the Google Cloud consoleand select your project.
2. Go to Monitoring> Metrics Explorer.
3. Click Select a Metric.
4. Choose Consumed APIas the resource type.
5. Choose API request count(serviceruntime.googleapis.com/api/request_count) as the metric.
6. Add filters to narrow down the results:
   • service : Specify the API service name (e.g., compute.googleapis.com).
   • response_code_class : Filter for 4xx or 5xx.
   • method : Specify the API method.
7. You can group the results to analyze the errors, for example, by: method , response_code , or service .

Frequently asked questions

Can customers create an allowlist of specific public IP addresses permitted to connect to the collector API endpoint, or is another form of client authorization supported?

Google SecOps doesn't support allowlisting specific IP addresses. Any connection with a valid API key can connect to the collector API endpoint.

Do you support standard HTTP GZIP compression (for example, Accept-Encoding: gzip header)?

Yes. The collector API endpoint supports standard HTTP GZIP compression using the Accept-Encoding: gzip header.

Need more help? Get answers from Community members and Google SecOps professionals.