Silent-host monitoring

This document explains the methods of how Google Security Operations silent-host monitoring ( SHM ) lets you identify hosts in your environment that have gone silent.

A silent host can signal potential collector stoppages.

Use a detection rule for SHM

We recommend that you configure a detection rule for SHM.

This method monitors UDM fields (such as hostname , ip , or mac ) and triggers an alert when expected values have not been received within a specified time period.

Example detection rules for SHM

This section contains example detection rules for SHM, which you can use or adapt for your specific needs. For example, you can modify the time period, use other fields (such as ip ), or split up the rule in other ways.

Detect silent hosts by hostname

The following example detects silent hosts by hostname :

• Scans 20-minute windows.
• Tracks hosts using ip , hostname , and mac .
• Alerts if no events have been received for a host in the last 10 minutes, but not silent in the last 20 minutes.

 rule shm_using_hostname {
 meta:

 events:
   $event.metadata.event_timestamp.seconds > timestamp.current_seconds() - 1200
   // $identifier_hash = hash.sha256(strings.concat($event.principal.ip[0], $event.principal.hostname, $event.principal.mac[0]))
   $silent_hostname = $event.principal.hostname
 match:
   $silent_hostname over 10m
 outcome:
   $max_event_time = max($event.metadata.event_timestamp.seconds)
   $max_diff = timestamp.current_seconds() - $max_event_time
 condition:
   $event and $max_diff > 600
} 

Detect silent collectors using ingestion labels

The following example detection rule for SHM:

• Scans 20-minute windows.
• Identifies silence using the ingestion_source ingestion label.
• Identifies values from a specified timezone (in this example, Asia/Calcutta) that have been silent for the last 10 minutes, but not silent in the last 20 minutes.

 rule shm_using_ingestion_label {
 meta:

 events:
   $event.metadata.event_timestamp.seconds > timestamp.current_seconds() - 1200
   $event.metadata.ingestion_labels.key = "ingestion_source"
   $silent_ingestion_source = $event.metadata.ingestion_labels.value
 match:
   $silent_ingestion_source over 20m
 outcome:
   $max_time_seconds = max($event.metadata.event_timestamp.seconds)
   $max_diff_seconds = timestamp.current_seconds() - $max_time_seconds
   $max_timestamp = timestamp.get_timestamp($max_time_seconds, "SECOND", "Asia/Calcutta")
   $current_timestamp = timestamp.get_timestamp(timestamp.current_seconds(), "SECOND", "Asia/Calcutta")
 condition:
   $event and $max_diff_seconds > 600
} 

Use Google Cloud Monitoring with ingestion labels for SHM

This method uses Google Cloud Monitoring to monitor log ingestion rates based on ingestion labels for SHM.

This section describes how to set up this method using Bindplane , which includes the following steps:

1. Configure Bindplane for SHM with Google Cloud Monitoring
2. Configure the Google Cloud Monitoring threshold for SHM

After you set up a logs pipeline that applies ingestion labels for SHM, you can set up Google Cloud Monitoring alerts per collector—for when the ingestion rate falls below a specified threshold. You can configure the alerts to go to a variety of places outside of Google SecOps and integrate the alerts into a workflow.

Benefits of this method:

• Monitors ingestion time, not event time.
• Leverages Cloud Monitoring's advanced alerting capabilities.

Downsides of this method:

• Requires a separate configuration outside of Google SecOps.
• Limited by the number of ingestion labels .

Configure Bindplane for SHM with Google Cloud Monitoring

The prerequisites to configure Bindplane for SHM with Google Cloud Monitoring are as follows:

• A deployed Bindplane server that is configured with a Google SecOps Standardization processor .
• The Google SecOps Standardization processor is configured to add a supported log_type and an ingestion label (for example, ingestion_source ).

To configure Bindplane for SHM with Google Cloud Monitoring, complete the following steps:

1. Send the hostname of the collector server as an attribute in each log entry.
2. On the Logtab, select Processors > Add Processors > Copy Field.
3. Configure the Copy Fieldprocessor:
   • Enter a short description for the resource.
   • Choose the Logs telemetry type.
   • Set the Copy From field to Resources .
   • Set the Resource field field to host.name .
   • Set the Copy To field field to Attributes .
   • Set the Attributes Field field, for example, to chronicle_ingestion_label["ingestion_source"] .

Configure the Google Cloud Monitoring threshold for SHM

Define a threshold based on your expected ingestion rate. Lower thresholds detect collector outages; higher thresholds detect upstream log gaps.

After you configure the Google Cloud Monitoring threshold for SHM, we recommend that you monitor the Chronicle Collector > Ingestion > Total Ingestion Log Countmetric. For detailed sample-setup instructions, go to Set up a sample policy to detect silent Google SecOps collection agents .

Use a Google SecOps dashboard for SHM

Use a Google SecOps dashboard to view daily counts for monitoring hosts that have gone silent.

This method is great for high-level daily overviews, but this method does not support alerts, and the results have a latency of up to 6 hours.

Need more help? Get answers from Community members and Google SecOps professionals.