Collect Microsoft IIS logs

Supported in:

This document explains how to collect Microsoft Internet Information Services (IIS) logs to Google Security Operations by using Bindplane. The parser first attempts to cleanse and normalize the input data by removing unnecessary characters and standardizing field names. Then, it uses a series of grok patterns to extract relevant fields from various Microsoft IIS log formats and maps them to the unified data model (UDM).

Before you begin

  • Ensure that you have a Google SecOps instance.
  • Ensure that you have a Windows 2016 or later.
  • If running behind a proxy, ensure firewall ports are open.

Get Google SecOps ingestion authentication file

  1. Sign in to the Google SecOps console.
  2. Go to SIEM Settings > Collection Agents.
  3. Download the Ingestion Authentication File. Save the file securely on the system where Bindplane will be installed.

Get Google SecOps customer ID

  1. Sign in to the Google SecOps console.
  2. Go to SIEM Settings > Profile.
  3. Copy and save the Customer IDfrom the Organization Detailssection.

Install the Bindplane agent on Windows

  1. Open the Command Promptor PowerShellas an administrator.
  2. Run the following command:

      msiexec 
      
     / 
     i 
      
     "https://github.com/observIQ/bindplane-agent/releases/latest/download/observiq-otel-collector.msi" 
      
     / 
     quiet 
     
    

Additional installation resources

Configure the Bindplane agent to ingest Syslog and send to Google SecOps

  1. Before configuring the YAML file, stop the observIQ Distro for Open Telemetry Collector Servicein the Services Panel.
  2. Access the configuration file:

    1. Locate the config.yaml file. Typically, it's in the /etc/bindplane-agent/ directory on Linux or in the installation directory on Windows.
    2. Open the file using a text editor (for example, nano , vi , or Notepad).
  3. Edit the config.yaml file as follows:

      receivers 
     : 
      
     iis 
     : 
      
     collection_interval 
     : 
      
     60s 
     processors 
     : 
      
     # Resourcedetection is used to add a unique (host.name) to the metric resource(s), allowing users to filter between multiple agent systems. 
      
     resourcedetection 
     : 
      
     detectors 
     : 
      
     [ 
     "system" 
     ] 
      
     system 
     : 
      
     hostname_sources 
     : 
      
     [ 
     "os" 
     ] 
      
     normalizesums 
     : 
      
     batch 
     : 
     exporters 
     : 
      
     chronicle/powershell 
     : 
      
     endpoint 
     : 
      
     malachiteingestion-pa.googleapis.com 
      
     # Adjust the path to the credentials file you downloaded in Step 1 
      
     creds 
     : 
      
     '/path/to/ingestion-authentication-file.json' 
      
     log_type 
     : 
      
     'IIS' 
      
     override_log_type 
     : 
      
     false 
      
     raw_log_field 
     : 
      
     body 
      
     customer_id 
     : 
      
     '<customer_id>' 
     service 
     : 
      
     pipelines 
     : 
      
     logs/winpowershell 
     : 
      
     receivers 
     : 
      
     - 
      
     iis 
      
     processors 
     : 
      
     - 
      
     resourcedetection 
      
     - 
      
     normalizesums 
      
     - 
      
     batch 
      
     exporters 
     : 
      
     [ 
     chronicle/iis 
     ] 
     
    
  4. Replace <customer_id> with the actual customer ID.

  5. Update /path/to/ingestion-authentication-file.json to the path where the authentication file was saved in the Get Google SecOps ingestion authentication file section.

  6. After saving the config.yaml file, startthe observIQ Distro for Open Telemetry Collector Service.

Restart the Bindplane agent to apply the changes

  • To restart the Bindplane agent in Windows, you can either use the Servicesconsole or enter the following command:

     net stop BindPlaneAgent && net start BindPlaneAgent 
    

UDM Mapping Table

Log field UDM mapping Logic
@timestamp
metadata.event_timestamp The timestamp of the event as recorded in the raw log.
@version
metadata.product_version The version of the IIS server.
AgentDevice
additional.fields.AgentDevice.value.string_value The device that generated the log.
AgentLogFile
additional.fields.AgentLogFile.value.string_value The name of the log file.
ASP.NET_SessionId
network.session_id The session ID of the user.
c-ip
principal.ip The IP address of the client.
Channel
security_result.about.resource.attribute.labels.Channel.value The channel where the event was logged.
ChannelID
security_result.about.resource.attribute.labels.ChannelID.value The ID of the channel where the event was logged.
Computer
target.hostname The hostname of the target machine.
cs-bytes
network.received_bytes The number of bytes received from the client.
cs-host
principal.hostname, principal.asset.hostname The hostname of the client.
cs-method
network.http.method The HTTP method used by the client.
cs-uri-query
target.url The query string of the URL requested by the client.
cs-uri-stem
target.url The path of the URL requested by the client.
cs-username
principal.user.user_display_name The username of the client.
cs-version
network.tls.version_protocol The HTTP version used by the client.
cs(Cookie)
Used to extract cookie information.
cs(Referer)
network.http.referral_url The URL that referred the client to the current page.
cs(User-Agent)
network.http.user_agent The user agent of the client.
csbyte
network.received_bytes The number of bytes received from the client.
cshost
principal.hostname, principal.asset.hostname The hostname of the client.
csip
principal.ip, principal.asset.ip The IP address of the client.
csmethod
network.http.method The HTTP method used by the client.
csreferer
network.http.referral_url The URL that referred the client to the current page.
csuseragent
network.http.user_agent The user agent of the client.
csusername
principal.user.user_display_name The username of the client.
csversion
network.tls.version_protocol The HTTP version used by the client.
date
Used to construct the event timestamp if the raw log timestamp is invalid.
description
security_result.description A description of the event.
devicename
target.hostname The hostname of the target machine.
dst_ip
target.ip, target.asset.ip The IP address of the target machine.
dst_port
target.port The port number of the target machine.
duration
The duration of the request in milliseconds.
EventEnqueuedUtcTime
additional.fields.EventEnqueuedUtcTime.value.string_value The time when the event was enqueued in UTC.
EventID
metadata.product_log_id The ID of the event.
EventProcessedUtcTime
additional.fields.EventProcessedUtcTime.value.string_value The time when the event was processed in UTC.
EventTime
metadata.event_timestamp The timestamp of the event.
EventType
metadata.product_event_type The type of the event.
file_path
target.file.full_path The full path of the file involved in the event.
FilterId
security_result.about.resource.attribute.labels.FilterId.value The ID of the filter.
FilterKey
security_result.about.resource.attribute.labels.FilterKey.value The key of the filter.
FilterName
security_result.about.resource.attribute.labels.FilterName.value The name of the filter.
FilterType
security_result.about.resource.attribute.labels.FilterType.value The type of the filter.
host
target.hostname The hostname of the target machine.
host.architecture
principal.asset.hardware.cpu_platform The architecture of the host machine.
host.geo.name
additional.fields.geo_name.value.string_value The geographical location of the host machine.
host.hostname
target.hostname, target.asset.hostname The hostname of the host machine.
host.id
observer.asset_id The ID of the host machine.
host.ip
principal.ip, principal.asset.ip The IP address of the host machine.
host.mac
principal.mac The MAC address of the host machine.
host.os.build
additional.fields.os_build.value.string_value The build number of the operating system on the host machine.
host.os.kernel
principal.platform_patch_level The kernel version of the operating system on the host machine.
host.os.name
additional.fields.os_name.value.string_value The name of the operating system on the host machine.
host.os.platform
principal.platform The platform of the operating system on the host machine.
host.os.version
principal.platform_version The version of the operating system on the host machine.
http_method
network.http.method The HTTP method used by the client.
http_response
network.http.response_code The HTTP response code.
http_status_code
network.http.response_code The HTTP status code of the response.
http_substatus
additional.fields.sc_substatus.value.string_value The HTTP substatus code of the response.
instance
additional.fields.instance.value.string_value The instance ID of the task.
intermediary_devicename
intermediary.hostname, intermediary.asset.hostname The hostname of the intermediary device.
json_message
The raw log message in JSON format.
kv_fields
Used to extract key-value pairs from the raw log message.
LayerKey
security_result.about.resource.attribute.labels.LayerKey.value The key of the layer.
LayerName
security_result.about.resource.attribute.labels.LayerName.value The name of the layer.
LayerId
security_result.about.resource.attribute.labels.LayerId.value The ID of the layer.
log.file.path
target.file.full_path The full path of the log file.
log.offset
metadata.product_log_id The offset of the event in the log file.
logstash.collect.host
observer.hostname The hostname of the machine that collected the log.
logstash.process.host
intermediary.hostname The hostname of the machine that processed the log.
logstash_json_message
The raw log message in JSON format.
message
security_result.description The raw log message.
ministry
additional.fields.ministry.value.string_value The ministry associated with the event.
name
The name of the entity.
NewValue
additional.fields.NewValue.value.string_value The new value of the configuration setting.
OldValue
additional.fields.OldValue.value.string_value The old value of the configuration setting.
port
principal.port The port number of the client.
priority_code
The priority code of the syslog message.
ProcessID
principal.process.pid The process ID of the process that generated the event.
ProviderGuid
security_result.about.resource.attribute.labels.ProviderGuid.value The GUID of the provider.
ProviderKey
security_result.about.resource.attribute.labels.ProviderKey.value The key of the provider.
ProviderName
security_result.about.resource.attribute.labels.ProviderName.value The name of the provider.
referrer_url
network.http.referral_url The URL that referred the client to the current page.
request_url
target.url The URL requested by the client.
s-computername
target.hostname The hostname of the target machine.
s-ip
target.ip, target.asset.ip The IP address of the target machine.
s-port
target.port The port number of the target machine.
s-sitename
additional.fields.sitename.value.string_value The name of the site.
sc-bytes
network.sent_bytes The number of bytes sent to the client.
sc-status
network.http.response_code The HTTP status code of the response.
sc-substatus
additional.fields.sc_substatus.value.string_value The HTTP substatus code of the response.
sc-win32-status
The Windows status code of the response.
scbyte
network.sent_bytes The number of bytes sent to the client.
scstatus
network.http.response_code The HTTP status code of the response.
severity
security_result.severity The severity of the event.
service.type
additional.fields.service_type.value.string_value The type of the service.
sIP
principal.ip, principal.asset.ip The IP address of the client.
sPort
principal.port The port number of the client.
sSiteName
additional.fields.sitename.value.string_value The name of the site.
src_ip
principal.ip, principal.asset.ip, observer.ip The IP address of the client.
src_port
principal.port The port number of the client.
sysdate
The date and time of the syslog message.
syslog_facility
security_result.severity_details The facility of the syslog message.
syslog_pri
The priority of the syslog message.
syslog_severity
security_result.severity_details The severity of the syslog message.
syslog_severity_code
The severity code of the syslog message.
tags
security_result.rule_name Tags associated with the event.
task
additional.fields.task.value.string_value The name of the task.
time
Used to construct the event timestamp if the raw log timestamp is invalid.
time-taken
The duration of the request in milliseconds.
uri_query
target.url The query string of the URL requested by the client.
user_agent
network.http.user_agent The user agent of the client.
UserName
target.user.userid The username of the user.
UserSid
target.user.windows_sid The Windows SID of the user.
Weight
security_result.about.resource.attribute.labels.Weight.value The weight of the filter.
win32_status
The Windows status code of the response.
xforwardedfor
The X-Forwarded-For header, containing a comma-separated list of IP addresses.
metadata.log_type IIS
network.direction INBOUND
metadata.vendor_name Microsoft
metadata.product_name Internet Information Server
metadata.event_type NETWORK_HTTP , USER_UNCATEGORIZED , GENERIC_EVENT , STATUS_UPDATE , USER_LOGOUT , USER_LOGIN
extensions.auth.type MACHINE

Need more help? Get answers from Community members and Google SecOps professionals.

Design a Mobile Site
View Site in Mobile | Classic
Share by: