Console
    Start free

    Collect Apache logs

    Supported in:

    This document explains how to ingest Apache logs to Google Security Operations using Bindplane agent.

    Apache HTTP Server is an open-source web server that serves HTTP requests and generates access and error logs. The parser handles both standard combined/common log formats and JSON-formatted logs, extracting HTTP request details, response codes, and client information.

    Before you begin

    Make sure you have the following prerequisites:

    • A Google SecOps instance
    • Windows Server 2016 or later, or Linux host with systemd
    • Network connectivity between the Bindplane agent and the Apache server
    • If running behind a proxy, ensure firewall ports are open per the Bindplane agent requirements
    • Privileged access to the Apache server (root or sudo)

    Get Google SecOps ingestion authentication file

    1. Sign in to the Google SecOps console.
    2. Go to SIEM Settings > Collection Agents.
    3. Download the Ingestion Authentication File.

    4. Save the file securely on the system where the Bindplane agent will be installed.

    Get Google SecOps customer ID

    1. Sign in to the Google SecOps console.
    2. Go to SIEM Settings > Profile.

    3. Copy and save the Customer IDfrom the Organization Detailssection.

    Install the Bindplane agent

    Install the Bindplane agent on your Windows or Linux operating system according to the following instructions.

    Windows installation

    1. Open Command Promptor PowerShellas an administrator.

    2. Run the following command:

        msiexec 
  
 / 
 i 
  
 "https://github.com/observIQ/bindplane-agent/releases/latest/download/observiq-otel-collector.msi" 
  
 / 
 quiet

    3. Wait for the installation to complete.

    4. Verify the installation by running:

       sc query observiq-otel-collector

      The service should show as RUNNING.

    Linux installation

    1. Open a terminal with root or sudo privileges.

    2. Run the following command:

       sudo  
sh  
-c  
 " 
 $( 
curl  
-fsSlL  
https://github.com/observiq/bindplane-agent/releases/latest/download/install_unix.sh ) 
 " 
  
install_unix.sh

    3. Wait for the installation to complete.

    4. Verify the installation by running:

       sudo  
systemctl  
status  
observiq-otel-collector

      The service should show as active (running).

    Additional installation resources

    For additional installation options and troubleshooting, see Bindplane agent installation guide .

    Configure Bindplane agent to ingest syslog and send to Google SecOps

    Locate the configuration file

    • Linux:

       sudo  
nano  
/etc/bindplane-agent/config.yaml

    • Windows:

       notepad "C:\Program Files\observIQ OpenTelemetry Collector\config.yaml"

    Edit the configuration file

    • Replace the entire contents of config.yaml with the following configuration:

        receivers 
 : 
  
 udplog 
 : 
  
 listen_address 
 : 
  
 "0.0.0.0:514" 
 exporters 
 : 
  
 chronicle/apache 
 : 
  
 compression 
 : 
  
 gzip 
  
 creds_file_path 
 : 
  
 '/etc/bindplane-agent/ingestion-auth.json' 
  
 customer_id 
 : 
  
 '<customer_id>' 
  
 endpoint 
 : 
  
 malachiteingestion-pa.googleapis.com 
  
 log_type 
 : 
  
 APACHE 
  
 raw_log_field 
 : 
  
 body 
 service 
 : 
  
 pipelines 
 : 
  
 logs/apache_to_chronicle 
 : 
  
 receivers 
 : 
  
 - 
  
 udplog 
  
 exporters 
 : 
  
 - 
  
 chronicle/apache

    Configuration parameters

    Replace the following placeholders:

    • Receiver configuration:

      • listen_address : IP address and port to listen on:
        • 0.0.0.0 to listen on all interfaces (recommended)
        • Port 514 is the standard syslog port (requires root on Linux; use 1514 for non-root)

    • Exporter configuration:

      • creds_file_path : Full path to ingestion authentication file:
        • Linux: /etc/bindplane-agent/ingestion-auth.json
        • Windows: C:\Program Files\observIQ OpenTelemetry Collector\ingestion-auth.json
      • customer_id : Customer ID copied from the Google SecOps console
      • endpoint : Regional endpoint URL:
        • US: malachiteingestion-pa.googleapis.com
        • Europe: europe-malachiteingestion-pa.googleapis.com
        • Asia: asia-southeast1-malachiteingestion-pa.googleapis.com
        • See Regional Endpoints for complete list

    Save the configuration file

    • After editing, save the file:
      • Linux: Press Ctrl+O , then Enter , then Ctrl+X
      • Windows: Click File > Save

    Restart the Bindplane agent to apply the changes

    • To restart the Bindplane agent in Linux, run the following command:

       sudo  
systemctl  
restart  
observiq-otel-collector

      1. Verify the service is running:

         sudo  
systemctl  
status  
observiq-otel-collector

      2. Check logs for errors:

         sudo  
journalctl  
-u  
observiq-otel-collector  
-f

    • To restart the Bindplane agent in Windows, choose one of the following options:

      • Command Prompt or PowerShell as administrator:

         net stop observiq-otel-collector && net start observiq-otel-collector

      • Services console:

        1. Press Win+R , type services.msc , and press Enter.
        2. Locate observIQ OpenTelemetry Collector.
        3. Right-click and select Restart.

        4. Verify the service is running:

           sc query observiq-otel-collector

        5. Check logs for errors:

            type 
  
 "C:\Program Files\observIQ OpenTelemetry Collector\log\collector.log"

    Configure syslog in Apache

    1. Sign in to the server hosting Apache using SSH.

    2. Create a file under /etc/rsyslog.d/ named 02-apache2.conf :

       vim  
/etc/rsyslog.d/02-apache2.conf

    3. Add the following code to the file:

       module(load="imfile" PollingInterval="10" statefile.directory="/var/spool/rsyslog")

input(type="imfile"
    File="/var/log/apache2/access.log"
    Tag="http_access"
    Severity="info"
    Facility="local6")

Local6.info @<BINDPLANE_IP>:<BINDPLANE_PORT>

module(load="imfile" PollingInterval="10" statefile.directory="/var/spool/rsyslog")

input(type="imfile"
    File="/var/log/apache2/error.log"
    Tag="http_error"
    Severity="error"
    Facility="local7")

Local7.error @<BINDPLANE_IP>:<BINDPLANE_PORT>
      • Replace <BINDPLANE_IP> and <BINDPLANE_PORT> with the IP address and port configured for the Bindplane agent.
      • If you are using TCP protocol, append an additional @ to the host line (for example, @@<BINDPLANE_IP>:<BINDPLANE_PORT> ).

    4. Restart rsyslog services:

       sudo  
service  
rsyslog  
restart

    UDM mapping table

    Log field UDM mapping Logic
    bytes
    		network.received_bytes Bytes received from the client.
    bytes
    		network.sent_bytes Bytes sent to the client.
    bytes_out
    		network.sent_bytes Bytes sent to the client.
    bytes_received
    		network.received_bytes Bytes received from the client.
    Content
    		network.http.method HTTP method extracted from the "Content" field.
    Content
    		target.url Target URL extracted from the "Content" field.
    cookie
    		additional.fields.value.string_value Value of the "cookie" field.
    dest_ip
    		target.ip IP address of the target.
    dest_name
    		target.hostname Hostname of the target.
    dest_port
    		target.port Port of the target.
    description
    		metadata.description Description of the event.
    duration_microseconds
    		additional.fields.value.string_value Value of the "duration_microseconds" field.
    file_full_path
    		target.file.full_path Full path of the target file.
    hostname
    		target.hostname Hostname of the target.
    http_content_type
    		additional.fields.value.string_value Value of the "http_content_type" field.
    http_host
    		principal.hostname Hostname of the principal.
    http_method
    		network.http.method HTTP method.
    http_referrer
    		network.http.referral_url HTTP referrer URL.
    http_user_agent
    		network.http.user_agent HTTP user agent.
    ID
    		metadata.id ID of the event.
    insertId
    		metadata.product_log_id Product log ID.
    ip
    		principal.ip IP address of the principal.
    jsonPayload.cIP
    		target.ip IP address of the target.
    jsonPayload.cPort
    		target.port Port of the target.
    jsonPayload.csBytes
    		network.sent_bytes Bytes sent to the client.
    jsonPayload.csMethod
    		network.http.method HTTP method.
    jsonPayload.csMimeType
    		target.file.mime_type MIME type of the target file.
    jsonPayload.csReferer
    		network.http.referral_url HTTP referrer URL.
    jsonPayload.csURL
    		target.url Target URL.
    jsonPayload.csUserAgent
    		network.http.user_agent HTTP user agent.
    jsonPayload.sHierarchy
    		additional.fields.value.string_value Value of the "sHierarchy" field.
    jsonPayload.sHostname
    		principal.hostname Hostname of the principal.
    jsonPayload.sIP
    		principal.ip IP address of the principal.
    jsonPayload.scBytes
    		network.received_bytes Bytes received from the client.
    jsonPayload.scHTTPStatus
    		network.http.response_code HTTP response code.
    jsonPayload.scResultCode
    		additional.fields.value.string_value Value of the "scResultCode" field.
    LastStatus
    		network.http.response_code HTTP response code.
    log_level
    		security_result.severity Severity of the security result.
    logName
    		security_result.category_details Category details of the security result.
    method
    		network.http.method HTTP method.
    pid
    		principal.process.pid Process ID of the principal.
    Port
    		target.port Port of the target.
    proto
    		network.application_protocol Application protocol.
    referer
    		network.http.referral_url HTTP referrer URL.
    RemoteHost
    		principal.ip IP address of the principal.
    RemoteUser
    		principal.user.userid User ID of the principal.
    resource.labels.instance_id
    		target.resource.product_object_id Product object ID of the target resource.
    resource.labels.project_id
    		target.resource.attribute.labels.value Value of the "project_id" label.
    resource.labels.zone
    		target.resource.attribute.cloud.availability_zone Availability zone of the target resource.
    resource.type
    		target.resource.resource_type Resource type of the target.
    response
    		network.http.response_code HTTP response code.
    SizeBytes
    		network.received_bytes Bytes received from the client.
    src_ip
    		principal.ip IP address of the principal.
    src_port
    		principal.port Port of the principal.
    ssl_cipher
    		network.tls.cipher TLS cipher.
    ssl_version
    		network.tls.version_protocol TLS version protocol.
    status
    		network.http.response_code HTTP response code.
    target
    		target.url Target URL.
    target_ip
    		target.ip IP address of the target.
    target_port
    		target.port Port of the target.
    time
    		metadata.event_timestamp Event timestamp.
    uri_path
    		target.process.file.full_path Full path of the target file.
    user
    		principal.user.userid User ID of the principal.
    useragent
    		network.http.user_agent HTTP user agent.
    version_protocol
    		network.tls.version_protocol TLS version protocol.
    Workername
    		principal.hostname Hostname of the principal.
    x_forwarded_for
    		Value of the "X-Forwarded-For" header.
    		metadata.log_type The value is set to "APACHE" in the parser code.
    		metadata.product_name The value is set to "Apache Web Server" in the parser code.
    		metadata.vendor_name The value is set to "Apache" in the parser code.
    		metadata.event_type The value is determined based on the presence of principal and target information. If both principal and target are present, the event type is set to "NETWORK_HTTP". If only principal is present, the event type is set to "STATUS_UPDATE". Otherwise, it is set to "GENERIC_EVENT".
    		additional.fields.key The key is set to "keep_alive", "duration_microseconds", "cookie", "http_content_type", "sHierarchy", "scResultCode" in the parser code based on the field.
    		target.port If the "proto" field is "HTTP", the port is set to 80. If the "proto" field is "HTTPS", the port is set to 443. If the "proto" field is "FTP", the port is set to 21.
    		target.resource.attribute.labels.key The key is set to "project_id" in the parser code.

    Need more help? Get answers from Community members and Google SecOps professionals.

    Design a Mobile Site
    View Site in Mobile | Classic
    Share by: