Collect Zscaler Webproxy logs

This document describes how you can export Zscaler Webproxy logs by setting up a Google Security Operations feed and how log fields map to Google SecOps Unified Data Model (UDM) fields.

For more information, see Data ingestion to Google SecOps overview .

A typical deployment consists of Zscaler Webproxy and the Google SecOps Webhook feed configured to send logs to Google SecOps. Each customer deployment can differ and might be more complex.

The deployment contains the following components:

• Zscaler Webproxy: The platform from which you collect logs.

• Google SecOps feed: The Google SecOps feed that fetches logs from Zscaler Webproxy and writes logs to Google SecOps.

• Google SecOps: Retains and analyzes the logs.

An ingestion label identifies the parser which normalizes raw log data to structured UDM format. The information in this document applies to the parser with the ZSCALER_WEBPROXY ingestion label.

Before you begin

Ensure you have the following prerequisites:

• Access to Zscaler Internet Access console. For more information, see Secure Internet and SaaS Access ZIA Help .
• Zscaler Webproxy 2024 or later
• All systems in the deployment architecture are configured with the UTC time zone.
• The API key which is needed to complete feed setup in Google Security Operations. For more information, see Setting up API keys .

Set up feeds

There are two different entry points to set up feeds in the Google SecOps platform:

• SIEM Settings > Feeds
• Content Hub > Content Packs

Set up feeds from SIEM Settings > Feeds

To configure multiple feeds for different log types within this product family, see Configure feeds by product .

To configure a single feed, follow these steps:

1. Go to SIEM Settings > Feeds.
2. Click Add New Feed.
3. On the next page, click Configure a single feed.
4. In the Feed namefield, enter a name for the feed; for example, Zscaler Webproxy Logs.
5. Select Webhookas the Source Type.
6. Select Zscaleras the Log Type.
7. Click Next.
8. Optional: Enter values for the following input parameters:
   1. Split delimiter: The delimiter that is used to separate the logs lines. Leave blank if a delimiter is not used.
   2. Asset namespace: The asset namespace.
   3. Ingestion labels: The label to be applied to the events from this feed.
9. Click Next.
10. Review your new feed configuration, and then click Submit.
11. Click Generate Secret Keyto generate a secret key to authenticate this feed.

Set up feeds from the Content Hub

Specify values for the following fields:

• Split delimiter: The delimiter that is used to separate log lines, such as \n .

Advanced options

• Feed Name: A prepopulated value that identifies the feed.
• Source Type: Method used to collect logs into Google SecOps.
• Asset namespace: The asset namespace .
• Ingestion labels: The label applied to the events from this feed.
• Click Next.
• Review the feed configuration in the Finalizescreen, and then click Submit.
• Click Generate Secret Keyto generate a secret key to authenticate this feed.

Set up Zscaler Webproxy

1. In the Zscaler Internet Access console, click Administration > Nanolog Streaming Service > Cloud NSS Feedsand then click Add Cloud NSS Feed.
2. The Add Cloud NSS Feedwindow appears. In the Add Cloud NSS Feedwindow, enter the details.
3. Enter a name for the feed in the Feed Namefield.
4. Select NSS for Webin NSS Type.
5. Select the status from the Statuslist to activate or deactivate the NSS feed.
6. Keep the value in the SIEM Ratedrop-down as Unlimited. To suppress the output stream due to licensing or other constraints, change the value.
7. Select Otherin the SIEM Typelist.
8. Select Disabledin the OAuth 2.0 Authenticationlist.
9. Enter a size limit for an individual HTTP request payload to the SIEM's best practice in Max Batch Size. For example, 512 KB.

10. Enter the HTTPS URL of the Chronicle API endpoint in the API URL in the following format:

     https://<CHRONICLE_REGION>-chronicle.googleapis.com/v1alpha/projects/<GOOGLE_PROJECT_NUMBER>/locations/<LOCATION>/instances/<CUSTOMER_ID>/feeds/<FEED_ID>:importPushLogs 
    

    • CHRONICLE_REGION : Region where your Chronicle instance is hosted. For example, US.
    • GOOGLE_PROJECT_NUMBER : BYOP project number. Obtain this from C4.
    • LOCATION : Chronicle region. For example, US.
    • CUSTOMER_ID : Chronicle customer ID. Obtain from C4.
    • FEED_ID : Feed ID shown on Feed UI on the new webhook created
    • Sample API URL:

     https://us-chronicle.googleapis.com/v1alpha/projects/12345678910/locations/US/instances/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/feeds/yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy:importPushLogs 
    

11. Click Add HTTP Header, and then add HTTP headers in the following format:

    • Header 1 : Key1: X-goog-api-key and Value1:API Key generated on Google Cloud BYOP's API Credentials.
    • Header 2 : Key2: X-Webhook-Access-Key and Value2:API secret key generated on webhook's "SECRET KEY".

12. Select Web Logsin the Log Typeslist.

13. Select JSONin the Feed Output Typelist.

14. Set Feed Escape Characterto , \ " .

15. To add a new field to the Feed Output Format,select Customin the Feed Output Typelist.

16. Copy-paste the Feed Output Formatand add new fields. Ensure the key names match the actual field names.

17. Following is the default Feed Output Format:

       
     \ 
     { 
      
     "sourcetype" 
      
     : 
      
     "zscalernss-web" 
     , 
      
     "event" 
      
     : 
      
     \ 
     { 
     "datetime" 
     : 
     "%d{yy}-%02d{mth}-%02d{dd} %02d{hh}:%02d{mm}:%02d{ss}" 
     , 
     "reason" 
     : 
     "%s{reason}" 
     , 
     "event_id" 
     : 
     "%d{recordid}" 
     , 
     "protocol" 
     : 
     "%s{proto}" 
     , 
     "action" 
     : 
     "%s{action}" 
     , 
     "transactionsize" 
     : 
     "%d{totalsize}" 
     , 
     "responsesize" 
     : 
     "%d{respsize}" 
     , 
     "requestsize" 
     : 
     "%d{reqsize}" 
     , 
     "urlcategory" 
     : 
     "%s{urlcat}" 
     , 
     "serverip" 
     : 
     "%s{sip}" 
     , 
     "requestmethod" 
     : 
     "%s{reqmethod}" 
     , 
     "refererURL" 
     : 
     "%s{ereferer}" 
     , 
     "useragent" 
     : 
     "%s{eua}" 
     , 
     "product" 
     : 
     "NSS" 
     , 
     "location" 
     : 
     "%s{elocation}" 
     , 
     "ClientIP" 
     : 
     "%s{cip}" 
     , 
     "status" 
     : 
     "%s{respcode}" 
     , 
     "user" 
     : 
     "%s{elogin}" 
     , 
     "url" 
     : 
     "%s{eurl}" 
     , 
     "vendor" 
     : 
     "Zscaler" 
     , 
     "hostname" 
     : 
     "%s{ehost}" 
     , 
     "clientpublicIP" 
     : 
     "%s{cintip}" 
     , 
     "threatcategory" 
     : 
     "%s{malwarecat}" 
     , 
     "threatname" 
     : 
     "%s{threatname}" 
     , 
     "filetype" 
     : 
     "%s{filetype}" 
     , 
     "appname" 
     : 
     "%s{appname}" 
     , 
     "app_status" 
     : 
     "%s{app_status}" 
     , 
     "pagerisk" 
     : 
     "%d{riskscore}" 
     , 
     "threatseverity" 
     : 
     "%s{threatseverity}" 
     , 
     "department" 
     : 
     "%s{edepartment}" 
     , 
     "urlsupercategory" 
     : 
     "%s{urlsupercat}" 
     , 
     "appclass" 
     : 
     "%s{appclass}" 
     , 
     "dlpengine" 
     : 
     "%s{dlpeng}" 
     , 
     "urlclass" 
     : 
     "%s{urlclass}" 
     , 
     "threatclass" 
     : 
     "%s{malwareclass}" 
     , 
     "dlpdictionaries" 
     : 
     "%s{dlpdict}" 
     , 
     "fileclass" 
     : 
     "%s{fileclass}" 
     , 
     "bwthrottle" 
     : 
     "%s{bwthrottle}" 
     , 
     "contenttype" 
     : 
     "%s{contenttype}" 
     , 
     "unscannabletype" 
     : 
     "%s{unscannabletype}" 
     , 
     "deviceowner" 
     : 
     "%s{deviceowner}" 
     , 
     "devicehostname" 
     : 
     "%s{devicehostname}" 
     , 
     "keyprotectiontype" 
     : 
     "%s{keyprotectiontype}" 
     \ 
     } 
     \ 
     } 
     
    

18. Select the timezone for the Timefield in the output file in the Timezonelist. By default, the timezone is set to your organization's time zone.

19. Review the configured settings.

20. Click Saveto test connectivity. If the connection is successful, a green tick accompanied by the message Test Connectivity Successful: OK (200)appears.

For more information about Google SecOps feeds, see Google SecOps feeds documentation . For information about requirements for each feed type, see Feed configuration by type .

If you encounter issues when you create feeds, contact Google SecOps support .

Supported Zscaler Webproxy log formats

The Zscaler Webproxy parser supports logs in JSON format.

Supported Zscaler Webproxy Sample Logs

• JSON

   {
      "event": {
        "ClientIP": "198.51.100.0",
        "action": "Allowed",
        "appclass": "Sales and Marketing",
        "appname": "Trend Micro",
        "bwthrottle": "NO",
        "clientpublicIP": "198.51.100.1",
        "contenttype": "Other",
        "datetime": "2024-05-06 10:56:04",
        "department": "Mid-Continent%20Companies",
        "devicehostname": "dummyhostname",
        "deviceowner": "dummydeviceowner",
        "dlpdictionaries": "None",
        "dlpengine": "None",
        "event_id": "7365838693731467265",
        "fileclass": "None",
        "filetype": "None",
        "hostname": "dummyhostname.com",
        "keyprotectiontype": "N/A",
        "location": "Road%20Warrior",
        "pagerisk": "0",
        "product": "NSS",
        "protocol": "HTTP_PROXY",
        "reason": "Allowed",
        "refererURL": "None",
        "requestmethod": "CONNECT",
        "requestsize": "606",
        "responsesize": "65",
        "serverip": "198.51.10.2",
        "status": "200",
        "threatcategory": "None",
        "threatclass": "None",
        "threatname": "None",
        "threatseverity": "None",
        "transactionsize": "671",
        "unscannabletype": "None",
        "url": "dummyurl.com:443",
        "urlcategory": "SSL - DNI - Bypass",
        "urlclass": "Bandwidth Loss",
        "urlsupercategory": "User-defined",
        "user": "abc@xyz.com",
        "useragent": "dummyuseragent",
        "vendor": "Zscaler"
      },
      "sourcetype": "zscalernss-web"
    } 
  

Field mapping reference

The following table lists the log fields of the ZSCALER_WEBPROXY log type and their corresponding UDM fields.

Need more help? Get answers from Community members and Google SecOps professionals.