Collect Cisco Umbrella Web Proxy logs

This document explains how to collect Cisco Umbrella Web Proxy logs to a Google Security Operations feed using AWS S3 bucket. The parser extracts fields from a CSV log, renaming columns for clarity and handling potential variations in the input data. It then uses included files ( umbrella_proxy_udm.include and umbrella_handle_identities.include ) to map the extracted fields to the UDM and process identity information based on the identityType field.

Before you begin

• Ensure that you have a Google SecOps instance.
• Ensure that you privileged access to AWS IAM and S3.
• Ensure that you have privileged access to Cisco Umbrella.

Configure a Cisco-managed Amazon S3 bucket

1. Sign in to the Cisco Umbrelladashboard.
2. Go to Admin > Log management.
3. Select Use a Cisco-managed Amazon S3 bucketoption.
4. Provide the following configuration details:
   • Select a region: select a region closer to your location for lower latency.
   • Select a retention duration: select the time period. The retention duration is 7, 14, or 30 days. After the selected time period, data is deleted and cannot be recovered. If your ingestion cycle is regular, use a shorter time period. You can change the retention duration at a later time.
5. Click Save.
6. Click Continueto confirm your selections and to receive activation notification.
   In the Activation completewindow that appears, the Access keyand Secret keyvalues are displayed.
7. Copy the Access keyand Secret keyvalues. If you lose these keys, you must regenerate them.
8. Click Got it > Continue.
9. A summary page displays the configuration and your bucket name. You can turn logging off or on as required by your organization. However, logs are purged based on the retention duration, regardless of new data getting added.

Optional: Configure user access keys for self-managed AWS S3 bucket

1. Sign in to the AWS Management Console .
2. Create a Userfollowing this user guide: Creating an IAM user .
3. Select the created User.
4. Select the Security credentialstab.
5. Click Create Access Keyin the Access Keyssection.
6. Select Third-party serviceas the Use case.
7. Click Next.
8. Optional: add a description tag.
9. Click Create access key.
10. Click Download CSV fileto save the Access Keyand Secret Access Keyfor later use.
11. Click Done.
12. Select the Permissionstab.
13. Click Add permissionsin the Permissions policiessection.
14. Select Add permissions.
15. Select Attach policies directly.
16. Search for and select the AmazonS3FullAccesspolicy.
17. Click Next.
18. Click Add permissions.

Optional: Configure a self-managed Amazon S3 bucket

1. Sign in to the AWS Management Console .

2. Go to S3.

3. Click Create bucket.

4. Provide the following configuration details:

   • Bucket name: provide a name for the Amazon S3 bucket.
   • Region: select a region.

5. Click Create.

Optional: Configure a bucket policy for self-managed AWS S3 bucket

1. Click the newly created bucket to open it.
2. Select Properties > Permissions.
3. In the Permissionslist, click Add bucket policy.

4. Enter the preconfigured bucket policy as follows:

    {
     "Version": "2008-10-17",
     "Statement": [
       {
         "Sid": "",
         "Effect": "Allow",
         "Principal": {
           "AWS": "arn:aws:iam::568526795995:user/logs"
         },
         "Action": "s3:PutObject",
         "Resource": "arn:aws:s3::: BUCKET_NAME 
   /*"
       },
       {
         "Sid": "",
         "Effect": "Deny",
         "Principal": {
           "AWS": "arn:aws:iam::568526795995:user/logs"
         },
         "Action": "s3:GetObject",
         "Resource": "arn:aws:s3::: BUCKET_NAME 
   /*"},
       {
         "Sid": "",
         "Effect": "Allow",
         "Principal": {
           "AWS": "arn:aws:iam::568526795995:user/logs"
         },
         "Action": "s3:GetBucketLocation",
         "Resource": "arn:aws:s3::: BUCKET_NAME 
   "
       },
       {
         "Sid": "",
         "Effect": "Allow",
         "Principal": {
           "AWS": "arn:aws:iam::568526795995:user/logs"
         },
         "Action": "s3:ListBucket",
         "Resource": "arn:aws:s3::: BUCKET_NAME 
   "
       }
     ]
   } 
   

   • Replace BUCKET_NAME with the Amazon S3 bucket name you provided.

5. Click Save.

Optional: Required Verification for self-managed Amazon S3 bucket

1. In the Cisco Umbrelladashboard, select Admin > Log management > Amazon S3.
2. In the Bucket namefield, specify your exact Amazon S3 bucket name, and then click Verify.
3. As part of the verification process, a file named README_FROM_UMBRELLA.txt is uploaded from Cisco Umbrella to your Amazon S3 bucket. You may need to refresh your browser in order to see the readme file when it is uploaded.
4. Download the README_FROM_UMBRELLA.txt file, and open it using a text editor.
5. Copy and save the unique Cisco Umbrellatoken from the file.
6. Go to the Cisco Umbrelladashboard.
7. In the Token numberfield, specify the token and click Save.
8. If successful, you get a confirmation message in your dashboard indicating that the bucket was successfully verified. If you receive an error indicating that your bucket can't be verified, re-check the syntax of the bucket name and review the configuration.

Configure a feed in Google SecOps to ingest the Cisco Umbrella Web Proxy logs

1. Go to SIEM Settings > Feeds.
2. Click Add new.
3. In the Feed namefield, enter a name for the feed; for example, Cisco Umbrella Web Proxy Logs.
4. Select Amazon S3 V2as the Source type.
5. Select Cisco Umbrella Web Proxyas the Log type.
6. Click Next.

7. Specify values for the following input parameters:

   • S3 URI: the bucket URI.
     • s3:/BUCKET_NAME
       • Replace BUCKET_NAME with the actual name of the bucket.
   • Source deletion options: select deletion option according to your preference.

8. Click Next.

9. Review your new feed configuration in the Finalizescreen, and then click Submit.

UDM Mapping Table

Log Field	UDM Mapping	Logic
ampDisposition	security_result.detection_fields[].value	The value of ampDisposition from the raw log.
ampMalware	security_result.detection_fields[].value	The value of ampMalware from the raw log.
ampScore	security_result.detection_fields[].value	The value of ampScore from the raw log.
avDetections	security_result.detection_fields[].value	The value of avDetections from the raw log.
blockedCategories	security_result.threat_name	The value of blockedCategories from the raw log.
certificateErrors	security_result.detection_fields[].value	The value of certificateErrors from the raw log.
contentType	security_result.detection_fields[].value	The value of contentType from the raw log.
destinationIp	target.ip	The value of destinationIp from the raw log.
destinationListID	security_result.detection_fields[].value	The value of destinationListID from the raw log.
dlpstatus	security_result.detection_fields[].value	The value of dlpstatus from the raw log.
externalIp	principal.ip	The value of externalIp from the raw log.
fileAction	security_result.detection_fields[].value	The value of fileAction from the raw log.
fileName	target.file.names	The value of fileName from the raw log.
identitiesV8	principal.hostname	The value of identitiesV8 from the raw log.
identity	principal.location.name	The value of identity from the raw log.
internalIp	principal.ip	The value of internalIp from the raw log.
isolateAction	security_result.detection_fields[].value	The value of isolateAction from the raw log.
referer	network.http.referral_url	The value of referer from the raw log.
requestMethod	network.http.method	The value of requestMethod from the raw log.
requestSize	security_result.detection_fields[].value	The value of requestSize from the raw log.
responseBodySize	security_result.detection_fields[].value	The value of responseBodySize from the raw log.
responseSize	security_result.detection_fields[].value	The value of responseSize from the raw log.
ruleID	security_result.rule_id	The value of ruleID from the raw log.
rulesetID	security_result.detection_fields[].value	The value of rulesetID from the raw log.
sha	security_result.about.file.sha256	The value of sha from the raw log.
statusCode	network.http.response_code	The value of statusCode from the raw log.
ts	timestamp	The value of ts from the raw log, parsed into a timestamp.
url	target.url	The value of url from the raw log.
userAgent	network.http.user_agent	The value of userAgent from the raw log.
verdict	security_result.detection_fields[].value	The value of verdict from the raw log.
warnstatus	security_result.detection_fields[].value	The value of warnstatus from the raw log. The value of collection_time from the raw log. Hardcoded to NETWORK_HTTP . Hardcoded to Cisco . Hardcoded to Umbrella . Hardcoded to UMBRELLA_WEBPROXY . Derived from the scheme of the URL field ( http or https ). Parsed from the userAgent field using a user-agent parsing library. The value of requestSize from the raw log, converted to an integer. The value of responseSize from the raw log, converted to an integer. Derived from the identity field when identityType (or identityTypeV8 with identitiesV8 ) indicates a user. Further parsed to extract user details like display name, first name, last name, and email address. Mapped from the verdict field: allowed or allowed -> ALLOW , other values -> BLOCK . If categories is not empty, set to NETWORK_CATEGORIZED_CONTENT . The value of categories from the raw log. Based on the verdict and potentially other fields. Usually Traffic allowed or Traffic blocked . If verdict is not allowed or blocked and statusCode is present, the summary is Traffic %{statusCode} .

Need more help? Get answers from Community members and Google SecOps professionals.