Console
    Start free

    Collect AWS CloudWatch logs

    Supported in:

    This document explains how to ingest AWS CloudWatch logs to Google Security Operations using Amazon S3.

    AWS CloudWatch is a monitoring and observability service that collects operational data in the form of logs, metrics, and events from AWS resources and applications. This integration uses Amazon Data Firehose to stream CloudWatch log data to an S3 bucket, which Google SecOps then ingests using an Amazon S3 V2 feed.

    Before you begin

    • A Google SecOps instance
    • Privileged access to the AWS Management Consolewith permissions to manage:
      • Amazon CloudWatch Logs(log groups, subscription filters)
      • Amazon Data Firehose(delivery streams)
      • Amazon S3(buckets)
      • AWS IAM(roles, policies, users)

    Configure AWS S3 bucket

    1. Create an Amazon S3 bucketfollowing this user guide: Creating a bucket .
    2. Save the bucket Nameand Regionfor future reference (for example, cwlogs-to-secops ).

    Configure the IAM role for Amazon Data Firehose

    Amazon Data Firehose requires an IAM role to write logs to your S3 bucket.

    Create the IAM policy

    1. In the AWS Console, go to IAM > Policies > Create policy.
    2. Select the JSONtab.

    3. Paste the following policy (replace cwlogs-to-secops with your actual bucket name):

        { 
  
 "Version" 
 : 
  
 "2012-10-17" 
 , 
  
 "Statement" 
 : 
  
 [ 
  
 { 
  
 "Sid" 
 : 
  
 "S3Delivery" 
 , 
  
 "Effect" 
 : 
  
 "Allow" 
 , 
  
 "Action" 
 : 
  
 [ 
  
 "s3:AbortMultipartUpload" 
 , 
  
 "s3:GetBucketLocation" 
 , 
  
 "s3:GetObject" 
 , 
  
 "s3:ListBucket" 
 , 
  
 "s3:ListBucketMultipartUploads" 
 , 
  
 "s3:PutObject" 
  
 ], 
  
 "Resource" 
 : 
  
 [ 
  
 "arn:aws:s3:::cwlogs-to-secops" 
 , 
  
 "arn:aws:s3:::cwlogs-to-secops/*" 
  
 ] 
  
 }, 
  
 { 
  
 "Sid" 
 : 
  
 "CloudWatchLogging" 
 , 
  
 "Effect" 
 : 
  
 "Allow" 
 , 
  
 "Action" 
 : 
  
 [ 
  
 "logs:PutLogEvents" 
  
 ], 
  
 "Resource" 
 : 
  
 "arn:aws:logs:*:*:log-group:/aws/kinesisfirehose/cwlogs-to-secops:log-stream:*" 
  
 } 
  
 ] 
 }

    4. Click Next.

    5. In the Policy namefield, enter FirehoseS3DeliveryPolicy .

    6. Click Create policy.

    Create the IAM role

    1. Go to IAM > Roles > Create role.
    2. Select Custom trust policy.

    3. Paste the following trust policy:

        { 
  
 "Version" 
 : 
  
 "2012-10-17" 
 , 
  
 "Statement" 
 : 
  
 [ 
  
 { 
  
 "Effect" 
 : 
  
 "Allow" 
 , 
  
 "Principal" 
 : 
  
 { 
  
 "Service" 
 : 
  
 "firehose.amazonaws.com" 
  
 }, 
  
 "Action" 
 : 
  
 "sts:AssumeRole" 
  
 } 
  
 ] 
 }

    4. Click Next.

    5. Search for and select FirehoseS3DeliveryPolicy.

    6. Click Next.

    7. In the Role namefield, enter FirehoseToS3Role .

    8. Click Create role.

    Create the Amazon Data Firehose stream

    1. Open the Kinesis consoleat https://console.aws.amazon.com/kinesis .
    2. In the navigation pane, select Amazon Data Firehose.
    3. Click Create Firehose stream.
    4. Under Choose source and destination, provide the following configuration:
      • Source: Select Direct PUT.
      • Destination: Select Amazon S3.
    5. In the Firehose stream namefield, enter cwlogs-to-secops .

    6. Under Transform records, in the Decompress source records from Amazon CloudWatch Logssection:

      1. Select Turn on decompression.
      2. Do notselect Turn on message extraction.

    7. Under Destination settings:

      • S3 bucket: Select the S3 bucket cwlogs-to-secops .
      • S3 bucket prefix(optional): Enter cloudwatch-logs/ .
      • S3 bucket error output prefix(optional): Enter firehose-errors/ .

    8. Under Buffer hints:

      • Buffer size: 5 MiB (default).
      • Buffer interval: 300 seconds (default).

    9. Under Advanced settings:

      • Server-side encryption: Optional. Enable if encryption is required.
      • Error logging: Select Enabled(recommended).
      • Permissions: Select Choose existing IAM role, then select FirehoseToS3Role .

    10. Click Create Firehose stream.

    11. Wait for the stream Statusto show Active.

    Configure the IAM role for CloudWatch Logs

    CloudWatch Logs requires an IAM role to send log data to the Firehose stream.

    Create the IAM policy

    1. Go to IAM > Policies > Create policy.
    2. Select the JSONtab.

    3. Paste the following policy (replace <region> and <account-id> with your AWS region and account ID):

        { 
  
 "Version" 
 : 
  
 "2012-10-17" 
 , 
  
 "Statement" 
 : 
  
 [ 
  
 { 
  
 "Effect" 
 : 
  
 "Allow" 
 , 
  
 "Action" 
 : 
  
 [ 
  
 "firehose:PutRecord" 
 , 
  
 "firehose:PutRecordBatch" 
  
 ], 
  
 "Resource" 
 : 
  
 "arn:aws:firehose:<region>:<account-id>:deliverystream/cwlogs-to-secops" 
  
 } 
  
 ] 
 }

    4. Click Next.

    5. In the Policy namefield, enter CWLtoFirehoseWritePolicy .

    6. Click Create policy.

    Create the IAM role

    1. Go to IAM > Roles > Create role.
    2. Select Custom trust policy.

    3. Paste the following trust policy (replace <region> with your AWS region):

        { 
  
 "Version" 
 : 
  
 "2012-10-17" 
 , 
  
 "Statement" 
 : 
  
 [ 
  
 { 
  
 "Effect" 
 : 
  
 "Allow" 
 , 
  
 "Principal" 
 : 
  
 { 
  
 "Service" 
 : 
  
 "logs.<region>.amazonaws.com" 
  
 }, 
  
 "Action" 
 : 
  
 "sts:AssumeRole" 
  
 } 
  
 ] 
 }

    4. Click Next.

    5. Search for and select CWLtoFirehoseWritePolicy.

    6. Click Next.

    7. In the Role namefield, enter CWLtoFirehoseRole .

    8. Click Create role.

    Create CloudWatch Logs subscription filter

    1. In the AWS Console, go to CloudWatch > Logs > Log groups.
    2. Select the target log group that you want to stream to Google SecOps.
    3. Select the Subscription filterstab.
    4. Click Create > Create Amazon Data Firehose subscription filter.
    5. Provide the following configuration details:
      • Destination: Select the Firehose stream cwlogs-to-secops .
      • Grant permission: Select the role CWLtoFirehoseRole .
      • Subscription filter name: Enter a descriptive name (for example, secops-all-events ).
      • Log format: Select Other.
      • Subscription filter pattern: Leave empty to send all events, or enter a filter pattern to send only specific events.

    6. Click Start streaming.

    Configure IAM user for Google SecOps

    Google SecOps needs an IAM user with access to the S3 bucket to ingest the delivered logs.

    1. Create a Userfollowing this user guide: Creating an IAM user .
    2. Select the created User.
    3. Select Security credentialstab.
    4. Click Create Access Keyin section Access Keys.
    5. Select Third-party serviceas Use case.
    6. Click Next.
    7. Optional: Add description tag.
    8. Click Create access key.
    9. Click Download .csv fileto save the Access Keyand Secret Access Keyfor future reference.
    10. Click Done.
    11. Select Permissionstab.
    12. Click Add permissionsin section Permissions policies.
    13. Select Add permissions.
    14. Select Attach policies directly.
    15. Search for AmazonS3FullAccesspolicy.
    16. Select the policy.
    17. Click Next.
    18. Click Add permissions.

    Configure a feed in Google SecOps to ingest AWS CloudWatch logs

    1. Go to SIEM Settings > Feeds.
    2. Click Add New Feed.
    3. On the next page, click Configure a single feed.
    4. Enter a unique name for the Feed name.
    5. Select Amazon S3 V2as the Source type.
    6. Select AWS CloudWatchas the Log type.
    7. Click Nextand then click Submit.
    8. Specify values for the following fields:
      • S3 URI: s3://cwlogs-to-secops/cloudwatch-logs/
      • Source deletion option: Select the deletion option according to your preference.
      • Maximum File Age: Include files modified in the last number of days (default is 180 days).
      • Access Key ID: User access key with access to the S3 bucket.
      • Secret Access Key: User secret key with access to the S3 bucket.
      • Asset namespace: The asset namespace .
      • Ingestion labels: The label to be applied to the events from this feed.
    9. Click Nextand then click Submit.

    UDM mapping table

    Log Field UDM Mapping Logic
    account
    		principal.user.userid The value of account from the raw log is mapped to the principal.user.userid field.
    account_id
    		principal.user.userid The value of account_id from the raw log is mapped to the principal.user.userid field.
    AlertId
    		metadata.product_log_id The value of AlertId from the raw log is mapped to the metadata.product_log_id field.
    arrivalTimestamp
    		metadata.event_timestamp The value of arrivalTimestamp from the raw log is converted to a timestamp and mapped to the metadata.event_timestamp field.
    attemptsMade
    		additional.fields The value of attemptsMade from the raw log is converted to a string and added as a key-value pair to the additional.fields with the key "Attempts Made".
    awsAccountId
    		principal.asset_id The value of awsAccountId from the raw log is prepended with "AWS Account id: " and mapped to the principal.asset_id field.
    billed_duration
    		additional.fields The value of billed_duration from the raw log is added as a key-value pair to the additional.fields with the key "billed_duration".
    BytesIn
    		network.received_bytes The value of BytesIn from the raw log is converted to an unsigned integer and mapped to the network.received_bytes field.
    cipher
    		network.tls.cipher The value of cipher from the raw log is mapped to the network.tls.cipher field.
    Ciphers
    		network.tls.client.supported_ciphers The value of Ciphers from the raw log is split by commas and each value is added to the network.tls.client.supported_ciphers array.
    cloudwatchLog
    		security_result.description The value of cloudwatchLog from the raw log is mapped to the security_result.description field.
    CloudAccountId
    		metadata.product_deployment_id The value of CloudAccountId from the raw log is mapped to the metadata.product_deployment_id field.
    CloudType
    		target.resource.attribute.cloud.environment The value of CloudType from the raw log determines the value of target.resource.attribute.cloud.environment . If CloudType is "gcp", the value is "GOOGLE_CLOUD_PLATFORM". If CloudType is "aws", the value is "AMAZON_WEB_SERVICES". If CloudType is "azure", the value is "MICROSOFT_AZURE".
    Context.Execution.Id
    		target.resource.attribute.labels The value of Context.Execution.Id from the raw log is added as a key-value pair to the target.resource.attribute.labels with the key "Context Id".
    Context.Execution.Name
    		target.resource.attribute.labels The value of Context.Execution.Name from the raw log is added as a key-value pair to the target.resource.attribute.labels with the key "Context Name".
    Context.Execution.RoleArn
    		target.resource.product_object_id The value of Context.Execution.RoleArn from the raw log is mapped to the target.resource.product_object_id field.
    descr
    		metadata.description The value of descr from the raw log, after removing extra whitespace, is mapped to the metadata.description field unless it is "-". If descr is empty, the value of log is used instead.
    destination.name
    		target.location.country_or_region The value of destination.name from the raw log is mapped to the target.location.country_or_region field.
    destination.properties.prefix
    		target.resource.attribute.labels The value of destination.properties.prefix from the raw log is added as a key-value pair to the target.resource.attribute.labels with the key "Destination properties prefix".
    detail.additionalEventData.configRuleArn
    		security_result.rule_id The value of detail.additionalEventData.configRuleArn from the raw log is mapped to the security_result.rule_id field.
    detail.additionalEventData.configRuleName
    		security_result.rule_name The value of detail.additionalEventData.configRuleName from the raw log is mapped to the security_result.rule_name field.
    detail.additionalEventData.managedRuleIdentifier
    		additional.fields The value of detail.additionalEventData.managedRuleIdentifier from the raw log is added as a key-value pair to the additional.fields with the key "managedRuleIdentifier".
    detail.additionalEventData.notificationJobType
    		additional.fields The value of detail.additionalEventData.notificationJobType from the raw log is added as a key-value pair to the additional.fields with the key "notificationJobType".
    detail.awsAccountId
    		principal.asset_id The value of detail.awsAccountId from the raw log is prepended with "AWS Account id: " and mapped to the principal.asset_id field.
    detail.awsRegion
    		principal.location.name The value of detail.awsRegion from the raw log is mapped to the principal.location.name field.
    detail.configRuleArn
    		security_result.rule_id The value of detail.configRuleArn from the raw log is mapped to the security_result.rule_id field.
    detail.configRuleName
    		security_result.rule_name The value of detail.configRuleName from the raw log is mapped to the security_result.rule_name field.
    detail.configurationItem.awsAccountId
    		principal.user.userid The value of detail.configurationItem.awsAccountId from the raw log is mapped to the principal.user.userid field.
    detail.configurationItem.awsRegion
    		target.location.country_or_region The value of detail.configurationItem.awsRegion from the raw log is mapped to the target.location.country_or_region field.
    detail.configurationItem.configuration.complianceType
    		security_result.summary The value of detail.configurationItem.configuration.complianceType from the raw log is mapped to the security_result.summary field.
    detail.configurationItem.configuration.targetResourceId
    		target.resource.attribute.labels The value of detail.configurationItem.configuration.targetResourceId from the raw log is added as a key-value pair to the target.resource.attribute.labels with the key "configurationItem configuration targetResourceId".
    detail.configurationItem.configuration.targetResourceType
    		target.resource.attribute.labels The value of detail.configurationItem.configuration.targetResourceType from the raw log is added as a key-value pair to the target.resource.attribute.labels with the key "configurationItem configuration targetResourceType".
    detail.configurationItem.configurationItemCaptureTime
    		_target.asset.attribute.creation_time The value of detail.configurationItem.configurationItemCaptureTime from the raw log is converted to a timestamp and mapped to the _target.asset.attribute.creation_time field.
    detail.configurationItem.configurationItemStatus
    		target.resource.attribute.labels The value of detail.configurationItem.configurationItemStatus from the raw log is added as a key-value pair to the target.resource.attribute.labels with the key "configurationItem configurationItemStatus".
    detail.configurationItem.configurationStateId
    		target.resource.attribute.labels The value of detail.configurationItem.configurationStateId from the raw log is converted to a string and added as a key-value pair to the target.resource.attribute.labels with the key "configurationItem configurationStateId".
    detail.configurationItem.resourceId
    		target.resource.id The value of detail.configurationItem.resourceId from the raw log is mapped to the target.resource.id field.
    detail.configurationItem.resourceType
    		target.resource.resource_subtype The value of detail.configurationItem.resourceType from the raw log is mapped to the target.resource.resource_subtype field.
    detail.configurationItemDiff.changedProperties.Configuration.configRuleList.1.updatedValue.configRuleArn
    		security_result.rule_id The value of detail.configurationItemDiff.changedProperties.Configuration.configRuleList.1.updatedValue.configRuleArn from the raw log is mapped to the security_result.rule_id field.
    detail.eventCategory
    		security_result.category_details The value of detail.eventCategory from the raw log is mapped to the security_result.category_details field.
    detail.eventID
    		metadata.product_log_id The value of detail.eventID from the raw log is mapped to the metadata.product_log_id field.
    detail.eventName
    		additional.fields The value of detail.eventName from the raw log is added as a key-value pair to the additional.fields with the key "Event Name".
    detail.eventSource
    		target.application The value of detail.eventSource from the raw log is mapped to the target.application field.
    detail.eventType
    		additional.fields The value of detail.eventType from the raw log is added as a key-value pair to the additional.fields with the key "Event Type".
    detail.eventVersion
    		metadata.product_version The value of detail.eventVersion from the raw log is mapped to the metadata.product_version field.
    detail.managementEvent
    		additional.fields The value of detail.managementEvent from the raw log is converted to a string and added as a key-value pair to the additional.fields with the key "detail managementEvent".
    detail.messageType
    		target.resource.attribute.labels The value of detail.messageType from the raw log is added as a key-value pair to the target.resource.attribute.labels with the key "Message Type".
    detail.newEvaluationResult.complianceType
    		security_result.summary The value of detail.newEvaluationResult.complianceType from the raw log is mapped to the security_result.summary field.
    detail.newEvaluationResult.configRuleInvokedTime
    		additional.fields The value of detail.newEvaluationResult.configRuleInvokedTime from the raw log is added as a key-value pair to the additional.fields with the key "newEvaluationResult_configRuleInvokedTime".
    detail.newEvaluationResult.evaluationResultIdentifier.evaluationResultQualifier.configRuleName
    		additional.fields The value of detail.newEvaluationResult.evaluationResultIdentifier.evaluationResultQualifier.configRuleName from the raw log is added as a key-value pair to the additional.fields with the key "newEvaluationResult_configRuleName".
    detail.newEvaluationResult.evaluationResultIdentifier.evaluationResultQualifier.resourceId
    		additional.fields The value of detail.newEvaluationResult.evaluationResultIdentifier.evaluationResultQualifier.resourceId from the raw log is added as a key-value pair to the additional.fields with the key "newEvaluationResult_resourceId".
    detail.newEvaluationResult.evaluationResultIdentifier.evaluationResultQualifier.resourceType
    		additional.fields The value of detail.newEvaluationResult.evaluationResultIdentifier.evaluationResultQualifier.resourceType from the raw log is added as a key-value pair to the additional.fields with the key "newEvaluationResult_resourceType".
    detail.newEvaluationResult.resultRecordedTime
    		additional.fields The value of detail.newEvaluationResult.resultRecordedTime from the raw log is added as a key-value pair to the additional.fields with the key "newEvaluationResult_resultRecordedTime".
    detail.oldEvaluationResult.configRuleInvokedTime
    		additional.fields The value of detail.oldEvaluationResult.configRuleInvokedTime from the raw log is added as a key-value pair to the additional.fields with the key "oldEvaluationResult_configRuleInvokedTime".
    detail.oldEvaluationResult.evaluationResultIdentifier.evaluationResultQualifier.configRuleName
    		additional.fields The value of detail.oldEvaluationResult.evaluationResultIdentifier.evaluationResultQualifier.configRuleName from the raw log is added as a key-value pair to the additional.fields with the key "oldEvaluationResult_configRuleName".
    detail.oldEvaluationResult.evaluationResultIdentifier.evaluationResultQualifier.resourceId
    		additional.fields The value of detail.oldEvaluationResult.evaluationResultIdentifier.evaluationResultQualifier.resourceId from the raw log is added as a key-value pair to the additional.fields with the key "oldEvaluationResult_resourceId".
    detail.oldEvaluationResult.evaluationResultIdentifier.evaluationResultQualifier.resourceType
    		additional.fields The value of detail.oldEvaluationResult.evaluationResultIdentifier.evaluationResultQualifier.resourceType from the raw log is added as a key-value pair to the additional.fields with the key "oldEvaluationResult_resourceType".
    detail.oldEvaluationResult.resultRecordedTime
    		additional.fields The value of detail.oldEvaluationResult.resultRecordedTime from the raw log is added as a key-value pair to the additional.fields with the key "oldEvaluationResult_resultRecordedTime".
    detail.readOnly
    		additional.fields The value of detail.readOnly from the raw log is converted to a string and added as a key-value pair to the additional.fields with the key "detail readOnly".
    detail.recipientAccountId
    		target.resource.attribute.labels The value of detail.recipientAccountId from the raw log is added as a key-value pair to the target.resource.attribute.labels with the key "Recipient Account Id".
    detail.recordVersion
    		metadata.product_version The value of detail.recordVersion from the raw log is mapped to the metadata.product_version field.
    detail.requestID
    		target.resource.attribute.labels The value of detail.requestID from the raw log is added as a key-value pair to the target.resource.attribute.labels with the key "Detail Request ID".
    detail.resourceType
    		target.resource.resource_subtype The value of detail.resourceType from the raw log is mapped to the target.resource.resource_subtype field.
    detail.s3Bucket
    		about.resource.name The value of detail.s3Bucket from the raw log is mapped to the about.resource.name field.
    detail.s3ObjectKey
    		target.resource.attribute.labels The value of detail.s3ObjectKey from the raw log is added as a key-value pair to the target.resource.attribute.labels with the key "s3ObjectKey".
    detail.userAgent
    		network.http.user_agent The value of detail.userAgent from the raw log is mapped to the network.http.user_agent field.
    detail.userIdentity.accessKeyId
    		target.user.userid The value of detail.userIdentity.accessKeyId from the raw log is mapped to the target.user.userid field.
    detail.userIdentity.accountId
    		metadata.product_deployment_id The value of detail.userIdentity.accountId from the raw log is mapped to the metadata.product_deployment_id field.
    detail.userIdentity.arn
    		target.user.userid The value of detail.userIdentity.arn from the raw log is mapped to the target.user.userid field.
    detail.userIdentity.principalId
    		principal.user.product_object_id The value of detail.userIdentity.principalId from the raw log is mapped to the principal.user.product_object_id field.
    detail.userIdentity.sessionContext.attributes.mfaAuthenticated
    		principal.user.attribute.labels The value of detail.userIdentity.sessionContext.attributes.mfaAuthenticated from the raw log is added as a key-value pair to the principal.user.attribute.labels with the key "mfaAuthenticated".
    detail.userIdentity.sessionContext.sessionIssuer.userName
    		target.user.user_display_name The value of detail.userIdentity.sessionContext.sessionIssuer.userName from the raw log is mapped to the target.user.user_display_name field.
    detail.userIdentity.type
    		principal.resource.type The value of detail.userIdentity.type from the raw log is mapped to the principal.resource.type field.
    detail-type
    		metadata.product_event_type The value of detail-type from the raw log is mapped to the metadata.product_event_type field.
    device
    		principal.asset.product_object_id The value of device from the raw log is mapped to the principal.asset.product_object_id field.
    digestPublicKeyFingerprint
    		target.file.sha1 The value of digestPublicKeyFingerprint from the raw log is mapped to the target.file.sha1 field.
    digestS3Bucket
    		principal.resource.name The value of digestS3Bucket from the raw log is mapped to the principal.resource.name field.
    digestS3Object
    		principal.asset.asset_id The value of digestS3Object from the raw log is prepended with "S3 Object: " and mapped to the principal.asset.asset_id field.
    digestSignatureAlgorithm
    		network.tls.cipher The value of digestSignatureAlgorithm from the raw log is mapped to the network.tls.cipher field.
    digestStartTime
    		metadata.event_timestamp The value of digestStartTime from the raw log is converted to a timestamp and mapped to the metadata.event_timestamp field.
    dimensions.VolumeId
    		additional.fields The value of dimensions.VolumeId from the raw log is added as a key-value pair to the additional.fields with the key "VolumeId".
    duration
    		additional.fields The value of duration from the raw log is added as a key-value pair to the additional.fields with the key "duration".
    errorCode
    		security_result.rule_name The value of errorCode from the raw log is mapped to the security_result.rule_name field.
    errorMessage
    		security_result.summary The value of errorMessage from the raw log is mapped to the security_result.summary field.
    executionId
    		principal.process.pid The value of executionId from the raw log is mapped to the principal.process.pid field.
    host
    		principal.hostname , principal.ip The value of host from the raw log, with hyphens replaced by dots, is parsed as an IP address and mapped to the principal.ip field if successful. Otherwise, it is mapped to the principal.hostname field.
    http_verb
    		network.http.method The value of http_verb from the raw log is converted to uppercase and mapped to the network.http.method field.
    kubernetes.container_hash
    		additional.fields The value of kubernetes.container_hash from the raw log is added as a key-value pair to the additional.fields with the key "container_hash".
    kubernetes.container_image
    		additional.fields The value of kubernetes.container_image from the raw log is added as a key-value pair to the additional.fields with the key "container_image".
    kubernetes.container_name
    		additional.fields The value of kubernetes.container_name from the raw log is added as a key-value pair to the additional.fields with the key "container_name".
    kubernetes.docker_id
    		principal.asset_id The value of kubernetes.docker_id from the raw log is prepended with "id: " and mapped to the principal.asset_id field.
    kubernetes.host
    		principal.hostname , principal.ip The value of kubernetes.host from the raw log, with hyphens replaced by dots, is parsed as an IP address and mapped to the principal.ip field if successful. Otherwise, it is mapped to the principal.hostname field.
    kubernetes.namespace
    		principal.namespace The value of kubernetes.namespace from the raw log is mapped to the principal.namespace field.
    kubernetes.namespace_name
    		principal.namespace The value of kubernetes.namespace_name from the raw log is mapped to the principal.namespace field.
    kubernetes.pod_id
    		principal.asset.asset_id The value of kubernetes.pod_id from the raw log is prepended with "pod_id: " and mapped to the principal.asset.asset_id field.
    kubernetes.pod_name
    		additional.fields The value of kubernetes.pod_name from the raw log is added as a key-value pair to the additional.fields with the key "pod name".
    lambdaArn
    		principal.hostname The value of lambdaArn from the raw log is mapped to the principal.hostname field.
    level
    		security_result.severity The value of level from the raw log determines the value of security_result.severity . If level is "Info", the value is "INFORMATIONAL". If level is "Error", the value is "ERROR". If level is "Warning", the value is "MEDIUM".
    log
    		metadata.description The value of log from the raw log is mapped to the metadata.description field if descr is empty.
    logFiles
    		about For each element in the logFiles array from the raw log, an about object is created with file.full_path set to s3Object , asset.hostname set to s3Bucket , and file.sha256 set to hashValue .
    log_processed.cause
    		security_result.summary The value of log_processed.cause from the raw log is mapped to the security_result.summary field.
    log_processed.ids
    		intermediary.hostname For each element in the log_processed.ids array from the raw log, an intermediary object is created with hostname set to the element's value.
    log_processed.level
    		security_result.severity The value of log_processed.level from the raw log is mapped to the security_result.severity field.
    log_processed.msg
    		metadata.description The value of log_processed.msg from the raw log is mapped to the metadata.description field.
    log_processed.ts
    		metadata.event_timestamp The value of log_processed.ts from the raw log is converted to a timestamp and mapped to the metadata.event_timestamp field.
    log_type
    		metadata.log_type The value of log_type from the raw log is mapped to the metadata.log_type field. This is a custom field added for context.
    logevent.message
    		security_result.description The value of logevent.message from the raw log is mapped to the security_result.description field. It is also parsed using grok to extract additional fields.
    logGroup
    		security_result.about.resource.name The value of logGroup from the raw log is mapped to the security_result.about.resource.name field.
    logStream
    		security_result.about.resource.attribute.labels The value of logStream from the raw log is added as a key-value pair to the security_result.about.resource.attribute.labels with the key "logStream".
    memory_used
    		additional.fields The value of memory_used from the raw log is added as a key-value pair to the additional.fields with the key "memory_used".
    metric_name
    		additional.fields The value of metric_name from the raw log is added as a key-value pair to the additional.fields with the key "metric_name".
    metric_stream_name
    		additional.fields The value of metric_stream_name from the raw log is added as a key-value pair to the additional.fields with the key "metric_stream_name".
    namespace
    		principal.namespace The value of namespace from the raw log is mapped to the principal.namespace field.
    owner
    		principal.user.userid The value of owner from the raw log is mapped to the principal.user.userid field.
    parameters
    		additional.fields The value of parameters from the raw log is added as a key-value pair to the additional.fields with the key "Parameters".
    Path
    		principal.process.file.full_path The value of Path from the raw log is mapped to the principal.process.file.full_path field.
    pid
    		principal.process.pid The value of pid from the raw log is mapped to the principal.process.pid field.
    PolicyName
    		security_result.rule_name The value of PolicyName from the raw log is mapped to the security_result.rule_name field.
    prin_host
    		principal.hostname The value of prin_host from the raw log is mapped to the principal.hostname field.
    principal_hostname
    		principal.hostname The value of principal_hostname from the raw log is mapped to the principal.hostname field.
    process
    		principal.application The value of process from the raw log is mapped to the principal.application field.
    rawData
    		additional.fields The value of rawData from the raw log is added as a key-value pair to the additional.fields with the key "Raw Data".
    Recommendation
    		security_result.detection_fields The value of Recommendation from the raw log is added as a key-value pair to the security_result.detection_fields with the key "Recommendation".
    referral_url
    		network.http.referral_url The value of referral_url from the raw log is mapped to the network.http.referral_url field.
    region
    		principal.location.name The value of region from the raw log is mapped to the principal.location.name field.
    resp_code
    		network.http.response_code The value of resp_code from the raw log is converted to an integer and mapped to the network.http.response_code field.
    resource_url
    		network.http.referral_url The value of resource_url from the raw log is mapped to the network.http.referral_url field.
    ResourceType
    		target.resource.resource_subtype The value of ResourceType from the raw log is mapped to the target.resource.resource_subtype field.
    response_body
    		additional.fields The value of response_body from the raw log is added as a key-value pair to the additional.fields with the key "Response body".
    Role
    		target.resource.product_object_id The value of Role from the raw log is mapped to the target.resource.product_object_id field.
    s3_bucket_path
    		target.file.full_path The value of s3_bucket_path from the raw log is mapped to the target.file.full_path field.
    sec_result.category
    		security_result.category The value of sec_result.category is derived from the parser logic. If descr contains "authentication is required", the value is "AUTH_VIOLATION".
    sec_result.description
    		security_result.description The value of sec_result.description is derived from the parser logic. It is set to the value of cloudwatchLog if present.
    sec_result.severity
    		security_result.severity The value of sec_result.severity is derived from the parser logic. It is set based on the value of severity or level .
    sec_result.summary
    		security_result.summary The value of sec_result.summary is derived from the parser logic. It is set to the value of log_processed.cause or errorMessage if present.
    security_result
    		security_result The security_result object is constructed from various fields and parser logic.
    serverId
    		additional.fields The value of serverId from the raw log is added as a key-value pair to the additional.fields with the key "server_id".
    severity
    		security_result.severity The value of severity from the raw log, converted to uppercase and normalized, is mapped to the security_result.severity field.
    Source
    		principal.hostname The value of Source from the raw log is mapped to the principal.hostname field.
    source
    		principal.hostname The value of source from the raw log is mapped to the principal.hostname field.
    SourceIP
    		principal.ip The value of SourceIP from the raw log is mapped to the principal.ip field.
    src_port
    		principal.port If src_port is "80", it is converted to an integer and mapped to the principal.port field, and network.application_protocol is set to "HTTP".
    stream
    		additional.fields The value of stream from the raw log is added as a key-value pair to the additional.fields with the key "stream".
    subscriptionFilters
    		security_result.about.resource.attribute.labels For each element in the subscriptionFilters array from the raw log, a key-value pair is added to the security_result.about.resource.attribute.labels with the key "subscriptionFilter" and the value from the array.
    support_contact
    		target.resource.attribute.labels The value of support_contact from the raw log is added as a key-value pair to the target.resource.attribute.labels with the key "Support Contact".
    t_ip
    		target.ip The value of t_ip from the raw log, after removing hyphens, is parsed as an IP address and mapped to the target.ip field if successful.
    time
    		metadata.event_timestamp The value of time from the raw log is converted to a timestamp and mapped to the metadata.event_timestamp field.
    timestamp
    		metadata.event_timestamp The value of timestamp from the raw log is converted to a timestamp using various formats and mapped to the metadata.event_timestamp field.
    tls
    		network.tls.version The value of tls from the raw log is mapped to the network.tls.version field.
    transferDetails.serverId
    		additional.fields The value of transferDetails.serverId from the raw log is added as a key-value pair to the additional.fields with the key "server_id".
    transferDetails.sessionId
    		network.session_id The value of transferDetails.sessionId from the raw log is mapped to the network.session_id field.
    transferDetails.username
    		principal.user.user_display_name The value of transferDetails.username from the raw log is mapped to the principal.user.user_display_name field.
    ts
    		metadata.event_timestamp The value of ts from the raw log, combined with the timezone if available, is converted to a timestamp and mapped to the metadata.event_timestamp field.
    type
    		metadata.product_event_type The value of type from the raw log is mapped to the metadata.product_event_type field.
    unit
    		additional.fields The value of unit from the raw log is added as a key-value pair to the additional.fields with the key "unit".
    url
    		target.url The value of url from the raw log is mapped to the target.url field.
    url_back_to_product
    		metadata.url_back_to_product The value of url_back_to_product from the raw log is mapped to the metadata.url_back_to_product field.
    User
    		principal.user.userid The value of User from the raw log is mapped to the principal.user.userid field.
    user
    		target.user.userid , metadata.event_type , extensions.auth.mechanism If user is present, metadata.event_type is set to "USER_LOGIN", extensions.auth.mechanism is set to "NETWORK", and the value of user is mapped to target.user.userid .
    value.count
    		additional.fields The value of value.count from the raw log is converted to a string and added as a key-value pair to the additional.fields with the key "count".
    value.max
    		additional.fields The value of value.max from the raw log is converted to a string and added as a key-value pair to the additional.fields with the key "max".
    value.min
    		additional.fields The value of value.min from the raw log is converted to a string and added as a key-value pair to the additional.fields with the key "min".
    value.sum
    		additional.fields The value of value.sum from the raw log is converted to a string and added as a key-value pair to the additional.fields with the key "sum".
    workflowId
    		additional.fields The value of workflowId from the raw log is added as a key-value pair to the additional.fields with the key "workflowId".

    Need more help? Get answers from Community members and Google SecOps professionals.

    Create a Mobile Website
    View Site in Mobile | Classic
    Share by: