Collect Tanium Stream logs

This document explains how to ingest Tanium Stream logs to Google Security Operations using Tanium Connect's native AWS S3 export functionality. Tanium Stream provides real-time endpoint telemetry, threat hunting data, and behavioral analytics in JSON format, which can be directly exported to S3 using Tanium Connect without requiring custom Lambda functions. The parser transforms raw JSON logs from Tanium Stream into a unified data model (UDM). It first normalizes common fields and then applies specific logic based on the "logType" or "eventType" to map relevant information into the appropriate UDM fields, handling various event types like network connections, user logins, process launches, and file modifications.

Before you begin

Make sure you have the following prerequisites:

• A Google SecOps instance
• Tanium Core Platform7.0 or later
• Tanium Streammodule installed and configured
• Tanium Connectmodule installed with valid license
• Tanium Threat Response3.4.346 or later (if using TR integration)
• Privileged access to Tanium Consolewith administrative rights
• Privileged access to AWS(S3, IAM)

Configure Tanium Stream service account

1. Sign in to the Tanium Console.
2. Go to Modules > Stream.
3. Click Settingsat the top right.
4. In the Service Accountsection, configure the following:
   • Service Account User: Select a user with appropriate Stream permissions.
   • Verifythe account has Connect User role privilege.
   • Confirmaccess to Stream data sources and endpoints.
5. Click Saveto apply the service account configuration.

Collect Tanium Stream prerequisites

1. Sign in to the Tanium Consoleas an administrator.
2. Go to Administration > Permissions > Users.
3. Create or identify a service account user with the following roles:
   • Stream Administratoror Stream Read Only Userrole.
   • Connect Userrole privilege.
   • Access to monitored computer groups (recommended: All Computersgroup).
   • Read Saved Questionpermission for Stream content sets.

Configure AWS S3 bucket and IAM for Google SecOps

1. Create Amazon S3 bucketfollowing this user guide: Creating a bucket
2. Save bucket Nameand Regionfor future reference (for example, tanium-stream-logs ).
3. Create a user following this user guide: Creating an IAM user .
4. Select the created User.
5. Select the Security credentialstab.
6. Click Create Access Keyin the Access Keyssection.
7. Select Third-party serviceas the Use case.
8. Click Next.
9. Optional: add a description tag.
10. Click Create access key.
11. Click Download CSV fileto save the Access Keyand Secret Access Keyfor later use.
12. Click Done.
13. Select the Permissionstab.
14. Click Add permissionsin the Permissions policiessection.
15. Select Add permissions.
16. Select Attach policies directly
17. Search for and select the AmazonS3FullAccesspolicy.
18. Click Next.
19. Click Add permissions.

Configure Tanium Connect AWS S3 destination

1. Sign in to the Tanium Console.
2. Go to Modules > Connect.
3. Click Create Connection.
4. Provide the following configuration details:
   • Name: Enter a descriptive name (for example, Stream Telemetry to S3 for SecOps ).
   • Description: Optional description (for example, Export endpoint telemetry and threat hunting data to AWS S3 for Google SecOps ingestion ).
   • Enable: Select to enable the connection to run on schedule.
5. Click Next.

Configure the connection source

1. In the Sourcesection, provide the following configuration details:
   • Source Type: Select Saved Question.
   • Saved Question: Select one of the following Stream-related saved questions:
     • Stream - Endpoint Eventsfor real-time endpoint telemetry.
     • Stream - Network Eventsfor network activity monitoring.
     • Stream - Process Eventsfor process execution tracking.
     • Stream - File Eventsfor file system activity.
     • Stream - Threat Hunting Datafor behavioral analytics.
   • Computer Group: Select All Computersor specific computer groups to monitor.
   • Refresh Interval: Set appropriate interval for data collection (for example, 5 minutesfor real-time telemetry).
2. Click Next.

Configure AWS S3 destination

1. In the Destinationsection, provide the following configuration details:
   • Destination Type: Select AWS S3.
   • Destination Name: Enter a unique name (for example, Google SecOps Stream S3 Destination ).
   • AWS Access Key: Enter the AWS access key from the CSV file downloaded in the AWS S3 configuration step.
   • AWS Secret Access Key: Enter the AWS secret access key from the CSV file downloaded in the AWS S3 configuration step.
   • Bucket Name: Enter your S3 bucket name (for example, tanium-stream-logs ).
   • Region: Select the AWS region where your S3 bucket is located.
   • Key Prefix: Enter a prefix for the S3 objects (for example, tanium/stream/ ).
2. Click Next.

Configure filters

1. In the Filterssection, configure data filtering options:
   • Send new items only: Select this option to send only new telemetry data since the last export.
   • Column filters: Add filters based on specific event attributes if needed (for example, filter by event type, process name, or threat indicators).
2. Click Next.

Format data for AWS S3

1. In the Formatsection, configure the data format:
   • Format: Select JSON.
   • Options:
     • Include headers: Deselect to avoid headers in JSON output.
     • Include empty cells: Select based on your preference.
   • Advanced Options:
     • File naming: Use default timestamp-based naming.
     • Compression: Select Gzipto reduce storage costs and transfer time.
2. Click Next.

Schedule the connection

1. In the Schedulesection, configure the export schedule:
   • Enable schedule: Select to enable automatic scheduled exports.
   • Schedule type: Select Recurring.
   • Frequency: Select Every 5 minutesfor near real-time telemetry data.
   • Start time: Set appropriate start time for the first export.
2. Click Next.

Save and verify connection

1. Review the connection configuration in the summary screen.
2. Click Saveto create the connection.
3. Click Test Connectionto verify the configuration.
4. If the test is successful, click Run Nowto perform an initial export.
5. Monitor the connection status in the Connect Overviewpage.

Configure a feed in Google SecOps to ingest Tanium Stream logs

1. Go to SIEM Settings > Feeds.
2. Click + Add New Feed.
3. In the Feed namefield, enter a name for the feed (for example, Tanium Stream logs ).
4. Select Amazon S3 V2as the Source type.
5. Select Tanium Streamas the Log type.
6. Click Next.
7. Specify values for the following input parameters:
   • S3 URI: s3://tanium-stream-logs/tanium/stream/
   • Source deletion options: Select deletion option according to your preference.
   • Maximum File Age: Include files modified in the last number of days. Default is 180 days.
   • Access Key ID: User access key with access to the S3 bucket.
   • Secret Access Key: User secret key with access to the S3 bucket.
   • Asset namespace: The asset namespace .
   • Ingestion labels: The label applied to the events from this feed.
8. Click Next.
9. Review your new feed configuration in the Finalizescreen, and then click Submit.

UDM Mapping Table

Log field	UDM mapping	Logic
additional.event__AlgorithmName	additional.fields.key: event AlgorithmName additional.fields.value.string_value: %{additional.eventAlgorithmName}	Value taken directly from the raw log field additional.event__AlgorithmName .
additional.event__AuthenticationPackageName	target.resource.name: %{additional.event__AuthenticationPackageName}	Value taken directly from the raw log field additional.event__AuthenticationPackageName .
additional.event__CallerProcessId	principal.process.pid: %{additional.event__CallerProcessId}	Value taken directly from the raw log field additional.event__CallerProcessId .
additional.event__CallerProcessName	principal.process.file.full path: %{additional.event _CallerProcessName}	Value taken directly from the raw log field additional.event__CallerProcessName .
additional.event__ClientProcessId	principal.process.pid: %{additional.event__ClientProcessId}	Value taken directly from the raw log field additional.event__ClientProcessId .
additional.event__ClientProcessStartKey	additional.fields.key: event ClientProcessStartKey additional.fields.value.string_value: %{additional.eventClientProcessStartKey}	Value taken directly from the raw log field additional.event__ClientProcessStartKey .
additional.event__CommandLine	target.process.command line: %{additional.event _CommandLine}	Value taken directly from the raw log field additional.event__CommandLine .
additional.event__ElevatedToken	additional.fields.key: event__ElevatedToken additional.fields.value.string_value: Yes/No	Value taken directly from the raw log field additional.event__ElevatedToken . If the value is "%%1842", it is replaced with "Yes". If the value is "%%1843", it is replaced with "No".
additional.event__FQDN	principal.hostname: %{additional.event__FQDN}	Value taken directly from the raw log field additional.event__FQDN .
additional.event__FailureReason	additional.fields.key: event FailureReason additional.fields.value.string_value: %{additional.eventFailureReason}	Value taken directly from the raw log field additional.event__FailureReason .
additional.event__ImpersonationLevel	additional.fields.key: event ImpersonationLevel additional.fields.value.string_value: %{additional.eventImpersonationLevel}	Value taken directly from the raw log field additional.event__ImpersonationLevel .
additional.event__IpAddress	target.ip: %{additional.event__IpAddress}	Value taken directly from the raw log field additional.event__IpAddress .
additional.event__IpPort	target.port: %{additional.event__IpPort}	Value taken directly from the raw log field additional.event__IpPort and converted to an integer.
additional.event__KeyLength	additional.fields.key: event KeyLength additional.fields.value.string_value: %{additional.eventKeyLength}	Value taken directly from the raw log field additional.event__KeyLength .
additional.event__KeyName	additional.fields.key: event KeyName additional.fields.value.string_value: %{additional.eventKeyName}	Value taken directly from the raw log field additional.event__KeyName .
additional.event__KeyType	additional.fields.key: event KeyType additional.fields.value.string_value: %{additional.eventKeyType}	Value taken directly from the raw log field additional.event__KeyType .
additional.event__LmPackageName	additional.fields.key: event LmPackageName additional.fields.value.string_value: %{additional.eventLmPackageName}	Value taken directly from the raw log field additional.event__LmPackageName .
additional.event__LogonGuid	target.resource.product_object id: %{additional.event _LogonGuid}	Value taken directly from the raw log field additional.event__LogonGuid with curly braces removed.
additional.event__LogonProcessName	target.process.file.full path: %{additional.event _LogonProcessName}	Value taken directly from the raw log field additional.event__LogonProcessName .
additional.event__LogonType	extensions.auth.auth details: Logon Type: %{additional.event _LogonType}	Value taken directly from the raw log field additional.event__LogonType .
additional.event__MandatoryLabel	additional.fields.key: event MandatoryLabel additional.fields.value.string_value: %{additional.eventMandatoryLabel}	Value taken directly from the raw log field additional.event__MandatoryLabel .
additional.event__NewProcessId	target.process.pid: %{additional.event__NewProcessId}	Value taken directly from the raw log field additional.event__NewProcessId .
additional.event__NewProcessName	target.process.file.full path: %{additional.event _NewProcessName}	Value taken directly from the raw log field additional.event__NewProcessName .
additional.event__ObjectServer	security_result.category details: %{additional.event _ObjectServer}	Value taken directly from the raw log field additional.event__ObjectServer .
additional.event__Operation	additional.fields.key: event Operation additional.fields.value.string_value: %{additional.eventOperation}	Value taken directly from the raw log field additional.event__Operation .
additional.event__ParentProcessId	principal.process.parent process.pid: %{additional.event _ParentProcessId}	Value taken directly from the raw log field additional.event__ParentProcessId .
additional.event__ParentProcessName	principal.process.parent_process.file.full path: %{additional.event _ParentProcessName}	Value taken directly from the raw log field additional.event__ParentProcessName .
additional.event__ProcessId	principal.process.pid: %{additional.event__ProcessId}	Value taken directly from the raw log field additional.event__ProcessId .
additional.event__ProcessName	principal.process.file.full path: %{additional.event _ProcessName}	Value taken directly from the raw log field additional.event__ProcessName .
additional.event__PrivilegeList	principal.user.attribute.permissions.name: %{additional.event__PrivilegeList}	Value taken directly from the raw log field additional.event__PrivilegeList .
additional.event__ProviderName	additional.fields.key: event ProviderName additional.fields.value.string_value: %{additional.eventProviderName}	Value taken directly from the raw log field additional.event__ProviderName .
additional.event__RestrictedAdminMode	additional.fields.key: event RestrictedAdminMode additional.fields.value.string_value: %{additional.eventRestrictedAdminMode}	Value taken directly from the raw log field additional.event__RestrictedAdminMode .
additional.event__ReturnCode	additional.fields.key: event ReturnCode additional.fields.value.string_value: %{additional.eventReturnCode}	Value taken directly from the raw log field additional.event__ReturnCode .
additional.event__RpcCallClientLocality	additional.fields.key: event RpcCallClientLocality additional.fields.value.string_value: %{additional.eventRpcCallClientLocality}	Value taken directly from the raw log field additional.event__RpcCallClientLocality .
additional.event__Service	security result.description: %{additional.event _Service}	Value taken directly from the raw log field additional.event__Service .
additional.event__Status	additional.fields.key: event Status additional.fields.value.string_value: %{additional.eventStatus}	Value taken directly from the raw log field additional.event__Status .
additional.event__SubStatus	additional.fields.key: event SubStatus additional.fields.value.string_value: %{additional.eventSubStatus}	Value taken directly from the raw log field additional.event__SubStatus .
additional.event__SubjectDomainName	principal.administrative domain: %{additional.event _SubjectDomainName}	Value taken directly from the raw log field additional.event__SubjectDomainName .
additional.event__SubjectLogonId	additional.fields.key: event SubjectLogonId additional.fields.value.string_value: %{additional.eventSubjectLogonId}	Value taken directly from the raw log field additional.event__SubjectLogonId .
additional.event__SubjectUserName	principal.user.user_display name: %{additional.event _SubjectUserName}	Value taken directly from the raw log field additional.event__SubjectUserName .
additional.event__SubjectUserSid	principal.user.windows sid: %{additional.event _SubjectUserSid}	Value taken directly from the raw log field additional.event__SubjectUserSid .
additional.event__TaskContentNew	additional.fields.key: event TaskContentNew additional.fields.value.string_value: %{additional.eventTaskContentNew}	Value taken directly from the raw log field additional.event__TaskContentNew .
additional.event__TaskName	additional.fields.key: event TaskName additional.fields.value.string_value: %{additional.eventTaskName}	Value taken directly from the raw log field additional.event__TaskName .
additional.event__TargetDomainName	target.administrative domain: %{additional.event _TargetDomainName}	Value taken directly from the raw log field additional.event__TargetDomainName .
additional.event__TargetLinkedLogonId	additional.fields.key: event TargetLinkedLogonId additional.fields.value.string_value: %{additional.eventTargetLinkedLogonId}	Value taken directly from the raw log field additional.event__TargetLinkedLogonId .
additional.event__TargetLogonId	additional.fields.key: event TargetLogonId additional.fields.value.string_value: %{additional.eventTargetLogonId}	Value taken directly from the raw log field additional.event__TargetLogonId .
additional.event__TargetOutboundDomainName	additional.fields.key: event TargetOutboundDomainName additional.fields.value.string_value: %{additional.eventTargetOutboundDomainName}	Value taken directly from the raw log field additional.event__TargetOutboundDomainName .
additional.event__TargetOutboundUserName	additional.fields.key: event TargetOutboundUserName additional.fields.value.string_value: %{additional.eventTargetOutboundUserName}	Value taken directly from the raw log field additional.event__TargetOutboundUserName .
additional.event__TargetSid	target.user.windows sid: %{additional.event _TargetSid}	Value taken directly from the raw log field additional.event__TargetSid .
additional.event__TargetUserName	target.user.userid: %{additional.event__TargetUserName}	Value taken directly from the raw log field additional.event__TargetUserName .
additional.event__TargetUserSid	target.user.windows sid: %{additional.event _TargetUserSid}	Value taken directly from the raw log field additional.event__TargetUserSid .
additional.event__TokenElevationType	additional.fields.key: event TokenElevationType additional.fields.value.string_value: %{additional.eventTokenElevationType}	Value taken directly from the raw log field additional.event__TokenElevationType .
additional.event__TransmittedServices	additional.fields.key: event TransmittedServices additional.fields.value.string_value: %{additional.eventTransmittedServices}	Value taken directly from the raw log field additional.event__TransmittedServices .
additional.event__VirtualAccount	additional.fields.key: event VirtualAccount additional.fields.value.string_value: %{additional.eventVirtualAccount}	Value taken directly from the raw log field additional.event__VirtualAccount .
additional.event__WorkstationName	target.hostname: %{additional.event__WorkstationName}	Value taken directly from the raw log field additional.event__WorkstationName .
additional.event_id	security_result.rule_name: EventID: %{additional.event_id}	Value taken directly from the raw log field additional.event_id and converted to a string.
additional.query	network.dns.questions.name: %{additional.query}	Value taken directly from the raw log field additional.query .
additional.response	network.dns.answers.name: %{additional.response}	Value taken directly from the raw log field additional.response .
metadata.description	metadata.description: %{metadata.description}	Value taken directly from the raw log field metadata.description .
metadata.eventTimestamp	metadata.event_timestamp.seconds: Extracted from %{metadata.eventTimestamp} metadata.event_timestamp.nanos: Extracted from %{metadata.eventTimestamp}	Seconds and nanoseconds are extracted from the raw log field metadata.eventTimestamp using date parsing.
metadata.eventType	metadata.product_event_type: %{metadata.eventType} metadata.event_type: %{metadata.eventType}	Value taken directly from the raw log field metadata.eventType .
metadata.logType	metadata.product_event_type: %{metadata.logType} metadata.event_type: %{metadata.logType}	Value taken directly from the raw log field metadata.logType .
network.applicationProtocol	network.application_protocol: %{network.applicationProtocol}	Value taken directly from the raw log field network.applicationProtocol .
network.direction	network.direction: %{network.direction}	Value taken directly from the raw log field network.direction .
network.ipProtocol	network.ip_protocol: %{network.ipProtocol}	Value taken directly from the raw log field network.ipProtocol .
principal.assetId	principal.asset_id: TANIUM:%{principal.assetId}	Value taken directly from the raw log field principal.assetId and prefixed with "TANIUM:".
principal.hostname	principal.hostname: %{principal.hostname}	Value taken directly from the raw log field principal.hostname .
principal.process.companySpecificParentProcessId	principal.process.product_specific_process_id: TANIUM:%{principal.assetId}:%{principal.process.companySpecificParentProcessId}	Value taken directly from the raw log field principal.process.companySpecificParentProcessId and formatted as "TANIUM:%{principal.assetId}:%{principal.process.companySpecificParentProcessId}".
principal.process.companySpecificProcessId	principal.process.product_specific_process_id: TANIUM:%{principal.assetId}:%{principal.process.companySpecificProcessId}	Value taken directly from the raw log field principal.process.companySpecificProcessId and formatted as "TANIUM:%{principal.assetId}:%{principal.process.companySpecificProcessId}".
principal.process.commandLine	target.process.command_line: %{principal.process.commandLine}	Value taken directly from the raw log field principal.process.commandLine with double quotes removed and hyphens replaced with ampersands.
principal.process.file.fullPath	target.process.file.full_path: %{principal.process.file.fullPath}	Value taken directly from the raw log field principal.process.file.fullPath .
principal.process.file.md5	target.process.file.md5: %{principal.process.file.md5}	Value taken directly from the raw log field principal.process.file.md5 and converted to lowercase.
principal.process.parentPid	principal.process.pid: %{principal.process.parentPid}	Value taken directly from the raw log field principal.process.parentPid only for PROCESS_LAUNCH events.
principal.process.pid	target.process.pid: %{principal.process.pid}	Value taken directly from the raw log field principal.process.pid only for PROCESS_LAUNCH events.
principal.process.productSpecificProcessId	principal.process.product_specific_process_id: TANIUM:%{principal.assetId}:%{principal.process.productSpecificProcessId}	Value taken directly from the raw log field principal.process.productSpecificProcessId and formatted as "TANIUM:%{principal.assetId}:%{principal.process.productSpecificProcessId}".
principal.user.groupid	principal.user.group_identifiers: %{principal.user.groupid}	Value taken directly from the raw log field principal.user.groupid .
principal.user.userid	principal.user.userid: %{principal.user.userid}	Value taken directly from the raw log field principal.user.userid .
src.ip	principal.ip: %{src.ip}	Value taken directly from the raw log field src.ip .
src.port	principal.port: %{src.port}	Value taken directly from the raw log field src.port and converted to an integer.
target.file.fullPath	target.file.full_path: %{target.file.fullPath}	Value taken directly from the raw log field target.file.fullPath .
target.file.md5	target.file.md5: %{target.file.md5}	Value taken directly from the raw log field target.file.md5 .
target.port	target.port: %{target.port}	Value taken directly from the raw log field target.port and converted to an integer.
target.registry.registryKey	target.registry.registry_key: %{target.registry.registryKey}	Value taken directly from the raw log field target.registry.registryKey .
target.registry.registryValue	target.registry.registry_value_data: %{target.registry.registryValue}	Value taken directly from the raw log field target.registry.registryValue .
target.user.userDisplayName	target.user.user_display_name: %{target.user.userDisplayName}	Value taken directly from the raw log field target.user.userDisplayName .
target.user.windowsSid	target.user.windows_sid: %{target.user.windowsSid}	Value taken directly from the raw log field target.user.windowsSid .
user-agent	network.http.user_agent: %{user-agent}	Value taken directly from the raw log field user-agent with double quotes removed.
N/A	extensions.auth.auth_mechanism: LOCAL/NETWORK/BATCH/SERVICE/UNLOCK/NETWORK_CLEAR_TEXT/NEW_CREDENTIALS/REMOTE_INTERACTIVE/CACHED_INTERACTIVE/MECHANISM_UNSPECIFIED	Determined by the parser code based on the value of additional.event__LogonType .
N/A	extensions.auth.type: MACHINE	Added by the parser code for USER_LOGIN and USER_LOGOUT events.
N/A	metadata.event_type: PROCESS_LAUNCH/NETWORK_CONNECTION/FILE_OPEN/FILE_DELETION/REGISTRY_MODIFICATION/USER_LOGIN/STATUS_UPDATE/USER_LOGOUT/PROCESS_MODULE_LOAD/PROCESS_TERMINATION/USER_CHANGE_PERMISSIONS/SCHEDULED_TASK_MODIFICATION/SCHEDULED_TASK_DISABLE/SCHEDULED_TASK_ENABLE/SCHEDULED_TASK_DELETION/SCHEDULED_TASK_CREATION/PROCESS_UNCATEGORIZED	Determined by the parser code based on the value of metadata.logType and additional.event_id .
N/A	metadata.log_type: TANIUM_TH	Hardcoded value added by the parser code.
N/A	metadata.product_name: Stream	Hardcoded value added by the parser code.
N/A	metadata.vendor_name: Tanium	Hardcoded value added by the parser code.
N/A	principal.hostname: %{principal_hostname}	Value taken from either principal.hostname or additional.event__FQDN .
N/A	principal.ip: %{srcIp}	Extracted from the raw log message using grok if src.ip is present.
N/A	security result.about.resource.name: %{additional.event _AuthenticationPackageName}	Value taken directly from the raw log field additional.event__AuthenticationPackageName for specific additional.event_id values.
N/A	security_result.category: AUTH_VIOLATION	Added by the parser code for specific additional.event_id values.
N/A	security_result.rule_name: EventID: %{additional.event_id}	Value taken directly from the raw log field additional.event_id and converted to a string.
N/A	target.hostname: %{query_host}	Extracted from the raw log field additional.query if it contains a hostname.
N/A	target.ip: %{dstIp}	Extracted from the raw log message using grok if src.ip is present.
N/A	target.ip: %{query_ip}	Extracted from the raw log field additional.query if it contains an IP address.
N/A	target.process.command_line: %{principal_process_commandLine}	Value taken from principal.process.commandLine if it is not empty.
N/A	target.process.file.full_path: %{principal_process_file_fullPath}	Value taken from principal.process.file.fullPath if it is not empty.
N/A	target.process.file.md5: %{principal_process_file_md5}	Value taken from principal.process.file.md5 if it is not empty.
N/A	target.process.product_specific_process_id: TANIUM:%{principal.assetId}:%{principal.process.productSpecificProcessId}	Value taken from principal.process.productSpecificProcessId and formatted as "TANIUM:%{principal.assetId}:%{principal.process.productSpecificProcessId}".
N/A	target.resource.resource_type: TASK	Added by the parser code for specific additional.event_id values.
N/A	timestamp.seconds: Extracted from %{metadata.eventTimestamp} timestamp.nanos: Extracted from %{metadata.eventTimestamp}	Seconds and nanoseconds are extracted from the raw log field metadata.eventTimestamp using date parsing.

Need more help? Get answers from Community members and Google SecOps professionals.