Collect SailPoint IAM logs
This document explains how you can ingest SailPoint Identity and Access Management (IAM) logs to Google Security Operations using Amazon S3. The parser handles the logs in JSON and XML formats, transforming them into the Unified Data Model (UDM). It differentiates between single UDM events (ProvisioningPlan, AccountRequest, SOAP-ENV), multi UDM events (ProvisioningProject), and UDM entities (Identity), applying specific parsing logic and field mappings for each, including generic event handling for non-XML data.
Before you begin
Make sure you have the following prerequisites:
- A Google SecOps instance.
- Privileged access to SailPoint Identity Security Cloud.
- Privileged access to AWS(S3, IAM, Lambda, EventBridge).
Collect SailPoint IAM prerequisites (IDs, API keys, org IDs, tokens)
- Sign in to the SailPoint Identity Security Cloud Admin Consoleas an administrator.
- Go to Global > Security Settings > API Management.
- Click Create API Client.
- Select Client Credentialsas the grant type.
- Provide the following configuration details:
- Name: Enter a descriptive name (for example,
Google SecOps Export API). - Description: Enter description for the API client.
- Scopes: Select
sp:scopes:all.
- Name: Enter a descriptive name (for example,
- Click Createand save the generated API credentials in a secure location.
- Record your SailPoint tenant base URL(for example,
https://tenant.api.identitynow.com). - Copy and save in a secure location the following details:
- IDN_CLIENT_ID.
- IDN_CLIENT_SECRET.
- IDN_BASE.
Configure AWS S3 bucket and IAM for Google SecOps
- Create Amazon S3 bucketfollowing this user guide: Creating a bucket
- Save bucket Nameand Regionfor future reference (for example,
sailpoint-iam-logs). - Create a Userfollowing this user guide: Creating an IAM user .
- Select the created User.
- Select Security credentialstab.
- Click Create Access Keyin section Access Keys.
- Select Third-party serviceas Use case.
- Click Next.
- Optional: Add description tag.
- Click Create access key.
- Click Download CSV fileto save the Access Keyand Secret Access Keyfor future reference.
- Click Done.
- Select Permissionstab.
- Click Add permissionsin section Permissions policies.
- Select Add permissions.
- Select Attach policies directly.
- Search for AmazonS3FullAccesspolicy.
- Select the policy.
- Click Next.
- Click Add permissions.
Configure the IAM policy and role for S3 uploads
- In the AWS console, go to IAM > Policies.
- Click Create policy > JSON tab.
- Copy and paste the following policy.
-
Policy JSON(replace
sailpoint-iam-logsif you entered a different bucket name):{ "Version" : "2012-10-17" , "Statement" : [ { "Sid" : "AllowPutObjects" , "Effect" : "Allow" , "Action" : "s3:PutObject" , "Resource" : "arn:aws:s3:::sailpoint-iam-logs/*" }, { "Sid" : "AllowGetStateObject" , "Effect" : "Allow" , "Action" : "s3:GetObject" , "Resource" : "arn:aws:s3:::sailpoint-iam-logs/sailpoint/iam/state.json" } ] } -
Click Next > Create policy.
-
Go to IAM > Roles > Create role > AWS service > Lambda.
-
Attach the newly created policy.
-
Name the role
SailPointIamToS3Roleand click Create role.
Create the Lambda function
- In the AWS Console, go to Lambda > Functions > Create function.
- Click Author from scratch.
-
Provide the following configuration details:
Setting Value Name sailpoint_iam_to_s3Runtime Python 3.13 Architecture x86_64 Execution role SailPointIamToS3Role -
After the function is created, open the Codetab, delete the stub and paste the following code (
sailpoint_iam_to_s3.py).#!/usr/bin/env python3 # Lambda: Pull SailPoint Identity Security Cloud audit events and store raw JSON payloads to S3 # - Uses /v3/search API with pagination for audit events. # - Preserves vendor-native JSON format for identity events. # - Retries with exponential backoff; unique S3 keys to avoid overwrites. import os , json , time , uuid , urllib.parse from urllib.request import Request , urlopen from urllib.error import URLError , HTTPError import boto3 S3_BUCKET = os . environ [ "S3_BUCKET" ] S3_PREFIX = os . environ . get ( "S3_PREFIX" , "sailpoint/iam/" ) STATE_KEY = os . environ . get ( "STATE_KEY" , "sailpoint/iam/state.json" ) WINDOW_SEC = int ( os . environ . get ( "WINDOW_SECONDS" , "3600" )) # default 1h HTTP_TIMEOUT = int ( os . environ . get ( "HTTP_TIMEOUT" , "60" )) IDN_BASE = os . environ [ "IDN_BASE" ] # e.g. https://tenant.api.identitynow.com CLIENT_ID = os . environ [ "IDN_CLIENT_ID" ] CLIENT_SECRET = os . environ [ "IDN_CLIENT_SECRET" ] SCOPE = os . environ . get ( "IDN_SCOPE" , "sp:scopes:all" ) PAGE_SIZE = int ( os . environ . get ( "PAGE_SIZE" , "250" )) MAX_PAGES = int ( os . environ . get ( "MAX_PAGES" , "20" )) MAX_RETRIES = int ( os . environ . get ( "MAX_RETRIES" , "3" )) USER_AGENT = os . environ . get ( "USER_AGENT" , "sailpoint-iam-to-s3/1.0" ) s3 = boto3 . client ( "s3" ) def _load_state (): try : obj = s3 . get_object ( Bucket = S3_BUCKET , Key = STATE_KEY ) return json . loads ( obj [ "Body" ] . read ()) except Exception : return {} def _save_state ( st ): s3 . put_object ( Bucket = S3_BUCKET , Key = STATE_KEY , Body = json . dumps ( st , separators = ( "," , ":" )) . encode ( "utf-8" ), ContentType = "application/json" , ) def _iso ( ts : float ) - > str : return time . strftime ( "%Y-%m- %d T%H:%M:%SZ" , time . gmtime ( ts )) def _get_oauth_token () - > str : """Get OAuth2 access token using Client Credentials flow""" token_url = f " { IDN_BASE . rstrip ( '/' ) } /oauth/token" data = urllib . parse . urlencode ({ 'grant_type' : 'client_credentials' , 'client_id' : CLIENT_ID , 'client_secret' : CLIENT_SECRET , 'scope' : SCOPE }) . encode ( 'utf-8' ) req = Request ( token_url , data = data , method = "POST" ) req . add_header ( "Content-Type" , "application/x-www-form-urlencoded" ) req . add_header ( "User-Agent" , USER_AGENT ) with urlopen ( req , timeout = HTTP_TIMEOUT ) as r : response = json . loads ( r . read ()) return response [ "access_token" ] def _search_events ( access_token : str , created_from : str , search_after : list = None ) - > list : """Search for audit events using SailPoint's /v3/search API""" search_url = f " { IDN_BASE . rstrip ( '/' ) } /v3/search" # Build search query for events created after specified time query_str = f 'created:">= { created_from } "' payload = { "indices" : [ "events" ], "query" : { "query" : query_str }, "sort" : [ "created" , "+id" ], "limit" : PAGE_SIZE } if search_after : payload [ "searchAfter" ] = search_after attempt = 0 while True : req = Request ( search_url , data = json . dumps ( payload ) . encode ( 'utf-8' ), method = "POST" ) req . add_header ( "Content-Type" , "application/json" ) req . add_header ( "Accept" , "application/json" ) req . add_header ( "Authorization" , f "Bearer { access_token } " ) req . add_header ( "User-Agent" , USER_AGENT ) try : with urlopen ( req , timeout = HTTP_TIMEOUT ) as r : response = json . loads ( r . read ()) # Handle different response formats if isinstance ( response , list ): return response return response . get ( "results" , response . get ( "data" , [])) except ( HTTPError , URLError ) as e : attempt += 1 print ( f "HTTP error on attempt { attempt } : { e } " ) if attempt > MAX_RETRIES : raise # exponential backoff with jitter time . sleep ( min ( 60 , 2 ** attempt ) + ( time . time () % 1 )) def _put_events_data ( events : list , from_ts : float , to_ts : float , page_num : int ) - > str : # Create unique S3 key for events data ts_path = time . strftime ( "%Y/%m/ %d " , time . gmtime ( to_ts )) uniq = f " { int ( time . time () * 1e6 ) } _ { uuid . uuid4 () . hex [: 8 ] } " key = f " { S3_PREFIX }{ ts_path } /sailpoint_iam_ { int ( from_ts ) } _ { int ( to_ts ) } _p { page_num : 03d } _ { uniq } .json" s3 . put_object ( Bucket = S3_BUCKET , Key = key , Body = json . dumps ( events , separators = ( "," , ":" )) . encode ( "utf-8" ), ContentType = "application/json" , Metadata = { 'source' : 'sailpoint-iam' , 'from_timestamp' : str ( int ( from_ts )), 'to_timestamp' : str ( int ( to_ts )), 'page_number' : str ( page_num ), 'events_count' : str ( len ( events )) } ) return key def _get_item_id ( item : dict ) - > str : """Extract ID from event item, trying multiple possible fields""" for field in ( "id" , "uuid" , "eventId" , "_id" ): if field in item and item [ field ]: return str ( item [ field ]) return "" def lambda_handler ( event = None , context = None ): st = _load_state () now = time . time () from_ts = float ( st . get ( "last_to_ts" ) or ( now - WINDOW_SEC )) to_ts = now # Get OAuth token access_token = _get_oauth_token () created_from = _iso ( from_ts ) print ( f "Fetching SailPoint IAM events from: { created_from } " ) # Handle pagination state last_created = st . get ( "last_created" ) last_id = st . get ( "last_id" ) search_after = [ last_created , last_id ] if ( last_created and last_id ) else None pages = 0 total_events = 0 written_keys = [] newest_created = last_created or created_from newest_id = last_id or "" while pages < MAX_PAGES : events = _search_events ( access_token , created_from , search_after ) if not events : break # Write page to S3 key = _put_events_data ( events , from_ts , to_ts , pages + 1 ) written_keys . append ( key ) total_events += len ( events ) # Update pagination state from last item last_event = events [ - 1 ] last_event_created = last_event . get ( "created" ) or last_event . get ( "metadata" , {}) . get ( "created" ) last_event_id = _get_item_id ( last_event ) if last_event_created : newest_created = last_event_created if last_event_id : newest_id = last_event_id search_after = [ newest_created , newest_id ] pages += 1 # If we got less than page size, we're done if len ( events ) < PAGE_SIZE : break print ( f "Successfully retrieved { total_events } events across { pages } pages" ) # Save state for next run st [ "last_to_ts" ] = to_ts st [ "last_created" ] = newest_created st [ "last_id" ] = newest_id st [ "last_successful_run" ] = now _save_state ( st ) return { "statusCode" : 200 , "body" : { "success" : True , "pages" : pages , "total_events" : total_events , "s3_keys" : written_keys , "from_timestamp" : from_ts , "to_timestamp" : to_ts , "last_created" : newest_created , "last_id" : newest_id } } if __name__ == "__main__" : print ( lambda_handler ()) -
Go to Configuration > Environment variables.
-
Click Edit > Add new environment variable.
-
Enter the environment variables provided in the following table, replacing the examples values with your values.
Environment variables
Key Example value S3_BUCKETsailpoint-iam-logsS3_PREFIXsailpoint/iam/STATE_KEYsailpoint/iam/state.jsonWINDOW_SECONDS3600HTTP_TIMEOUT60MAX_RETRIES3USER_AGENTsailpoint-iam-to-s3/1.0IDN_BASEhttps://tenant.api.identitynow.comIDN_CLIENT_IDyour-client-id(from step 2)IDN_CLIENT_SECRETyour-client-secret(from step 2)IDN_SCOPEsp:scopes:allPAGE_SIZE250MAX_PAGES20 -
After the function is created, stay on its page (or open Lambda > Functions > your-function).
-
Select the Configurationtab.
-
In the General configurationpanel, click Edit.
-
Change Timeoutto 5 minutes (300 seconds)and click Save.
Create an EventBridge schedule
- Go to Amazon EventBridge > Scheduler > Create schedule.
- Provide the following configuration details:
- Recurring schedule: Rate(
1 hour). - Target: Your Lambda function
sailpoint_iam_to_s3. - Name:
sailpoint-iam-1h.
- Recurring schedule: Rate(
- Click Create schedule.
(Optional) Create read-only IAM user & keys for Google SecOps
- Go to AWS Console > IAM > Users.
- Click Add users.
- Provide the following configuration details:
- User: Enter
secops-reader. - Access type: Select Access key – Programmatic access.
- User: Enter
- Click Create user.
- Attach minimal read policy (custom): Users > secops-reader > Permissions > Add permissions > Attach policies directly > Create policy.
-
JSON:
{ "Version" : "2012-10-17" , "Statement" : [ { "Effect" : "Allow" , "Action" : [ "s3:GetObject" ], "Resource" : "arn:aws:s3:::sailpoint-iam-logs/*" }, { "Effect" : "Allow" , "Action" : [ "s3:ListBucket" ], "Resource" : "arn:aws:s3:::sailpoint-iam-logs" } ] } -
Name =
secops-reader-policy. -
Click Create policy > search/select > Next > Add permissions.
-
Create access key for
secops-reader: Security credentials > Access keys. -
Click Create access key.
-
Download the
.CSV. (You'll paste these values into the feed).
Configure a feed in Google SecOps to ingest SailPoint IAM logs
- Go to SIEM Settings > Feeds.
- Click + Add New Feed.
- In the Feed namefield, enter a name for the feed (for example,
SailPoint IAM logs). - Select Amazon S3 V2as the Source type.
- Select SailPoint IAMas the Log type.
- Click Next.
- Specify values for the following input parameters:
- S3 URI:
s3://sailpoint-iam-logs/sailpoint/iam/ - Source deletion options: Select deletion option according to your preference.
- Maximum File Age: Include files modified in the last number of days. Default is 180 days.
- Access Key ID: User access key with access to the S3 bucket.
- Secret Access Key: User secret key with access to the S3 bucket.
- Asset namespace: The asset namespace .
- Ingestion labels: The label applied to the events from this feed.
- S3 URI:
- Click Next.
- Review your new feed configuration in the Finalizescreen, and then click Submit.
Supported SailPoint IAM Sample Logs
- JSON
{
"org"
:
"dummyorg"
,
"pod"
:
"prd99-useast1"
,
"created"
:
"2023-11-30T21:52:00.624Z"
,
"id"
:
"acbae614ff5de03be074d42256b366030e1d171155f2bcb486d16c78514f3114"
,
"action"
:
"AUTHENTICATION-105"
,
"type"
:
"AUTH"
,
"actor"
:
{
"name"
:
"user_auth_id"
},
"target"
:
{
"name"
:
"user_target_id"
},
"stack"
:
"oathkeeper"
,
"trackingNumber"
:
"332f85c52e9c4664b19775e692660d45"
,
"ipAddress"
:
"192.0.2.1"
,
"details"
:
"332f85c52e9c4664b19775e692660d45"
,
"attributes"
:
{
"pod"
:
"prd99-useast1"
,
"org"
:
"dummyorg"
,
"sourceName"
:
"AuthnProvider"
,
"info"
:
"LOGIN_SUCCESS_SAML"
,
"hostName"
:
"dummy-auth-host.com"
},
"objects"
:
[
"AUTHENTICATION"
],
"operation"
:
"REQUEST"
,
"status"
:
"PASSED"
,
"technicalName"
:
"AUTHENTICATION_REQUEST_PASSED"
,
"name"
:
"Request Authentication Passed"
,
"synced"
:
"2023-11-30T21:52:00.720Z"
,
"_type"
:
"event"
,
"_version"
:
"v7"
}
UDM mapping table
| Log field | UDM mapping | Logic |
|---|---|---|
action
|
metadata.description
|
The value of the action
field from the raw log. |
actor.name
|
principal.user.user_display_name
|
The value of the actor.name
field from the raw log. |
attributes.accountName
|
principal.user.group_identifiers
|
The value of the attributes.accountName
field from the raw log. |
attributes.appId
|
target.asset_id
|
"App ID: " concatenated with the value of the attributes.appId
field from the raw log. |
attributes.attributeName
|
additional.fields[0].value.string_value
|
The value of the attributes.attributeName
field from the raw log, placed within an additional.fields
object. The key is set to "Attribute Name". |
attributes.attributeValue
|
additional.fields[1].value.string_value
|
The value of the attributes.attributeValue
field from the raw log, placed within an additional.fields
object. The key is set to "Attribute Value". |
attributes.cloudAppName
|
target.application
|
The value of the attributes.cloudAppName
field from the raw log. |
attributes.hostName
|
target.hostname
, target.asset.hostname
|
The value of the attributes.hostName
field from the raw log. |
attributes.interface
|
additional.fields[2].value.string_value
|
The value of the attributes.interface
field from the raw log, placed within an additional.fields
object. The key is set to "Interface". |
attributes.operation
|
security_result.action_details
|
The value of the attributes.operation
field from the raw log. |
attributes.previousValue
|
additional.fields[3].value.string_value
|
The value of the attributes.previousValue
field from the raw log, placed within an additional.fields
object. The key is set to "Previous Value". |
attributes.provisioningResult
|
security_result.detection_fields.value
|
The value of the attributes.provisioningResult
field from the raw log, placed within a security_result.detection_fields
object. The key is set to "Provisioning Result". |
attributes.sourceId
|
principal.labels[0].value
|
The value of the attributes.sourceId
field from the raw log, placed within a principal.labels
object. The key is set to "Source Id". |
attributes.sourceName
|
principal.labels[1].value
|
The value of the attributes.sourceName
field from the raw log, placed within a principal.labels
object. The key is set to "Source Name". |
auditClassName
|
metadata.product_event_type
|
The value of the auditClassName
field from the raw log. |
created
|
metadata.event_timestamp.seconds
, metadata.event_timestamp.nanos
|
The value of the created
field from the raw log, converted to timestamp if instant.epochSecond
is not present. |
id
|
metadata.product_log_id
|
The value of the id
field from the raw log. |
instant.epochSecond
|
metadata.event_timestamp.seconds
|
The value of the instant.epochSecond
field from the raw log, used for timestamp. |
ipAddress
|
principal.asset.ip
, principal.ip
|
The value of the ipAddress
field from the raw log. |
interface
|
additional.fields[0].value.string_value
|
The value of the interface
field from the raw log, placed within an additional.fields
object. The key is set to "interface". |
loggerName
|
intermediary.application
|
The value of the loggerName
field from the raw log. |
message
|
metadata.description
, security_result.description
|
Used for various purposes, including setting the description in metadata and security_result, and extracting XML content. |
name
|
security_result.description
|
The value of the name
field from the raw log. |
operation
|
target.resource.attribute.labels[0].value
, metadata.product_event_type
|
The value of the operation
field from the raw log, placed within a target.resource.attribute.labels
object. The key is set to "operation". Also used for metadata.product_event_type
. |
org
|
principal.administrative_domain
|
The value of the org
field from the raw log. |
pod
|
principal.location.name
|
The value of the pod
field from the raw log. |
referenceClass
|
additional.fields[1].value.string_value
|
The value of the referenceClass
field from the raw log, placed within an additional.fields
object. The key is set to "referenceClass". |
referenceId
|
additional.fields[2].value.string_value
|
The value of the referenceId
field from the raw log, placed within an additional.fields
object. The key is set to "referenceId". |
sailPointObjectName
|
additional.fields[3].value.string_value
|
The value of the sailPointObjectName
field from the raw log, placed within an additional.fields
object. The key is set to "sailPointObjectName". |
serverHost
|
principal.hostname
, principal.asset.hostname
|
The value of the serverHost
field from the raw log. |
stack
|
additional.fields[4].value.string_value
|
The value of the stack
field from the raw log, placed within an additional.fields
object. The key is set to "Stack". |
status
|
security_result.severity_details
|
The value of the status
field from the raw log. |
target
|
additional.fields[4].value.string_value
|
The value of the target
field from the raw log, placed within an additional.fields
object. The key is set to "target". |
target.name
|
principal.user.userid
|
The value of the target.name
field from the raw log. |
technicalName
|
security_result.summary
|
The value of the technicalName
field from the raw log. |
thrown.cause.message
|
xml_body
, detailed_message
|
The value of the thrown.cause.message
field from the raw log, used to extract XML content. |
thrown.message
|
xml_body
, detailed_message
|
The value of the thrown.message
field from the raw log, used to extract XML content. |
trackingNumber
|
additional.fields[5].value.string_value
|
The value of the trackingNumber
field from the raw log, placed within an additional.fields
object. The key is set to "Tracking Number". |
type
|
metadata.product_event_type
|
The value of the type
field from the raw log. |
_version
|
metadata.product_version
|
The value of the _version
field from the raw log. |
|
N/A
|
metadata.event_timestamp
|
Derived from instant.epochSecond
or created
fields. |
|
N/A
|
metadata.event_type
|
Determined by parser logic based on various fields, including has_principal_user
, has_target_application
, technicalName
, and action
. Default value is "GENERIC_EVENT". |
|
N/A
|
metadata.log_type
|
Set to "SAILPOINT_IAM". |
|
N/A
|
metadata.product_name
|
Set to IAM
. |
|
N/A
|
metadata.vendor_name
|
Set to "SAILPOINT". |
|
N/A
|
extensions.auth.type
|
Set to "AUTHTYPE_UNSPECIFIED" in certain conditions. |
|
N/A
|
target.resource.attribute.labels[0].key
|
Set to "operation". |
Need more help? Get answers from Community members and Google SecOps professionals.

