Collect AWS IAM logs

This document explains how to ingest AWS IAM logs to Google Security Operations. The parser transforms raw JSON formatted logs into a structured Unified Data Model (UDM). It extracts relevant fields like user details, role information, permissions, and timestamps, mapping them to corresponding UDM fields for consistent security analysis.

Before you begin

• Ensure that you have a Google SecOps instance.
• Ensure that you have privileged access to AWS.

Configure AWS IAM and S3

1. Create an Amazon S3 bucketfollowing this user guide: Creating a bucket .
2. Save the bucket Nameand Regionfor later use.
3. Create a user following this user guide: Creating an IAM user .
4. Select the created User.
5. Select the Security credentialstab.
6. Click Create Access Keyin the Access Keyssection.
7. Select Third-party serviceas the Use case.
8. Click Next.
9. Optional: add a description tag.
10. Click Create access key.
11. Click Download CSV fileto save the Access Keyand Secret Access Keyfor later use.
12. Click Done.
13. Select the Permissionstab.
14. Click Add permissionsin the Permissions policiessection.
15. Select Add permissions.
16. Select Attach policies directly
17. Search for and select the AmazonS3FullAccesspolicy.
18. Click Next.
19. Click Add permissions.

Configure CloudTrail to capture IAM logs

1. Sign in to the AWS Management Console .
2. In the search bar, type and select CloudTrailfrom the services list.
3. Click Create trail.
4. Provide a Trail name; for example, IAMActivityTrail .
   • Apply trail to all regions: select Yesto capture activities across all regions.
   • Storage location: select the S3 bucket created earlier or create a new one.
   • S3 Bucket: enter a name for the S3 bucket; for example, iam-logs-bucket .
   • Select Create a new IAM role(if not created earlier).
   • Management events: select Readand Writeto capture both read and write events on IAM resources.
   • Data events: enable S3 and Lambdadata events.
5. Click Createto create the trail.

Configure CloudTrail to Export Logs to S3

1. Go to Services > S3.
2. Select the S3 bucketwhere CloudTrail logs are stored; for example, iam-logs-bucket .
3. Ensure that CloudTrailhas the right permissions to write logs to the bucket.

4. Add the following policy if it's not already present:

     { 
     
    "Version" 
    : 
     
    "2012-10-17" 
    , 
     
    "Statement" 
    : 
     
    [ 
     
    { 
     
    "Sid" 
    : 
     
    "CloudTrailS3Access" 
    , 
     
    "Effect" 
    : 
     
    "Allow" 
    , 
     
    "Principal" 
    : 
     
    { 
     
    "Service" 
    : 
     
    "cloudtrail.amazonaws.com" 
     
    }, 
     
    "Action" 
    : 
     
    "s3:PutObject" 
    , 
     
    "Resource" 
    : 
     
    "arn:aws:s3:::your-bucket-name/AWSLogs/*" 
     
    } 
     
    ] 
    } 
    
   

5. Enable Versioning on the S3 bucket to ensure that logs are stored with multiple versions.

6. Go to Properties > Bucket Versioning > Enable.

Optional: Configure Lambda for real-time export

1. Go to the AWS Lambda Console.
2. Click Create function.
3. Select Author from Scratch.
4. Set the function name as ExportIAMLogsToS3 .
5. Select a Python 3.xruntime.

6. Assign an IAM Role to the function that has permissions to:

     { 
     
    "Version" 
    : 
     
    "2012-10-17" 
    , 
     
    "Statement" 
    : 
     
    [ 
     
    { 
     
    "Effect" 
    : 
     
    "Allow" 
    , 
     
    "Action" 
    : 
     
    [ 
     
    "logs:GetLogEvents" 
    , 
     
    "logs:FilterLogEvents" 
    , 
     
    "logs:DescribeLogGroups" 
    , 
     
    "logs:DescribeLogStreams" 
     
    ], 
     
    "Resource" 
    : 
     
    "*" 
     
    }, 
     
    { 
     
    "Effect" 
    : 
     
    "Allow" 
    , 
     
    "Action" 
    : 
     
    [ 
     
    "s3:PutObject" 
     
    ], 
     
    "Resource" 
    : 
     
    "arn:aws:s3:::your-bucket-name/*" 
     
    } 
     
    ] 
    } 
    
   

7. Use the following Pythoncode to fetch IAM logsand upload them to S3:

     import 
     
    boto3 
    import 
     
    gzip 
    from 
     
    io 
     
    import 
    BytesIO 
    s3 
    = 
    boto3 
    . 
    client 
    ( 
    's3' 
    ) 
    logs 
    = 
    boto3 
    . 
    client 
    ( 
    'logs' 
    ) 
    def 
     
    lambda_handler 
    ( 
    event 
    , 
    context 
    ): 
    log_group 
    = 
    event 
    [ 
    'logGroup' 
    ] 
    log_stream 
    = 
    event 
    [ 
    'logStream' 
    ] 
    log_events 
    = 
    logs 
    . 
    get_log_events 
    ( 
    logGroupName 
    = 
    log_group 
    , 
    logStreamName 
    = 
    log_stream 
    , 
    startFromHead 
    = 
    True 
    ) 
    log_data 
    = 
    " 
    \n 
    " 
    . 
    join 
    ([ 
    event 
    [ 
    'message' 
    ] 
    for 
    event 
    in 
    log_events 
    [ 
    'events' 
    ]]) 
    # Compress and upload to S3 
    compressed_data 
    = 
    gzip 
    . 
    compress 
    ( 
    log_data 
    . 
    encode 
    ( 
    'utf-8' 
    )) 
    s3 
    . 
    put_object 
    ( 
    Bucket 
    = 
    'your-s3-bucket-name' 
    , 
    Key 
    = 
    'iam-logs/ 
    {log_stream} 
    .gz' 
    , 
    Body 
    = 
    compressed_data 
    ) 
    
   

• Replace your-s3-bucket-name with your actual bucket name.

Configure Lambda Trigger for CloudWatch Logs

1. In the Lambda Console, go to Designer.
2. Choose Add Trigger > CloudWatch Logs.
3. Select the CloudWatch Logslog group associated with your IAM logs; for example, /aws/cloudtrail/ .
4. Click Add.

Set up feeds

There are two different entry points to set up feeds in the Google SecOps platform:

• SIEM Settings > Feeds > Add New
• Content Hub > Content Packs > Get Started

How to set up the AWS IAM feed

1. Click the Amazon Cloud Platformpack.
2. Locate the AWS IAMlog type.

3. Specify the values in the following fields.

   • Source Type: Third party API
   • Username: Username to authenticate with
   • Secret: Secret to authenticate with

   Advanced options

   • Feed Name: A prepopulated value that identifies the feed.
   • Asset Namespace: Namespace associated with the feed.
   • Ingestion Labels: Labels applied to all events from this feed.

4. Click Create feed.

For more information about configuring multiple feeds for different log types within this product family, see Configure feeds by product .

UDM Mapping Table

Log Field	UDM Mapping	Logic
Arn	entity.entity.resource.name	Directly mapped from the Arn field for USER and RESOURCE entity types. For GROUP entity type, it's mapped from Group.Arn .
AssumeRolePolicyDocument	entity.entity.resource.attribute.permissions.name	Directly mapped from the AssumeRolePolicyDocument field, but only for RESOURCE entity type.
CreateDate	entity.entity.user.attribute.creation_time	Directly mapped from the CreateDate field and converted to Chronicle's timestamp format for USER entity type.
CreateDate	entity.entity.resource.attribute.creation_time	Directly mapped from the CreateDate field and converted to Chronicle's timestamp format for RESOURCE entity type.
Group.Arn	entity.entity.resource.name	Directly mapped from the Group.Arn field for GROUP entity type.
Group.CreateDate	entity.entity.group.attribute.creation_time	Directly mapped from the Group.CreateDate field and converted to Chronicle's timestamp format.
Group.GroupID	entity.entity.group.product_object_id	Directly mapped from the Group.GroupID field.
Group.GroupName	entity.entity.group.group_display_name	Directly mapped from the Group.GroupName field.
Group.GroupName	entity.entity.group.email_addresses	Directly mapped from the Group.GroupName field.
Group.Path	entity.entity.group.attribute.labels.value	Directly mapped from the Group.Path field, the key is hardcoded to path
IsTruncated	entity.entity.group.attribute.labels.value	Directly mapped from the IsTruncated field and converted to string, the key is hardcoded to is_truncated
Marker	entity.entity.group.attribute.labels.value	Directly mapped from the Marker field, the key is hardcoded to marker
PasswordLastUsed	entity.entity.user.last_login_time	Directly mapped from the PasswordLastUsed field and converted to Chronicle's timestamp format.
Path	entity.entity.user.attribute.labels.value	Directly mapped from the Path field for USER entity type, the key is hardcoded to path .
Path	entity.entity.resource.attribute.labels.value	Directly mapped from the Path field for RESOURCE entity type, the key is hardcoded to path .
PermissionsBoundary.PermissionsBoundaryArn	entity.entity.resource.attribute.labels.value	Directly mapped from the PermissionsBoundary.PermissionsBoundaryArn field, the key is hardcoded to permissions_boundary_arn .
PermissionsBoundary.PermissionsBoundaryType	entity.entity.resource.attribute.labels.value	Directly mapped from the PermissionsBoundary.PermissionsBoundaryType field, the key is hardcoded to permissions_boundary_type .
RoleID	entity.entity.resource.product_object_id	Directly mapped from the RoleID field.
RoleLastUsed.LastUsedDate	entity.entity.resource.attribute.labels.value	Directly mapped from the RoleLastUsed.LastUsedDate field, the key is hardcoded to role_last_used_date .
RoleLastUsed.Region	entity.entity.location.name	Directly mapped from the RoleLastUsed.Region field.
RoleName	entity.entity.resource.attribute.roles.name	Directly mapped from the RoleName field.
Tags.Key	entity.entity.user.attribute.labels.key	Used as the key for the labels field within the user attribute.
Tags.Value	entity.entity.user.attribute.labels.value	Used as the value for the labels field within the user attribute.
UserID	entity.entity.user.product_object_id	Directly mapped from the UserID field.
UserName	entity.entity.user.userid	Directly mapped from the UserName field.
Users.Arn	relations.entity.resource.name	Directly mapped from the Users.Arn field within the user relation.
Users.CreateDate	relations.entity.user.attribute.creation_time	Directly mapped from the Users.CreateDate field within the user relation and converted to Chronicle's timestamp format.
Users.PasswordLastUsed	relations.entity.user.last_login_time	Directly mapped from the Users.PasswordLastUsed field within the user relation and converted to Chronicle's timestamp format.
Users.Path	relations.entity.user.attribute.labels.value	Directly mapped from the Users.Path field within the user relation, the key is hardcoded to path .
Users.PermissionsBoundary.PermissionsBoundaryArn	relations.entity.resource.attribute.labels.value	Directly mapped from the Users.PermissionsBoundary.PermissionsBoundaryArn field within the user relation, the key is hardcoded to permissions_boundary_arn .
Users.PermissionsBoundary.PermissionsBoundaryType	relations.entity.resource.attribute.labels.value	Directly mapped from the Users.PermissionsBoundary.PermissionsBoundaryType field within the user relation, the key is hardcoded to permissions_boundary_type .
Users.UserID	relations.entity.user.product_object_id	Directly mapped from the Users.UserID field within the user relation.
Users.UserName	relations.entity.user.userid	Directly mapped from the Users.UserName field within the user relation.
N/A	entity.metadata.collected_timestamp	Populated with the event timestamp from the raw log.
N/A	entity.metadata.vendor_name	Hardcoded to AWS .
N/A	entity.metadata.product_name	Hardcoded to AWS IAM .
N/A	entity.metadata.entity_type	Determined based on the presence of specific fields: USER if UserID exists, RESOURCE if RoleName exists, and GROUP if Group.GroupName exists.
N/A	entity.entity.resource.resource_subtype	Set to User for USER and GROUP entity types.
N/A	entity.entity.resource.resource_type	Set to ACCESS_POLICY for RESOURCE entity type.
N/A	entity.entity.resource.attribute.cloud.environment	Hardcoded to AMAZON_WEB_SERVICES .
N/A	relations.entity_type	Hardcoded to USER for user relations within a group.
N/A	relations.relationship	Hardcoded to MEMBER for user relations within a group.
N/A	relations.direction	Hardcoded to UNIDIRECTIONAL for user relations within a group.
N/A	relations.entity.resource.resource_subtype	Hardcoded to User for user relations within a group.

Need more help? Get answers from Community members and Google SecOps professionals.