Collect AWS API Gateway access logs

This document explains how to ingest Amazon API Gateway access logs to Google Security Operations using AWS CloudWatch Logs and Kinesis Data Firehose. Amazon API Gateway provides REST and HTTP APIs to build and manage APIs at scale. Access logs help monitor API usage and troubleshoot issues. This integration streams these logs into Google SecOps for analysis and monitoring.

Before you begin

Make sure you have the following prerequisites:

• Google SecOps instance
• Privileged access to AWS

Enable Amazon API Gateway Access Logging (to CloudWatch Logs)

1. Go to AWS Console > API Gateway.

2. For REST APIs:

   • Open your Stage > Logs/Tracing > enable Access logging.

   • Log format: Select JSON.

       { 
       
      "requestId" 
      : 
      "$context.requestId" 
      , 
       
      "ip" 
      : 
      "$context.identity.sourceIp" 
      , 
       
      "requestTime" 
      : 
      "$context.requestTime" 
      , 
       
      "httpMethod" 
      : 
      "$context.httpMethod" 
      , 
       
      "routeKey" 
      : 
      "$context.routeKey" 
      , 
       
      "status" 
      : 
      "$context.status" 
      , 
       
      "protocol" 
      : 
      "$context.protocol" 
      , 
       
      "responseLength" 
      : 
      "$context.responseLength" 
      , 
       
      "integrationLatency" 
      : 
      "$context.integrationLatency" 
      , 
       
      "error" 
      : 
      "$context.error.message" 
      } 
      
     

   • CloudWatch Logs log group: Choose or create a log group (for example, /aws/apigateway/access ).

3. For HTTP APIs:

   • Select your API > Monitor > Logging.
   • Select Stage > Edit.
   • Enable Access logging.
   • Use the same JSON log format as above.
   • CloudWatch Logs log group: Choose or create a log group (for example, /aws/apigateway/access ).

4. Click Save.

Configure a Feed in Google SecOps to Ingest Amazon API Gateway logs

1. Go to SIEM Settings > Feeds.
2. Click + Add New Feed.
3. In the Feed namefield, enter Amazon API Gateway - CloudWatch via Firehose .
4. Select Amazon Data Firehoseas the Source type.
5. Select Amazon API Gatewayas the Log type.
6. Click Next.
7. Specify values for the following input parameters:
   • Split delimiter: Optional n
   • Asset namespace: aws.api_gateway
   • Ingestion labels: For example, source=apigw_access , aws_region=<your-region>
8. Click Next.
9. Review the feed configuration and click Submit.
10. In the feed Details, click Generate Secret Keyand copy the Secret Key.
11. Copy the Feed HTTPS endpoint URLfrom Endpoint Information.
12. Create a Google Cloud API keyin APIs & Services > Credentials > Create credentials > API key, and restrict it to Google SecOps API.
13. Copy and save the API key in a secure location.

Configure Amazon Kinesis Data Firehose (Direct to Google SecOps)

1. In the AWS Console, go to Kinesis > Data Firehose > Create delivery stream.
2. Provide the following configuration details:
   • Source: Select Direct PUT or other sources.
   • Destination: Choose HTTP endpoint.
   • HTTP endpoint URL: Enter ENDPOINT_URL?key=API_KEY (use the Feed HTTPS endpoint URL and the API key from the previous step).
   • HTTP method: Select POST.
   • Access key: Paste the Secret Key generated in the feed.
   • Buffering hints: Set Buffer size= 1 MiB, Buffer interval= 60 seconds.
   • Compression: Select Disabled.
   • S3 backup: Select Disabled.
   • Leave retryand loggingsettings as default.
3. Click Create delivery stream. (For example, cwlogs-to-secops .)

Configure IAM Permissions and Subscribe the Log Group

1. In the AWS Console, go to IAM > Policies > Create policy > JSON.
2. 1. In the JSON editor, enter the following policy:

     { 
     
    "Version" 
    : 
     
    "2012-10-17" 
    , 
     
    "Statement" 
    : 
     
    [ 
     
    { 
     
    "Effect" 
    : 
     
    "Allow" 
    , 
     
    "Action" 
    : 
     
    [ 
     
    "firehose:PutRecord" 
    , 
     
    "firehose:PutRecordBatch" 
     
    ], 
     
    "Resource" 
    : 
     
    "arn:aws:firehose:<region>:<account-id>:deliverystream/cwlogs-to-secops" 
     
    } 
     
    ] 
    } 
    
   

   • Replace <region> and <account-id> with your AWS Region and account ID.

3. Name the policy CWLtoFirehoseWrite and click Create policy.

4. Go to IAM > Roles > Create role.

5. Select Custom trust policyand enter the following:

     { 
     
    "Version" 
    : 
     
    "2012-10-17" 
    , 
     
    "Statement" 
    : 
     
    [ 
     
    { 
     
    "Effect" 
    : 
     
    "Allow" 
    , 
     
    "Principal" 
    : 
     
    { 
     
    "Service" 
    : 
     
    "logs.<your-region>.amazonaws.com" 
     
    }, 
     
    "Action" 
    : 
     
    "sts:AssumeRole" 
     
    } 
     
    ] 
    } 
    
   

6. Attach the policy CWLtoFirehoseWrite to the role.

7. Name the role CWLtoFirehoseRole and click Create role.

8. Go to CloudWatch > Logs > Log groups.

9. Select the API Gatewaylog group you created earlier.

10. Open the Subscription filterstab and click Create.

11. Choose Create Amazon Kinesis Data Firehose subscription filter.

12. Configure the following:

    • Destination: Delivery stream cwlogs-to-secops .
    • Grant permission: Role CWLtoFirehoseRole .
    • Filter name: Enter all-events .
    • Filter pattern: Leave empty to send all events.

13. Click Start streaming.

Need more help? Get answers from Community members and Google SecOps professionals.