Collect Zendesk CRM logs
Supported in:
This document explains how to ingest Zendesk Customer Relationship Management (CRM) logs to Google Security Operations using Amazon S3.
Before you begin
Make sure you have the following prerequisites:
- A Google SecOps instance.
- Privileged access to Zendesk.
- Privileged access to AWS(S3, Identity and Access Management (IAM), Lambda, EventBridge).
Get Zendesk prerequisites
- Confirm plan & role
- You must be a Zendesk Adminto create API tokens / OAuth clients. The Audit Logs APIis available only on Enterprise plan. (If your account isn't Enterprise, skip
audit_logsinRESOURCES.)
- You must be a Zendesk Adminto create API tokens / OAuth clients. The Audit Logs APIis available only on Enterprise plan. (If your account isn't Enterprise, skip
- Turn on API token access (one-time)
- In the Admin Center, go to Apps and integrations > APIs > API configuration.
- Enable Allow API token access.
- Generate an API token (for Basic auth)
- Go to Apps and integrations > APIs > API tokens.
- Click Add API token > (optionally) add Description > Save.
- Copy and save the API tokennow (you won't be able to view it again).
- Save the admin emailthat will authenticate with this token.
- Basic-auth format used by the Lambda:
email_address/token:api_token
- Basic-auth format used by the Lambda:
- (Optional) Create an OAuth client (for Bearer auth instead of API token)
- Go to Apps and integrations > APIs > OAuth clients > Add OAuth client.
- Fill in the Name, Unique Identifier(auto), Redirect URLs(can be placeholder if you only mint tokens with API), and Save.
- Create an access token for the integration and grant the minimum scopesrequired by this guide:
-
tickets:read(for Incremental Tickets) -
auditlogs:read(for Audit Logs; Enterprise only) - If unsure,
readalso works for read-only access.
-
- Copy the access token(paste into
ZENDESK_BEARER_TOKEN) and record the client ID/secretsecurely (for future token refresh flows).
-
Record your Zendesk base URL
- Use
https://<your_subdomain>.zendesk.com(paste intoZENDESK_BASE_URLenv var).
What to copy & save for later
- Base URL(for example,
https://acme.zendesk.com) - Email Addressof the administrator user (for API token auth)
- API Token(if using
AUTH_MODE=token) - or OAuth access token(if using
AUTH_MODE=bearer) - (Optional): OAuth client id/secretfor lifecycle management
- Use
Configure AWS S3 bucket and IAM for Google SecOps
- Create Amazon S3 bucketfollowing this user guide: Creating a bucket
- Save bucket Nameand Regionfor future reference (for example,
zendesk-crm-logs). - Create a Userfollowing this user guide: Creating an IAM user .
- Select the created User.
- Select the Security credentialstab.
- Click Create Access Keyin section Access Keys.
- Select Third-party serviceas Use case.
- Click Next.
- Optional: Add a description tag.
- Click Create access key.
- Click Download .CSV fileto save the Access Keyand Secret Access Keyfor future reference.
- Click Done.
- Select Permissionstab.
- Click Add permissionsin section Permissions policies.
- Select Add permissions.
- Select Attach policies directly.
- Search for AmazonS3FullAccesspolicy.
- Select the policy.
- Click Next.
- Click Add permissions.
Configure the IAM policy and role for S3 uploads
- In the AWS console, go to IAM > Policies.
- Click Create policy > JSON tab.
- Copy and paste the following policy.
-
Policy JSON(replace
zendesk-crm-logsif you entered a different bucket name):{ "Version" : "2012-10-17" , "Statement" : [ { "Sid" : "AllowPutObjects" , "Effect" : "Allow" , "Action" : "s3:PutObject" , "Resource" : "arn:aws:s3:::zendesk-crm-logs/*" }, { "Sid" : "AllowGetStateObject" , "Effect" : "Allow" , "Action" : "s3:GetObject" , "Resource" : "arn:aws:s3:::zendesk-crm-logs/zendesk/crm/state.json" } ] } -
Click Next > Create policy.
-
Go to IAM > Roles > Create role > AWS service > Lambda.
-
Attach the newly created policy.
-
Name the role
ZendeskCRMToS3Roleand click Create role.
Create the Lambda function
- In the AWS Console, go to Lambda > Functions > Create function.
- Click Author from scratch.
-
Provide the following configuration details:
Setting Value Name zendesk_crm_to_s3Runtime Python 3.13 Architecture x86_64 Execution role ZendeskCRMToS3Role -
After the function is created, open the Codetab, delete the stub and paste the following code (
zendesk_crm_to_s3.py).#!/usr/bin/env python3 import os , json , time , base64 from urllib.request import Request , urlopen from urllib.error import HTTPError , URLError import boto3 S3_BUCKET = os . environ [ "S3_BUCKET" ] S3_PREFIX = os . environ . get ( "S3_PREFIX" , "zendesk/crm/" ) STATE_KEY = os . environ . get ( "STATE_KEY" , "zendesk/crm/state.json" ) BASE_URL = os . environ [ "ZENDESK_BASE_URL" ] . rstrip ( "/" ) # e.g. https://your_subdomain.zendesk.com AUTH_MODE = os . environ . get ( "AUTH_MODE" , "token" ) . lower () # token|bearer EMAIL = os . environ . get ( "ZENDESK_EMAIL" , "" ) API_TOKEN = os . environ . get ( "ZENDESK_API_TOKEN" , "" ) BEARER = os . environ . get ( "ZENDESK_BEARER_TOKEN" , "" ) RESOURCES = [ r . strip () for r in os . environ . get ( "RESOURCES" , "audit_logs,incremental_tickets" ) . split ( "," ) if r . strip ()] MAX_PAGES = int ( os . environ . get ( "MAX_PAGES" , "20" )) LOOKBACK = int ( os . environ . get ( "LOOKBACK_SECONDS" , "3600" )) # 1h default HTTP_TIMEOUT = int ( os . environ . get ( "HTTP_TIMEOUT" , "60" )) HTTP_RETRIES = int ( os . environ . get ( "HTTP_RETRIES" , "3" )) s3 = boto3 . client ( "s3" ) def _headers () - > dict : if AUTH_MODE == "bearer" and BEARER : return { "Authorization" : f "Bearer { BEARER } " , "Accept" : "application/json" } if AUTH_MODE == "token" and EMAIL and API_TOKEN : token = base64 . b64encode ( f " { EMAIL } /token: { API_TOKEN } " . encode ()) . decode () return { "Authorization" : f "Basic { token } " , "Accept" : "application/json" } raise RuntimeError ( "Invalid auth settings: provide token (EMAIL + API_TOKEN) or BEARER" ) def _get_state () - > dict : try : obj = s3 . get_object ( Bucket = S3_BUCKET , Key = STATE_KEY ) b = obj [ "Body" ] . read () return json . loads ( b ) if b else { "audit_logs" : {}, "incremental_tickets" : {}} except Exception : return { "audit_logs" : {}, "incremental_tickets" : {}} def _put_state ( st : dict ) - > None : s3 . put_object ( Bucket = S3_BUCKET , Key = STATE_KEY , Body = json . dumps ( st , separators = ( "," , ":" )) . encode ( "utf-8" ), ContentType = "application/json" , ) def _http_get_json ( url : str ) - > dict : attempt = 0 while True : try : req = Request ( url , method = "GET" ) for k , v in _headers () . items (): req . add_header ( k , v ) with urlopen ( req , timeout = HTTP_TIMEOUT ) as r : return json . loads ( r . read () . decode ( "utf-8" )) except HTTPError as e : if e . code in ( 429 , 500 , 502 , 503 , 504 ) and attempt < HTTP_RETRIES : ra = 1 + attempt try : ra = int ( e . headers . get ( "Retry-After" , ra )) except Exception : pass time . sleep ( max ( 1 , ra )) attempt += 1 continue raise except URLError : if attempt < HTTP_RETRIES : time . sleep ( 1 + attempt ) attempt += 1 continue raise def _put_page ( payload : dict , resource : str ) - > str : ts = time . gmtime () key = f " { S3_PREFIX } / { time . strftime ( '%Y/%m/ %d /%H%M%S' , ts ) } -zendesk- { resource } .json" s3 . put_object ( Bucket = S3_BUCKET , Key = key , Body = json . dumps ( payload , separators = ( "," , ":" )) . encode ( "utf-8" ), ContentType = "application/json" , ) return key def fetch_audit_logs ( state : dict ): """GET /api/v2/audit_logs.json with pagination via `next_page` (Zendesk).""" next_url = state . get ( "next_url" ) or f " { BASE_URL } /api/v2/audit_logs.json?page=1" pages = 0 written = 0 last_next = None while pages < MAX_PAGES and next_url : data = _http_get_json ( next_url ) _put_page ( data , "audit_logs" ) written += len ( data . get ( "audit_logs" , [])) last_next = data . get ( "next_page" ) next_url = last_next pages += 1 return { "resource" : "audit_logs" , "pages" : pages , "written" : written , "next_url" : last_next } def fetch_incremental_tickets ( state : dict ): """Cursor-based incremental export: /api/v2/incremental/tickets/cursor.json (pagination via `links.next`).""" next_link = state . get ( "next" ) if not next_link : start = int ( time . time ()) - LOOKBACK next_link = f " { BASE_URL } /api/v2/incremental/tickets/cursor.json?start_time= { start } " pages = 0 written = 0 last_next = None while pages < MAX_PAGES and next_link : data = _http_get_json ( next_link ) _put_page ( data , "incremental_tickets" ) written += len ( data . get ( "tickets" , [])) links = data . get ( "links" ) or {} next_link = links . get ( "next" ) last_next = next_link pages += 1 return { "resource" : "incremental_tickets" , "pages" : pages , "written" : written , "next" : last_next } def lambda_handler ( event = None , context = None ): state = _get_state () summary = [] if "audit_logs" in RESOURCES : res = fetch_audit_logs ( state . get ( "audit_logs" , {})) state [ "audit_logs" ] = { "next_url" : res . get ( "next_url" )} summary . append ( res ) if "incremental_tickets" in RESOURCES : res = fetch_incremental_tickets ( state . get ( "incremental_tickets" , {})) state [ "incremental_tickets" ] = { "next" : res . get ( "next" )} summary . append ( res ) _put_state ( state ) return { "ok" : True , "summary" : summary } if __name__ == "__main__" : print ( lambda_handler ()) -
Go to Configuration > Environment variables.
-
Click Edit > Add new environment variable.
-
Enter the environment variables provided in the following table, replacing the example values with your values.
Environment variables
Key Example value S3_BUCKETzendesk-crm-logsS3_PREFIXzendesk/crm/STATE_KEYzendesk/crm/state.jsonZENDESK_BASE_URLhttps://your_subdomain.zendesk.comAUTH_MODEtokenZENDESK_EMAILanalyst@example.comZENDESK_API_TOKEN<api_token>ZENDESK_BEARER_TOKEN<leave empty unless using OAuth bearer>RESOURCESaudit_logs,incremental_ticketsMAX_PAGES20LOOKBACK_SECONDS3600HTTP_TIMEOUT60 -
After the function is created, stay on its page (or open Lambda > Functions > your-function).
-
Select the Configurationtab.
-
In the General configurationpanel, click Edit.
-
Change Timeoutto 5 minutes (300 seconds)and click Save.
Create an EventBridge schedule
- Go to Amazon EventBridge > Scheduler > Create schedule.
- Provide the following configuration details:
- Recurring schedule: Rate(
1 hour). - Target: Your Lambda function
zendesk_crm_to_s3. - Name:
zendesk_crm_to_s3-1h.
- Recurring schedule: Rate(
- Click Create schedule.
(Optional) Create read-only IAM user & keys for Google SecOps
- Go to AWS Console > IAM > Users > Add users.
- Click Add users.
- Provide the following configuration details:
- User: Enter
secops-reader. - Access type: Select Access key – Programmatic access.
- User: Enter
- Click Create user.
- Attach minimal read policy (custom): Users > secops-reader > Permissions > Add permissions > Attach policies directly > Create policy.
-
JSON:
{ "Version" : "2012-10-17" , "Statement" : [ { "Effect" : "Allow" , "Action" : [ "s3:GetObject" ], "Resource" : "arn:aws:s3:::zendesk-crm-logs/*" }, { "Effect" : "Allow" , "Action" : [ "s3:ListBucket" ], "Resource" : "arn:aws:s3:::zendesk-crm-logs" } ] } -
Name =
secops-reader-policy. -
Click Create policy > search/select > Next > Add permissions.
-
Create access key for
secops-reader: Security credentials > Access keys. -
Click Create access key.
-
Download the
.CSV. (You'll paste these values into the feed).
Configure a feed in Google SecOps to ingest Zendesk CRM logs
- Go to SIEM Settings > Feeds.
- Click + Add New Feed.
- In the Feed namefield, enter a name for the feed (for example,
Zendesk CRM logs). - Select Amazon S3 V2as the Source type.
- Select Zendesk CRMas the Log type.
- Click Next.
- Specify values for the following input parameters:
- S3 URI:
s3://zendesk-crm-logs/zendesk/crm/ - Source deletion options: Select deletion option according to your preference.
- Maximum File Age: Include files modified in the last number of days. Default is 180 days.
- Access Key ID: User access key with access to the S3 bucket.
- Secret Access Key: User secret key with access to the S3 bucket.
- Asset namespace: The asset namespace .
- Ingestion labels: The label applied to the events from this feed.
- S3 URI:
- Click Next.
- Review your new feed configuration in the Finalizescreen, and then click Submit.
Need more help? Get answers from Community members and Google SecOps professionals.
