Collect Jenkins logs

Overview

This parser extracts key information such as timestamps, user IDs, source IPs, actions, and object IDs from JSON and SYSLOG formatted logs. It uses grok patterns to match various log message formats, handling variations in structure, and populates a unified data model (UDM) with the extracted fields. The parser also categorizes events based on the presence of user or IP information.

Before you begin

Ensure that you have the following prerequisites:

• Google SecOps instance.
• Privileged access to Google Cloud IAM.
• Privileged access to Google Cloud Storage.
• Privileged access to Jenkins.

Create a Google Cloud Storage Bucket

1. Go to Cloud Storage.
2. Create a new bucket. Choose a unique name and appropriate region.
3. Ensure the bucket has proper access controls (for example, only authorized service accounts can write to it).

Create a Google Cloud Service account

1. Go to IAM & Admin > Service Accounts.
2. Create a new service account. Give it a descriptive name (for example, jenkins-logs).
3. Grant the service account the Storage Object Creatorrole on the GCS bucket you created in the previous step.
4. Create an SSH key for your service account: Create and delete service account keys .

5. Download a JSON key file for the service account.

Install Google Cloud Storage plugin in Jenkins

1. Go to Manage Jenkins > Plugins.
2. Select Available plugins.
3. Search for the Google Cloud Storageplugin.
4. Install the plugin and restart Jenkins if required.

Install Google OAuth Credentials Plugin in Jenkins

1. Go to Manage Jenkins > Plugins.
2. Select Available plugins
3. Search for the Google OAuth Credentialsplugin.
4. Install the plugin and restart Jenkins if required.

Configure Jenkins to authenticate with Google Cloud

1. Go to Manage Jenkins > Credentials > System.

2. Click add Add Credentials.

3. Kind: select Google Service Account from private key.

4. Project name: set a name for the credentials.

5. Upload the JSON key file you obtained during the Google Cloud Service account creation.

6. Click Create.

Configure Jenkins logs to upload Google SecOps

1. In the Jenkins job configuration, add Google Storage Build Log Uploadin post-build actions, with the following parameters:
   • Google Credentials: The name of your Google credentials you created in the previous step.
   • Log Name: The name of the file to store the Jenkins build log, under the specified storage path.
   • Storage Location: The name of the bucket where you want to upload your logs. The bucket must be accessible to the service account you created.
2. Test the log upload.

Set up feeds

To configure a feed, follow these steps:

1. Go to SIEM Settings > Feeds.
2. Click Add New Feed.
3. On the next page, click Configure a single feed.
4. In the Feed namefield, enter a name for the feed; for example, Jenkins Logs.
5. Select Google Cloud Storage V2as the Source type.
6. Select Jenkinsas the Log type.
7. Click Get Service Accountas the Chronicle Service Account.

8. Specify values for the following input parameters:

   • Storage Bucket URI: Google Cloud storage bucket URL in gs://my-bucket/<value> format.
   • Source deletion options: select deletion option according to your preference.

9. Click Create Feed.

UDM Mapping Table

Log Field	UDM Mapping	Logic
act	security_result.action_details	Extracted from msg1or msg2fields. Represents the action performed. Leading whitespace is removed.
data	principal.user.useridOR principal.ipOR metadata.description	If datamatches an IP address pattern, it maps to principal.ip. If it matches a username pattern, it maps to principal.user.userid. Otherwise, it maps to metadata.description.
msg1	target.asset.product_object_idOR security_result.action_details	Used to extract objectand act. If a / is present, it is split into objectand act. If » is present, it is split into objectand act. Otherwise, it is treated as actand potentially further parsed.
msg2	metadata.descriptionOR security_result.action_details	If present, initially mapped to metadata.description. If it contains "completed:", the value after is extracted and mapped to security_result.action_details.
object	target.asset.product_object_id	Extracted from msg1. Represents the object acted upon.
object_id	target.resource.attribute.labels.value	Extracted from objectif a / is present. Represents a more specific object identifier. The key is hardcoded as "Plugin Name".
src_ip	principal.ip	Extracted from messageor data. Represents the source IP address.
user	principal.user.userid	Extracted from messageor data. Represents the user associated with the event.
	metadata.event_timestamp	Copied from the calculated @timestampfield.
	metadata.event_type	Determined by parser logic. Set to USER_UNCATEGORIZEDif useris present, STATUS_UNCATEGORIZEDif src_ipis present, and GENERIC_EVENTotherwise.
	metadata.product_name	Hardcoded as Jenkins.
	metadata.product_version	Hardcoded as Jenkins.
	metadata.vendor_name	Hardcoded as JENKINS.
	metadata.event_timestamp	Constructed from year, month, day, time, and ampmfields.

Need more help? Get answers from Community members and Google SecOps professionals.