Collect Google Kubernetes Engine logs

This document describes how you can collect Google Kubernetes Engine logs by setting up a Google SecOps feed and how log fields map to Google SecOps Unified Data Model (UDM) fields. This document also lists the supported log types and event types for Google Kubernetes Engine.

For more information, see Data ingestion to Google SecOps .

A typical deployment consists of Google Kubernetes Engine and the Google SecOps feed configured to send logs to Google SecOps. Each customer deployment might differ and might be more complex.

The deployment contains the following components:

• Google Cloud: The Google Cloud services and products from which you collect logs.

• Google Kubernetes Engine. The Google Kubernetes Engine platform from which you collect logs.

• Google SecOps. Google SecOps retains and analyzes the logs from Google Kubernetes Engine.

An ingestion label identifies the parser which normalizes raw log data to structured UDM format. The information in this document applies to the Google Kubernetes Engine parser with the following ingestion label: KUBERNETES_NODE

Before you begin

• Ensure that you have a Google Administrator account.

• Verify whether you have the required permissions to perform the following tasks:

  • Create or access a Google Cloud project.
  • Enable the Google Kubernetes Engine cluster. For more information, see Deploy an app to a GKE cluster

• Ensure that all systems in the deployment architecture are configured in the UTC time zone.

• Verify the log types that the Google SecOps parser supports. For information about supported Google Kubernetes Engine resource types, see Supported Google Kubernetes Engine resource types .

Configure Google Cloud for ingestion

To ingest KUBERNETES_NODE logs to Google SecOps, follow the steps on the Ingest Google Cloud data to Google SecOps page.

If you encounter issues when you ingest KUBERNETES_NODE logs, contact Google Security Operations support .

If you encounter issues when you create feeds, contact Google Security Operations support .

Supported Google Kubernetes Engine resource types

The following table lists the resources types that the Google Kubernetes Engine parser supports:

Supported Google Kubernetes Engine log formats

The Google Kubernetes Engine parser supports logs in JSON format.

Supported Google Kubernetes Engine sample logs

• JSON:

   {
    "protoPayload": {
      "@type": "type.googleapis.com/google.cloud.audit.AuditLog",
      "authenticationInfo": {
        "principalEmail": "user@dummy.com"
      },
      "requestMetadata": {
        "callerIp": "198.51.100.1",
        "callerSuppliedUserAgent": "google-cloud-sdk gcloud/415.0.0 command/gcloud.beta.container.clusters.create invocation-id/077f3f1330ae433a8c7d13161bbcc643 environment/None environment-version/None interactive/True from-script/False python/3.9.12 term/ (Windows NT 10.0.22621),gzip(gfe)",
        "requestAttributes": {
          "time": "2023-01-25T06:43:46.203881095Z",
          "auth": {}
        },
        "destinationAttributes": {}
      },
      "serviceName": "container.googleapis.com",
      "methodName": "google.container.v1beta1.ClusterManager.CreateCluster",
      "authorizationInfo": [
        {
          "permission": "container.clusters.create",
          "granted": true,
          "resourceAttributes": {}
        }
      ],
      "resourceName": "projects/sccfindings29396/zones/us-east1-d/clusters/gke",
      "request": {
        "cluster": {
          "subnetwork": "projects/sccfindings29396/regions/us-east1/subnetworks/default",
          "name": "gke",
          "networkConfig": {},
          "defaultMaxPodsConstraint": {},
          "monitoringConfig": {
            "componentConfig": {
              "enableComponents": [
                "SYSTEM_COMPONENTS"
              ]
            }
          },
          "shieldedNodes": {},
          "loggingConfig": {
            "componentConfig": {
              "enableComponents": [
                "SYSTEM_COMPONENTS",
                "WORKLOADS"
              ]
            }
          },
          "initialClusterVersion": "1.24.8-gke.2000",
          "locations": [
            "us-east1-d"
          ],
          "nodePools": [
            {
              "maxPodsConstraint": {},
              "management": {
                "autoRepair": true,
                "autoUpgrade": true
              },
              "config": {
                "diskType": "pd-standard",
                "machineType": "e2-medium",
                "oauthScopes": [
                  "https://www.googleapis.com/auth/devstorage.read_only",
                  "https://www.googleapis.com/auth/logging.write",
                  "https://www.googleapis.com/auth/monitoring",
                  "https://www.googleapis.com/auth/service.management.readonly",
                  "https://www.googleapis.com/auth/servicecontrol",
                  "https://www.googleapis.com/auth/trace.append"
                ],
                "metadata": {
                  "disable-legacy-endpoints": "true"
                },
                "imageType": "COS_CONTAINERD",
                "diskSizeGb": 30
              },
              "upgradeSettings": {
                "maxSurge": 1
              },
              "name": "default-pool",
              "initialNodeCount": 1
            }
          ],
          "network": "projects/sccfindings29396/global/networks/default",
          "releaseChannel": {
            "channel": "REGULAR"
          },
          "ipAllocationPolicy": {
            "useIpAliases": true
          },
          "addonsConfig": {
            "gcePersistentDiskCsiDriverConfig": {
              "enabled": true
            },
            "httpLoadBalancing": {},
            "networkPolicyConfig": {
              "disabled": true
            },
            "horizontalPodAutoscaling": {}
          },
          "masterAuthorizedNetworksConfig": {}
        },
        "@type": "type.googleapis.com/google.container.v1alpha1.CreateClusterRequest",
        "parent": "projects/sccfindings29396/locations/us-east1-d"
      },
      "response": {
        "selfLink": "https://container.googleapis.com/v1alpha1/projects/165675416793/zones/us-east1-d/operations/operation-1674629028274-a0ec9b53",
        "status": "RUNNING",
        "operationType": "CREATE_CLUSTER",
        "targetLink": "https://container.googleapis.com/v1alpha1/projects/165675416793/zones/us-east1-d/clusters/gke",
        "@type": "type.googleapis.com/google.container.v1alpha1.Operation",
        "startTime": "2023-01-25T06:43:48.274343825Z",
        "name": "operation-1674629028274-a0ec9b53"
      },
      "resourceLocation": {
        "currentLocations": [
          "us-east1-d"
        ]
      },
      "policyViolationInfo": {
        "orgPolicyViolationInfo": {}
      }
    },
    "insertId": "6dqtvadkv96",
    "resource": {
      "type": "gke_cluster",
      "labels": {
        "project_id": "sccfindings29396",
        "cluster_name": "dummy_customer_name",
        "location": "us-east1-d"
      }
    },
    "timestamp": "2023-01-25T06:43:48.355321416Z",
    "severity": "NOTICE",
    "logName": "projects/sccfindings29396/logs/cloudaudit.googleapis.com%2Factivity",
    "operation": {
      "id": "operation-1674629028274-a0ec9b53",
      "producer": "container.googleapis.com",
      "first": true
    },
    "receiveTimestamp": "2023-01-25T06:43:48.837791874Z"
  } 
  

Field mapping reference

The following sections explain how the Google Security Operations parser maps Google Kubernetes Engine log fields to Google Security Operations Unified Data Model (UDM) fields.

Field mapping reference: KUBERNETES_NODE event identifier to UDM event type

The following table lists the KUBERNETES_NODE event identifiers and their corresponding UDM event types. The mapping to an UDM event type is based on the protopayload.methodname log field, which is considered as the event identifier.

The following table lists the KUBERNETES_NODE event identifiers and their corresponding UDM event types for mappings that aren't based on the protopayload.methodname log field.

Field mapping reference: KUBERNETES_NODE Common Fields

The following table lists the common fields of the KUBERNETES_NODE log type and their corresponding UDM fields.

Field mapping reference: KUBERNETES_NODE log fields to UDM fields

The following table lists the log fields of the KUBERNETES_NODE log type and their corresponding UDM fields.

What's next

• Data ingestion to Google SecOps

Need more help? Get answers from Community members and Google SecOps professionals.