Unexpected Child Shell

This document describes a threat finding type in Security Command Center. Threat findings are generated by threat detectors when they detect a potential threat in your cloud resources. For a full list of available threat findings, see Threat findings index .

Overview

Container Threat Detection observed a process that unexpectedly spawned a child shell process. This event might indicate that an attacker is trying to abuse shell commands and scripts.

How to respond

To respond to this finding, do the following:

Step 1: Review finding details

1. Open an Unexpected Child Shell finding as directed in Reviewing findings . The details panel for the finding opens to the Summarytab.

2. On the Summarytab, review the information in the following sections:

   • What was detected, especially the following fields:
     • Parent process: the process that unexpectedly created the child shell process.
     • Child process: the child shell process.
     • Arguments: the arguments provided to the child shell process binary.
     • Environment variables: the environment variables of the child shell process binary.
     • Containers: the name of the container.
     • Containers URI: the image URI of the container.
     • Kubernetes pods: the Pod name and namespace.
   • Affected resource, especially the following fields:
     • Resource display name: the name of the affected resource.
     • Resource full name: the full resource name of the cluster. The full resource name includes the following information:
       • The project that contains the cluster: projects/ PROJECT_ID
       • The location in which the cluster is located: either zone/ ZONE or locations/ LOCATION
       • The name of the cluster: projects/ CLUSTER_NAME
   • Related links, especially the following fields:
     • VirusTotal indicator: link to the VirusTotal analysis page.

3. Click the JSONtab and note the following fields:

+ processes : an array containing all processes related to the finding. This array includes the child shell process and the parent process. + resource : + project_display_name : The name of the project that contains the assets. + sourceProperties : + VM_Instance_Name : the name of the GKE node where the Pod executed.

Step 2: Review cluster and node

1. In the Google Cloud console, go to the Kubernetes clusterspage.

   Go to Kubernetes clusters

2. On the Google Cloud console toolbar, select the project listed in resource.project_display_name , if necessary.

3. Select the cluster listed in resource.name . Note any metadata about the cluster and its owner.

4. Click the Nodestab. Select the node listed in VM_Instance_Name .

5. Click the Detailstab and note the container.googleapis.com/instance_id annotation.

Step 3: Review Pod

1. In the Google Cloud console, go to the Kubernetes Workloadspage.

   Go to Kubernetes Workloads

2. On the Google Cloud console toolbar, select the project that you noted in the Resource full name( resource.name ) of the cluster in the finding summary, if necessary.

3. Click Show system workloads.

4. Filter the list of workloads by the cluster name that you noted in Resource full name( resource.name ) of the finding summary and, if necessary, the pod Namespace( kubernetes.pods.ns ) that you noted.

5. Click the workload name that matches the value of the VM_Instance_Name property that you noted in the finding JSON earlier. The Pod detailspage opens.

6. On the Pod detailspage, note any information about the Pod that might help you resolve the threat.

Step 4: Check logs

1. In the Google Cloud console, go to Logs Explorer.

   Go to Logs Explorer

2. On the Google Cloud console toolbar, select the project listed in resource.project_display_name .

3. Set Select time rangeto the period of interest.

4. On the page that loads, do the following:

   1. Find Pod logs for Pod_Name by using the following filter:
      • resource.type="k8s_container"
      • resource.labels.project_id=" resource.project_display_name "
      • resource.labels.location=" location "
      • resource.labels.cluster_name=" cluster_name "
      • resource.labels.namespace_name=" Pod_Namespace "
      • resource.labels.pod_name=" Pod_Name "
   2. Find cluster audit logs by using the following filter:
      • logName="projects/ resource.project_display_name /logs/cloudaudit.googleapis.com%2Factivity"
      • resource.type="k8s_cluster"
      • resource.labels.project_id=" resource.project_display_name "
      • resource.labels.location=" location "
      • resource.labels.cluster_name=" cluster_name "
      • Pod_Name
   3. Find GKE node console logs by using the following filter:
      • resource.type="gce_instance"
      • resource.labels.instance_id=" instance_id "

Step 5: Investigate the running container

If the container is still running, it might be possible to investigate the container environment directly.

1. Go to the Google Cloud console.

   Open Google Cloud console

2. On the Google Cloud console toolbar, select the project listed in resource.project_display_name .

3. Click Activate Cloud Shell.

4. Obtain GKE credentials for your cluster by running the following commands.

   For zonal clusters, run the following:

      
   gcloud  
   container  
   clusters  
   get-credentials  
    cluster_name 
     
   --zone  
    location 
     
   --project  
    resource.project_display_name 
    
   

   For regional clusters, run the following:

      
   gcloud  
   container  
   clusters  
   get-credentials  
    cluster_name 
     
   --region  
    location 
     
   --project  
    resource.project_display_name 
    
   

5. To launch a shell within the container environment, run the following:

      
   kubectl  
    exec 
     
   --namespace = 
    Pod_Namespace 
     
   -ti  
    Pod_Name 
     
   -c  
    Container_Name 
     
   --  
   /bin/sh 
   

   This command requires the container to have a shell installed at /bin/sh .

   To view all processes running in the container, run the following command in the container shell:

      
   ps  
   axjf 
   

   This command requires the container to have /bin/ps installed.

Step 6: Research attack and response methods

1. Review MITRE ATT&CK framework entries for this finding type: Command and Scripting Interpreter: Unix Shell .
2. Check the SHA-256 hash value for the binary flagged as malicious on VirusTotal by clicking the link in VirusTotal indicator . VirusTotal is an Alphabet-owned service that provides context on potentially malicious files, URLs, domains, and IP addresses.
3. To develop a response plan, combine your investigation results with the MITRE research and VirusTotal analysis.

Step 7: Implement your response

The following response plan might be appropriate for this finding, but might also impact operations. Carefully evaluate the information you gather in your investigation to determine the best way to resolve findings.

• Contact the owner of the project with the compromised container.
• Stop or delete the compromised container and replace it with a new container .

What's next

• Learn how to work with threat findings in Security Command Center .
• Refer to the Threat findings index .
• Learn how to review a finding through the Google Cloud console.
• Learn about the services that generate threat findings .