Automating Forensic Artifact Collection with Splunk and GRR

Recently I had the need for GRR to collect forensic artifacts when a Splunk alert was triggered. The point of this is to collect the forensics data when a incident ticket is generated to save IR staff time and eliminate redundant redundant tasks.

Example Scenario
When a pre-defined malicious event is seen, Splunk will send an email with event details to the ticketing system and IR folks will investigate. One of the first step in the example below is to acquire the files in question with GRR. To save time we will want to automate the collection of evidence.

AV does a horrible job of detecting malicious scripts like JS.Proslikefan.B (and anything malicious in general). However, with the help of WLS this is simple to detect and alert on. Splunk search:

`wlslogs` (EventID=4688 OR EventID=592) InternalName=wscript.exe BaseFileName!=wscript.exe

To briefly explain this alert, JS.Proslikefan maintains persistent by executing a 'random filename.lnk' file in the startup folder. The LNK file executes a randomly named copy of 'wscript.exe' in the appdata folder along with the malicious script. When a person logs on to an infected machine it would generate a WLS event like this (snippet):

BaseFileName="udpbat.exe" InternalName="wscript.exe" CommandLine="C:\Users\tupac\AppData\Roaming\avseda\udpbat.exe  C:\Users\tupac\AppData\Roaming\avseda\vnyqxluw.js"

Process Overview 
1. Splunk alert finds execution of 'wscript.exe' when BaseFileName is not 'wscript.exe'.
2. Splunk alert launches 'wrapper.py' which then launches 'grrRemoteGetFile.py'. 
3. 'grrRemoteGetFile.py' sends GRR an API request to acquire files in question.
4. Profit.


Splunk uses its own python version which doesn't have modules like 'requests'. Rather than installing modules into Splunk's python, we can just use a wrapper which will use the system default version of python. 

Creating the Splunk Alert
Run the search and when you're satisfied your search has minimal false positives, save it as an Alert.

When going through the alert wizard check the 'enable' box under Run a script and enter wrapper.py

wrapper.py (Mashed together from a few examples on the Splunk forums):

 #!/usr/bin/python   
 import gzip, os, sys, csv   
 from subprocess import call   
   
 python_executable = "/usr/bin/python"   
 real_script = "/opt/splunk/bin/scripts/grrRemoteGetFile.py"   
   
 for envvar in ("PYTHONPATH", "LD_LIBRARY_PATH"):   
  if envvar in os.environ:   
   del os.environ[envvar]   
   
 def openany(p):   
  if p.endswith(".gz"):   
   return gzip.open(p)   
  else:   
   return open(p)   
   
 results_file = sys.argv[8]  
   
 for row in csv.DictReader(openany(results_file)):   
  my_command = [ python_executable, real_script, row["host"], ]   
  call(my_command)   

The wrapper script does the following: 

• Remove environment path and LD_LIBRARY_PATH
• Opens the splunk search results (unzip's and reads csv for host value) .
• Execute 'grrRemoteGetFile.py' with the systems default python along with the hostname that triggered the alert.

Splunk will pass 9 variables to the script when it executes. Variable 8 contains the path to the gzip'd search results in csv format. The other variables are documented here.


grrRemoteGetFile.py 

 #!/usr/bin/python    
 import sys, json, urllib2, base64, requests   
 from requests.auth import HTTPBasicAuth   
   
 hostname = sys.argv[2]  
   
 grrserver = 'https://grrserver:8000'   
 username = 'Tupac'   
 password = 'isAlive'   
   
 base64string = base64.encodestring('%s:%s' % (username, password)).replace('\n', '')   
 authheader = "Basic %s" % base64string   
   
 index_response = requests.get(grrserver, auth=HTTPBasicAuth(username, password))   
 csrf_token = index_response.cookies.get("csrftoken")   
   
 headers = {   
  "Authorization": authheader,   
  "x-csrftoken": csrf_token,   
  "x-requested-with": "XMLHttpRequest"   
  }   
   
 data = {   
  "hostname": hostname,   
  "paths": ["%%users.appdata%%\Roaming\*\*.{js,exe}",   
            "%%users.appdata%%\Roaming\Microsoft\Windows\Start Menu\Programs\Startup\*.lnk"],    
  "pathtype": "OS"    
  }   
   
 response = requests.post(grrserver + "/api/clients/" + hostname + "/flows/remotegetfile",   
        headers=headers, data=json.dumps(data),   
        cookies=cookies, auth=HTTPBasicAuth(username, password))   

'grrRemoteGetFile.py' will start a FileFinder flow on the 'hostname' variable passed to it by the 'wrapper.py' script. When IR staff is able to review the ticket, the files will be available in GRR to download and review.

In Summary...
This is just a basic example to demonstrate how all the pieces fit together. There are some really cool things you can do with these tools to automate stuff. Some things I've been playing with:

Automatically launch Incident Response Collector ($MFT, Registry, Browser History, etc.) and full memory image when a known bad MD5 or static indicator is seen in Splunk.

Utilize WLS's hash tracking to automatically submit new binaries to internal malware analysis tools. Splunk alert would be:

`wlslogs` (EventID=4688 OR EventID=592) NewHash=True

In the wrapper.py, you would add row["NewProcessName"] and pass it to grrRemoteGetFile.py to download (instead of the static path in the example above).

Get any executable downloaded from the internet and send it to internal malware analysis tools.

`wlslogs` (EventID=4688 OR EventID=592) Zone=3

Get any compressed file attachment opened from Outlook email and send to internal malware analysis tools.

`wlslogs` (EventID=4688 OR EventID=592) CreatorProcessName=OUTLOOK BaseFileName=winzip*

If you have any examples/suggestions on automation with GRR and WLS/Splunk, share them on the GRR user group, I'm really interested to hear what other folks have done.