Splunking with Sysmon Series Part 2: Tuning
This Splunk tutorial is a continuation of my previous Sysmon article, Splunking with Sysmon Part 1: The Setup. In part 1, I went over the basics of getting Sysmon installed in your environment and forwarding to Splunk.
This second part will help you to take your initial configuration, either Modular Sysmon or SwiftonSecurity’s configuration, and tune it to your environment. This initial tuning can help reduce a significant amount of the events Sysmon generates.
I will be using the full Splunk Boss of the Soc Version 2 Dataset to demonstrate the potential of tuning your Sysmon Splunk logs. It was used for the Splunk Boss of the Soc events in 2018 and 2019. I am using this dataset because it was an untuned Sysmon configuration that generated over one million Sysmon events.
The searches I use are available on my Github, and I will reference a few of them throughout this blog.
The tuning I am going over in this blog is to reduce the amount of every day expected noise, not to fine tune so that only important events are logged. To start, we need to get a good handle on which EventCodes are generating the majority of alerts. My first search to start is:
My example from botsv2 is below:
Looking at the EventCodes, EventCode 1 (process creation) and 5 (process termination) are generating the most events and are what I would look at tuning first. For EventCode 1, I would run the following searches and find events that are worth evaluating for tuning.
The screenshot below shows a similar search, and from it, I would consider tuning all three ParentImages seen. The most important part of the process is doing a risk analysis on whatever you are considering tuning. For example, if I excluded “C:\Program Files\SplunkUniversalForwarder\bin\splunkd.exe” from my Sysmon configuration, I would generate over 400,000 less events, but also lose insight into any processes that may spawn from that process. Only exclude processes where you determine that the benefits outweigh the risks. The same is said for the initial configuration, as they tune out a lot of noise as well.
In my example I will tune out the “C:\Program Files\SplunkUniversalForwarder\bin\splunkd.exe”, “C:\Windows\System32\csrss.exe”, and “C:\Program Files (x86)\Symantec\Symantec Endpoint Protection\12.1.7266.6800.105\Bin\ccSvcHst.exe” processes.
Sysmon Configuration Modifications
Sysmon configurations follow a basic XML format that generally follows below:
It starts with a RuleGroup that has an “or” relation, and then it is divided into each EventType. The EventTypes either are onmatch=”excluded” or onmatch=”include”. When the onmatch=”exclude”, it will log everything from that EventType except for what is explicitly excluded. When the onmatch=”include”, it will only include logs from the EventType that match rules and will not log anything else. A misconfiguration with “include” rules will cause missing events. The EventTypes are matched below to their EventCode.
The rules will be applied with any of the following conditions depending on how you want the rule applied. Some of the more common conditions are:
Example config changes
The two main Sysmon configurations are very different in how you modify the rules. Below are examples from each to model how to write and apply the configuration changes. I am using the “image”, “is”, and “contains” conditions in the examples. One caution to take into consideration with the image and contains is that they can be exploited.
Below, I exclude splunkd.exe based on “image”, which will exclude any process that is named “splunkd.exe”, so I could run “C:\Users\Dusty\Desktop\splunkd.exe” and it would not log. Same thing with contains, as long as the process contains the exclusion string it will not log.
With the SwiftonSecurity configuration, the XML is already completely built. You will just need to add the rules that you want to apply to the configuration. To apply the Process Creation (EventCode 1) rules from my Lookup Submit above, I would add the following lines to the configuration xml file:
Modular Sysmon rules are added to their own xml file in folders for each EventCode. For the exclusions noted above, I would create a new file named “exclude_hurricane_labs.xml” in the folder “1_process_creation” with the following contents:
If the EventType is different, simply change the <ProcessCreate onmatch=”exclude”> to the EventType you want to exclude and save it in the proper folder. This allows you to easily keep track of any changes made and reduces the impact of a typo. To add these rules to the configuration, you will need to re-run the powershell script below to generate the new configuration xml.
The amount of tuning you decide to do has to be carefully considered from a risk management mindset. By adopting the initial configurations, you are already reducing your visibility some, so this is just a step to take to gear the configuration to your specific environment. This larger scale tuning can help to reduce your license usage and increase the speed of searches.
In the next few parts of this series, I will go through specific examples of risks and detections possible using Sysmon.
About Hurricane Labs
Hurricane Labs is a dynamic Managed Services Provider that unlocks the potential of Splunk and security for diverse enterprises across the United States. With a dedicated, Splunk-focused team and an emphasis on humanity and collaboration, we provide the skills, resources, and results to help make our customers’ lives easier.
For more information, visit www.hurricanelabs.com and follow us on Twitter @hurricanelabs.