Splunking with Sysmon Series Part 1: The Setup

Sysmon (System Monitor) is a system monitoring and logging tool that is a part of the Windows Sysinternals Suite. It generates much more detailed and expansive logs than the default Windows logs, and it provides a great, free alternative to many of the Endpoint Detection and Response (EDR) solutions available. If you are curious about the potential of Sysmon, I did a short talk showing some of its capabilities.

This blog series will help prepare you to get Sysmon up and running, deploy it in your environment, and forward the event logs to your Splunk indexers.

Configuration and installation

Sysmon has a simple installation, although there are a few decisions you will need to make before you prepare to install it in your environment. You’ll need to decide on an initial configuration, deployment method, and forwarding mechanism.

The first major choice is the initial configuration of your Sysmon build. It is possible to make your own configuration, but it takes a good understanding of what logs you want to generate and can create a decent amount of unpredictable logs.

There are two popular configurations that are easy to deploy and have done a lot of the initial legwork, and both are great choices to start with.

SwiftOnSecurity

SwiftOnSecurity has a simplified one file configuration that is great to start out with to see what is possible with Sysmon. You can download it on GitHub and easily install Sysmon with it to be up and running in a few minutes.

The advantages of this configuration are that it is simple to modify, roll out changes, and keep up to date. The main disadvantage is that, as you tune your configuration to your environment, the one file deployment makes it difficult to keep track of those changes. So for a very simple, one-shot configuration it works great, but the second popular configuration is much better suited to a well-tuned environment.

If you decide to use SwiftOnSecurity’s configuration, I would recommend the following configuration additions, as the default configuration will not exclude Splunk processes and can create a large amount of events:

To the section titled: <ProcessCreate onmatch="exclude">.

Modular Sysmon

Modular Sysmon, by Olaf Hartong, is more complex than Swift’s, but is not overwhelming. An important aspect of Modular Sysmon is that many of the rules are mapped to the Mitre ATT&CK framework. So each event will have a RuleName field showing the ATT&CK mapping like below:

What it does well is that all of the configurations are based in folders for each Event Id with separate files based on including or excluding events from the configuration. The configuration is available on GitHub and requires a few more steps to install.

Since it is not based on a single XML file, there are a couple of PowerShell scripts required to build the configuration file each time you update the configuration. From inside the Modular Sysmon directory, run the following commands to build the xml configuration:

The advantages to Modular Sysmon is that you can create a file for each Event ID you want to modify the configuration file and have a copy of it outside the main location. That way, when the base configuration is updated, it is easy to put those files back in after the upgrade. Also, since all the changes you make are in their own files, it is easy to keep track of what modifications were in the default configuration and what were created by you.

The Splunk changes I recommend for SwiftOnSecurity’s configuration are included by default in Modular Sysmon.

Installation

After choosing your Sysmon configuration, the installation on a single machine is easy.

Download Sysmon from Sysinternals, unzip the folder, and copy the configuration file into the folder. As an administrator, open up a command prompt or PowerShell window, change into the Sysmon directory, and execute the following command:

Deployment

The next choice is how you intend to deploy Sysmon to the endpoints in your environment.

When using Splunk, I highly recommend having a test bed of select, in-use computers in a variety of departments. Deploy Sysmon to that test group first for a week or two in order to generate a good amount of logs and create an initial tuning configuration to deploy company wide. Sysmon can generate a large amount of events when not tuned, and it could overwhelm a license that is too small. I will discuss how I approach tuning in Part 2 of this blog.

The simplest way to deploy Sysmon is with a Group Policy enforced Scheduled Task to run a PowerShell or Batch script. There are a few considerations to take into account with this approach, as if it is done poorly it can result in information disclosure that can potentially allow a malicious actor to circumvent your Sysmon logging.

I have a PowerShell script on GitHub that can serve as an example deployment script. The script will check to see if Sysmon is installed and which version is running on the endpoint. If the version matches what is expected in the script, it will update the configuration on the system. If not, it will uninstall the current version and install the new version with the current configuration file provided.

One way to deploy the script and configuration file is to have a scheduled task run the script from a shared folder with the configuration file in that same folder. Have the task run as the Builtin System account (NT Authority\System) or another user created for this purpose and limit the shared folder access to only computer accounts (or said user), and the privileged users that will be modifying the script and configuration. This will prevent users from being able to see and circumvent the Sysmon logs and will maintain security on the configuration files. Running this as a regularly scheduled task (as often as you would make changes to the configuration) will allow the endpoints to maintain the updated version without much maintenance. An example of this is below:

Splunk Configuration

There are two main ways to get your Sysmon logs to your Splunk indexers. I would recommend using the Splunk Universal Forwarder, but if your environment is unsuitable for it, Windows Event Forwarding also works.

Deploying Splunk Universal Forwarders (UF) to all endpoints and using that to ingest Sysmon logs to your Splunk Indexers is the preferred method. This option allows for Splunk to ingest more than just Windows Logs from the endpoints and offers more control over what is sent. To send your Sysmon logs to the Forwarder configuration, simply add the below configuration, plus any modifications you may want to make, to your input.conf file:

If the Splunk UF does not work well in your environment, Windows Event Forwarding (WEF) can also be used to send the Symon logs from endpoints to a centralized server for collecting logs. This can be used for just Sysmon logs or for all Windows logs, if you prefer. This is set up through Group Policy and allows you to specify which Event IDs you want to send. You can find the Microsoft documentation for Windows Event Forwarding here.

Once WEF is set up, you would have to use a Splunk Universal Forwarder on the WEF collectors to send the Forwarded logs to Splunk.

Lastly, add the Splunk Add-On for Microsoft Sysmon to your Splunk deployment: https://splunkbase.splunk.com/app/1914/. By default, the events will have a sourcetype of xmlwineventlog and source of WinEventLog:Microsoft-Windows-Sysmon/Operational. Below is a list of EventCodes and their meanings:

ID Event

1 Process Create

2 File creation time

3 Network connection detected

4 Sysmon service state change (cannot be filtered)

5 Process terminated

6 Driver Loaded

7 Image loaded

8 CreateRemoteThread detected

9 RawAccessRead detected

10 Process accessed

11 File created

12 Registry object added or deleted

13 Registry value set

14 Registry object renamed

15 File stream created

16 Sysmon configuration change (cannot be filtered)

17 Named pipe created

18 Named pipe connected

19 WMI filter

20 WMI consumer

21 WMI consumer filter

22 DNS query

23 File Delete

Conclusion

Hopefully, this blog post helped you learn a little about Sysmon and settle any fears you may have with deploying it. It is an incredibly powerful tool, but one that requires some work to get ready for a full production deployment.

If you have any more questions feel free to reach out to me on Twitter at @DustyMMiller. Stay tuned for part 2 of my Sysmon blog where I will go through my process of tuning a Sysmon configuration with Splunk.