Your Splunk Guide for Smooth Sailing with CEF Field Extractions
One of the more common log formats you’ll run into when importing data into Splunk is the ArcSight Common Event Format (CEF). A unique feature of CEF is its ability to support custom extensions, which allows for vendor flexibility when looking to log data that is otherwise not handled by a defined field in CEF. This flexibility, however, does require some additional Splunk configuration (or an addition to a Splunk app) in order for the field names and values to match up properly.
What it looks like in Splunk
For this example, I’ll be using Check Point firewall logs, which I’ll send in CEF format via the Check Point Log Exporter since it’s a data source available in our lab environment. That said, this approach will work for most products that log in CEF format.
When looking at these logs, you’ll see a few fields that begin with “cs”:
You should immediately recognize a relationship between these two fields: the cs2Label field is the name of the values of the field cs2. In other words, the field=value pair in Splunk should ultimately end up as cs2Label=cs2.
Assuming you’re already using a Splunk app–and these fields aren’t already created–you’ll want to create a local props/transforms configuration to handle these field extractions. These are search-time operations, so the configuration only needs to exist on a search head.
The following are two example .conf files I’ll typically use for this:
Props:
Transforms:
With these configurations in place, either do a debug/refresh or restart Splunk on your search head. Then, you’ll see fields for Peer_Gateway and Rule_Name exist in the search results:
Depending on your data and how it’s formatted, it’s possible you may need to slightly adjust the REGEX in the transforms.conf file, but this should be a good starting point.
Next, you’ll proceed to add any additional knowledge objects to your data source. Since these field extractions leverage the REPORT-<your config name> method, you can continue to add additional knowledge objects using other methods available in the sequence of search-time operations, such as field aliases and calculated fields.
Conclusion
Hopefully this approach makes working with CEF formatted logs in Splunk easier. I’ve used this process many times myself, and writing this makes it easier for me (and you) to find it later. Happy Splunking!
About Hurricane Labs
Hurricane Labs is a dynamic Managed Services Provider that unlocks the potential of Splunk and security for diverse enterprises across the United States. With a dedicated, Splunk-focused team and an emphasis on humanity and collaboration, we provide the skills, resources, and results to help make our customers’ lives easier.
For more information, visit www.hurricanelabs.com and follow us on Twitter @hurricanelabs.
