Oh No! My JSON Keys and Values are Separated! How Can I Extract Them For My Searches?

Splunk will be able to parse the fields timestamp, level, message, user, user{}.id, user{}.username, and ip_address with their corresponding values. So, you would be able to run the search index=donuts | table timestamp level message user{}.username ip_address and get the following results:

Easy, right? But what happens when the keys and values are not on the same line in the log? If the JSON log is formatted with a nested list of keys and values, then Splunk will extract it like so:

The fields will be moreInformation{}.key and moreInformation{}.value, not DonutId, DonutType, or DonutToppings. In a table this might be enough for your needs, but you may want to run searches that coordinate the keys to their values more explicitly. 

We get this automatically in Splunk, where the keys and values that we want are separated and not explicitly paired with each other.

And our goal is for these key/value pairs to be explicitly set for the searches we want to run: DonutId = 1234567

Our solution is to use a combination of the mvfind and mvindex eval functions: https://docs.splunk.com/Documentation/SCS/current/SearchReference/MultivalueEvalFunctions The mvfind function will help us identify which key we want to use for the field name, while the mvindex function will associate the value to the key we are using for a field. What happens is that new meaningful fields are created that you can now use in your search.

The search would look like this:

You would do this for each field that you want to use from the nested JSON. Find the key, then associate the corresponding value using the index location within the JSON. You can do this within the search or use the eval commands in a Calculated Field to reuse the fields easily in other searches.