Splunk Tutorial: CRUDing a KV Store in Splunk Using Python
In a previous blog series, I covered how to create and CRUD a KV Store using Splunk’s SPL (Search Processing Language). Feel free to check out the various pieces and parts of that tutorial here: Creating and CRUDing a KV Store in Splunk: Part 1 and Part 2.
In this blog post I’m going to cover how to do the same thing using Python. The blog post is available, as well as the subsequent screencasts. So, feel free to read, watch or do both.
In order to follow along with this you should download the Splunk Python SDK.
Export the PYTHONPATH
You can place the Splunk SDK folder where you want, but you will need to add the folder to your Python path in order to run the examples:
Using .splunkrc
In this example, I’m going to use a .splunkrc file to store my credentials. The .splunkrc file is a handy way for us to store our credentials when we connect to Splunk through our Python script.
You don’t have to use a .splunkrc file, but its easier than having to write this every time we want to execute a file, see below:
Example .splunkrc
Where to put .splunkrc
The location of the .splunkrc file will depend on whether or not you’re following along using Windows.
On Windows, you will put the .splunkrc in C:\Users\currenusername\.splunkrc
If you are on Linux or OSX place it in ~/.splunkrc
How does this work?
When we run our Python file, Splunk is going to check and see if user credentials have been passed into the command line. If not, it will then check if a .splunkrc file exists.
Inside of your Splunk SDK folder, there is an examples folder. This is where we will add our task_collection.py and add the following:
Importing connect will allow us to actually connect to Splunk. We will pass our credentials to it shortly. Next, we will try to import parse, which will be used to pull out our credentials from command line arguments or the .splunkrc file. If it cannot be imported that means you did not successfully export the SDK to your PYTHONPATH, as described above.
We then define the main() function where we will set up an opts variable from which we will pull out our user credentials. We will also set the owner to nobody and the app context to search. We will then connect to Splunk using our opts.kwargs.
Next, in the main() function, we will set up our collection name and set the collection using service.kvstore.
We will also check if the collection exists and if it does not, then we will create it:
You can then read the data from KV Store collection, using the query() function:
Finally, you will want to run the following, in order to execute the main() function, when you run the Python script from the command line:
Once you’ve added everything, save the file.
Create
To create, or insert, new data in your collection add the following to the Python script inside of the main() function above where we are printing out the collection data using the query() function:
All we are doing is inserting JSON to add a new value to KV Store.
Save the script and then run it on the command line. You should get back something like this:
Read
As described above, we can use query() to read the data from our KV Store. In our example we are specifically running this (see below) to print out the results:
Update
Currently, we don’t have an update in our file. Go ahead and comment out the collection.data.insert that currently exists and then add:
The key to updating is to first provide a string version of the _key you want to update and then as the second parameter include all the fields you want to update from your KV Store.
Save your file and rerun it, and you should see the updated values in the output.
Delete
We can either delete a specific row based on the _key:
Or we can delete the entire collection:
So, what can we do with this information? We could CRUD our KV Store from outside of Splunk using the Python SDK, or we could create a custom REST endpoint, modify our data and the CRUD the collection in some way.
In Closing
With all of this information, you should have a pretty solid understand of how to create a new KV store collection, as well as a lookup definition that allows us to communicate with our collection through the Splunk query language. We also covered how to CRUD our KV store collection through the Splunk query language as well as doing the same through Python. If you have any questions, feel free to leave them below in the comments section.
About Hurricane Labs
Hurricane Labs is a dynamic Managed Services Provider that unlocks the potential of Splunk and security for diverse enterprises across the United States. With a dedicated, Splunk-focused team and an emphasis on humanity and collaboration, we provide the skills, resources, and results to help make our customers’ lives easier.
For more information, visit www.hurricanelabs.com and follow us on Twitter @hurricanelabs.
