Splunk Custom Endpoints Part 2: Cleaning the Data

In Part 1 of this series we set up the basics in creating our custom endpoints. First, we set up our custom_endpoint.py and set up our Send class. We then set up our restmap.conf to map our Send class to the /send endpoint. Finally, we set up web.conf to allow POSTing to our custom endpoint of /send. Now that we have the basics out of the way, we’ll start testing our custom endpoint.

Again, as in the previous blog post, I’ve listed the subsequent screencast links below and included them throughout the post.

• Video 4: Setting up the Dashboard
• Video 5: Setting up the JavaScript
• Video 6: POST Request in JavaScript

POSTing

Now that we have everything set up, try going to: https://<hostname&gt;:<splunkd_port>/services/admin/users/<username>

In my case, <username> will be batman.

When we created our user, it only had the user role attached to it. Here you should see something like this:

Our custom endpoint that we set up is https://<hostname&gt;:<splunkd_port>/services/send – try going to that endpoint in your browser. If everything works as expected you should get back a “Success!” Message.

Now go back to https://<hostname&gt;:<splunkd_port>_port>/services/admin/users/batman and you should now see both a user and admin role attached:

We have successfully POSTed to our custom endpoint and modified the roles of a user in Splunk.

How did this work?

If you look back in the custom_endpoint.py file you will see that it has hardcoded values that simply POST the data, which results in an automatic role change. If you wanted to change your user back to the user role only instead of user and admin then you would need to change the following line in your custom_endpoint.py file:

to

Then go back to the /send custom endpoint and it will change the user role. Of course, this is only for testing purposes and wouldn’t be very efficient at changing our user’s roles. Next, we will be working to automate this process.

A Dose of Reality

At this point you may be wondering why we are creating a custom endpoint for another endpoint that already exists. As a real life example, this is probably overkill, since we could just open up /services/admin/users/<user> to allow us to POST to it directly. However, even going with that approach we should be mindful of any potential security concerns that may pose. Realistically, if a pre-existing endpoint doesn’t allow you to POST to it it’s probably worth taking a moment to consider why.

Creating Our Dashboard

Now that we know we can post to it, let’s create a basic form in the Splunk user interface (UI).

Next, we’ll create our dashboard. If you’re interested, check out the screencast for Setting up the dashboard here. In Splunk’s UI go into the Search and Reporting app, click ‘Dashboards’ in the top menu and then click ‘Create New Dashboard’. Fill out the form to create your new dashboard. You can call it whatever you like. I am calling mine “Custom Endpoint”.

Once you’ve created your new dashboard click on ‘Edit’ < ‘Edit Source’ and replace the Simple XML with the following:

We haven’t created our custom_endpoint.js file but we will do that here shortly. Save your changes. Your dashboard should look like this:

Set Up the JavaScript

Now we need to create our custom_endpoint.js file that we’re referencing in our dashboard and you can see the related screencast for that here. On the command line go to the Search app’s appserver/static folder. Create those folders if they do not already exist. Inside of the static folder create a new file called custom_endpoint.js. In custom_endpoint.js add the necessary code to post to our custom endpoint.

Add the following to the custom_endpoint.js file:

Now restart Splunkweb ($SPLUNKHOME/bin/splunk restart splunkweb) in order for the file to show up. Once you restart Splunkweb open up the console window in your browser and refresh the dashboard. You should see the console output of “This is working” in the console.

Now that it’s working, remove the console message and add the following inside the function (here’s the POST Request in JavaScript screencast for you too):

In order to see the changes run _bump in the URL like so:

Here is what we’re doing:

1. First, we listen for our submit button click event.
2. Next, we prevent the default button behavior (refreshing the page) with e.preventDefault()
3. Then create a new service variable, which will allow us to POST data to our custom endpoint. We then use the serialized array data from our form.
4. Create data variable that takes the serialized data of the form.
5. Finally, we are logging out the format of the data.

The serialized data you’re submitting will appear as the following in the console window:

We want to take that object and pass it to our custom endpoint. Next we will set up our service function. Place the following inside of the click function:

Here is what we are doing:

1. We are POSTing to our custom endpoint of send and passing the data variable that we defined earlier. The function takes two parameters – error and response.
2. Inside of the function we’ll check to see if an error is thrown and if one is we log it out, otherwise we log out the successful response from the server.
3. Empty out the postResponseBox, which is where our success message will go. This will prevent duplicate success messages from appearing on subsequent POSTs.
4. Reset the form, essentially clearing out all the fields with $(‘#userPOSTForm’)[0].reset();.
5. Finally, we remove the success message after 4 seconds.

Save the file.

Now that we have the service set up and ready to POST. It would be good if we can actually see how the data will look to our Python script. Therefore, we will be modifying our custom_endpoint.py file to simply take the data we send it and return it back to us. This will allow us to see the format the data is in, which will allow us to clean it up and modify it. We will do that in Part 3!