How to Build Automatic Identity Lookup in Splunk

One of the critical components of configuring the Splunk Enterprise Security Suite involves building lookup tables for identities. This post will provide an overview of the steps required to get started with automatic identity lookup building using LDAP or Active Directory. This is by no means a comprehensive document of all possible configurations and options, but it should provide enough information to get you going in the right direction.

More often than not, you will already have much of this information already existing in another directory, such as Active Directory or LDAP. Fortunately, Splunk can be configured to automatically pull data from these sources and build the appropriate lookup tables for Enterprise Security.

Step 1: Configure the Splunk Supporting Add-on for Active Directory (SA-ldapsearch) to query your LDAP/Active Directory environment

Don’t be fooled by the name – just because this app is called the Splunk Support for Active Directory doesn’t mean that Active Directory is required. This app can be used to query any LDAP server. Download and install the Splunk Support for Active Directory app.

The following instructions were written for version 1.x of the app, which requires Java to function. The configuration should be similar on version 2.x, which no longer has the Java requirement.

As part of the app setup, configure ldap.conf for the domains you want to query:

Sample:

Step 2: Use ldapsearch to generate the lookup table

These instructions are based on the steps found in the Splunk Enterprise Security Installation manual.

Substitute the domain (configured in ldap.conf) in the search as follows:

If referencing the Splunk documentation, leave the outputlookup command off since this lookup doesn’t exist yet.

If you are using an LDAP directory that is not Active Directory, this can still be queried using a similar method, but the table will be using different values depending on the LDAP structure.

Step 3: Create the lookup table file

Lookup table

The base lookup table file can be created by uploading a skeleton file through SplunkWeb. You will want to make sure that the destination app is the SplunkEnterpriseSecuritySuite and that the sharing permissions are set to Global (object should appear in all Apps).

​Creating the lookup table file (uploading a skeleton file)

The header of the CSV should be as follows:

Setting the appropriate permissions

File saved successfully

Step 4: Create the lookup table definition

The second link is used for configuring lookup definitions

The lookup table definition will be used in a saved search to generate the lookup table. The destination app for this lookup table definition should again be the SplunkEnterpriseSecuritySuite. The name can be whatever you would like the lookup definition to be called (ad_identities generally works fine for Active Directory). The type should be file-based, and the lookup file should be set to the CSV you uploaded in step 3.

Lookup definition created, permissions set to private

You will want to make sure that the sharing permissions are set to Global on this as well.

Correct permissions displayed 

Step 5: Create the lookup table and verify that it is updated on the filesystem

Run the same search as Step 2, but add the | outputlookup <lookup_name> command to the search. This will output the results to the lookup table specified.

Splunk search outputting lookup table

Verify that this is functioning properly by checking the file on the filesystem, which should be $SPLUNK_HOME/etc/apps/SplunkEnterpriseSecuritySuite/lookups if you have followed the steps up to this point.

Step 6: save the search as a report

This is assuming that the lookup table is being generated properly. Also, this can be scheduled at a later point in time to automatically update the lookup table.

Step 7: Configure Identities in ES: Navigate to App: Enterprise Security -> Configuration -> Data Enrichment -> Identity Management

Identity Management Configuration

Create a new identity, being careful to identify the type as “identity” and the source to the lookup definition you created earlier, which will be lookup://<lookup_name>. This should be enabled by default once it’s created, but verify that this is the case in the web interface.

Adding a new identities to the identity manager

Step 8: create the merged identity file

Before you create the merged identity file, wait approximately 5 minutes for Splunk to automatically detect the change in the identity configuration.The following search will show you the status: index=_internal source=*python_modular_input.log *identit*.

Step 9: Verify that the identities_expanded.csv file was updated on the filesystem of the search head

This should be in $SPLUNK_HOME/etc/apps/SA-IdentityManagement/lookups by default.

Current modification date/time on identities_expanded.csv

Step 10: Schedule the report you created earlier to run on a semi-regular basis

This is being done assuming all is working as expected. Also, as a final note, don’t forget to disable the sample identities that are enabled by default Enterprise Security.

Original Post Date: March 19, 2015 by Tom Kopchak. Updated: March 8, 2016.