How to Build Automatic Identity Lookup in Splunk
One of the critical components of configuring the Splunk Enterprise Security Suite involves building lookup tables for identities. This post will provide an overview of the steps required to get started with automatic identity lookup building using LDAP or Active Directory. This is by no means a comprehensive document of all possible configurations and options, but it should provide enough information to get you going in the right direction.
More often than not, you will already have much of this information already existing in another directory, such as Active Directory or LDAP. Fortunately, Splunk can be configured to automatically pull data from these sources and build the appropriate lookup tables for Enterprise Security.
Step 1: Configure the Splunk Supporting Add-on for Active Directory (SA-ldapsearch) to query your LDAP/Active Directory environment
Don’t be fooled by the name – just because this app is called the Splunk Support for Active Directory doesn’t mean that Active Directory is required. This app can be used to query any LDAP server. Download and install the Splunk Support for Active Directory app.
The following instructions were written for version 1.x of the app, which requires Java to function. The configuration should be similar on version 2.x, which no longer has the Java requirement.
As part of the app setup, configure ldap.conf for the domains you want to query:
Step 2: Use ldapsearch to generate the lookup table
These instructions are based on the steps found in the Splunk Enterprise Security Installation manual.
Substitute the domain (configured in ldap.conf) in the search as follows:
If referencing the Splunk documentation, leave the outputlookup command off since this lookup doesn’t exist yet.
If you are using an LDAP directory that is not Active Directory, this can still be queried using a similar method, but the table will be using different values depending on the LDAP structure.
Step 3: Create the lookup table file
The base lookup table file can be created by uploading a skeleton file through SplunkWeb. You will want to make sure that the destination app is the SplunkEnterpriseSecuritySuite and that the sharing permissions are set to Global (object should appear in all Apps).
Creating the lookup table file (uploading a skeleton file)
The header of the CSV should be as follows:
Setting the appropriate permissions
File saved successfully
Step 4: Create the lookup table definition
The second link is used for configuring lookup definitions
The lookup table definition will be used in a saved search to generate the lookup table. The destination app for this lookup table definition should again be the SplunkEnterpriseSecuritySuite. The name can be whatever you would like the lookup definition to be called (ad_identities generally works fine for Active Directory). The type should be file-based, and the lookup file should be set to the CSV you uploaded in step 3.
Lookup definition created, permissions set to private
You will want to make sure that the sharing permissions are set to Global on this as well.
Correct permissions displayed
Step 5: Create the lookup table and verify that it is updated on the filesystem
Run the same search as Step 2, but add the | outputlookup <lookup_name> command to the search. This will output the results to the lookup table specified.
Splunk search outputting lookup table
Verify that this is functioning properly by checking the file on the filesystem, which should be $SPLUNK_HOME/etc/apps/SplunkEnterpriseSecuritySuite/lookups if you have followed the steps up to this point.
Step 6: save the search as a report
This is assuming that the lookup table is being generated properly. Also, this can be scheduled at a later point in time to automatically update the lookup table.
Step 7: Configure Identities in ES: Navigate to App: Enterprise Security -> Configuration -> Data Enrichment -> Identity Management
Identity Management Configuration
Create a new identity, being careful to identify the type as “identity” and the source to the lookup definition you created earlier, which will be lookup://<lookup_name>. This should be enabled by default once it’s created, but verify that this is the case in the web interface.
Adding a new identities to the identity manager
Step 8: create the merged identity file
Before you create the merged identity file, wait approximately 5 minutes for Splunk to automatically detect the change in the identity configuration.The following search will show you the status: index=_internal source=*python_modular_input.log *identit*.
Step 9: Verify that the identities_expanded.csv file was updated on the filesystem of the search head
This should be in $SPLUNK_HOME/etc/apps/SA-IdentityManagement/lookups by default.
Current modification date/time on identities_expanded.csv
Step 10: Schedule the report you created earlier to run on a semi-regular basis
This is being done assuming all is working as expected. Also, as a final note, don’t forget to disable the sample identities that are enabled by default Enterprise Security.
Original Post Date: March 19, 2015 by Tom Kopchak. Updated: March 8, 2016.
About Hurricane Labs
Hurricane Labs is a dynamic Managed Services Provider that unlocks the potential of Splunk and security for diverse enterprises across the United States. With a dedicated, Splunk-focused team and an emphasis on humanity and collaboration, we provide the skills, resources, and results to help make our customers’ lives easier.
For more information, visit www.hurricanelabs.com and follow us on Twitter @hurricanelabs.