The Secret of Serverless Applications: Splunking Access Logs Using AWS

So, what is a “serverless” application?

In case you haven’t heard, there’s a cool new phrase going around the cloud-based programming world: “serverless applications”. Of course, this phrase is quite a misnomer (as is a lot in this post-fact world that we live in), but the idea is still neat, perhaps even novel.

In short, a “serverless” application is one where you (the developer / administrator) don’t have to manage the server. This is convenient if you’re not interested in managing a whole server just to deploy code that runs based on “triggers”, such as a new file being uploaded. Serverless computing has often been called “Platform-as-a-Service” (PaaS), in the same vein as “Software-as-a-Service” (SaaS, such as Office 365), and “Infrastructure-as-a-Service” (IaaS, such as AWS EC2 or Azure).

Lambda is a pretty neat tool

AWS provides a few different tools for working with serverless applications. One of the more interesting ones is a service called “Lambda”, which can be considered to be a “Function-as-a-Service”.

Similar to typical serverless applications, the “server” and OS part of the stack are managed for you by the vendor (AWS). Unlike PaaS applications, however, the platform itself is managed for you, and you get the luxury of only having to write code. Lambda supports a few different languages for you to write your “function”, and then provides a slew of ways to trigger the function. Some of these include:

• AWS SNS (Simple Notification Service)
• AWS S3 (Simple Storage Service)
• Alexa (this is actually how you write your own Alexa “skills”)

Lambda even lets you use the Amazon API Gateway to expose your functions as HTTP endpoints, which we’ll discuss in a separate blog post at a later date.

Enter: Amazon’s Serverless Application Repository

More recently (that is to say, toward the end of 2017), Amazon released the “Serverless Application Repository”, which is a fancy web-based tool for deploying templatized serverless applications. Splunk happened to be one of the launch-time partners, and provided a fairly useful application for collecting the access logs from an AWS “application load balancer” (ALB).

We run a few of these in AWS, so we thought this was an excellent opportunity to test out the Serverless Application Repository. We tested this using the ALB in front of our Splunk search head cluster (because of course).

Putting it to the test

We followed the instructions in Splunk’s blog post (Note: Serverless Application Repository is no longer in beta, so you can just access it from your AWS console), and the setup was very straightforward. The only oddity we encountered was that the Serverless Application Repository created an S3 bucket for us, but we still needed to add permissions for the ALB to write logs there – you can follow the instructions in Amazon’s documentation for enabling access logging in the ALB, but you’ll just be modifying an existing bucket rather than creating a new one.

Shortly after this was all deployed, we started receiving access logs for our SHC in Splunk, in a format the Splunk Add-on for AWS recognized. It was altogether a very simple process, and we’re looking forward to seeing what other interesting things appear in the Serverless Application Repository in the coming months.