How to improve your pentesting results

There are a number of questions floating around about the value penetration testing provides businesses and security operations. This blog post will serve to answer a few of these frequent flyer questions, including:

• Why is a penetration test useful?
• Why is it cheating to say: “Here’s an IP, tell us everything you see”?
• Why is it better to review results alongside the pentesters?
• What’s this purple team of which you speak?

Onward!

Why is a pentest useful to a SOC?

Penetration testing (pentesting) is the most common way for a business to determine vulnerabilities in a system, a network, a web application, or a particular service in their environment that the security team needs to focus on hardening. Imperva defines a pentest as: “a simulated cyber attack against your computer system to check for exploitable vulnerabilities.”

Pentesting is useful to a security operations center (SOC) because these tests can be used to identify gaps in active alerting, and they help generate the discussion with the customer or IT/Security team to develop ways of filling those gaps. They’re also useful to SOC teams through inadvertently testing incident response protocols, the analysts’ capabilities for analysis and decision-making, and the existing rules to generate alerts to the SOC.

Not only can the results of a pentest engage the IT, security, and/or SOC teams, it can also engage C-level executives and other management across an organization through the reevaluation of strategic risk, disaster recovery planning, incident response, and security posture focus.

Don’t cheat on the test

Pentests often come with their own host of problems depending on a number of factors, including whether it’s an internal or external test and what the scope of the test is.

More often than not, only a select few people within an organization know the pentesting is taking place and what devices or IPs those tests are coming from. That is to say, not everyone on the IT or security team will know the tests are taking place, usually as an additional testing of internal security mechanisms and processes for handling breaches, attacks, and vulnerability exploitation.

Occasionally, as SOC analysts, we’re given the directive by our customers to “keep an eye out” for specific things during a penetration test being performed against their environment, such as an IP that the pentesters are using or a device that is being targeted. The problem with this approach is that the customer isn’t getting what they’re paying for from our SOC services and are simultaneously sabotaging their own security posture growth.

This pre-approved approach is the equivalent of cheating on a test–you’re basically being given answers to the questions that haven’t been asked yet.

Being told “192.168.1.220 is going to be doing some hinky stuff on our network between Monday and Thursday this week; please let us know when you see it” circumvents the test by outing the attacker before they’ve done anything. Putting the security team(s) on high alert for anything out of the ordinary versus reacting to something that is intended to appear natural creates a state of artificial results instead of organically grown detection and analysis methodologies.

This kind of request for analysis and incident response only serves to “check the box” on the auditor’s compliance report for “tested and security mechanisms are being developed.”

Post-mortem vs active detection

The preferred way to handle pentesting from a SOC perspective is through a post-mortem on the results of the penetration test after the activity has ended.

Performing a post-mortem on a pentest is a valuable tool in the infosec arsenal to help harden systems that have been compromised, but it isn’t the only way to approach the task. A post-mortem is best performed alongside the pen test team, the IT/security team at the organization, and one or two analysts involved in the analysis and creation of new rules.

This side-by-side approach to pentesting and involving the pentesters in the conversation means the team can:

• • Ask the pentesters questions about the tools they used and how they got where they did specifically outside of the report (though the report should detail this).
  • Recreate the scenario in which the attacker gains access to a system–which is just as important as knowing what they did and how–to try detecting it from a security growth perspective.
  • Involve the pentesters in developing detection methods in order to better understand the creative human element of hacking, how to best concentrate efforts, and ultimately where defenses can be strengthened.

This information is valuable for troubleshooting existing mechanisms that are believed to be capable of detecting the attacks. Discussions need to take place covering the pentest report so each side can present their ideas for enhancing their detection capability, incident response changes, tools for analysis, and so on.

Reviewing the results and providing guidelines for system changes to mitigate attacks through the seen vectors in the pen test is far more valuable than the equivalent of “spray and pray” with a search like “index=* src=192.168.1.220 OR src_ip=192.168.1.220” between X and Y dates to “see what you can see.”

We often find out through active SOC alerting that a customer is having a penetration test performed once we’ve successfully detected part or all of the activity and notified the customer security team. This is an organic, natural way of determining where security gaps are within a system because an existing mechanism detected the activity, an analyst thought it was suspicious, and so they notified the security team. Naturally, this doesn’t cover all of the attack vectors and is the equivalent of “getting lucky” when there are better ways to build security mechanisms to detect and alert on these occurrences.

Where does purple teaming come into play?

This post has talked specifically about how pentesting and reporting/detection affects the blue team, but an expansion on defending a network exists in the form of “purple teaming.”

Purple teaming is a combination of the common nomenclature in the infosec world where the blue team is defense and the red team is offense. The purple team is a combination where both sides can see how the other operates in tandem. SecureAuth spells it out nicely: “Purple Teams are (as their name would suggest) a single group of people who do both Red and Blue testing and securing of a company.”

Purple teams should not be a replacement for both of the red and blue team operations within your organization, but should instead serve to enhance the existing capabilities of those individuals or teams.

Oftentimes in the IT and cybersecurity realm we become “Jack of all trades, master of none,” and it doesn’t serve to promote an environment of growth, especially when it comes to security. No one is ever going to be a master of both securing a network and attacking one, which is why the blue team and red team exist independently. The landscape on both sides is ever-evolving and adapting to the tactics and defenses being used.

By leveraging a group of individuals who understand both sides of the cybersecurity fence, organizations can be better equipped to deal with how the attacks are actually perpetrated–and how best to defend, detect, and mitigate those threats. Not only can purple team exercises help strengthen your security posture as an organization, but it also allows your team to integrate new methodologies into your regular audits–and they can improve their networking in cybersecurity circles through this collaboration.

Conclusion

Overall, pentesting activities taking place in your organization seems beneficial on its own, but it’s not a true, overall test of your secure defense strategy. Alerting your SOC to the possibility that you’ll see events from a specific source is also not helping you do more than check a box.

The difference here is between testing your preparedness when facing unknown actors and testing your preparedness when alerted in advance. Your assessment will be more accurate if your existing procedures are allowed to function without the advance notice. Further, hindsight is always 20-20, and discussing what could have happened is still a valid discussion that needs to take place–making the post-mortem of a penetration test that much more important.

The key to a stronger security posture with regard to penetration testing is having mitigations in place through behavioral analysis, better firewall rules, better detection capabilities, and a strong incident response process.