Automated vs Manual Pentesting

With automation and Artificial Intelligence (AI) on the rise, it's becoming increasingly important to adapt to the tools and techniques that are released every day. By failing to adapt, it quickly becomes easy to fall behind in each new era of technology. While it may be tempting to adopt AI primarily for workforce reduction and cost savings, it's essential to keep in mind the irreplaceable value of human oversight on a project.

In the world of pentesting, automation is mandatory to be effective at enumerating an environment and exploiting vulnerabilities. Nearly every tool in a pentester's arsenal is designed to automate some part of the engagement. Given this reliance on automation, it raises the question: if every pentester uses automated tools in their engagements, why not take it a step further and leverage AI?

Setting aside my personal bias of wanting to keep my job, there are stronger and more compelling reasons we should avoid doing this.

Automated Exploits Can Crash Servers

First of all, automation can help to identify critical vulnerabilities, but sometimes these exploits can crash a server. For example, the ZeroLogon vulnerability (CVE-2020-1472) allows an attacker to gain elevated privileges on a domain controller (DC) by setting its machine password to null.

While this can be an easy win in an Active Directory (AD) environment, it also places the DC in an unstable state until the password is restored. A skilled manual tester would document the findings rather than exploit it. If the DC crashes before the password is replaced, it can trigger a denial-of-service for the entire AD infrastructure.

Once that happens, the organization will incur far greater costs by spending more time and money identifying the root cause and fixing the disruption than they would have spent on a manual pentest.

No Coordination Between Red/Blue Teams

Once the enumeration phase is complete, exploitation needs to be coordinated with key personnel and scheduled during off hours to maintain network availability. It's important to coordinate with a trusted agent – serving as a liaison between the pentesters and the organization's defenders – to provide the most value out of your pentest.

This approach allows the defending organization to know what attacks are being run at specific times, limit damage and risk to the business, and control the activities the pentesters are allowed to perform. Additionally, maintaining accurate timestamps also helps the organization's defenders identify where the attacks are occurring in their logs and develop new alerts to fire the next time it occurs.

In contrast, an automated solution will have its playbook to run, but it will be difficult to control when an exploit is conducted, and what type of attacks it can or can't perform in your environment.

False Sense of Security

Automated pentesting can also provide a false sense of security. If an automated solution runs its playbook, it will reliably find the vulnerabilities it's programmed for. But it won't go out of its way to find hidden misconfigurations if it's not been explicitly told to look for them.

For example, let's say there's an old printer in your environment that has an embedded web service - with default administrator credentials that have never been changed. When you log in, you start enumerating the webpage looking for any sensitive information. Then under the Networking tab, you discover an LDAP server address.

A pentester can execute a Passback Attack by starting a quick network listener for LDAP traffic, tell the printer to talk to the attacker's LDAP server instead, and they can retrieve that masked password. Now they have new network credentials to pass around the environment. See How to Hack Through a Pass-Back Attack: MFP Hacking Guide for a full breakdown of this attack.

When working with local web services on printers, there’s not a standard webpage layout that every printer uses. This makes it extremely difficult to automate this type of enumeration and an automated tool may not find this misconfiguration without some level of manual intervention.

Final Thoughts

Ultimately, automation in pentesting is vitally important for enumerating and exploiting networks. Without tools like Nmap, Nessus, and Metasploit, pentesting would be a grueling process that could take weeks or months to perform manually. However, for a pentest to be truly valuable, it's important to always have human oversight. Having an experienced tester behind the keyboard ensures that scanning and exploiting are conducted safely. That way you can be sure that servers won’t crash unexpectedly, and you will always know what attacks are being performed. This human element provides assurance that your infrastructure was tested thoroughly and responsibly – something automation or AI alone cannot guarantee.

At least, for now.