Oct 3, 20176 min read

Vulnerability Assessment vs. Penetration Test: Key Differences and Objectives

Updated: Oct 4, 2017

*This blog entry was originally published on March 13, 2017 on the original Polito Blog by Ben Hughes. It was re-posted on October 3, 2017 due to migrating to a new blog platform.

On occasion we encounter questions about the differences between a vulnerability assessment and penetration test. While the end goal in both cases is improving an organization’s risk management and security posture, sometimes the two terms are mistakenly used interchangeably. Marketing hype and confusion aside, while there is not a single authoritative definition of “vulnerability assessment” and “penetration test”, there is a common understanding among infosec industry practitioners (still leaving room for debate about the finer points) regarding what each type of assessment generally entails and what differentiates one from the other.

A “vulnerability assessment” identifies vulnerabilities, quantifies and categorizes them based on factors such as their relative risk levels, and provides mitigation and remediation recommendations based on best practices and any relevant patches or standards. An example would be running a Nessus vulnerability scan against systems, attempting to verify any identified OS and software vulnerabilities and their corresponding real-world risk levels, and formatting the results into a detailed report. Validation of automatically identified vulnerabilities is generally limited due to being as passive as possible, avoiding actual exploitation attempts.

In contrast, a “penetration test” is a simulation of a real-world attacker that models typical attack tools, techniques, and objectives. Ideally, a penetration test includes as many stages of the attack kill chain as allowed, including reconnaissance, delivery, exploitation, and actions on objectives – just as a malicious targeted attack would. While it may follow preliminary vulnerability assessment activity, a penetration test goes beyond a vulnerability assessment to not just scan for known (e.g. publicly disclosed) vulnerabilities, but to find and exploit vulnerabilities whether previously known or not. The purpose of exploitation (which should be handled in an authorized and responsible proof of concept, “do no harm” approach) is to confirm whether a potential vulnerability is exploitable in the real-world, and accordingly determine the level of risk it poses to the organization. Vulnerability scans often focus on known vulnerabilities and baseline assessment standards such as PCI compliance and the OWASP Top 10 Vulnerabilities, but skilled penetration testers will creatively find and exploit vulnerabilities regardless of whether they show up in an existing testing plugin database, checklist, or framework.

Additionally, if authorized, penetration testers may also leverage social engineering techniques in order to gain unauthorized access to targeted systems and simulate additional stages of the cyber kill chain. An example would be sending a phishing email to quietly run proof-of-concept "malware" such as a backdoor on a victim employee's workstation, then performing additional "malicious" actions from that initial foothold just as a real-world attacker would. This is notably different from merely running a vulnerability scan, and this level of penetration testing is sometimes distinguished as “red teaming.”

Some examples may be helpful to illustrate the differences between a vulnerability assessment and penetration test. During a vulnerability assessment scan, Nessus’ generic plugins may not detect a critical vulnerability in a custom web application; however, a skilled penetration tester may identify a flaw in the application’s business logic that could lead to a complete compromise of its data. An Nmap scan to identify and fingerprint unexpected accessible systems, open ports, and running services may be conducted to find and assess relevant vulnerabilities. Examples of such findings stemming from a simple Nmap scan could include items such as end of life OS and software installations, a forgotten FTP server running outdated FTP software and still serving confidential files, or an internal system with RDP available to the internet due to a misconfigured firewall rule. Penetration testing could take the next steps to attempt to exploit the outdated server with public exploit code, access the FTP server and steal confidential files, and brute force a remote login to the internal system.

Some caveats are in order. First, while the terminology should always be kept distinct, penetration tests and vulnerability assessments are not necessarily mutually exclusive. Penetration testing activity often naturally follows vulnerability assessment activity. In other words, vulnerability assessments including vulnerability scans often point penetration testers in the right direction as a starting point. For example, a standard Nessus vulnerability scan against a web application may reveal some potential vulnerabilities that could then be explored further during a penetration testing stage. Penetration testers could develop proof of concept exploits for specific vulnerabilities, for confirmation and risk level analysis purposes. Similarly, a Burp Suite Professional automated scan may be fairly classified as a vulnerability assessment, but follow up tasks to confirm and try to exploit identified vulnerabilities with proof-of-concept tests identified vulnerabilities crosses into penetration testing. Often an automated vulnerability scan’s findings don’t get it completely right, but raise a red flag and provide enough technical details about the target and potential vulnerability to guide subsequent penetration testing. Additionally, we have increasingly seen requests for and interest in engagements deemed a “Vulnerability Assessment and Penetration Test (VAPT)”, which includes both a vulnerability assessment and penetration test. This is a preferred approach to conflating one term as encompassing both types of assessments, or referring to one as the other.

The following lists some key differentiators between a typical vulnerability assessment and a typical penetration test. Note that these are generalizations that we have repeatedly observed while practicing both types of assessments. Note how such differences can complement each other.

Typical Use Cases

Vulnerability Assessment - Formal vulnerability management, compliance, routine audits.

Penetration Test - Formal vulnerability management, identifying and remediating vulnerabilities not discovered by vulnerability scanning alone, determining risk of real-world attack vectors, threat modeling.

Approach

Vulnerability Assessment - Tends to center on automated scanning.

Penetration Test - Automated scanning supplemented by additional tools and manual techniques (whatever it takes for vulnerability identification and exploitation). Utilizes multiple stages of the attacker kill chain.

Quantity vs. Quality

Vulnerability Assessment - Enumerates as many vulnerabilities as possible.

Penetration Test - Focuses on finding and exploiting a few high quality vulnerabilities.

Example Tools

Vulnerability Assessment - Standard scanners such as Nessus, OpenVAS, Nmap, and Burp Suite Pro’s Active Scanner.

Penetration Test - Vulnerability assessment tools plus additional testing and exploitation tools such as Burp Suite Pro’s Intruder and Repeater, SQLmap, Metasploit, custom code, and other tools including the offensive arsenal provided by Kali.

Speed

Vulnerability Assessment - Relatively fast.

Penetration Test - Relatively slow.

Accuracy

Vulnerability Assessment - False positives, false negatives, and incomplete findings are more likely.

Penetration Test - False positives can be weeded out and are ultimately unlikely, findings can be confirmed and exploited.

LOE

Vulnerability Assessment - Mostly automated, sometimes supplemented by some manual testing and confirmation.

Penetration Test - Some automated, but emphasis on manual testing and exploitation.

Types of Vulnerabilities Identified

Vulnerability Assessment - Known vulnerabilities, “low hanging fruit”, some incomplete findings.

Penetration Test - Known vulnerabilities but also unknown (0-day) vulnerabilities, especially in custom infrastructure/apps.

Validation and Exploitation of Identified Vulnerabilities

Vulnerability Assessment - Limited validation and no exploitation.

Penetration Test - Additional testing and exploitation leading to validation of vulnerabilities and their real-world exploitability.

Noise Level / Detectability

Vulnerability Assessment - High.

Penetration Test - Low-Medium.

Risk of Damage to Production Systems

Vulnerability Assessment - Very low risk, unless scan traffic volume becomes an issue and is not throttled properly (e.g. unexpected DoS condition).

Penetration Test - Slightly higher risk, though exploits are typically limited in scope and designed to be a proof of concept.

We understand that the differences are not always this distinct, and like the overarching concept of risk, actual engagements tend to utilize a sliding scale between vulnerability assessment features and penetration test features. That is one reason why properly scoping an engagement and defining these terms and underlying features is so important.

Both approaches offer significant value for an organization seeking to identify and mitigate any major vulnerabilities, implement formal vulnerability and risk management programs, and ultimately enhance its overall security posture against a variety of cyber threats. Ideally, every organization should be regularly conducting both vulnerability assessments and penetration tests to safeguard their systems and data. Periodic vulnerability scans are a cornerstone of a successful vulnerability and risk management program, but to better prepare an organization’s defenses against real-world attacks, penetration tests are essential. Put another way, a vulnerability assessment by itself is necessary, but not sufficient to reveal your organization’s weaknesses and take proactive action to implement high-impact security controls – a penetration test is the next critical step.

Polito, Inc. offers a wide range of security consulting services including penetration testing, vulnerability assessments, incident response, digital forensics, and more. If your business or your clients have any cyber security needs, contact our experts and experience what Masterful Cyber Security is all about.