Blog
Penetration testing

6 types of penetration testing & which is best for your business

Olya Osiagina
Author
Olya Osiagina
Head of Demand Generation

Key Points

If you are thinking about performing a penetration test on your organization, you might be interested in learning about the different types of penetration tests available. With that knowledge, you'll be better equipped to define the scope for your project, hire the right expert and, ultimately, achieve your security objectives.

TL;DR of the different types of penetration testing

  1. Network penetration testing: The most common and crucial type of penetration test
  2. Automated penetration testing: Also known as vulnerability scanning
  3. Web application penetration testing: For uncovering vulnerabilities across websites and web applications
  4. Cloud penetration testing: For testing the security posture of your cloud environment
  5. Social engineering pen tests: For testing the resilience of your personnel to social engineering attacks (e.g. phishing assessments)
  6. Red teaming: An advanced assessment that can take months to complete

What is penetration testing?

Penetration testing, commonly referred to as “pen testing”, is a technique that simulates real-life attacks on your IT systems to find weaknesses that could be exploited by hackers. These assessments are carried out by penetration testers or 'ethical hackers'.

Whether to comply with security regulations such as ISO 27001, gain customer and 3rd party trust, or achieve your own peace of mind, penetration testing is an effective method used by modern organizations to strengthen their cyber security posture and prevent data breaches.  

How often should pen testing be conducted?

Due to the complexity and cost of a penetration test, is it usually only carried out on an annual basis. That's why automated and continuous solutions like vulnerability scanning are so important; they help you keep your systems secure between annual penetration tests. Learn more about the differences between penetration testing vs vulnerability scanning.

6 types of pen testing to strengthen cybersecurity

The type of penetration testing you need depends on your unique IT environment. We have summarized the most common types to help you find out which you would benefit from the most.

1. Network penetration testing

As the name suggests, a network penetration test aims to identify weaknesses in your network infrastructure, be that on the premises or in cloud environments. It is one of the most common and crucial tests to perform to ensure the security of your business-critical data.

Network penetration testing covers a broad range of checks, including insecure configurations, encryption vulnerabilities, and missing security patches in order to determine the steps a hacker could take to attack your organization. Security professionals often categorize network penetration tests into two different perspectives: external and internal penetration testing.

External penetration testing

External penetration testing involves searching for vulnerabilities that could be exploited by any attacker with access to the internet.

In this type of network penetration test, penetration testers are trying to get access to your business critical systems and data in order to determine how an attacker without any prior access or knowledge would be able to target your organization.

You can think of an external penetration test as being performed from the perspective of an "outsider".

Internal penetration testing

In contrast, internal penetration testing is concerned with testing your internal corporate environment.

This type of pen testing considers scenarios in which an attacker has managed to gain an initial foothold within your corporate network, for example by exploiting a vulnerability in one of your internet-facing systems, or through the use of social engineering.

In this case, the test is performed from an “insider” perspective, with an objective of finding a way to steal sensitive information or disrupting the operations of an organization.

When to carry out network penetration testing

External network penetration testing is the first type of penetration testing organizations usually perform. Generally speaking, external weaknesses are considered to pose a more serious threat than internal. For one thing, a hacker has to overcome an external security barrier before accessing your internal networks and pivoting to other systems.

If you haven't conducted any kind of penetration testing before, an external or “perimeter” test is often the best place to start, as the perimeter is the easiest thing for attackers to get to. If you have trivial vulnerabilities in your internet-facing infrastructure, that's where the hackers will start.

2. Automated penetration testing

Understandably, as penetration tests can be costly and infrequent (only run once or twice per year), many people naturally wonder if automated penetration testing is feasible.  

While it's not possible to fully automate a penetration test (as there will always be an element of manual work conducted by skilled professionals), it's similarly impossible for humans to manually check for every vulnerability that exists, there are simply too many.

That's where vulnerability scanning comes in. With these tools, you can schedule scans, get rapidly tested for many thousands of weaknesses, and be notified of your results in a variety of channels and formats. It's no wonder that vulnerability scanners form a critical part of a penetration tester's toolkit.  

When you combine continuous vulnerability scanning with an annual penetration test, you can rest assured that your systems are covered by a robust and comprehensive cyber security program. To see the tool in action, try our demo below or take our Pro Plan for a spin with our 14-day free trial.

Find your weaknesses,
before the hackers do

Try Intruder for free

3. Web application penetration testing

Web application penetration testing attempts to uncover security vulnerabilities across websites and web applications, such as e-commerce platforms, content management systems, and customer relationship management software.

This type of pen test deals with reviewing the entire web application's security, including its underlying logic and custom functionalities, to identify vulnerabilities and prevent data breaches.  

Some of the common vulnerabilities detected during a web app penetration test include database injections, cross-site scripting (XSS), and broken authentication.

If you are interested in learning more about different types of web application weaknesses, their severity and how you can prevent them, the Open Web Application Security Project (OWASP) Top 10 is a great place to start. Every few years OWASP publishes information about the most frequent and dangerous web application flaws, basing its findings on the data collected from many thousands of applications.

When to carry out a web application penetration test

Considering the prevalence of web applications in modern organizations, and the valuable information that they transmit and store, it is unsurprising that they are an attractive target to cyber criminals. A report found that almost one-in-ten vulnerabilities in internet-facing applications are considered high or critical risk.

For this reason, any organization that is developing or managing their own internet-facing applications should strongly consider conducting web application penetration testing as part of their web app security testing program.

4. Cloud penetration testing

In a cloud penetration test, the tester will start with no access to the cloud environment or limited access if they're testing an assumed breach scenario – for example assuming an attacker has compromised a server in the cloud environment, or a specific user. They'll then try to escalate their privileges to reach other parts of the cloud environment, trying to access sensitive data.

Different manual methods tools (such as black box penetration testing vs gray box penetration testing) may be used, depending on the type of cloud service and the provider. However, since you don't actually own the cloud infrastructure, platform or software as an entity but rather as a service, there are strict legal restrictions and technical challenges to a cloud penetration tests.

As a cloud user, your focus is on ‘security in the cloud' and not ‘security of the cloud'. What you are responsible for, and therefore what can be included in the scope of a pentest, is dictated by the shared responsibility model.

When to carry out cloud penetration tests

Businesses often won't have in-house cloud or security experts when they start their shift to cloud services. This means teams might be quickly and unknowingly creating security holes that are hard to monitor and fix. A cloud penetration test would give you in-depth insight into your cloud security posture to help you verify whether your ongoing vulnerability management efforts are working.

5. Social engineering pen tests

In comparison to previously described penetration testing types, which focus on finding weaknesses in technology, social engineering attempts to compromise the security of an organization by exploiting human psychology.

It can take a variety of forms and could be executed both remotely, for example by trying to obtain sensitive information from users through phishing emails or phone calls, or on-site, in which case a penetration tester will attempt to gain access to a physical facility (otherwise known as 'physical penetration testing'). In all cases, an objective of this penetration test is to manipulate individuals, usually the company's employees, to give away valuable information.

The success of a social engineering penetration test largely depends on the information gathered in the “reconnaissance” phase, which involves researching targeted individuals or an organization by using publicly accessible open source intelligence (OSINT).

After building a more precise image of their target, a penetration tester can use discovered information to proceed with the creation of a tailored attack strategy.

One of the most common attack vectors in social engineering is a phishing attack, usually delivered by email. When performing a phishing attack, a penetration tester does not necessarily stop when an unsuspecting employee clicks on a malicious link, but can go further, attempting to steal user credentials and get access to an employee's laptop.

Such attacks can be extremely successful, especially when performed by experienced penetration testers. Take a look at some phishing attempts made against the Intruder team.  

When to carry out social engineering penetration testing

Social engineering penetration testing is not as widely adopted as network or web application testing. However, if your organization is already doing regular security awareness training, conducting social engineering tests can be a great addition to your arsenal for identifying and fixing security issues in your operations.

6. Red teaming  

This advanced technique has its origin in military training exercises. It is designed to challenge an organization's security, processes, policies and plans by adopting an adversarial mindset.

In contrast, Blue Teaming, otherwise known as “defensive security”, involves detecting and withstanding Red Team attacks as well as real-life adversaries.

Red Teaming combines digital, social and physical domains to implement comprehensive real-life attack scenarios. As such, Red Teaming can be considered a distinct operation from penetration testing, but since its tasks span all of the types of penetration testing described above, we thought it was worth mentioning it in this article.

An objective of a standard penetration test is to find as many vulnerabilities as possible within a given timeframe. The breath of this test is naturally limited by the scope of work; but real-life adversaries don't have such artificial restrictions to follow.

As a result, even if an organization regularly performs penetration tests and vulnerability scans, it can still be exposed to more sophisticated attacks such as where social engineering and internal network weaknesses are chained together.

This is where Red Teaming comes in. It assesses an organization's environment as a whole, understanding how all parts function together. It then applies critical thinking to discover new vulnerabilities that attackers can exploit, helping the organization to assess its response to real-world attacks.

When to carry out a red team assessment

Compared to the standard penetration test, which lasts several days or weeks, Red Team assessments generally take much longer, in some cases several months to complete.

Due to its complex nature, it is a rather rare operation, typically performed by larger organizations or by government contractors with well-established security programs.

Black box vs Gray box vs White box pen testing methods

You may have heard of terms like "black box penetration testing", "gray box penetration testing", and "white box penetration testing". These terms are supposed to refer to the amount of information shared with testers prior to an engagement. For example, source code is "typically" provided in white box penetration tests, but not in black box pen tests.  

The problem with these terms though is that there is no strict definition of what they mean, so often as soon as a client asks if you can do a white box penetration test the immediate follow up question is, sure, but what information do you want to include? At which point, you might as well just define what information will be provided anyway without these vague terms, since no penetration test is really conducted on zero information.  

The rationale behind these terms is that you might want to understand what someone can figure out on their own without any information. That might be a valid instinct, but it's worth questioning this mindset.

There's a saying in cyber security that “obscurity is not security” - which means you shouldn't hope that something is secure just because someone doesn't know or can't guess something.

A pen-tester armed with more information might be very quickly able to tell you that something isn't secure, while an uninformed one might take way longer to figure it out. And guess what – you're the one paying for their time.

So while we wouldn't say "scenario" type pen tests are never right, it's worth thinking about why you are doing it and what you really want from the exercise. Do you want to be more secure, or do you want to see how long it takes someone to figure it out?  

A good pen-testing company will usually help guide you by asking for the information that makes the most sense for the budget you have available, or the scenario you're asking them to test.

To conclude

Penetration testing is a broad discipline that encompasses different techniques, so it is important to understand the relative risks that your organization is facing in order to choose the most appropriate type. Once you have decided on the type of pentest you want to conduct, the next logical step is to choose the right company or tool for your project; if you need help getting started, we have written a helpful guide on how to choose the right pentesting company and penetration testing tools.

At Intruder, we offer a combination of penetration testing and continuous vulnerability scanning with our automated penetration tool. Start your 14 day free trial today.

Thanks to Daniel Thatcher (Intruder Security Research Engineer)


Get our free

Ultimate Guide to Vulnerability Scanning

Learn everything you need to get started with vulnerability scanning and how to get the most out of your chosen product with our free PDF guide.

Sign up for your free 14-day trial

7 days free trial
Pen tests help you find weaknesses in your systems before hackers do. Explore the types of pen testing & find the most effective one for your security needs.
back to BLOG

6 types of penetration testing & which is best for your business

Olya Osiagina

If you are thinking about performing a penetration test on your organization, you might be interested in learning about the different types of penetration tests available. With that knowledge, you'll be better equipped to define the scope for your project, hire the right expert and, ultimately, achieve your security objectives.

TL;DR of the different types of penetration testing

  1. Network penetration testing: The most common and crucial type of penetration test
  2. Automated penetration testing: Also known as vulnerability scanning
  3. Web application penetration testing: For uncovering vulnerabilities across websites and web applications
  4. Cloud penetration testing: For testing the security posture of your cloud environment
  5. Social engineering pen tests: For testing the resilience of your personnel to social engineering attacks (e.g. phishing assessments)
  6. Red teaming: An advanced assessment that can take months to complete

What is penetration testing?

Penetration testing, commonly referred to as “pen testing”, is a technique that simulates real-life attacks on your IT systems to find weaknesses that could be exploited by hackers. These assessments are carried out by penetration testers or 'ethical hackers'.

Whether to comply with security regulations such as ISO 27001, gain customer and 3rd party trust, or achieve your own peace of mind, penetration testing is an effective method used by modern organizations to strengthen their cyber security posture and prevent data breaches.  

How often should pen testing be conducted?

Due to the complexity and cost of a penetration test, is it usually only carried out on an annual basis. That's why automated and continuous solutions like vulnerability scanning are so important; they help you keep your systems secure between annual penetration tests. Learn more about the differences between penetration testing vs vulnerability scanning.

6 types of pen testing to strengthen cybersecurity

The type of penetration testing you need depends on your unique IT environment. We have summarized the most common types to help you find out which you would benefit from the most.

1. Network penetration testing

As the name suggests, a network penetration test aims to identify weaknesses in your network infrastructure, be that on the premises or in cloud environments. It is one of the most common and crucial tests to perform to ensure the security of your business-critical data.

Network penetration testing covers a broad range of checks, including insecure configurations, encryption vulnerabilities, and missing security patches in order to determine the steps a hacker could take to attack your organization. Security professionals often categorize network penetration tests into two different perspectives: external and internal penetration testing.

External penetration testing

External penetration testing involves searching for vulnerabilities that could be exploited by any attacker with access to the internet.

In this type of network penetration test, penetration testers are trying to get access to your business critical systems and data in order to determine how an attacker without any prior access or knowledge would be able to target your organization.

You can think of an external penetration test as being performed from the perspective of an "outsider".

Internal penetration testing

In contrast, internal penetration testing is concerned with testing your internal corporate environment.

This type of pen testing considers scenarios in which an attacker has managed to gain an initial foothold within your corporate network, for example by exploiting a vulnerability in one of your internet-facing systems, or through the use of social engineering.

In this case, the test is performed from an “insider” perspective, with an objective of finding a way to steal sensitive information or disrupting the operations of an organization.

When to carry out network penetration testing

External network penetration testing is the first type of penetration testing organizations usually perform. Generally speaking, external weaknesses are considered to pose a more serious threat than internal. For one thing, a hacker has to overcome an external security barrier before accessing your internal networks and pivoting to other systems.

If you haven't conducted any kind of penetration testing before, an external or “perimeter” test is often the best place to start, as the perimeter is the easiest thing for attackers to get to. If you have trivial vulnerabilities in your internet-facing infrastructure, that's where the hackers will start.

2. Automated penetration testing

Understandably, as penetration tests can be costly and infrequent (only run once or twice per year), many people naturally wonder if automated penetration testing is feasible.  

While it's not possible to fully automate a penetration test (as there will always be an element of manual work conducted by skilled professionals), it's similarly impossible for humans to manually check for every vulnerability that exists, there are simply too many.

That's where vulnerability scanning comes in. With these tools, you can schedule scans, get rapidly tested for many thousands of weaknesses, and be notified of your results in a variety of channels and formats. It's no wonder that vulnerability scanners form a critical part of a penetration tester's toolkit.  

When you combine continuous vulnerability scanning with an annual penetration test, you can rest assured that your systems are covered by a robust and comprehensive cyber security program. To see the tool in action, try our demo below or take our Pro Plan for a spin with our 14-day free trial.

Find your weaknesses,
before the hackers do

Try Intruder for free

3. Web application penetration testing

Web application penetration testing attempts to uncover security vulnerabilities across websites and web applications, such as e-commerce platforms, content management systems, and customer relationship management software.

This type of pen test deals with reviewing the entire web application's security, including its underlying logic and custom functionalities, to identify vulnerabilities and prevent data breaches.  

Some of the common vulnerabilities detected during a web app penetration test include database injections, cross-site scripting (XSS), and broken authentication.

If you are interested in learning more about different types of web application weaknesses, their severity and how you can prevent them, the Open Web Application Security Project (OWASP) Top 10 is a great place to start. Every few years OWASP publishes information about the most frequent and dangerous web application flaws, basing its findings on the data collected from many thousands of applications.

When to carry out a web application penetration test

Considering the prevalence of web applications in modern organizations, and the valuable information that they transmit and store, it is unsurprising that they are an attractive target to cyber criminals. A report found that almost one-in-ten vulnerabilities in internet-facing applications are considered high or critical risk.

For this reason, any organization that is developing or managing their own internet-facing applications should strongly consider conducting web application penetration testing as part of their web app security testing program.

4. Cloud penetration testing

In a cloud penetration test, the tester will start with no access to the cloud environment or limited access if they're testing an assumed breach scenario – for example assuming an attacker has compromised a server in the cloud environment, or a specific user. They'll then try to escalate their privileges to reach other parts of the cloud environment, trying to access sensitive data.

Different manual methods tools (such as black box penetration testing vs gray box penetration testing) may be used, depending on the type of cloud service and the provider. However, since you don't actually own the cloud infrastructure, platform or software as an entity but rather as a service, there are strict legal restrictions and technical challenges to a cloud penetration tests.

As a cloud user, your focus is on ‘security in the cloud' and not ‘security of the cloud'. What you are responsible for, and therefore what can be included in the scope of a pentest, is dictated by the shared responsibility model.

When to carry out cloud penetration tests

Businesses often won't have in-house cloud or security experts when they start their shift to cloud services. This means teams might be quickly and unknowingly creating security holes that are hard to monitor and fix. A cloud penetration test would give you in-depth insight into your cloud security posture to help you verify whether your ongoing vulnerability management efforts are working.

5. Social engineering pen tests

In comparison to previously described penetration testing types, which focus on finding weaknesses in technology, social engineering attempts to compromise the security of an organization by exploiting human psychology.

It can take a variety of forms and could be executed both remotely, for example by trying to obtain sensitive information from users through phishing emails or phone calls, or on-site, in which case a penetration tester will attempt to gain access to a physical facility (otherwise known as 'physical penetration testing'). In all cases, an objective of this penetration test is to manipulate individuals, usually the company's employees, to give away valuable information.

The success of a social engineering penetration test largely depends on the information gathered in the “reconnaissance” phase, which involves researching targeted individuals or an organization by using publicly accessible open source intelligence (OSINT).

After building a more precise image of their target, a penetration tester can use discovered information to proceed with the creation of a tailored attack strategy.

One of the most common attack vectors in social engineering is a phishing attack, usually delivered by email. When performing a phishing attack, a penetration tester does not necessarily stop when an unsuspecting employee clicks on a malicious link, but can go further, attempting to steal user credentials and get access to an employee's laptop.

Such attacks can be extremely successful, especially when performed by experienced penetration testers. Take a look at some phishing attempts made against the Intruder team.  

When to carry out social engineering penetration testing

Social engineering penetration testing is not as widely adopted as network or web application testing. However, if your organization is already doing regular security awareness training, conducting social engineering tests can be a great addition to your arsenal for identifying and fixing security issues in your operations.

6. Red teaming  

This advanced technique has its origin in military training exercises. It is designed to challenge an organization's security, processes, policies and plans by adopting an adversarial mindset.

In contrast, Blue Teaming, otherwise known as “defensive security”, involves detecting and withstanding Red Team attacks as well as real-life adversaries.

Red Teaming combines digital, social and physical domains to implement comprehensive real-life attack scenarios. As such, Red Teaming can be considered a distinct operation from penetration testing, but since its tasks span all of the types of penetration testing described above, we thought it was worth mentioning it in this article.

An objective of a standard penetration test is to find as many vulnerabilities as possible within a given timeframe. The breath of this test is naturally limited by the scope of work; but real-life adversaries don't have such artificial restrictions to follow.

As a result, even if an organization regularly performs penetration tests and vulnerability scans, it can still be exposed to more sophisticated attacks such as where social engineering and internal network weaknesses are chained together.

This is where Red Teaming comes in. It assesses an organization's environment as a whole, understanding how all parts function together. It then applies critical thinking to discover new vulnerabilities that attackers can exploit, helping the organization to assess its response to real-world attacks.

When to carry out a red team assessment

Compared to the standard penetration test, which lasts several days or weeks, Red Team assessments generally take much longer, in some cases several months to complete.

Due to its complex nature, it is a rather rare operation, typically performed by larger organizations or by government contractors with well-established security programs.

Black box vs Gray box vs White box pen testing methods

You may have heard of terms like "black box penetration testing", "gray box penetration testing", and "white box penetration testing". These terms are supposed to refer to the amount of information shared with testers prior to an engagement. For example, source code is "typically" provided in white box penetration tests, but not in black box pen tests.  

The problem with these terms though is that there is no strict definition of what they mean, so often as soon as a client asks if you can do a white box penetration test the immediate follow up question is, sure, but what information do you want to include? At which point, you might as well just define what information will be provided anyway without these vague terms, since no penetration test is really conducted on zero information.  

The rationale behind these terms is that you might want to understand what someone can figure out on their own without any information. That might be a valid instinct, but it's worth questioning this mindset.

There's a saying in cyber security that “obscurity is not security” - which means you shouldn't hope that something is secure just because someone doesn't know or can't guess something.

A pen-tester armed with more information might be very quickly able to tell you that something isn't secure, while an uninformed one might take way longer to figure it out. And guess what – you're the one paying for their time.

So while we wouldn't say "scenario" type pen tests are never right, it's worth thinking about why you are doing it and what you really want from the exercise. Do you want to be more secure, or do you want to see how long it takes someone to figure it out?  

A good pen-testing company will usually help guide you by asking for the information that makes the most sense for the budget you have available, or the scenario you're asking them to test.

To conclude

Penetration testing is a broad discipline that encompasses different techniques, so it is important to understand the relative risks that your organization is facing in order to choose the most appropriate type. Once you have decided on the type of pentest you want to conduct, the next logical step is to choose the right company or tool for your project; if you need help getting started, we have written a helpful guide on how to choose the right pentesting company and penetration testing tools.

At Intruder, we offer a combination of penetration testing and continuous vulnerability scanning with our automated penetration tool. Start your 14 day free trial today.

Thanks to Daniel Thatcher (Intruder Security Research Engineer)


Release Date
Level of Ideal
Comments
Before CVE details are published
🥳
Limited public information is available about the vulnerability.

Red teamers, security researchers, detection engineers, threat actors have to actively research type of vulnerability, location in vulnerable software and build an associated exploit.

Tenable release checks for 47.43% of the CVEs they cover in this window, and Greenbone release 32.96%.
Day of CVE publish
😊
Vulnerability information is publicly accessible.

Red teamers, security researchers, detection engineers and threat actors now have access to some of the information they were previously having to hunt themselves, speeding up potential exploit creation.

Tenable release checks for 17.12% of the CVEs they cover in this window, and Greenbone release 17.69%.
First week since CVE publish
😐
Vulnerability information has been publicly available for up to 1 week.

The likelihood that exploitation in the wild is going to be happening is steadily increasing.

Tenable release checks for 10.9% of the CVEs they cover in this window, and Greenbone release 20.69%.
Between 1 week and 1 month since CVE publish
🥺
Vulnerability information has been publicly available for up to 1 month, and some very clever people have had time to craft an exploit.

We’re starting to lose some of the benefit of rapid, automated vulnerability detection.

Tenable release checks for 9.58% of the CVEs they cover in this window, and Greenbone release 12.43%.
After 1 month since CVE publish
😨
Information has been publicly available for more than 31 days.

Any detection released a month after the details are publicly available is decreasing in value for me.

Tenable release checks for 14.97% of the CVEs they cover over a month after the CVE details have been published, and Greenbone release 16.23%.

With this information in mind, I wanted to check what is the delay for both Tenable and Greenbone to release a detection for their scanners. The following section will focus on vulnerabilities which:

  • Have CVSSv2 rating of 10
  • Are exploitable over the network
  • Require no user interaction

These are the ones where an attacker can point their exploit code at your vulnerable system and gain unauthorised access.

We’ve seen previously that Tenable have remote checks for 643 critical vulnerabilities, and OpenVAS have remote checks for 450 critical vulnerabilities. Tenable release remote checks for critical vulnerabilities within 1 month of the details being made public 58.4% of the time, but Greenbone release their checks within 1 month 76.8% of the time. So, even though OpenVAS has fewer checks for those critical vulnerabilities, you are more likely to get them within 1 month of the details being made public. Let’s break that down further.

In Figure 10 we can see the absolute number of remote checks released on a given day after a CVE for a critical vulnerability has been published. What you can immediately see is that both Tenable and OpenVAS release the majority of their checks on or before the CVE details are made public; Tenable have released checks for 247 CVEs, and OpenVAS have released checks for 144 CVEs. Then since 2010 Tenable have remote released checks for 147 critical CVEs and OpenVAS 79 critical CVEs on the same day as the vulnerability details were published. The number of vulnerabilities then drops off across the first week and drops further after 1 week, as we would hope for in an efficient time-to-release scenario.

Figure 10: Absolute numbers of critical CVEs with a remote check release date from the date a CVE is published

While raw numbers are good, Tenable have a larger number of checks available so it could be unfair to go on raw numbers alone. It’s potentially more important to understand the likelihood that OpenVAS or Tenable will release a check of a vulnerability on any given day after a CVE for a critical vulnerability is released. In Figure 11 we can see that Tenable release 61% their checks on or before the date that a CVE is published, and OpenVAS release a shade under 50% of their checks on or before the day that a CVE is published.

Figure 11: Percentage chance of delay for critical vulnerabilities

So, since 2010 Tenable has more frequently released their checks before or on the same day as the CVE details have been published for critical vulnerabilities. While Tenable is leading at this point, Greenbone’s community feed still gets a considerable percentage of their checks out on or before day 0.

I thought I’d go another step further and try and see if I could identify any trend in each organisations release delay, are they getting better year-on-year or are their releases getting later? In Figure 12 I’ve taken the mean delay for critical vulnerabilities per year and plotted them. The mean as a metric is particularly influenced by outliers in a data set, so I expected some wackiness and limited the mean to only checks released 180 days prior to a CVE being published and 31 days after a CVE being published. These seem to me like reasonable limits, as anything greater than 6 months prior to CVE details being released is potentially a quirk of the check details and anything after a 1-month delay is less important for us.

What can we take away from Figure 12?

  • We can see that between 2011 and 2014 Greenbone’s release delay was better than that of Tenable, by between 5 and 10 days.
  • In 2015 things reverse and for 3 years Tenable is considerably ahead of Greenbone by a matter of weeks.
  • But, then in 2019 things get much closer and Greenbone seem to be releasing on average about a day earlier than Tenable.
  • For both the trendline over an 11-year period is very close, with Tenable marginally beating Greenbone.
  • We have yet to have any data for 2021 for OpenVAS checks for critical show-stopper CVEs.
Figure 12: Release delay year-on-year (lower is better)

With the larger number of checks, and still being able to release a greater percentage of their remote checks for critical vulnerabilities Tenable could win this category. However, the delay time from 2019 and 2020 going to OpenVAS, and the trend lines being so close, I am going to declare this one a tie. It’s a tie.

The takeaway from this is that both vendors are getting their checks out the majority of the time either before the CVE details are published or on the day the details are published. This is overwhelmingly positive for both scanning solutions. Over time both also appear to be releasing remote checks for critical vulnerabilities more quickly.

Written by

Olya Osiagina

Recommended articles

Ready to get started with your 14-day trial?
try for free