With global cyber crime costs expected to reach $10.5 trillion annually by 2025, it comes as little surprise that the risk of attack ...
back to BLOG

Mind the gap – how to ensure your vulnerability detection methods are up to scratch

Andy Hornegold

With global cyber crime costs expected to reach $10.5 trillion annually by 2025, it comes as little surprise that the risk of attack is companies’ biggest concern globally. To help businesses uncover and fix the vulnerabilities and misconfigurations affecting their systems, there is an (over)abundance of solutions available.

But be aware, they may not give you a full and continuous view of your weaknesses if used in isolation. With huge financial gains to be had from each successful breach, hackers do not rest in their hunt for flaws and use a wide range of tools and scanners to help them in their search. Beating these criminals means staying one step ahead and using the most comprehensive and responsive vulnerability detection support you can.

We’ll go through each solution and explain how you can maintain your vigilance.  Of course, vulnerability management is just one step businesses must take to prevent a breach; there’s also proper asset management, employee training and incident response to consider but this article will cover scanning and penetration testing specifically.

Vulnerability scanning

A vulnerability scanner checks your systems for security flaws that can be used to steal data or sensitive information, or generally cause disruption to your business. Depending on your needs, you can deploy scanners to keep an eye on any area of your system from your external or internal infrastructure to your web apps and endpoints, as well as any authenticated or unauthenticated areas of your website.

They do have their limitations however.

Firstly, vulnerability scans can only ever report on what they find in the moment. If you’re running them infrequently then you could easily miss new vulnerabilities that have been introduced in between scans. This is why it’s important to ensure you have a vulnerability management solution in place that can give you continuous visibility of your systems and help you to prioritise and fix any security issues.

And with some scanning vendors there can also be a bit of a waiting game to play while they release checks for new vulnerabilities. This often happens when an exploit is made public before a vulnerability's details are. Fortunately, some solutions – such as Intruder Vanguard - take a faster, more proactive approach, finding the proof-of-concept exploit, breaking it down, and then checking all its customers - often before the scanning vendors have started their checks.

The other challenge with some vulnerability scanning tools is that they are often not tailored to your business and security posture. This is because most have to be generic so that they can be applied to any environment. Vulnerability scanners find it hard to handle bespoke/custom services or applications because they haven't seen them before and therefore cannot extract meaningful results.  As a result, they can often produce false positives, which in turn can lead to wasted time and resources trying to fix non-existent issues.

To avoid this, you need a solution which takes into account your specific environment i.e. the types of systems you have deployed, the configuration of these systems, the data stored within them, and the mitigating controls you have in place. Then, it needs to use this information to ensure it only presents you with issues that have a tangible impact on your security.

How can you make that possible? By adding human expertise.

While a scan will find a vulnerability and report it, it won’t carry out a full "impact review" to show what the actual risk is of someone successfully exploiting the vulnerability.  Penetration tests, however, will.

Penetration tests

A penetration test (also known as a pen test) is a simulated cyber attack carried out by ethical hackers on your systems to identify vulnerabilities that could be exploited by malicious attackers. This helps you to understand not only what needs to be fixed but also the potential impact of an attack on your business.

However, there are major issues with using this as your sole vulnerability detection method.

Firstly, while in depth, penetration testing only covers a point in time. With 20 new vulnerabilities identified every day, your penetration test results are likely to be out of date as soon you receive the report.

Not only that but reports can take as long as six months to produce because of the work involved, as well as several months to digest and action.

They can be very expensive - often costing thousands of pounds each time.

With hackers finding more sophisticated methods to break into your systems, what is the best modern solution to keep you one step ahead?

A hybrid of vulnerability scanning and penetration testing

In order to gain the most comprehensive picture of your security posture, you need to combine automated vulnerability scanning and human-led penetration testing.

Intruder Vanguard does just that, bringing security expertise and continuous coverage together to find what other scanners can’t. It fills the gap between traditional vulnerability management and point in time penetration tests, to provide a continuous watch over your systems. With the world's leading security professionals on hand, they’ll probe deeper, find more vulnerabilities, and provide advisories on their direct impact on your business to help you keep attackers at bay.

The threat of attack is rising, don't leave yourself vulnerable. Choose continuous, comprehensive coverage from Intruder Vanguard.

Release Date
Level of Ideal
Before CVE details are published
Limited public information is available about the vulnerability.

Red teamers, security researchers, detection engineers, threat actors have to actively research type of vulnerability, location in vulnerable software and build an associated exploit.

Tenable release checks for 47.43% of the CVEs they cover in this window, and Greenbone release 32.96%.
Day of CVE publish
Vulnerability information is publicly accessible.

Red teamers, security researchers, detection engineers and threat actors now have access to some of the information they were previously having to hunt themselves, speeding up potential exploit creation.

Tenable release checks for 17.12% of the CVEs they cover in this window, and Greenbone release 17.69%.
First week since CVE publish
Vulnerability information has been publicly available for up to 1 week.

The likelihood that exploitation in the wild is going to be happening is steadily increasing.

Tenable release checks for 10.9% of the CVEs they cover in this window, and Greenbone release 20.69%.
Between 1 week and 1 month since CVE publish
Vulnerability information has been publicly available for up to 1 month, and some very clever people have had time to craft an exploit.

We’re starting to lose some of the benefit of rapid, automated vulnerability detection.

Tenable release checks for 9.58% of the CVEs they cover in this window, and Greenbone release 12.43%.
After 1 month since CVE publish
Information has been publicly available for more than 31 days.

Any detection released a month after the details are publicly available is decreasing in value for me.

Tenable release checks for 14.97% of the CVEs they cover over a month after the CVE details have been published, and Greenbone release 16.23%.

With this information in mind, I wanted to check what is the delay for both Tenable and Greenbone to release a detection for their scanners. The following section will focus on vulnerabilities which:

These are the ones where an attacker can point their exploit code at your vulnerable system and gain unauthorised access.

We’ve seen previously that Tenable have remote checks for 643 critical vulnerabilities, and OpenVAS have remote checks for 450 critical vulnerabilities. Tenable release remote checks for critical vulnerabilities within 1 month of the details being made public 58.4% of the time, but Greenbone release their checks within 1 month 76.8% of the time. So, even though OpenVAS has fewer checks for those critical vulnerabilities, you are more likely to get them within 1 month of the details being made public. Let’s break that down further.

In Figure 10 we can see the absolute number of remote checks released on a given day after a CVE for a critical vulnerability has been published. What you can immediately see is that both Tenable and OpenVAS release the majority of their checks on or before the CVE details are made public; Tenable have released checks for 247 CVEs, and OpenVAS have released checks for 144 CVEs. Then since 2010 Tenable have remote released checks for 147 critical CVEs and OpenVAS 79 critical CVEs on the same day as the vulnerability details were published. The number of vulnerabilities then drops off across the first week and drops further after 1 week, as we would hope for in an efficient time-to-release scenario.

Figure 10: Absolute numbers of critical CVEs with a remote check release date from the date a CVE is published

While raw numbers are good, Tenable have a larger number of checks available so it could be unfair to go on raw numbers alone. It’s potentially more important to understand the likelihood that OpenVAS or Tenable will release a check of a vulnerability on any given day after a CVE for a critical vulnerability is released. In Figure 11 we can see that Tenable release 61% their checks on or before the date that a CVE is published, and OpenVAS release a shade under 50% of their checks on or before the day that a CVE is published.

Figure 11: Percentage chance of delay for critical vulnerabilities

So, since 2010 Tenable has more frequently released their checks before or on the same day as the CVE details have been published for critical vulnerabilities. While Tenable is leading at this point, Greenbone’s community feed still gets a considerable percentage of their checks out on or before day 0.

I thought I’d go another step further and try and see if I could identify any trend in each organisations release delay, are they getting better year-on-year or are their releases getting later? In Figure 12 I’ve taken the mean delay for critical vulnerabilities per year and plotted them. The mean as a metric is particularly influenced by outliers in a data set, so I expected some wackiness and limited the mean to only checks released 180 days prior to a CVE being published and 31 days after a CVE being published. These seem to me like reasonable limits, as anything greater than 6 months prior to CVE details being released is potentially a quirk of the check details and anything after a 1-month delay is less important for us.

What can we take away from Figure 12?

Figure 12: Release delay year-on-year (lower is better)

With the larger number of checks, and still being able to release a greater percentage of their remote checks for critical vulnerabilities Tenable could win this category. However, the delay time from 2019 and 2020 going to OpenVAS, and the trend lines being so close, I am going to declare this one a tie. It’s a tie.

The takeaway from this is that both vendors are getting their checks out the majority of the time either before the CVE details are published or on the day the details are published. This is overwhelmingly positive for both scanning solutions. Over time both also appear to be releasing remote checks for critical vulnerabilities more quickly.

Written by

Andy Hornegold

Recommended articles

Ready to get started with your 14-day trial?
try for free