Eliminate visibility gaps and make issues actionable for your security and dev team.
back to BLOG

Gotta catch em all: Bug hunting explained

James Harrison

Flaws in your software or applications may just be functional – or they could become a serious security issue. That’s why we offer a unique bug hunting service to add to your arsenal and help weed out these security flaws.

Not all bugs are created equal – only some bugs will create a vulnerability that can be exploited by hackers, such as by compromising user authentication, authorization of access rights and privileges, or the confidentiality of your data. Security testers and bug hunters look for these types of flaws. Think of it like this: all men are humans, but not all humans are men; all security bugs are defects, not all defects are security bugs.

What does Intruder hunt for?

The types of vulnerabilities detected by our Bug Hunting service includes everything from simple misconfigurations which leave sensitive data exposed, to exploiting complex attack chains which attackers could use to gain control over your systems.

How does bug hunting find these flaws?

Bug hunting looks to find these flaws by continuously testing your external infrastructure and web applications over the term of your agreed contract in a similar way to penetration testing, in that it finds serious weaknesses which scanners alone might not detect.

But unlike a traditional point-in-time pen test, bug hunting is continuous. Bug hunting runs over the term of a contract, often paid for per day, while penetration testing is a scheduled, one-off test of your digital security to identify weaknesses and vulnerabilities. For example, at Intruder if you have 10 days of bug hunting on account, we’ll hunt for these security vulnerabilities spread across the year.

Do I need bug hunting as well as a penetration test?

With a single penetration test, the analysis of your security could be out of date within a day as new vulnerabilities are released so frequently. If one of your developers makes a change or reconfiguration to you application, as soon as a new vulnerability comes out you can’t be sure if it’s still secure. Bug hunting is ongoing and continuous, using the latest techniques as and when they emerge, so your application is checked and tested across the year.

Can I choose what I want tested?

Bug hunting can be more efficient than traditional penetration testing, only looking for exploitable vulnerabilities with proven impact. For example, if you've got a SQL injection, what does that mean for you? Bug hunting goes further – if it finds a SQL injection, it will prove it’s exploitable by extracting sensitive data from the database in the same way as an attacker would. Only then will it be reported when the impact has been proven. If the SQL injection can be used to extract information, it's very serious and impact has been proven, and we’ll report it as an advisory.

Do I need bug hunting if I use a scanner?

Bug hunting is designed to find secret but serious weaknesses which scanners alone can’t detect. For example, authorization weaknesses are typically very difficult for scanners to detect because they don’t have the business context needed to recognize that the data in a particular response is sensitive. But the human eye can detect these weaknesses.

Who does the bug hunting?

Bug hunting is done by qualified penetration testers in accordance with the certification bodies we work with such as Crest and Offensive Security.

Why is this important?

Bug hunting won’t report on theoretical vulnerabilities with tenuous exploits with no public information. Although we’ll test and look for them, we’ll only report the ones that we can prove with an exploit. Bug hunting is also efficient because it only looks for high and critical impact weaknesses. So, no time is wasted reporting on things like missing headers and weak ciphers, which you’ll probably already know about from the scanner.

Ask an expert: Bug hunting in action

With Dan Andrew, Intruder Head of Security

Bug Hunting can help you prioritize fixes on the most critical of your vulnerabilities first, and it can discover weaknesses which you don’t even know about, by employing penetration testing techniques which automated scanners alone can’t find. Perhaps the best way to explain Bug Hunting is with an example...

Imagine you have a large digital estate. You’re under-resourced. Your team has to prioritize the most critical weaknesses to fix first, because vulnerabilities crop up faster than you can fix the ones you already have. Hidden amongst the many weaknesses in products you use is one which allows attackers to exploit a Local File Inclusion weakness from an unauthenticated perspective, but only in certain application configurations.

Our Bug Hunting team steps in and discovers the weakness on a target in a vulnerable configuration, and they exploit the weakness to prove it can practically be exploited, and not just in theory. So, they prove that attackers can access a local configuration file which contains credentials for one of your databases, and they gain access to the database using these credentials, which contains sensitive information on one of your customers.

Once impact is proven, the weakness is reported to you as a critical risk advisory – now you can focus on a fix for a vulnerability that you now know needs prioritizing above others, because it’s proven to be exploitable and the impact is clear and high risk.

Critical issues and advisories clearly displayed on the Intruder dashboard

Need to know

What is it?

Our bug hunting pits your external targets against our team of expert penetration testers to try and find weaknesses and exposures. We focus on finding the high impact attack chains that could have significant impact on your business if not found and fixed.

What’s included?

How is it different from a penetration test?

What do you focus on?

This is completely up to you – whatever you think will help us discover any weaknesses.

How do you report?

We send out advisories for any security issues uncovered during the process. If no issues are found, we’ll let you know too.

Who’s it for?

Bug hunting is a bolt-on service available to Premium and Vanguard customers, booked by the day. It’s a hybrid service, which means it’s a blend of vulnerability management and pen testing rolled into one. You’ll see the output from bug hunting advisories alongside any issues discovered by the Intruder scanner.

If you want more information about Bug Hunting or the other features available with our Premium or Vanguard plans, check out our pricing page or contact our Sales team.

Release Date
Level of Ideal
Before CVE details are published
Limited public information is available about the vulnerability.

Red teamers, security researchers, detection engineers, threat actors have to actively research type of vulnerability, location in vulnerable software and build an associated exploit.

Tenable release checks for 47.43% of the CVEs they cover in this window, and Greenbone release 32.96%.
Day of CVE publish
Vulnerability information is publicly accessible.

Red teamers, security researchers, detection engineers and threat actors now have access to some of the information they were previously having to hunt themselves, speeding up potential exploit creation.

Tenable release checks for 17.12% of the CVEs they cover in this window, and Greenbone release 17.69%.
First week since CVE publish
Vulnerability information has been publicly available for up to 1 week.

The likelihood that exploitation in the wild is going to be happening is steadily increasing.

Tenable release checks for 10.9% of the CVEs they cover in this window, and Greenbone release 20.69%.
Between 1 week and 1 month since CVE publish
Vulnerability information has been publicly available for up to 1 month, and some very clever people have had time to craft an exploit.

We’re starting to lose some of the benefit of rapid, automated vulnerability detection.

Tenable release checks for 9.58% of the CVEs they cover in this window, and Greenbone release 12.43%.
After 1 month since CVE publish
Information has been publicly available for more than 31 days.

Any detection released a month after the details are publicly available is decreasing in value for me.

Tenable release checks for 14.97% of the CVEs they cover over a month after the CVE details have been published, and Greenbone release 16.23%.

With this information in mind, I wanted to check what is the delay for both Tenable and Greenbone to release a detection for their scanners. The following section will focus on vulnerabilities which:

These are the ones where an attacker can point their exploit code at your vulnerable system and gain unauthorised access.

We’ve seen previously that Tenable have remote checks for 643 critical vulnerabilities, and OpenVAS have remote checks for 450 critical vulnerabilities. Tenable release remote checks for critical vulnerabilities within 1 month of the details being made public 58.4% of the time, but Greenbone release their checks within 1 month 76.8% of the time. So, even though OpenVAS has fewer checks for those critical vulnerabilities, you are more likely to get them within 1 month of the details being made public. Let’s break that down further.

In Figure 10 we can see the absolute number of remote checks released on a given day after a CVE for a critical vulnerability has been published. What you can immediately see is that both Tenable and OpenVAS release the majority of their checks on or before the CVE details are made public; Tenable have released checks for 247 CVEs, and OpenVAS have released checks for 144 CVEs. Then since 2010 Tenable have remote released checks for 147 critical CVEs and OpenVAS 79 critical CVEs on the same day as the vulnerability details were published. The number of vulnerabilities then drops off across the first week and drops further after 1 week, as we would hope for in an efficient time-to-release scenario.

Figure 10: Absolute numbers of critical CVEs with a remote check release date from the date a CVE is published

While raw numbers are good, Tenable have a larger number of checks available so it could be unfair to go on raw numbers alone. It’s potentially more important to understand the likelihood that OpenVAS or Tenable will release a check of a vulnerability on any given day after a CVE for a critical vulnerability is released. In Figure 11 we can see that Tenable release 61% their checks on or before the date that a CVE is published, and OpenVAS release a shade under 50% of their checks on or before the day that a CVE is published.

Figure 11: Percentage chance of delay for critical vulnerabilities

So, since 2010 Tenable has more frequently released their checks before or on the same day as the CVE details have been published for critical vulnerabilities. While Tenable is leading at this point, Greenbone’s community feed still gets a considerable percentage of their checks out on or before day 0.

I thought I’d go another step further and try and see if I could identify any trend in each organisations release delay, are they getting better year-on-year or are their releases getting later? In Figure 12 I’ve taken the mean delay for critical vulnerabilities per year and plotted them. The mean as a metric is particularly influenced by outliers in a data set, so I expected some wackiness and limited the mean to only checks released 180 days prior to a CVE being published and 31 days after a CVE being published. These seem to me like reasonable limits, as anything greater than 6 months prior to CVE details being released is potentially a quirk of the check details and anything after a 1-month delay is less important for us.

What can we take away from Figure 12?

Figure 12: Release delay year-on-year (lower is better)

With the larger number of checks, and still being able to release a greater percentage of their remote checks for critical vulnerabilities Tenable could win this category. However, the delay time from 2019 and 2020 going to OpenVAS, and the trend lines being so close, I am going to declare this one a tie. It’s a tie.

The takeaway from this is that both vendors are getting their checks out the majority of the time either before the CVE details are published or on the day the details are published. This is overwhelmingly positive for both scanning solutions. Over time both also appear to be releasing remote checks for critical vulnerabilities more quickly.

Written by

James Harrison

Recommended articles

Ready to get started with your 14-day trial?
try for free