What is penetration testing? The ultimate guide
What is penetration testing?
Penetration testing is a security process that simulates real-life attacks on your IT systems to find weaknesses that could be exploited by hackers.
Whether you’re trying to comply with regulations like ISO 27001, build trust with customers and suppliers, or just want to be sure your IT infrastructure is secure, penetration testing is a proven method to strengthen your cyber security posture and prevent data breaches.
Why would you use a penetration test?
There are several reasons why you could or should use penetration tests to evaluate your security, but here are some of the most common:
- Customers need proof of your security posture or accreditation with SOC 2 or ISO 27001
- You’re operating in a regulated industry such as finance or energy that require regular audits
- You want to reassure stakeholders that you’re secure from a specific attack
- You’re responsible for the security of your organization, and you want to prove that an attacker can’t bypass current controls
- You’ve been breached and want to check the access vector has been closed
- You’ve implemented new security controls and want to check they’re working
What are the different types of penetration test?
While penetration tests often use the same methodology, they vary according to the IT environment you’re trying to secure.
1. Network penetration testing
As the name suggests, a network penetration test identifies any weaknesses in your on-premises or cloud network infrastructure such as insecure configurations, encryption vulnerabilities, and missing security patches. It’s a very broad term that can cover:
- Internal infrastructure assessments
- External (internet-facing) infrastructure assessments
- Web application testing
- API testing
- Bespoke-protocol assessments
- Man-in-the-middle assessments
- SIP testing
It’s often simplified into external and internal tests. External penetration testing searches for vulnerabilities that could be exploited by an attacker via the internet. Think of it from the perspective of an "outsider". Internal penetration testing checks your internal environment and looks at how an attacker can gain a foothold into your corporate network, for example by exploiting a vulnerability in one of your internet-facing systems using social engineering (more on this later).
Generally speaking, external weaknesses pose a more serious threat. For one thing, a hacker must overcome an external security barrier before accessing your internal networks to access other systems. If you haven’t run any penetration testing before, an external test is often the best place to start, as the perimeter is the easiest thing for attackers to target. If you have trivial vulnerabilities in your internet-facing infrastructure, that’s where the hackers will start. You can read more in our guide to network penetration testing.
2. Web application penetration testing
Web application penetration testing uncovers vulnerabilities across websites and web applications, such as e-commerce platforms, content management systems, and CRM software. This reviews your entire web app security, including underlying logic and custom functionalities, to prevent data breaches.
Common vulnerabilities include database injections, cross-site scripting (XSS), and broken authentication. If you’re interested in learning more about the different types of web application weaknesses, their severity and how you can prevent them, the Open Web Application Security Project (OWASP) Top 10 is a great place to start.
Considering how many businesses now use web applications, and the valuable information they transmit and store, it is unsurprising that they are an attractive target for cyber criminals. Almost one-in-ten vulnerabilities in internet-facing applications are considered high or critical risk. For this reason alone, organizations that develop or use their own web applications should consider web application security testing.
3. Cloud penetration testing
Cloud penetration testing is designed to find security issues in your cloud services, but since you don’t actually own the cloud infrastructure, platform or software as an entity but rather as a service, there are legal and technical restrictions on what you can actually test. Certain aspects of cloud security are controlled and handled by the cloud provider and the customer is responsible for the others, based on the Service Level Agreement (SLA) between you and your cloud service provider (CSP).
For instance, the cloud provider is not responsible for custom policies you’ve configured in your cloud account. Similarly, you're not responsible for patching the underlying infrastructure used by serverless functions, or the physical security of the data centers managed by the cloud providers. As a cloud user, your focus is on ‘security in the cloud’ and not ‘security of the cloud’. What you are responsible for, and therefore what can be included in the scope of a pentest, is dictated by the shared responsibility model.
News headlines are littered with examples of large companies leaking large amounts of data, so cloud pentesting aims to identify improper storage and handling of data in the cloud. Databases, object stores, logs and repositories can all be made public inadvertently, and attackers are continually scanning the entire internet for these sorts of exposures. Our research has shown how quickly data exposed to the internet gets accessed.
Using passwords that are weak or reused frequently can make cloud accounts vulnerable to password-guessing attacks, so pentesters will check for weak credentials. Unpatched vulnerabilities are an easy point of entry for hackers, and many use automated tools to find them, so penetration testers will also check for any out-of-date, third-party software. Read more in our dedicated cloud penetration testing guide.
4. Red teaming
Red team testing and penetration tests have different objectives and outcomes – they shouldn’t be confused or combined. Penetration tests aim to identify all vulnerabilities (or as many as possible within the engagement timeframe) that exist within the systems in scope.
Conversely, a Red Team acts offensively, imitating a specific threat actor, starting with minimal internal knowledge about the target organization, with the goal of achieving a mutually agreed objective. This could be compromising a specific system on the network, carrying out a specific action on a target system, or gaining access to a specific email inbox.
The goal is not to identify every vulnerability, but to see whether a specific attack is possible, and whether the internal Blue (defensive) Team or security controls can detect, disrupt and prevent the attack.
5. Social engineering
While not a test as such, it’s worth mentioning social engineering as it is sometimes included as part of the more comprehensive penetration tests. Social engineering is designed to see if employees stick to security policies and practices. It can show how easily a hacker could convince them to break security rules to provide access to sensitive information, or by using publicly accessible open-source intelligence (as detailed in the OSINT framework). The customer should also get a better understanding of how successful their security training is, and how the organization stacks up, security-wise, compared to their peers. You can read about some infamous examples here.
Ask the expert: What does a penetration test involve?
With Intruder Security Expert, Ben Marr
How are point-in-time and continuous scans carried out?
Point-in-time assessments are set for a fixed period, determined in the initial scoping of works. This will ascertain the size of the application and complexity, and provide an estimate of the time and effort required to give complete coverage of the application. When it comes to the engagement itself, the test will go through a series of phases. At a basic level, these include information gathering, enumeration, active testing, exploitation of identified issues, and reporting.
Continuous testing is done cyclically throughout the year, rather than just once. This, in my opinion, gives a much better representation of the target’s security posture as your business operates and grows. The engagement will follow the same phases and perform the same level of testing, but importantly will also include those new features and/or systems which have been created. With a point-in-time assessment, these wouldn’t get tested until the next annual test, potentially leaving you vulnerable to weaknesses for a very long time.
What’s the difference between black, gray or white box testing?
These refer to the amount of information shared with the tester before the engagement. In a white-box assessment, the tester will be given enough information to help them perform the test as thoroughly as possible, such as source code, schematics of the systems, and so forth.
A black box assessment is where no information is shared prior to the test beyond the scope and anything needed for authorization. This most accurately simulates what an attacker will be facing. In my opinion, for a penetration test where the testing window is finite and short, this isn’t the best way to go. It restricts the value of the pentest by arbitrarily making it more difficult for the tester to ensure that they have given the target adequate coverage.
Gray box assessments are where most penetration tests fall. Here, some information is shared before the engagement that will help the test to run smoothly. This could be the tester being walked through the capabilities of the application or given a demo, accounts to authenticate, or some system schematics.
The most important thing, regardless of the level of information given, is that the tester has a point of contact who knows the project well and has technical knowledge of the target. This means problems and issues can be resolved quickly, resulting in a more accurate and thorough test, and ultimately a more useful report.
What do testers look for?
While the tester is running through the phases of the engagement, they will be looking for anything present within the target(s) that can be a tangible risk. Where that risk could affect either the confidentiality, integrity or availability of the target. This could be as simple as a identifying an exposed development API which lacks authentication that's exposed to the internet, or as involved as extracting data from an uploaded Git repository and identifying vulnerabilities within the extracted code that may not have been identified through normal testing. The most important thing is that any risk that's identified is relevant to the target and to the client’s business.
How can you be sure there's no risk to my environment?
During a penetration test, there will always be some level of risk to the environment. However, it's important that the risk is managed throughout the engagement cycle. Firstly, by agreeing what types of testing are included/excluded within the rules of engagement, agreeing the target(s) which are in scope, and maintaining an open dialogue throughout the engagement so that everyone is clear and voices any concerns. Where possible, testing should be conducted in an environment that mirrors the live environment but is segregated to prevent putting users at risk.
Even with the best intentions, the worst can still happen. For example, while testing a banking app in a staging environment, I inadvertently took down the production banking system by inputting the characters “AAAA”. An oversight by the developers missed key validations on their API, and the back end wasn’t sandboxed sufficiently. This meant testing data could reach production without the testers and application owner knowing. As soon as the app owner noticed, we agreed to immediately de-scope the affected area to prevent further availability issues. This meant we could continue testing and eliminate any further outages.
This shows why it’s so important to have an open dialogue and do as much as you can to manage any risk to the business, so that when issues like this occur, they can be dealt with immediately.
What do pen testers find that an automated web app scanner could miss?
Broadly speaking, no two web applications are the same because they can be built entirely differently. This means that a risk for one application may not be a risk for another, and this makes it incredibly difficult to automate effectively. Humans understand nuance, and are also very good at identifying abnormalities that would get ignored by tooling which mostly relies on binary yes/no observations.
A simple example would be an exposed API endpoint. A scanner would see that there is an endpoint which is available and attempt to test for various vulnerabilities, but it misses the important context of: should it be available? A human would look at this, and then figure out if it affects the target’s security posture, or if this information could be used elsewhere for a more targeted and effective attack.
What kind of report can I expect?
Reports are traditionally created as a standalone document and distributed as a PDF after the engagement. Each company has their own reporting style dictating the flow and look of a document. However, they will all include key information, such as the agreed scope, methodologies used, a high-level summary of the target’s posture, and an itemised list of identified security shortfalls.
This will include a rating of the vulnerabilities, a description of what was identified including evidence where appropriate, and most importantly – remediation advice. Pentesting vendors should have a sample report so you can check that it meets your reporting needs.
What’s next for penetration testing?
By Product Lead, Andy Hornegold
Automation will continue to improve and enhance penetration testing as cyber security becomes more complex, and there’s a big shift towards application-layer security testing as people hand responsibility for underlying infrastructure to cloud-providers.
The big question is how AI will impact penetration testing, but it’s hard to say for sure at this point. Cyber security vendors have been using machine learning for some time now, but haven’t been able to produce positive use cases in penetration testing (although it has been useful for Blue Team activities).
In general, the training data hasn’t been available for penetration testing to build a good
model to conduct a full penetration test, since the environments are often very different. Whether AI is applied also raises ethical questions – penetration testing is an offensive security function and handing decision making over to an automated process should be considered carefully and treated with caution.
Like a golf caddy helping a player choose the right clubs and offering his or her expertise and wisdom, hopefully AI will complement rather than replace consultants, and help those responsible for cyber security to react more quickly to changing environments. Time will tell.
How Intruder can help
Pentesting encompasses so many different techniques, it’s important to understand what risks you’re facing and what you hope to achieve before you choose the most appropriate type, tool and tester. If you need more help to get started, we’ve also got handy guides on how to choose the right pentesting company and the best penetration testing tools.
Penetration tests can seem expensive even if they’re only done once or twice a year, so you may wonder if they’re worth the cost, or if there are cheaper ways of doing them – including if they can be automated. While it’s not possible to fully automate a penetration test yet (some manual tasks will always need to be done by skilled professionals with expert knowledge and intuition), it’s also impossible for humans to manually check for every vulnerability – there are simply too many.
This is where continuous vulnerability scanning can help. These automated tools schedule regular scans, rapidly test for many thousands of weaknesses, and notify you of any issues – which is why scanners are a critical tool in the pentesters armory.
Intruder offers a combination of penetration testing and automated vulnerability scanning so you can protect your digital systems with proven, comprehensive security solutions.
If you’d like to see Intruder in action, take our Pro plan for a spin with a free 14-day trial or get a demo of our Premium plan. With Premium, your attack surface is scanned every day to check if new services have gone live or offline. If the scan identifies a new target, a licence will be assigned (if available) and the target will automatically be scanned for vulnerabilities. If you don’t have a licence, we’ll let you know. After every scan, your dashboard will be updated with a summary and any new results. You can also add bug hunting to your plan to pit your external targets against our team of penetration testers. Pro or Premium – we’ve got you covered.
- Raw CVE Coverage
- Risk Rating Coverage
- Remote Check Types
- Check Publication Lead Time
- Local/Authenticated vs Remote Check Prioritisation
- Software Vendor & Package Coverage
- Headline Vulnerabilities of 2021 Coverage
- Analysis Decisions
Red teamers, security researchers, detection engineers, threat actors have to actively research type of vulnerability, location in vulnerable software and build an associated exploit.
Tenable release checks for 47.43% of the CVEs they cover in this window, and Greenbone release 32.96%.
Red teamers, security researchers, detection engineers and threat actors now have access to some of the information they were previously having to hunt themselves, speeding up potential exploit creation.
Tenable release checks for 17.12% of the CVEs they cover in this window, and Greenbone release 17.69%.
The likelihood that exploitation in the wild is going to be happening is steadily increasing.
Tenable release checks for 10.9% of the CVEs they cover in this window, and Greenbone release 20.69%.
We’re starting to lose some of the benefit of rapid, automated vulnerability detection.
Tenable release checks for 9.58% of the CVEs they cover in this window, and Greenbone release 12.43%.
Any detection released a month after the details are publicly available is decreasing in value for me.
Tenable release checks for 14.97% of the CVEs they cover over a month after the CVE details have been published, and Greenbone release 16.23%.
With this information in mind, I wanted to check what is the delay for both Tenable and Greenbone to release a detection for their scanners. The following section will focus on vulnerabilities which:
- Have CVSSv2 rating of 10
- Are exploitable over the network
- Require no user interaction
These are the ones where an attacker can point their exploit code at your vulnerable system and gain unauthorised access.
We’ve seen previously that Tenable have remote checks for 643 critical vulnerabilities, and OpenVAS have remote checks for 450 critical vulnerabilities. Tenable release remote checks for critical vulnerabilities within 1 month of the details being made public 58.4% of the time, but Greenbone release their checks within 1 month 76.8% of the time. So, even though OpenVAS has fewer checks for those critical vulnerabilities, you are more likely to get them within 1 month of the details being made public. Let’s break that down further.
In Figure 10 we can see the absolute number of remote checks released on a given day after a CVE for a critical vulnerability has been published. What you can immediately see is that both Tenable and OpenVAS release the majority of their checks on or before the CVE details are made public; Tenable have released checks for 247 CVEs, and OpenVAS have released checks for 144 CVEs. Then since 2010 Tenable have remote released checks for 147 critical CVEs and OpenVAS 79 critical CVEs on the same day as the vulnerability details were published. The number of vulnerabilities then drops off across the first week and drops further after 1 week, as we would hope for in an efficient time-to-release scenario.
While raw numbers are good, Tenable have a larger number of checks available so it could be unfair to go on raw numbers alone. It’s potentially more important to understand the likelihood that OpenVAS or Tenable will release a check of a vulnerability on any given day after a CVE for a critical vulnerability is released. In Figure 11 we can see that Tenable release 61% their checks on or before the date that a CVE is published, and OpenVAS release a shade under 50% of their checks on or before the day that a CVE is published.
So, since 2010 Tenable has more frequently released their checks before or on the same day as the CVE details have been published for critical vulnerabilities. While Tenable is leading at this point, Greenbone’s community feed still gets a considerable percentage of their checks out on or before day 0.
I thought I’d go another step further and try and see if I could identify any trend in each organisations release delay, are they getting better year-on-year or are their releases getting later? In Figure 12 I’ve taken the mean delay for critical vulnerabilities per year and plotted them. The mean as a metric is particularly influenced by outliers in a data set, so I expected some wackiness and limited the mean to only checks released 180 days prior to a CVE being published and 31 days after a CVE being published. These seem to me like reasonable limits, as anything greater than 6 months prior to CVE details being released is potentially a quirk of the check details and anything after a 1-month delay is less important for us.
What can we take away from Figure 12?
- We can see that between 2011 and 2014 Greenbone’s release delay was better than that of Tenable, by between 5 and 10 days.
- In 2015 things reverse and for 3 years Tenable is considerably ahead of Greenbone by a matter of weeks.
- But, then in 2019 things get much closer and Greenbone seem to be releasing on average about a day earlier than Tenable.
- For both the trendline over an 11-year period is very close, with Tenable marginally beating Greenbone.
- We have yet to have any data for 2021 for OpenVAS checks for critical show-stopper CVEs.
With the larger number of checks, and still being able to release a greater percentage of their remote checks for critical vulnerabilities Tenable could win this category. However, the delay time from 2019 and 2020 going to OpenVAS, and the trend lines being so close, I am going to declare this one a tie. It’s a tie.
The takeaway from this is that both vendors are getting their checks out the majority of the time either before the CVE details are published or on the day the details are published. This is overwhelmingly positive for both scanning solutions. Over time both also appear to be releasing remote checks for critical vulnerabilities more quickly.