Vulnerability Assessment versus Penetration Testing

Vulnerability Assessment v Penetration Testing, George Stewart tries to clear up the confusion.



Even in professional circles, there often seem to be differences of opinion over the value of Penetration Testing and Vulnerability Assessment. This can lead to gigs with over-prescribed scopes, resulting in poor value to the client. Let’s talk about the difference, why each is important and, hopefully, help you choose between the two in future.

What's the difference?

Vulnerability Assessment and Penetration Testing are basically two different ways of detecting weaknesses in an organisation’s security systems. In turn, they have different sets of pros and cons and come from an opposing stance.

Let’s use the analogy of testing a home security system to illustrate the difference…

In Vulnerability Assessments, the focus is on breadth over depth. The aim is to find all the problems in a system’s security, any deviations from best practice and make recommendations – but not to assess the potential impact. When assessing a home security system you might check that the windows have bars that aren’t rusty, there’s a dog out the back that barks at strangers, security cameras inside the house and that the back-to-base alarm system triggers an alarm to your security company.

Penetration Testing is the opposite: the focus is on depth over breadth. You’re the outsider and you want to see if you can get in. You don’t necessarily try to find all the security holes, only those that get you closer to your final goal. In our home security example, your objective is to steal the TV. You ignore the rusty bars, drug the dog, jump the back fence, wear a balaclava and take the TV before the security guards arrive.

Pen Testing is especially useful for testing your internal networks, as a Vulnerability Assessment will find so many holes that it’s unlikely you’ll be able to plug them all – so you need to be in the position of knowing the priorities to fix. Pen Testing finds holes that a Vulnerability Assessment won’t, because it’s not limited to technology factors; it looks at the way systems are actually used.

When taking a holistic view of security, Pen Testing can involve social engineering: looking not only at a system, but how people are using and managing it. Given appropriate board-level approval, Pen Testers will pit themselves against the organisation and its employees – trying to exploit human error, snafus and mere opportunity. These engagements provide the ultimate test of whether your security is up to scratch, and offer insight into systemic issues facing your organisation.

Different strategies and mindsets

Each exercise calls for a different strategy and mindset. Vulnerability Assessments are mostly done annually under a typical security policy. They are always done the same way: thoroughly, methodical, holistically. Pen Tests, on the other hand, call for thinking outside the box: ingenuity, daring, imagination. Plus, by their very nature, they can only address limited potential targets.

Pen Testing makes many organisations nervous. It can result in red faces: lackadaisical work practices on the part of the IT department; insufficient spending on security by the business. But Pen Tests shouldn’t be circumscribed; imagine asking a security expert to review your home for insurance purposes then tell them they can only try to get in through the steel front door...

Would you rather be a boring, methodical vulnerability assessor in a grey cardigan, or an intrepid, outrageous member of a Penetration Testing Tiger Team? Just goes to show it takes all sorts!

About the Author

George Stewart is the Team Leader for our Assurance business unit, where he manages many of our security penetration / assurance testing engagements.

He is a Crest Registered Tester (CRT) and has a degree in Software Engineering with Honours from the Australian National University. He is also experienced in audit being a registered PCI-DSS QSA and having previously held an IRAP assessor certification, has completed assessments or provided advice for a variety of government and national organisations, Internet service providers and payment gateways.