April 5, 2021

The Cyber Rants Podcast

Episode #23 - Penetration Testing: What You Need to Know (Part 4)

This week in the final part of our Penetration Test Mini-Series, the guys discuss the realities of automated vs. manual penetration tests and what those terms actually mean. They also talk about timeframes, approaches, and situations that seem to cause some confusion for companies undergoing their first penetration test. The team also gets into Virtual CISO Penetration Testing and the VCISO Pen Testing Requirements.

Pick up your copy of Cyber Rants on Amazon.
Looking to take your Cyber Security to the next level? Visit us at www.silentsector.com.
Be sure to rate the podcast, leave us a review, and subscribe!

welcome to the cyber rants podcast where we're all about sharing the forbidden secrets and slightly

embellished truths about corporate cyber security programs we're ranting we're raving and we're

telling you the stuff that nobody talks about on their fancy website and trade show giveaways

all to protect you from cyber criminals and now here's your hosts mike rotondo

zach fuller and lauro chavez hello and welcome to the cyber ants podcast this is your co-host

zach fuller joined by mike rotondo and lauro chavez and we are finishing up our series today

on penetration testing hope you've learned some from the previous episodes i've talked a lot about

different types of pen test qualities factors and considerations so we will

continue to dive a little bit deeper today and as such we are going to do a very

brief overview of the news and exploits so we can dive right into the topic of penetration testing

mike zach so just got four quick headlines for you today first headline is that billions of records

have been hacked already make cyber security a priority or risk disaster warns an analyst

basically there's been 55 billion data records that have been compromised since tok since 2005

however in the last year alone for 2021 31 billion data records have been compromised that's up

171 percent from the previous year so it's getting worse new bugs could let hackers bypass spectre

attack mitigations on linux systems they found two new vulnerabilities on linux-based operating

systems that if excess if successful exploited could let attackers circumvent mitigations for

speculative speculative attacks such as spectre uh their cve 2020 27 170 and cve 2020 27 171

this i put in here because i couldn't resist ziggy ransomware admin announced it will refund

the victims who paid the ransom victims that paid the ransom should contact you by email yeah

at ziggy ransomware at sec mail.pro to be refunded in about two weeks

victims have to provide the payment receipt and the computer's unique id

so yeah i never thought i'd see that anyway microsoft exchange attacks increase

while wannacry gets a restart yup wannacry's back company saw 57 rise in ransomware attacks over

the past six months at a global level there's a constant monthly increase of nine percent since

the beginning of the year we thought 2020 was bad from a uh your perspective looks like 2021

is going to be bad from a tech perspective as far as attacks go don't make you want to cry exactly

i cannot believe they're giving money back i don't know if this change of heart should

warrant some kindness from us but maybe not um for uh for exploits this week we've just got one

the vs ftpd uh product that offers secure and fast ftp server for unix-like systems has a

remote denial of service which is really interesting so there is a really really

cool python 3 script that was developed and it will video us your ftp service so make sure if

you've got vsfpd running that you're upgrading to version 304 or greater and that is it for exploits

this week great thank you both well i'm excited about my refund uh that's i we've heard about

we've heard about ransomware customer service improving over time but this is this is next level

yeah do you want your money back in monopoly money or would you prefer

cash you want boomer paper i'm just kidding that they they want the receipt

you know so but you know the other part of the article when i wanted to say that you know they're

not being all that altruistic basically they got paid in bitcoin and bitcoins value has gone up

so much so like yeah we can refund the original amount and take the skim off the top so basically

sure they still make money yeah i'm i'm still i still don't don't believe it but we'll see well

well let's talk about pen testing and answer everybody's burning questions how does that sound

sure let's put some fire out yeah yeah we've uh uh covered it a lot over the last few episodes as far

as just considerations and such go and i think behind all this people are probably starting to

see that you could look at it as kind of a good better best approach right so there's there's

no one-size-fits-all there's not necessarily a right answer um and people are going to try to

sell you all kinds of stuff so um that being said i can i can say with almost certainty that that

that kind of commoditized rinse repeat canned approach is most the time not very beneficial

but then beyond that there are varying levels so one of the one of the questions that we get a lot

is around manual penetration testing versus automated and i think a lot of people from

outside the industry at no no fault of their own but they've heard this this thing about oh your

pen test needs to be manual and so they ask oh is it manual or what percentage of manual work is

done without fully understanding what that means um of course there's a bunch of great information

about this in the book um but lauro do you want to dive in a little bit and kind of explain your

your theory on this and what's what and is there a better option uh on one side or the other

sure zach i i think there's a lot of convolution around these these terms they're getting i think

they get thrown around you know indiscretionally and and it doesn't it doesn't help anybody

so so let's let's talk about when when someone asks you if you're if you're a cybersecurity

professional or you're owning a business you're getting a questionnaire or something and they said

did you do a manual test or a automated test what they probably are asking you is did you

get a a tool to run a series of tests or you know a scan basically did you get a

an automated scan from one of your tools or technologies or did you actually assign a third

party with a human resource that had tools and techniques at their disposal in order to

conduct the test i think that's really where the muster meets the brand right i think when

when you get when someone gets questioned that when i say hey did you do a manual on automated

i think that's probably what they're trying to ascertain is did you get a printout from

a tool or did you actually have a knowledgeable accredited certified practitioner conduct the test

now let's let's look at the terms in general how they apply to penetration testing as

as the the the job would entail right from from my perspective and and we talk we do talk about

this in the book um i think it's it's it's quite comical to me so i'll try to delete my humor

aside for the moment but let's i always and in the book i talk about this and and i i also use the um

the analogy of of changing your tires okay and so if i were to change my tires manually i would need

to take the lug nuts with my fingers and undo them right if i'm going to use an automated approach

well then i'm going to then technically take the impact wrench or even a manual crescent wrench

and i'm going to to loosen those lug nuts right now some people would still say that

that's a manual approach right because i'm manually exerting work it's just a kind of a

backed off stage of manual right how manual do you want to get reference and automated would

be if i had you know a tire company or maybe a mobile tire company just come to my house

and do my my tires for me right and all i had to do was swipe my card and that's it that was what

i would consider to be an automated tire change so when we're in the world of pin testing okay we're

going to use we're going to use tool star disposal it's like i have the impact wrench sitting there

why am i going to try to take off the lug nuts with my fingers just so i can prove a point that

i'm really strong and and my bones don't break when i try to do this it's going to be impossible

right if there's 100 foot pounds of torque on the bolt and the same in pin testing work it's

going to take fundamentally longer for me to do a bunch of nmap scans and a bunch of manual

msf open you know openmsf exploit validation especially if we're talking about like a class

c right a classy subnet 254 hosts that we need to check okay why would i cost my client more money

in saying that this testing period because i have to do everything manually right quote-unquote um

is going to take me you know 700 hours to do versus if i use the tools at my disposal that

automation and intelligent i.t has given us over the years just to cut that time down and still be

able to get the same type of data so we're going to do a blend of both right as as the term goes

so i'm going to use automated techniques because they're fast and they deliver information quickly

i don't have to wait around for them but i'm also going to do manual tasks like validating

that the vulnerability is actually a vulnerability and it in and provides some form of an attack

surface and is not just a misconfiguration that you need to patch there's a complete difference

between an exploitable vulnerability and a vulnerability that's simply a misconfiguration

so we're going to do that manual validation we're also going to to do manual tasks around like web

app calls and things like that we're going to you know get in the command line and do curl

things right we're going to maybe write python scripts that's going to be some manual tasks to

support other automated tasks right i'm gonna i'm gonna spend some time manually creating the script

but when i push the button it's gonna run right so there are a blend of both that's gonna happen and

so i kind of think that there's a there's a little bit of a misunderstanding in in the industry about

what this term means and also how it's applied in in actual penetration testing work versus

what maybe asked in a questionnaire or by a human in conversation about a business

connecting right and and proof of uh a technical assessment of penetration tests being conducted

and and if it followed nisk protocol right and i think that's really what they're trying to get at

mike you've got anything yeah i think i think there's just so many nebulous terms

out there that the standard just really isn't defined and nist is as close as we get and

so and the problem is that a lot of times the people requesting it don't have a full grasp

of what they really need not anybody that's listening to this podcast of course because

you're all intelligent human beings but um you know they a lot of people out there just don't

know what they really want or need so you've got somebody in sales side who throws a bunch of

technical jargons out there that sound really cool and they say okay great that's commoditized it's

you know within my budget and that works and they wind up with a scam as opposed to a pen test um

so i i think the problem is that you know we don't have a terminology a glossary that that mixes you

know we've all moved to company to company where you one of the first things you get is access to

the lexicon so you understand what everybody's talking about because each each company has

their own language um in certain things so um but yeah i mean the standards would be nice nist is as

close as we get to a standard but you know again we have different terms that have evolved over

the years and if you've been around as long as i have you've seen terms come and go and change

the building of the pyramids the creation of the wheel exactly you know when we made those first

computers out of stone um you know that's when it was all manual pen testing it was all manual yeah

but uh but yeah nowadays if somebody tells you they do a hundred percent manual run because

they still haven't completed their first pen test that they started working on 15 years ago because

that's what it would take right i mean you have to use tools but that doesn't mean it's an automated

pen test i mean imagine if you try to what does metasploit have in it like something like 4 000

exploits or something at any are you any given time i mean if you tried to build every single

one of those yourself without a tool it would be that would be your career right there it would be

it take it would take too long and so i i think that to me this spawned out of sales right where

they they tried to get they were trying to they were trying to ramp prices based on terminology

it's like well we can sell you an automated test for this or a manual test for this right and and

it i see that that's typically where a lot of the new terminology is coming from

yeah is is really just around that i hate to say bernasey and marketing paradigm

right where where they're trying to use words that create a different response in the human

but you know it doesn't really hold ground to any practitioners who do this work because we

i mean that's the beauty of automation that's the beauty of technology is that we we create

these tools so we don't have to do it again we've invented the wheel once right so we don't have to

try to figure out how we roll around you know i would have you know figured out the propeller

blade first you know so but in any case that's what's important is that we create these automated

tools that we can use that provide value um and and cost savings to our clients like you know

again that that 10-year pin test is going to be expensive versus one it's going to take me a week

right well let's talk about the term threat hut real quick that sounds like a video game to me so

this is like a whole can of worms it is devsecops and zero trust and

sassy and all the terms that are really just describing something yeah it does

it's like you should have a nintendo uh with with some some plastic guns playing threat huh

running through a computer and you're shooting at it yeah yeah i mean it technically is shooting at

moving targets though right yeah well some of the malware some of the stories that we've been

reading lately or the news stories some of the stuff is just you know exists in memory only you

know there's no there's nothing to find but traces of the fact that it's been there but some scary

stuff out there that they've been developing from a malware perspective yeah oh sure absolutely yeah

i think i think we'll always see this this asymmetric type activity right always increase

in its upward evolution yeah be i mean you know right because it has to um you know they they

they see defenses and then they you know they work around them based on the tools and techniques that

we're using to defend right so i think it's just an ever-evolving it's an ever-evolving battle

right i mean it's just you get you throw more cats and dogs in and that's all you're doing right

you know it's just it just gets bigger just becomes the brawl um so you know how do we how do

we you know how do we really defend against that and and i you know i think that's a good that's a

good segue to to to ongoing penetration testing versus point in time right which is i think

i don't even want to dive into the threat hunt can of worms really because it's i i could pull

out the soapbox in a lectern and chat about this for a really long time a whole podcast on that

i got a week's i could i can i could be like a threat a threat hunt is nothing more than a than

a pen test on steroids change my mind right and i would love to see what people have to say about

this but again i believe that was a marketing term that was thrown out um in order to provide a a

seemingly more seemingly more um excited response in in in their sales targets right but let's talk

about what what really sticks and that's ongoing penetration testing versus versus point in time

right and and that's what we typically see is everybody's usually point time right because they

have their own box right absolutely right they got they got cis or they have pci or whatever and it

says you gotta get that test done right so they're gonna get one done once a year and you know

usually we find it's the end of the year right the month like in november

we've waited all this time let's hurry and get it done right and so they're they're

really not looking at the intent of what a penetration test is supposed to serve you from

a risk perspective right i mean they're they're not looking at it from that perspective at all

they're just like framework says we gotta get a pen test what's that we gotta get one

right do we need an automated or manual right so and that's okay right you're doing something

and we're we're never going to discredit that right you're doing something that's important do

something right it's better than doing nothing but a point in time pen test is never going to give

you an accurate risk assessment or risk ranking on your environment as a whole right it's going to

tell you whatever you have in scope you're one web app maybe you got 40 and you're just like well we

can't afford to do all 40 the shoes we want right so you're going to do one web app in one time

of the year and you're going to say yeah we had to pin test it you know this this this one app

was fine okay well how often is your agile spreads you know how are how some of you making changes

to that web app there's a big difference um with with having ongoing risk assessment services right

penetration testing activities versus a point in time and if you don't have those types of things

and you know you made a good point earlier mike about the output of pen test is important

because it's gonna it's gonna find some trivial misconfigurations on your web app as an example

right well why did they why did you deploy with these trivial misconfigurations in the first place

are you considering this for your business is that okay to do are you saying that having okay

so we've talked about this before in other pin test episodes right the x frame options headers

not set is not an exploitable vulnerability and 99 of the cases you're going to find

but but basic entry little noobs in this game we're going to call it out because i'm going

to try to get a bug down here when i try to show them impressive on a report right

but let's get past all that and say why did you deploy a web application with the x-frame options

that are missing in the first place is it because you had a compensating control in the javascript

or is it just because it was too hard it required too many dev hours there's a reason there or was

it negligence right but like you said mike that that information can come out of that pin test

and say there was no exploitable findings right nothing serious but you had some trivial things

why did they get deployed like that can we go back to the document set and say in our

software delivery you know standard we need to add a place for doing this type of ongoing pen

test right now it can just be if you're just a vet if you're doing the right devsecops

right which is what this is and you're like oh we probably should deploy code with all this

crap in it yeah you should probably have a you know in the military we call it certification

and accreditation so you certify that code before it goes live that certification process includes

a pen test right we want to run it through some form of probably an automated scanner

right to just determine or you know something like a a a three letter i won't use their name but

one of the three letters big tools that they have that they sell that are hundreds of thousands of

dollars you could run your code through that and validate if you've made any mistakes in

this change before you go live or you can go live to a staging environment and have a third party

pin test it it depends there's a million ways to do this process but that's the benefit of ongoing

activities you don't have to call it an ongoing pen test right no one's going to come in and just

threat hunt for the rest of your life right to throw the tiki turns around but pen test like

processes can be built in and baked into your everyday operations so that you are conducting

seemingly ongoing campaigns the other thing too is the cleanup from the fine why do you still have

ssl 2.0 out acceptable i mean you know why is that protocol i'm still running you know where you can

accept those why is it still why is it still being found in this game why is this ssl 3.0 being still

being found why what you know why is that stuff and not cleaned up right right can't be looking

for your age so you find this stuff and then you don't don't fix it well it only takes you know

15 minutes to make that change well it doesn't make the change right so what's the document say

and i want to ask a lot of it you know it's not it's not our place right at the time to you know

obviously right but i in my head i'm thinking like i want to ask a client like what is your

what does your um what does your data set or your data standard say about having these types

of vulnerabilities is it okay to use right um what does your encryption policy say is it okay to use

ssl version two and version three right is that still acceptable for your business right and i

already know the argument for some organizations be like well we have we have clients that have

older machines and they might not be able to connect our application right so that's

always right the reason for leaving those opening however i will tell you that all modern browsers

okay anybody who's not using a windows xp machine from 2002 is on a modern browser chrome firefox

doesn't matter right the safari they're all going to prefer a pki modernized approved security

connection over they're always going to prove that over something weird you know what i mean so

again we have that conversation with with older versions of tls and ssl and older versions of um

uh older versions of your connection oriented methodologies too and it's like it shouldn't

a great example 80 and 443 people leave 80 open and they're like well we would redirect to 443

what browser in the last five years is ever kind of going to not warn you when you connect

to a web application over port 80 it's going to say you can't go here this is bad like there's

no security like you can't go to this website and so i don't understand the argument right based on

modern technology well isn't it that microsoft came out with an edge because they send down

i.e that their edge won't even connect to port 80 anymore or they're moving to that in some way sure

literally i mean even if port 80 is open traffic is allowed it still won't allow it to connect

sometimes you do hear the um the company's justification is oh well we have three clients

that are still using aol you know whatever they're they're worried about i'm one of them i'm one of

them i mean it's almost at that point it's almost like we'll send them all new laptops and get them

on a new browser and it would uh be cheaper to do that than suffering you know our clients

what's that i'm guessing they're not your high dollar clients it's kind of like music you can

use a war analogy because north korea still uses biplanes we shouldn't send f-22s to south korea

so it's crazy but that's you know you you hear it i think a little too often um that oh well we you

know our clients have these old old browsers we're worried about them being able to connect

well let's bring it we ran into a place where there was a lot of customer facing

websites right where they would be able to get quotes for different things and um

you know it's like well we don't know what the end users are using you know it's then we can't

we can't control people and it's like well at a certain point you have to protect the house right

you know your 16 year old keeps forgiving his keys it doesn't mean you leave the

door unlocked every night that's a great analogy man that's that's perfect actually

that's perfect if we all join the same team and everybody gets their systems right and

then all the these people on deprecated systems will be forced to upgrade because they won't be

able to do anything they're trying to do and then the world would be a better place

for a while for about five minutes yeah five minutes so to circle back right i i hate i hate to

i hate to ruin the uh their rhetoric because i'm having a great time but i know everybody was

like what's the point well the point is is that ongoing is going to be point time any day a week

our recommendations you should have a third party doing you know um we have manual and automated

penetration testing at least quarterly right you should you should have a third party looking

at yourself quarterly if you if you really want to understand what your like risk period is and

mike's going to jump in any minute and say and internal penetration testing too because we see

everybody skip out on that yep and everybody skips on that and after every major release yep and of

all the you know you just gave that headline of all those billions of records that have been

leaked and i want to tell you something okay all of the information that everybody needs to break

into your organization is probably out there in pieces okay what what what you're up against is

the time window okay that data is so vast to look through right now that's what takes time it's like

the proverbial needle in a haystack the needle's there we know it's there it's

a matter of the time it takes to dig for it right and so that is what you're up against

with all of these records that are now out there in mass they are there for everybody to look at

everybody to gain information on that's going to contain password hashes common passwords

email addresses right they're going to have a lot of stuff about your organization that's there it

might be old but you need to ask yourself is you know is that time window that you're up against

worth the you know worth having the the uptick in in cyber security's you know related tasks and

you don't have to hire that third party okay just there's a book that i know a couple smart guys and

one dude with a beard wrote it's the least of the three but in any case you can get that book and

it'll tell you about all the stuff that you can put in your everyday operations processes to like

simulate a lot of this work and reduce your risk quite efficiently and cheaply

well you know another thing too that i think is forgotten a lot is as people start to go back

to offices uh don't forget about your wireless environment too and that seems to be one of the

one of the weakest points of an office environment at least from what we've seen in testing that you

know we've we've done with clients so don't forget about that if you're still running wep

it's timed upgrade yeah because you might have you might have clients with older laptops that

you know don't use you're still running the web you need to just

grab your waps put them in a box send them back to the manufacturer

go practice the phrase do you want fries with that yep how to work off only the only fax machines

moving forward yeah well well this has been good you know i hope everybody learned a lot there's a

lot to unpack when it comes to pen testing and a lot of misconceptions and confusion out there so

our goal was to help remediate some of that if you're looking for for some more information

check out nist uh 800 115 is the is the penetration test methodology recommended

so it doesn't have to get too fancy i mean you can kind of see the fundamentals of these activities

and how they work um through a industry accepted standard with nist and um that'll help you gain

a better understanding but um yeah just know that penetration testing is important if you

you learned one thing out of this uh little series here it's something that that everybody should be

doing it's critical to understand your real risk right so you might do a cyber risk assessment

against a framework like nist or css cis controls but uh that's not gonna do it right

i mean just because it's in writing and people say that they're doing something doesn't mean

that the attacker from the outside uh doesn't see things entirely different so that said any final

thoughts just remember that the attacker out there has nothing better to do all day

but figure out how to attack you so if you get to a static point you think all right we're secure

you're not you're secure at that point but unless you continue to evolve remember the

enemy's always going to evolve so you have to keep that in mind as you move forward there's no

all right we've achieved everything it doesn't happen so keep that happen

sand castles man it's sand castles on the beach right the waves are coming you better you gotta

have to reinforce the castle the way the water's gonna take it away yeah well good stuff well

thanks for listening everyone please subscribe to the podcast rate us let us know your comments

reach out i would love to hear from you we're always looking for new topics that you care about

and that's what's most important so hope this helped you out and check out cyberrants.com if

you haven't already there's some information about the book and such and uh podcast episodes are also

on our website at silentsector.com have a great day pick up your copy of the cyber rants book on

amazon today and if you're looking to take your cyber security program to the next level visit us

online at www.silentsector.com join us next time for another edition of the cyber rants podcast

Episode #23 - Penetration Testing: What You Need to Know (Part 4)

Send Us Your Questions & Rants!