Small-town street scene in Illinois American flag flapping in breeze by huge painted American flag fading from brick wall

Episode #20 - Penetration Tests: What You Need to Know (Part 1)

Penetration tests are vital for nearly every organization to see how secure they really can be. While the demand for them is higher than ever, it can be a bit tricky on deciding whether manual penetration testing or automated penetration testing is best for you. This week, the guys answer questions and give their own advice on how to guide yourself through the world of Penetration Tests.

Pick up your copy of Cyber Rants on Amazon.
Looking to take your Cyber Security to the next level? Visit us at
Be sure to rate the podcast, leave us a review, and subscribe! 


welcome to the cyber rants podcast where we're all  about sharing the forbidden secrets and slightly  
embellished truths about corporate cyber security  programs we're ranting we're raving and we're  
telling you the stuff that nobody talks about  on their fancy website and trade show giveaways  
all to protect you from cyber criminals  and now here's your hosts mike rotondo  
zach fuller and lauro chavez hello and welcome to  the cyber ants podcast this is your co-host zach  
fuller joined by lauro chavez and mike rotondo and  we have a multi-part series for you here moving  
forward this is going to be about penetration  testing cyber risk assessments the things that  
you need to know how many parts will be well  that's up in the air right now it depends on how  
long we want to talk about it i guess but there's  a lot to be said and there's a lot of confusion  
in the world of penetration testing and risk  assessment so we want to clear that up but  
before we do mike you want to kick us off with  the news yeah so this is the headlines for this  
week i tried to not go too heavy on solar winds in  exchange but some of it couldn't be avoided today  
first thing is our evil anderson group threatens  to launch ddos attacks call journalists and  
business partners idea behind this is that if they  contact your business partner and let you know  
that you're uh being hacked and have ransomware  that you're gonna be more willing to pay so  
cool criminals keeping that before yeah understand  companies out there in the wild so solarwinds just  
keeps getting worse new strain of malware found  infecting victims uh sisa is now saying you take  
18 months to clean up this mess i'm going to guess  it's closer to two or three years but there's more  
variance out there a malware can exploit new flaw  in intel cpus to launch side channel attacks more  
reverse engineering going on a great deal of  employees have inappropriate access to sensitive  
data this is kind of interesting because  this dovetails off of the remote workforce  
and in this article it's it leaders expressed  concern about inappropriate or malicious access  
to applications and data 47 are concerned about  malicious actors impersonating employees and 41  
are concerned about inappropriate access  to sensitive information the bigger thing  
is that 76 of employees had inappropriate  access to sensitive data files and 76 were  
granted inappropriate access to sensitive files  within the past year um either that's all right  
we're in a pandemic and we shouldn't get things  done or people are getting sloppy and lazy but  
sloppy and lazy we need to tie that stuff down  and uh protect your data because it still matters  
huge rise in vet hackers submitting  vulnerabilities during cobit 19 apparently  
hackers did have time on their hands bug bounties  increased uh misconfigure vulnerabilities rising  
by 310 percent and submissions for both improper  access control and privilege privilege escalation  
went up by 53 percent so this is interesting  exposed passwords gave hackers access to 150  
000 cameras there's a camera company called  prakata that apparently does things like tesla  
hospitals women's health clinics  psychiatrics hospitals jails  
and even their own their cameras were hacked and  exposed if you're not living under a rock this  
might be new to you microsoft exchange attacks  caused panic as criminals go shell collecting  
big microsoft exchange for on-prem hacks going on  out there the most decision most decision makers  
plan to increase spending on cyber security this  year that's good news for companies like silent  
sector so we like to hear that 66 percent of them  are expecting to increase their spending on on  
cyber security access and because of issues with  remote working and fishing and ransomware attacks  
there's an f5 patch out there for a critical big  ip pre-off rc ebug there's an additional 10 apt  
groups out there that are working on the exchange  servers most of them are espionage groups one  
of them is a crypto mining campaign the top 10  cyber security vulnerabilities of 2020 came out  
uh and cve 2019 19871 and citrix pat server passed  transversal flaw was number one u.s schools face  
cyber number face record number of security  incidents in 2020. it's bad enough the kids would  
have to work from home can't can't ditch can't  take snow days and now we got security problems  
and lastly there's a new linux backdoor called red  xor likely operated by chinese nation state it's  
an undocumented backdoor targeting linux systems  masquerading as a poll kit damon so the only thing  
today on uh exploit database that i think is  probably worth uh note is golden ftp server has  
a pass password i guess i guess it's we'll call it  a misconfiguration of how the password is managed  
how that authentication connection is handled  by golden ftp server and so there is an a buffer  
overflow that exists in that process the the  exploit's been tested and verified however there  
is a disclaimer that in the case that it apps that  you know it it runs but it doesn't it doesn't it  
doesn't root but there's still a vulnerable server  that apparently the ftp server will no longer work  
anymore so if you're trying this exploit out make  sure you're doing it in a poc and for everybody  
using golden ftp server make sure you're updating  past four seven and that's it that's all i got  
that's uh worthy of news today for exploitation  well outstanding let's dive into the deep topic  
of penetration testing which is a wonderful  activity to do and everybody should do it  
and there are lots of different kinds and i'm  sure over the next few episodes we will get into  
the nitty-gritty details of a lot of information  that people should know about penetration testing  
and selecting what the right penetration test is  and such but before we dive into that let's talk  
a bit about the state of the market and and maybe  why there's so much confusion and this is this is  
going to be some ranting here at least on my part  but um for my for my shoes what i see is that the  
penetration testing industry there's there's  certainly been a lot of demand for it  
and it's hard to find good talent to actually do  penetration testing and it can get expensive and  
there are a lot of other things but everybody's  scrambling and trying to figure out how do we  
deal with this in the marketplace as an  industry right and so as a result a lot of  
commoditized kind of canned automated rinse and  repeat one size fits all you could describe them  
in a whole bunch of different ways but i think  you get the gist uh solutions have come out there  
and offerings and so the penetration testing world  you can find all kinds of stuff and now there are  
a lot of options out there for the consumer of  penetration testing it doesn't matter if you're  
talking about your internal or external network  or web application or whatever the case may be  
there are options that can get  confusing for people out there so today  
we want to talk a little bit about what's what  right what to identify uh the difference between  
these kind of commoditized can penetration  test versus one that actually has a has some  
expertise behind it and is done in a more  meticulous manner rather than just with the  
outputs of a tool and can reports and  such so with that laurel of course you  
run and oversee or pen testing activities for  silent sector i mean what's your view on the  
market on what people should look for as far as uh  penetration tests you know what is what is enough  
and what is overkill who are you even talking to i  thought i just did janitorial work first well you  
do that as well but you know ah okay man of many  talents you mean the other stuff i got this well  
it's certainly a good question and it's  this is certainly a good topic to get on  
and it will certainly not be laid to rest in our  short time today but i think i think there's a lot  
going on and like you said there's i think what  i see the most of is and mike sees this too is a  
lot of the canned reports where an organization is  you know typically it's it's two ways right either  
they're they're trying to get a they're trying  to do business with a with a large organization  
they're saying when was your last pin test and  then they just don't answer and then go try to  
get a pen test hurry up quick try to get a pen  test sort of thing and what will happen or or you  
know um they're being you know required that of of  some internal process right then same thing right  
we need to hurry well let's go get a pen test well  if you're in a hurry for one you have the time to  
you know really kind of understand or or  read cyber rants and and try to understand  
what the differences are and what you should be  asking to receive as part of your pen test it's  
it's really easy to get caught up in you know a  very cheap test that'll that'll come out and it'll  
just be a canned report um spit out you might as  well call it a vulnerability scan it's going to  
call out items like your you know http only flag  is not set uh in your web application or you've  
got x-frame options not set and you're gonna get  hacked tomorrow and so these these scanners will  
put out this information the golden poodle attacks  the golden poodle attacks yeah right poodle yeah i  
see that too you've got 64-bit block cipher still  enabled oh you're going down you know and my point  
is that first off no modern browser is going to  prefer to connect using a 64-bit sli cipher this  
is 2021 okay i mean this is an internet explorer  4 although there may be some people out there  
still using it but um it's it's it's not fair to  these organizations that are paying good money  
even if it's cheap right even if you're getting a  canned report you're still paying a couple grand  
okay um it's not so it's a bag of chips all  right so you know you're still paying good money  
and you're getting you're getting garbage you're  getting garbage that you still gotta sip through  
and you're gonna you're going to send this pen  test report over to your developers and they're  
going to look at you like wait what why do i  have to fix this again i don't understand now  
that's not always the case okay now a lot of times  if you've got like a big misconfiguration like  
you've got postgresql exposed or sql exposed or  you've got you know a web route with no password  
or something that automated scanners going  to find that and be like oh i found something  
those are certainly worth knowing and they're  certainly worth finding right i mean and so if  
you've never done anything you don't know anything  you know that's not a bad approach to just at  
least start with something that's a vulnerability  scan right that's you can at least know what  
what exists out there um but you know when when  it comes to getting i guess the bang for your buck  
there is some reading i think and studying  required to understand first off the difference  
between the types of tests that i'm going to get  from different organizations via the canned report  
or you know an actual manual penetration test or  even a cyber risk assessment campaign or a threat  
hunt there's a there's a whole bunch of different  terms that are getting thrown around out there  
and there's not a lot of clarity on what they mean  and so you want to make sure that first off you  
understand what type of test you need really if  it's going to meet internal requirements right if  
you're looking at a governance framework what do  you need are they asking for internal external is  
it is it a client or a partner and they're saying  you need to have a web app and an api test and  
you need to be external and internal and  a physical test so understand that first  
i think that's really important  and then i think it's important to  
understand what what attack surface you're trying  to understand which is where where my point comes  
in i don't i don't you know mike will assert this  i don't look at the governance piece first because  
to me that's kind of a check block i look at how  do we defend this this technical asset deployment  
that this company is using to front their  business that's how i approach these things  
so i look at the problem a little different  than um you know made an automated scanner's  
going to look at it as scope and so i think that's  where you get more into a cyber risk assessment  
type of testing or a threat where they're  going beyond what the scanner is telling  
them is there and they're looking at the  architecture that you've built your business on  
that is sustaining your your applications  and software or widgets that you're selling  
or whatever places you're driving customers  and we're going to look at that architecture  
we're going to find a weakness a capability  to to take advantage of that and and typically  
anything good is sometimes not found i mean  there's some good stuff found on the scanner  
but just not anymore okay again this isn't  2004. so a lot of stuff found in the scanner  
or from an automated perspective at least  that i've been seeing it's kind of garbage  
okay i use it as a i use it as a as a point of of  data intelligence but i certainly don't bet all my  
my work on on what's coming out of a qualis or a  rapid seven or a nessus or anything other scanner  
it doesn't matter um even an in-map scan okay  that that just because there's ports open there's  
always going to be ports open right so that's just  the first step but i believe that a lot of the  
stuff coming out of the industry today it  is just that it's just an automated can scan  
i don't know micah you've seen that so i i think  one thing that has to be addressed is the scanner  
doesn't understand what compensating controls  you have in place the scanner doesn't understand  
the architecture of your infrastructure the  scanner doesn't understand how things work  
together it is looking for and looking  for for what may be a vulnerability and  
you know these things are built to detect  vulnerabilities right they're they're looking  
for something vulnerable it doesn't mean it  actually exists or is exploitable and that's  
where you know you get a lot of those poodle  things and and then those other the you know the  
the 64-bit cypher they were talking about being  you know called out as uh critical things to fix  
and it's they're not taking into account what's  really necessary to secure your environment and  
and i think what's happening also is this is  where you know a we've got one side of the  
house where the business just wants to check the  box and say yes we've had a vulnerability scheme  
there are those people out there that also are  very concerned about getting the vulnerability  
scan and really want to know what's wrong  there are those that want the plausible  
deniability to say we had a vulnerability scan  by xyz company and they didn't find anything  
that they call the pen test but i think there  also comes in play where the msp comes in place  
you know manager a doesn't really know much  about it or pen testing goes to the msp and go  
that's managing their instructions to say we  need a pen test msp goes okay we can do that  
for you and really just does a you know a scan  for them or very automated some kind of tool  
that does the work and there's really no logic  or thought behind it and i gotta be honest i've  
seen some very bad canned reports from very big  companies that read just they're just horrible um  
that there's really nothing behind them and i'm  sure they're spending a whole lot of money to  
get those things done it's got three letters in  the name doesn't it yeah it does yeah hey now
not pointing fingers but uh for the three  letters you only point one finger and that's  
the middle one yeah you know i you know one  of the things that i i see a lot too is that  
you know we see this all the  time where a b2b tech company  
is told like you said before lauro it's told  by their client hey you gotta have a pen test  
so they go get a pen test um and they have you  know various options they can go with kind of  
the the you know the cheap quick canned automated  approach um but the problem that they see and a  
lot of times they don't know better right going  into this maybe this is the first time they've  
done they've dealt with this or they're you know  emerging company or whatever the case may be but  
they go in and they get this pen test and  it shows them a whole bunch of stuff so  
they're like oh wow i got my i got some value out  because this pen test report has a bunch of stuff  
on it granted like you said that those things  haven't been validated manually by an expert  
looking at that stuff um but also that now their  customer says well did you remediate everything  
that was found on the pen test it's like now  they got to go do everything versus versus  
so it actually ends up costing them so much  more and time and and resources to go do all  
that remediation so they saved a little  money up front and now they just steered  
their whole dev team away from you know or  their network team whatever it is away from  
their core you know business operations and  creating you know new new um features and such all  
to just do this this remediation when it turns out  it what was pointed out on the on the automated  
results aren't actually exploitable areas of that  environment and but they but now that it's it says  
that and that's the report they got now their  client expects them to get get all this stuff  
done because a lot of times too they make that  mistake of giving the whole pentest report to your  
client instead of a letter of attestation that's  written specifically for that pen test which a lot  
of companies aren't aren't given out these days  unfortunately but um so they're sharing all this  
now the next thing that you know we've seen happen  of course is um they share this information with  
the you know fortune 500 or whatever that you  know they're trying to land their this this  
big contract with and their security team  looks at and says okay we have questions  
let's talk let's talk about the penetration test  report um and then their the ability for their  
their tester to really go through and detail  um can really make or break the kind of the  
image of that organization through the the  vendor reading process so if they can't get  
through explanations about the environment and  why the certain findings were labeled at certain  
levels of risk it can become a real problem and it  can undermine their their credibility so um what  
what may seem like a deal in the short run uh can  can certainly cost a lot you couldn't have said  
it better scammers you know there are scammers  out there for everything you know they're trying  
cyber security bug hunts the same thing  like i can't tell you how many clients  
we've had over the years approach me and say hey  i got i got a random email by a bug hunter that  
you know found these things and wants to get  paid none of it is any good it's all the crap  
we're talking about it's like you know your cookie  doesn't have the secure attribute set to it pay me  
you know i made a meme if you follow me on  linkedin i dropped a meme the other day about the  
spongebob basically you know picking up the paper  and saying it's like hey i found an xss bug on  
your site pay me and it's like he throws it in the  trash and warms his hands with it that's really  
all it's good for it is complete crap and so you  you know for you know two things right first off  
understand the company you're getting uh you're  getting you're paying for it for a pen test right  
know the tester are they outsourcing it to  somebody ask them i i've also we've dealt with  
sorry to rant here but we've dealt with want to  be cyber security companies that say they offer  
penetration testing when you ask them what  team tools do they use they have no answers  
well we don't know we're looking into that oh  what vulnerability scanners did you guys do  
we're looking into that right well it sounds to  me like you don't have a plan you don't you're  
not doing anything but you want to offer services  because right now you know again scammers right  
and some businesses that appear to be legit can  also be attempting to scam you um as as this  
as this kind of you know the need for  cyber security rises in our country  
and all of the people now want to get onto it  and now all these kind of shell companies are  
opening up say oh we offer this we offer  that and the truth is they don't offer  
anything that's going to be worth your money  or your time um you know so make sure you know  
the company make sure that the tester can  answer questions and replay attacks for you  
if they can't if they can't give you instructions  to replay or replay an attack surface for you  
that they were able to identify then it's  it's hocus pocus okay it's smoke and mirrors  
number two just like zach said don't send  them your pen test report um because they're  
gonna that's got sensitive data in it about your  organization even if no high-risk items are found  
it's still got ip's dns it's gonna have all kinds  of data if the company that you're doing business  
with from a penetration test perspective or threat  hunt does not offer a letter of attestation for  
their testing services you probably shouldn't  use them and you certainly shouldn't send that  
report you know try to get away with only if it's  all you have it's all you have but try to get  
away with just sending the dates and the table of  contents well there's also questions about sending  
your pen test report in the mail i mean you're  sending confident highly confidential data  
yeah you know to some company absolutely and then  what are they doing with it right just like in the  
headlines you said you know so many people have  unauthorized access to data it's because what  
what are they doing right no no offense to the  governance teams out there that are trying to do  
the right thing for for third party um assurance  right they're reaching out third party saying hey  
you've had a pen test this year cool send me the  report okay so shame on them for after for being  
data hoarders and shame on you for sending that  report over emo but they're gonna stick it in a  
sharepoint site and everybody's gonna have access  to it exactly you know it that's educate yourself  
there's plenty of data out there on what is what  a pen test is there's plenty articles you can  
you know there's a site called google and if you  put in one of the pen test it'll it'll spit back  
how do you spell that i'm gonna write that down  i think it's b-u-t-t actually i heard there's  
a book uh called cyber rants that i think is uh  quite a bit of stuff about pen find  
it on amazon today um yeah that's that's  for sure you know another thing too in all  
seriousness a good good resource for people to  look at is nist 800 115 so google nist 800-115  
and that's the nist recommended methodology for  penetration testing and it covers all kinds of  
different pen tests and such but it's it's a base  you know methodology that that you know good good  
pen testers are gonna follow right you don't  have to you don't have to reinvent the wheel or  
anything like that it's readily out there national  institute of standards and technology it's um  
it's open source it's there for  you so check out nist 800 115  
and it's you know do do some research get to  know what a pen test is and our goal with this  
series too is to help you better understand what's  what in this crazy world of penetration testing so  
with that said mike lauro any final  thoughts always remember never forget  
that's it yeah and i just want to you know educate  yourself on what you what you really need what is  
the goal of your pen test are you just checking  a box or do you really want to know what's wrong  
but really investigate the company that you're  looking into that wants to do it for you
msps you know aren't security companies um  so you know talk to a trusted resource but  
do educate yourself just learn about this stuff  um realize cyber security sometimes seems really  
scary but it is not the movie swordfish um and  it isn't what was that super man movie anyway  
um it's not the one with richard pryor creating  the giant machine in the basement it's probably  
most of our artists too young for that anyway um  i remember that yeah yeah i do yeah and swordfish  
yeah the the seven had the seven headed hydra  that he hid in the ftp server at his old college  
yeah that's a good one that's a good tool for pen  testing still using a super good tool yeah plus  
you have those you know seven monitors hooked up  together that was really cool too well you have  
to have that to do you have to have that well it's  hard to look at a dos prompt in just one 24 inch
you got to have you know your terminal  dude you got to have it stretched out  
big it's got to overlay like six  screens that way you can read it  
you know that's one of the biggest misnomers  out there is that people think pentas is all  
this graphical really cool stuff in  reality it's just reading text yes
that's a fire it's just a hydrant it's a water  hose it's nothing really fancy but it is it's  
data science right i mean so people who do it  who do it professionally who do it for real um  
treat it as data science and nothing more right  it's it's obtaining a either you know the data's  
there to support exploitation or it's not um if  your test and your company's giving you a bunch  
of assumptions to make then don't use them black  and white is or it isn't right and and it's kind  
of like the matrix you can take the the red pill  or the blue pill it just depends on whether you  
really want to know or not you can buy half  of each and drink a beer and see what happens  
i would not take those with alcohol i'd  recommend against both both matrix pills um  
and so would your doctor well that said thank  you for listening hope you enjoyed our rants  
more to come on this topic and please rate the  podcast let us know your comments questions  
reach out and we will cover future topics of  interest as we go along here have a great day  
thank you take care everybody pick up your copy of  the cyber rants book on amazon today and if you're  
looking to take your cyber security program to the  next level visit us online at  
join us next time for another  edition of the cyber rants podcast