Small-town street scene in Illinois American flag flapping in breeze by huge painted American flag fading from brick wall

Episode #22 - Penetration Testing: What You Need to Know (Part 3)

This week, the guys continue their penetration testing discussion, covering the following common questions:

How often should your organization conduct a penetration test?
What's the right approach, red team penetration testing or purple team penetration testing?
What should you see in your penetration test reports?

Pick up your copy of Cyber Rants on Amazon.
Looking to take your Cyber Security to the next level? Visit us at
Be sure to rate the podcast, leave us a review, and subscribe! 


welcome  to the cyber ants podcast this is your co-host  
zach fuller joined by mike rotondo and  lauro chavez and today we are continuing our  
mini-series about penetration testing and have  some great topics and considerations to cover  
but before that why don't you kick us off  with the news in the usual fashion mike hello  
everybody and uh welcome to the news today  first we're going to start out with russian  
hacker pleads guilty to planting malware in tesla  gigafactory apparently um scott was an employee  
uh working on a tesla gigafactory it was offered  a million dollars a bitcoin if they facilitated an  
attack also he planned to carry out a distributed  denial of service attack which would have diverted  
the company's attention paving way for another  attack watch your employees firm search to patches  
attackers exploit critical f5 bugs uh there's  a new cbe out there 20 21-229-86 it's a flaw  
in rest-based eye control management interface  which could allow for authentication bypass and  
remote code execution it's at a rating of 9.8  popular remote lesson monitoring program can be  
exploited to attack student pcs this is kind of  disconcerting there's a new teaching tool called  
net up vision pro which contained vulnerabilities  it could be exploited basically you weren't even  
you were not able to encrypt the network traffic  and students would automatically start sending  
screenshots to the teacher whether they knew it  or expected to do so or not um we've got another  
disgruntled employee an admin with an axe to grind  sent to prison for wiping microsoft user accounts  
uh this is basically everybody's worst nightmare  32 year old employee was fired went back to india  
while he was outside the us he deleted 80 percent  of the employers microsoft 365 accounts needless  
to say company ground to a halt for a while um and  he's going to jail for two years so there's that  
back in the wayback machine everybody  remember melissa the first big virus  
managed to disrupt hundreds of networks including  those of microsoft this was back in march of 1999  
at least a hundred thousand workplace computers  were affected understanding remote security remote  
work security that's the world we're in today and  really what it comes down to it's an interesting  
little article uh but it's basically a breach even  one employee's laptop is bad but it breached the  
entire company's infrastructure and potentially  everyone connected to it is much worse so  
what's happening is companies find it more  difficult to keep remote employees secure than to  
prevent malware on a remote laptop from infecting  infrastructure this is added because i simply like  
saying purple fox root kit purple fox root kit  can now spread itself to other windows computers  
there's 90 thousand incidents that have  been spotted in the wild through 2020 in the  
beginning of 2021 uh it's got a novel spreading  technique uses using indiscriminate port scanning  
and exploitation of exposed smb services with weak  passwords and hashes how many times you get those  
internal scans you go oh it's just smb we're not  going to worry about fixing that well guess what  
purple fox root kit it's looking for you um  phishing leads to breach a california state  
controller this one i kind of laugh  at because i mean it's serious but  
basically what happened is there was a phishing  attack that exploited wound up exploiting uh the  
state controller's office of california fishers  had access for more than 24 hours the intruders  
used that time to steal social security numbers  and sensitive files of thousands of state workers  
on the send targeted phishing messages to at least  nine thousand other workers in their contacts  
the reason i kind of laugh is that is because  in october 2020 someone in their great wisdom  
at the california department technology decided to  exclude managers and supervisors from being fish  
tested uh using dynamic testing they had to know  everything about it so basically they they gutted  
their testing sys gutted their testing and wound  up getting hacked because of it uh black kingdom  
ransomware hunting unpatched microsoft exchange  servers yep that's still happening there's another  
another attack out there they're trying to  exploit anybody that has not patched and if you  
haven't patched yet i'm sorry two-thirds of the  large firms attacked as kobe 19 hamper security  
from the 27 percent of businesses experience  attacks at least once a week with phishing at  
83 an impersonation 27 most common a fifth 21 of  those reporting attacks end up losing money data  
or other assets as a result and 35 reported other  negative consequences such as business instruction  
or a loss of staff productivity a critical cisco  jabber bug could let attackers hack remote systems  
it allows you to execute arbitrary programs  on the undulating underlying operating system  
cause the ddos intercept protected traffic or  access sensitive information and with that let's  
go ahead and end out the headlines go ahead lauro  purple fox root kit i like that that's kind of  
cool yeah the clever clever names yeah yeah like  he's like like you said if you haven't patched  
already time to give up i.t in the exploit world  this week we really only have one to be of concern  
about um and that's with an online ide it's an  integrated development environment basically  
a sas platform that allows you development  capabilities it's called cody dad and there  
was a remote code execution that was provided via  python to um as a module for metasploit fantastic  
all you need to do is get a demo account and  you need to be authenticated for the rce to work  
but that's super easy to get through the  through the application so this has been  
communicated to cody dad and they've hopefully  been remediating this but just as a cautionary  
entail to everybody be careful of be wary of  using online platforms to do your development  
on okay they're not they're not all you know  really really doing their security testing  
or penetration testing which you know it you know  brings us brings us to the circle of conversation  
today but that's that's really i think um all to  be aware of this week for exploits in the wild  
that are going to be weaponized in metasploit  pro or metallic framework if you uh like the  
free version well as we continue our discussion  about penetration testing our our goal through  
these podcasts of course is to answer a lot  of common questions that come up that we hear  
about these various topics and so today we want  to dive into a few more topics about penetration  
testing and just address various approaches and  reporting and such but before we dive into that  
how often do you think somebody should  get a penetration test that's that's a  
very very valid concern that a lot of people  have and and what would you advise them  
when they're wondering what what frequency is  appropriate for our organization great entryway  
i think for and certainly something we hear a  lot there's a couple ways to answer this question  
so if you're going to ask somebody who is  doing this for a living they're going to  
tell you the more frequent the better there are  products out there that will actually let you do  
round-the-clock live campaign testing in your  own networks um and so you know there you can  
be as proactive as you want to what i will say is  that annually is probably just now scratching the  
bare minimum and that that seems to be what the  theme was for a long time because um traditionally  
these these were more complicated and and you know  there was a lot more convolutedness in what was  
occurred and there was a lot less canned reports  back in the day so they felt once once a week or  
once a year was was adequate i think in a lot  of the governance frameworks nist and you know  
pci and things like that that would require an  annual pen test and that that's probably great  
so you know i guess it comes back to what's your  intent is your intent to check a box then once a  
year is probably frequent enough but if you're  concerned and you're not sleeping well at night  
because you know you've got development staff that  are making changes or you've got infrastructure  
changes that are going on or you're migrating  things from an on-prem environment to the cloud  
and there's there's changes happening that aren't  giving you that high confidence of a low-risk  
environment that you may be used to having and so  that may something like that may cause you to to  
ramp the frequency in your pin testing um i think  quarterly is is really going to be the standard um  
what what frameworks like pci will throw in  some i call it magician's language to kind  
of allow you to you know essentially draw your  own i say you know draw your own firing squad so  
they'll say things like upon significant  changes right so pin testing should occur  
annually and upon significant changes here's the  thing is that pci and we don't know about four yet  
but they won't they won't necessarily uh define  what a significant change is nor can they in your  
environment right that's up for your business to  decide so most businesses we say they choose not  
to define it and therefore they're not held to any  accountability of any kind now if they define a  
significant change as like you know you know an  updated jquery library in your web application  
for example that would denote a pin test  right under the guise of the pci language  
and so again you draw your firing squad  so you can you can write yourself into a  
really really expensive around-the-clock  penetration testing routine however while that  
business people may look at that as costly  all your security governance and financial  
risk people are going to look at that as um  proactive right we're we're we we have now a  
very current risk snapshot of the environment  and all the changes that are happening if you're  
just trying to check the block and you know you  don't really care about the business that you're  
serving then maybe annually is going to be your  number to go for and if you're part of an i.t  
department and you've got a lot of changes going  on and you really care about understanding the  
the protective posture of the organization at  any given time then you know you may want to  
at least do quarterly maybe monthly maybe you  should define those significant changes and  
what they mean to you if you're if you're bound by  pci and if you're really really trying to step it  
up above everybody else then you know you can grab  one of your products deploy it and you can you can  
perform around the clock pin testing on everything  it seems i would include hipaa as well in that um  
as far as yeah absolutely mike thank you for that  good point yeah hipaa's gonna be uh requiring that  
i think doesn't sock um require that as well stop  acquiring has a pentax requirement but it doesn't  
delineate how frequently other than angling but  i think that needs to be updated as well if you  
truly care about what you're doing and you truly  have sensitive data the the desire would be that  
you would be proactive and testing it minimum  quarterly one of the things that we see in  
out there in the in the wild you know different  industries it's interesting because we work with  
you know variety different industries but i think  the ones that are most proactive are really those  
b2b technology companies where other  other organizations are relying on their  
products and their environment they  seem to be a little bit more um frequent  
in their testing and analysis and such right  and i think part of that too it goes to a bigger  
a bigger picture bigger discussion about risk in  general and what the organization is willing to  
uh allow or or accept um the reason being with  us take a b2b technology company for example  
software as a service platform let's just  say um their their software is their brand  
right so if that goes their brand goes the company  goes it's all it's all done right the credibility  
out the window versus um let's take like a um  maybe a regional credit union or bank for example  
where there it's it's not so much you know the  entity still is a brand bigger than any one  
application or any one platform that they're using  um so the considerations may be maybe different  
right or healthcare companies same type same type  of thing so it doesn't doesn't make it right to do  
it do it less frequently for sure but um you know  of course all companies have limited resources but  
um i would encourage everybody listening that  uh to to uh do something right doing something  
is always better than doing nothing and it blows  my mind but there's still lots of companies out  
there who have never had a penetration test or  haven't had one in the last three to five years  
um it's startling how many haven't really yeah  it's almost like if they're not beholden to some  
uh compliance requirement or some client  requirement um there there are a lot of  
organizations that just they're not taking it on  their own initiative to do them so we still hear  
that argument well i've got nothing that anybody  wants to steal and that's yeah this is dangerous  
crazy well well that's um good overview you know  some good considerations on timing and such when  
you should get these things done um generally  speaking too you probably get better rates if you  
do these activities uh earlier in the year rather  than wait till the end of the year fourth quarter  
just because like we from being in the business we  see everybody jumping through hoops to get their  
pen test done the last minute before the end  of the year and as such the higher demand they  
cost more right so it's it's better to uh kind  of space them out throughout the year plan be  
proactive rather than trying to jump through hoops  to meet a compliance requirement that said you  
know there's a handful of different approaches and  i figured touch on that for a little bit i mean  
what are your opinions on red team approach versus  a purple team approach i mean you want to start by  
describing both in the differences and then  just dive in a little bit because i know  
it's very common in our industry for pen testing  to just kind of happen in a vacuum company will  
say okay give me the targets we're going to do  the work and we're going to send you a report  
but i don't think that's the best best  approach always do you want to touch on  
the differences and and pros and cons of each  yeah yeah absolutely so traditionally speaking  
in a mature organization you'd have a you'd have  a defensive team that that was you know maybe  
running your security operation center inside or  you know had some security telemetry that they're  
looking at you know whether you know there's a  lot of products we i don't want to name right now  
um in any case that's going to be your blue  team all right and traditionally those are the  
the defensive security team members that are  you know watching the boundary watching the  
log files watching things happen firewall managing  changes those sorts of things that are kind of in  
in that that role of defensive cyber security um  another mature perspective is to also have a red  
team and those would be another team of that  would simulate attackers and they would help  
the blue team understand what indicators  of attack look like and what indicators of  
compromise look like and so in in the penetration  testing in the threat hunting world that's where  
the that's kind of where the the desire is to be  as part of the red team essentially right and and  
what what silence sector does is is a version  of both right we call it a purple team approach  
um it's it's been you know defined prior to us of  course but but essentially it's a blend right we  
don't offer it in a vacuum and we recommend that  that's how the the technical assessment threat  
hunt or the penetration test where you're going to  refer to it or however far you're going to go with  
the assessment should be referred to  as and what that is essentially is a  
is a team-based approach so that we can train the  internal response teams now you're blue you might  
have a mature cyber security organization okay  you may just be a small b2b and you've got you  
know two network guys and a server guy that's your  blue team okay they've probably because they're  
they're jack of all trades and they're running  the small shop they've got a lot of logging tools  
at their disposal and they've probably got some  parsing going on so they're already running traps  
right for basic things that they just need  to know to keep the business running right  
um and that's a good thing about it it's one of  those things where i always feel bad for for those  
really really smart one it shop you know humans  that are that are just keeping everything going  
um because they need reprieve right they need  to they need a team of three or four or five  
to really do the job well but they've got a  lot of capability because they're so used to  
handling things that they you know they'll ride  a lot of traps for all kinds of things and not  
really understand that this is actually part of a  blue team exercise right and so when the website  
goes down and you you had some random script on  a server that was just randomly pinging a you  
know an item on the page that's going to tell  you that something strange has happened and so  
it could be an indicator of attack it could  be an indicator of compromise it could just be  
you know that the hosting provider went down  but that's sort of the what we're looking at  
when we when we when we conduct these exercises so  purple team is certainly the the approach that i  
would consider that you should prefer for those  that are asking any any teams that conduct any  
exercises or technical assessments against your  assets they should work with you and they should  
they should let you know when they're starting  they should also let you know when they find  
things right you shouldn't have to wait 40  hours or 60 hours beyond the end of the test  
period to obtain results that say hey you have  this critical vulnerability that i was able to get  
a you know a shell out of um or you know create  accounts or all these you know random things  
they should let you know immediately and  then you should also um turbo team exercises  
are also really good because you can schedule  these events and so the the teams inside right  
your networking guy your it guy or if you're  in a mature organization your blue team can now  
know when the testing period starts and they can  start looking for those indicators of compromise  
or indicators of attack and then they can they  can then you know go through their incident  
response protocol in order to you know pretend  the war game is real and what they would do to  
silence the attack right and so you get you get  a kind of a both benefit you're not only you're  
meeting requirements from a compliance you're  you're actually getting a princess to look at that  
attack surface that's in the target scope but  now you're also working to train the internal i.t  
staff and other staff to to know what this type of  traffic this aggressive scanning and these these  
types of inquiries to the electronic assets look  like at a telemetry level at the log files and  
what types of dashboards they may have the  day or or or logger irrigation they have a day  
and you can train them to see what that  looks like so when the testing period is over  
and something happens in the wild they're better  suited to acknowledge that what what this is  
what's happening and then and then if necessary  uh activate instant response score to call or even  
shut those those activities down so i i certainly  think you know purple team is the way to go  
um mike what do you what do you see um  is that typical what how how you feel  
about yeah for stockton i think it's evolving  right a long time ago it was just penetration  
test was just a commodity you paid someone to come  do it and then they looked at the results in the  
pre-charts and said thank you very much we'll see  you in a year but no i think the engagement of the  
internal team is is really very beneficial um for  for two reasons one they understand what you're  
doing and why and where the attack vectors are  it's an educational process and two it's really  
testing their ability to respond and we see that  to varying degrees of success and and failure um  
a lot of people put so much faith in their tools  that they fail to understand the human element of  
it and that's where the purple team will expose  those weaknesses just like testing your dr plan  
or your ir plan right you need to test your daily  attack plan your attack vector plan you know that  
that sort of thing you need to be attacked you  need to be able to test everything all the time  
because things just don't automatically work  right they never they never automatically work  
and and a lot of times um we've seen in the small  b2b world that they have they have better success  
writing something custom and small than buying  some big tool it's usually the the custom small  
thing that um i mean a great example of slow  post okay there there's really not a lot of of  
tools that will i did they can prevent the attack  but they won't identify it in a stream of traffic  
um and one of you know somebody we know wrote  a small script to just look to see if the web  
you know just should check the website up or  down and that was if they weren't able to tell  
that slow post was the denial of service that  was occurring but they knew that the site was  
down right and i mean it was just they didn't  need anything fancy it was a you know it was a  
15 minute script you know that they just ran right  so you know it's better to be clever sometimes  
with what you have then you know to certainly try  to spend on on big tools yeah exactly and you know  
relying you know just to bring back solar  winds actually as a case in point that was  
just a standard update that wound up blowing up  infrastructure all over the place things don't  
work the way they're supposed to all the time  so yeah trusted software not so trusted right  
but yeah a script that you write yourself it  takes you 15 minutes to write you can test  
it generally works hard to explain you know what  you put into it i call that organic sort of yeah  
i think there's a lot of hype in in the cyber  security industry uh around all the stuff that you  
should have all this stuff that you know or great  features and functionality but the reality is  
that um companies especially when you look at  mid-market and emerging size companies i mean  
they need their results they they need it's not  so much about being efficient as it is about being  
effective right they need to be doing the right  things um they don't necessarily need all the  
bells and whistles i think there's a lot to be  said there but you know so those are those are  
some really good considerations around frequency  of testing what type of approach and then finally  
you know another another thing that comes  up people ask well what should i get out  
of a penetration test you know i went  through this paid my money what do i get  
for it and of course that comes down to really  reporting and and hopefully some ongoing  
consultation right and and hopefully those things  are customized to organization and not just a  
a canned approach but um what do you see as far as  the most effective you know pieces of a report the  
most meaningful documentation um what do you see  that being for the client the sad thing is that  
a lot of these reports that we put a lot of time  into and a lot of people put a lot of time into  
everybody reads the first three pages and that's  it they look at the pretty pretty color graphs and  
a lot of it gets wasted i really encourage people  to actually dig into the meat of these reports  
yeah good point yeah no no no good point mike  and and here's the thing is that most of the  
most leadership that's they don't all they  care about is like did they find anything that  
you know is gonna is gonna lose our business  today right i mean that's kind of the common  
thing that leadership wants to know right do  we have something that's exploitable today  
the technical teams typically are you know when  something's not found but you know even us right  
we always encourage them like in the reports  but you know here i'll go i'll go ahead and  
pull out the lectern for a minute because i  have a lot of complaints in this department so i  
i didn't bring the complaint box because  i couldn't pull it with a one ton truck  
so you know i have a lot of animosity towards  some security organizations that just give you  
a report that just pulls out some crap like like  you know your x your x options you know aren't set  
you know for cross-site scripting right your cross  origins you you know or or your your secure cookie  
settings aren't there or http 80 is allowed on  the site even those are redirect it's like why  
don't you why don't you take a moment to go out  back and slap yourself and then come back and try  
to give the organization something of value and i  think that's really the biggest thing here is that  
that report should include something valuable to  you okay now there's always going to be trivial  
misconfigurations that are found okay and we've  talked about this in previous episodes where  
there's a big difference between something that's  vulnerable and something that's exploitable which  
is why you know they're in the news that mike  gives we're always going to hear about you know  
patches that need to come out that the vendors  have found themselves that say oh this could cause  
you know remote code execution okay there's not  an exploit out today for that that doesn't mean  
that something couldn't be developed but there's  a lot of time and man hours and research and  
there's a lot that goes into the development of  exploit okay it's not something that you can just  
proof in there it is um you know depending  on the type of the type of software involved  
sometimes you can do that right sometimes there's  a 20-minute script you can write that'll work  
a lot of times it's a you know 100 or 200 hour  endeavor and you've got to have access to some  
source code so you want meaningful things things  that are exploitable right things that i can i  
have a shell script today for that i could  drop and i have a payload to drop into meta  
plate today and point it at your system and i get  a shell that i can log in with right now right  
now that's what matters right that's what should  matter and that's what all these threat huds in  
the pen test should be looking for and the report  in the first three pages should not only say that  
um but if none of that's there it should give you  you know excellent marks for having a defensive  
depth cyber security program right and and and  protocols that you put in place um and then should  
also include the reports that are going to have  all those details all that you know the trivial  
misconceptions that may be there i'm not saying  we're not even us we're gonna let you know that  
yes these trivial misconfigurations for found  your web server is still open on port 80 and it  
does a redirect the 443 there's no criticality  exploits attack vectors that are going to cause  
you any harm here however it would probably be of  best practice to go ahead and consider how modern  
browsers works to go ahead and close that port 80  and not even offer the redirect anymore just force  
everything on 443. that's not our decision that's  an architectural model change that's something  
they should do but i'm not going to call this out  as a high vulnerability um or a critical thing  
saying that you know oh your website can get  hacked now because you allow it on port 80. so  
okay i'll stop mike what i miss obviously at that  point i would say you know if you've got it close  
anything that you don't need open first of all  let's start with that you know if you don't need  
port 80 close 480 and there's no reason travis  should be over 480 anyway um but no i mean yeah  
i mean we've seen unfortunately the position that  we've seen we we have some professional paranoia  
and we've seen a lot of bad things because that's  why we're here we're focused on the bad things and  
fix them making good things right so um yeah  i i i think you spot on laurel i think that's  
some very good very good points there's one other  thing i want to bring up real quick and we don't  
have time to talk to about it in depth today but  internal pen testing really needs to be something  
that needs to be considered because what we're  seeing is a lot of these bulletproof you know palo  
altos and f5s and although the f5 has issues right  now um but anyway these bulletproof perimeters  
and then the inside is a soft candy you know like  an m so um we need to start talking about internal  
fan testing and people need to start investing in  that as well now good point man we we don't see  
enough of that and when we do get an opportunity  i mean that's where all the fun happens from a pen  
test perspective um on the internal pen testing  everybody is all put all their focus on the hard  
crusted exterior it's just like the death star  right no one considered the little port no one  
considered that right and i'm sure there was some  auditor that was like you know i don't think this  
is i think this could be smaller because you know  it's a womp rat could get in here and you know  
technically a a missile might be able to come in  here so certainly look at the internal pin testing  
and and one final thing is is on that report  right what you should get the report but  
you don't want to share that report with third  parties because even though it may not have found  
the exportable items it's still going to have  intimate details about your organization about  
your technical controls about your technical  assets you don't want to just give it away  
right um so you should always whatever  organization that you work with should provide you  
a letter of attestation for that pen test that's  a publicly digestible document that says we're a  
third-party firm this is what we profess this is  what we do and these are our certifications and  
this is what we conducted against the environment  and we have tested it you know it was a low risk  
or whatever right we found so that's what you  should be providing to you should get that as  
part of your pin test report so that you can  provide that to third parties um customers  
clients may be asking for it excellent point  don't don't share your penetration test reports  
and so it's so amazing how many people say oh yeah  that this uh prospect wanted it so he sent it to  
him yeah and let me let me let me jump in too  and just just to point this out like when when  
it doesn't matter if it's a governance  security person or yeah some prospect  
they're going to put it on a sharepoint site on a  public drive someplace in their organization where  
who knows who has access to it keep that in mind  yeah might as well just work it over to the cyber  
criminals yeah well well good stuff and i think  there's a lot more to talk about here i mean we  
this is uh this is a deep deep topic but hopefully  we're answering some of your uh questions for  
those of you listening um and getting through some  of the common uh things that we hear uh out there  
and uh hopefully it helps you so thank you so  much for joining us uh please please rate the  
podcast subscribe and um that uh will help us too  if you provide some some insight about what you  
want to hear what questions you have we're happy  to answer them thanks a lot and have a great day  
take care take care pick up your copy of the cyber  rants book on amazon today and if you're looking  
to take your cyber security program to the next  level visit us online at  
join us next time for another  edition of the cyber rants podcast