00:00.00 Jonathan Johnson But you notre the fresh or you can' hear me hold up I Do apparently you don't appreciate the fresh cut man if you're wearing a hat damn Rip R P and piece. Yeah, that was a sign from God for sure. 00:00.00 Andy Robbins Now. 00:05.71 Jared Atkinson Well I Yeah yeah I don't. 00:10.93 dcppodcast You you see minded. 00:13.26 Jared Atkinson Ah, you're blurry. 00:15.25 Andy Robbins Yeah, you just what you just went like super blurry there for a second. Yeah. 00:19.98 Jonathan Johnson But that corn fed Texas editor. 00:21.73 Jared Atkinson Um, like literally right? literally right? when you said do you see mine. You just like it's like it looks like ah it's like it looks like we're ah it looks like. 00:27.19 Andy Robbins Yeah, yeah, like wick Pixel which Pixel is his hair. 00:30.93 Jonathan Johnson I can definitely tell even though Luke's pixel later right now he still looks pissed i. 00:35.45 Andy Robbins I know. 00:39.83 Jared Atkinson It's like looks like we're filming you but we didn't have permission. 00:45.26 Jonathan Johnson I damn I'm muted thing see I bet you Jared I bet you I bet you. 00:45.57 dcppodcast So much better isn't it. 00:48.50 Andy Robbins Wow. 00:50.75 Jared Atkinson I oh okay I noticed when I'm talking though like I don't get very much movement in my little line like look at Johnny's is like off the freaking charts. 00:52.66 Andy Robbins Wow. Yeah, your waveform goes flat. 01:00.76 Jonathan Johnson Yeah man I got some I got some lungs. 01:01.00 dcppodcast Um, yeah, mine is. Mine's like yours shared. 01:03.75 Jared Atkinson Ah, yours is still better than mine like I don't see any movement at all. 01:09.39 dcppodcast Yeah I mean I turned Johnny down in post. So. 01:09.68 Andy Robbins So I do on yours I see I see movement on yours. Wow. 01:12.28 Jared Atkinson Okay, Johnny's is like let's just say Johnny in the in the past ten seconds Johnny's clipped 1 2 3 4 5 six seven times in the past ten seconds so have fun with that. 01:14.19 Jonathan Johnson It's all about the Girth you know I'm saying. 01:18.88 dcppodcast Tony's is clipping. 01:25.22 Jonathan Johnson Well. 01:25.43 Andy Robbins 2 big facts, big facts. You got audio New York jared on my case. 01:30.24 Jared Atkinson Um, let's do this thing? yeah. 01:34.18 dcppodcast All right? We're good to go So everyone wants to kick it off. It was fine before don't. 01:37.95 Jared Atkinson How's that is that better. 01:42.13 Jared Atkinson All right? How's that is that back to normal. 01:45.50 Andy Robbins Same. 01:46.00 dcppodcast It's it's the I can't tell that anything's changing. 01:48.99 Jared Atkinson Okay, all right Johnny take us. Yeah. 01:51.13 Jonathan Johnson Take us away is this the last season of or this the last episode of the season. 01:55.10 Jared Atkinson I Don't know I don't like the season construct personally. 01:57.90 dcppodcast So okay, too bad. 02:01.32 Jonathan Johnson Um, so I just got to know because I was going to start off that way is this the last episode of season or no all right? cool. Yeah now with Andy someone with more Twitter followers for sure. Um, she. 02:05.21 dcppodcast Probably not. But. 02:09.77 Jared Atkinson Don't want to end it on this. You know. 02:09.94 Andy Robbins Yeah, now you want to you want to you want to end it on a high note. You don't want to enter it with me creature. 02:16.93 dcppodcast Wow. So. 02:21.33 Jonathan Johnson I'll start it off. Um, let me know when are we good? Are you recording or is it good to go Luke cool. 02:25.19 dcppodcast Yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah, yeah. 02:32.31 Jared Atkinson Okay, start, please just stop. 02:32.38 Andy Robbins What is happening right now. 02:33.60 Jonathan Johnson I let's go I get started everybody thanks for joining us on this beautiful Friday for another episode of the dcp podcast today with us is Andy Robbins um Andy has joined us before and it's an honor to have him again. 02:45.21 Andy Robbins Hello. 02:51.26 Jonathan Johnson Thanks for joining today Andy it's also we were talking just a little bit ago how it's very rare if you look in the background and he's in Seattle and it's actually not raining so it's very rare for that. 02:51.31 Andy Robbins Thanks, It's an honor to be here. 03:01.90 Andy Robbins Yeah, this is the first sunny day. The the Seattle area has had in about 45 years so got to take advantage of it for sure. 03:08.67 Jonathan Johnson Thing I think today. Um I know you and I like in Dms have talked about prevention and detection and how to couple those together I think that's going to be generally the the focus of today's podcast as as it's. 03:24.00 Andy Robbins Yeah. 03:26.74 Jonathan Johnson Not very much talked about I think in general of how detection actually helps prevention and vice versa. Um, so yeah, did you have any beginning thoughts on that before we kind of like dive in. Yeah yeah. 03:34.95 Andy Robbins Sure Yeah I mean like there's like where do you even start like with that topic right? Okay, cool all right? Okay, so I yeah so I mean I think this is kind of like an evolving. 03:39.57 Jared Atkinson Um, it's your monologue. It's like Saturday night live you get a monologue at the beginning to tell us. 03:46.44 Jonathan Johnson And don't do this with many guests man so feel feel special. 03:54.86 Andy Robbins Set of thoughts around this concept of detection of prevention of like the foundation of our discipline. What it what it is exactly that we're trying to do how we do that and whether or not we actually know if we're. Effective. So I think that as time goes on the actual problem that I've been trying to talk about has become more clear and it's one of those things and it's it's always been like easy to feel or easy to kind of like. Try to make an analogy about but it's been kind of hard to put into words but I think like the 1 problem that I think we can like anchor the conversation around and where I see common ground. Between detection and prevention. There's 1 word that that I think this kind of all lands on that word is accountability so what exactly is accountability accountability is the validation of trust so I tell Johnny to go do something I trust that he's going to go do it. But how do I hold him accountable to that I have to I have to verify that he did the thing that I want that I asked him to do right? Otherwise there is no accountability and without accountability. you really can't have trust and you really can't have long-term solutions to important problems. so I think I'll so I'll stop there I've got a lot more thoughts on that. But maybe that can be like a good starting point to have other people jump in. 05:46.58 Jonathan Johnson Jared. Do you want to go to you and I go cool. Yeah I mean I definitely agree I think with any process with insecurity. This should be accountability. Um earlier I sent you your tweet about like the accountability graph I think that is just kind of like perfectly set to move for today as well. I think it's. Um, one way that you can set accountability for detections a couple of ways I think it's kind of like in my head 1 of the main purposes of red teams um is to see what detections have been put into place and see what the organization process is of that time and then see if it could be. 06:15.98 Andy Robbins Oh yeah. 06:19.30 Jonathan Johnson And see if it can be improved I mean everything can be enhanced and improved What can be automated. Um, what needs a more manual process, etc, etc. Um, and I I think that's I think that's correct. But I think there's also a point of accountability for for prevention that may be harder to test as well right? So if we put a. 06:32.70 Andy Robbins Oh yeah. 06:38.50 Jonathan Johnson Prevention control in place I think oftentimes. It's one of those like um, hey we we've let's just use example Microsoft documentation told us that this would stop this? Um, and we've we've turned it on cool now we can just move on when there's not really a hey, let's come back to that and see if that's actually working. Um. I'm a big fan of like the asr rules that Microsoft um provides um actually I use them on my own computers today and the other day I couldn't like load Rocket leak because it didn't have it wasn't like the binary was too new and it just wouldn't load so I to turn it off Jared's about the flex about how good he is at Rocket League aren't you. 07:10.82 Andy Robbins Um, yeah. 07:11.35 Jared Atkinson Um, let's score the best off of I'm better than you so that? yeah. 07:15.96 Jonathan Johnson Um, okay, well okay there it is um so I'm a big fan of those but I I think I think the issue there is not a lot of people know they exist and there's not a lot of education around how to leverage those correctly and then um also not a much education on how to troubleshoot if those. 07:19.50 Andy Robbins And he can prove it. 07:35.10 Jonathan Johnson Don't work properly because um, actually with those rules. There are actual audit logs that are triggered. So if you wanted to actually hold that quote unquote accountable or test if that's working run something and expect it to be blocked if it's blocked. There should be in a log event or an audit event somewhere I'm a big fan of that. 07:48.58 Andy Robbins yeah yeah I liked what you said earlier about like you know, ah how exactly do you hold detection accountable how exactly do you hold prevention accountable and you like you made the point that like prevention and it might actually be more difficult. To hold prevention accountable. So I want to try to make a little analogy here and just like Zoom out and like recontextualize you know what it is exactly what we're talking about so what exactly is infosec infosec is a collection of people which includes. Vendors independent consultants companies that have internal programs trying to do their thing so it's people people trying to do a thing. What is the thing that we're trying to do. We're trying to secure information. Okay, that's taut logical but that's basically what it is. So as a group of people trying to do a thing I think that there's precedent and analogy to be made and for me the thing that comes to mind most immediately is the history of food safety. So if you rewind. One hundred and fifty years ago it's a big problem is people are getting sick and dying from drinking milk that has like bacteria in it or has gone bad or whatever. So eventually, you know Louis Pasteur discovers that if you. Heat up the milk to a certain temperature. It kills all the bacteria and it I don't know maybe it increases the shelf life of the milk and it makes it safer overall so there you've got a problem and you've got a solution the solution being the pasteurization process. Then to hold that pasteurization process accountable food processors will have sampling that they do whenever there's a a production of milk going through their facility so you get a whole bunch of milk that comes in on a truck. They got to pasteurize it. They got to package it palletize it ship it out during that process after they do the pasteurization they'll do samples of that batch and they'll look at the samples and they'll say was the pasteurization process effective for this batch and if they find. That it was not effective. They might throw the entire batch out and lose all of that and if it was effective then they have a good check on the pasteurization process and they can trust that that pasteurization process was actually effective so that's a specific. 10:37.57 Andy Robbins Example of like 1 organization doing a thing having accountability on that thing. But then what's the greater effect of that. The greater effect is it wasn't just Louis Pasteur working on this problem. There were other people working on this problem as well. A lot of those people. Were con artists and they were selling like you know magical elixir solutions to the problem of getting sick from Maybe you knew it was from the milk and maybe you didn't know it was from the milk. So. The greater effect is that after pasteurization was introduced the room for all those con artists started to evaporate obviously there are still con artists around in the food industry still but Louis Pasteur's solution. It has. Stood the test of time we still use it today probably going to be using it forever and so what you have is this incredibly effective solution that has changed the fundamentals of food safety and I think that. Johnny you said earlier you know that we can hold detection accountable. We can hold prevention accountable. Maybe it's a little bit different on prevention I think that right now what we're seeing in our field and I think this started maybe a couple of years ago is there is. A huge appetite and need for accountability I think that's why we have the miter attack framework I think it's why we have called Dara I think it's why whenever someone publishes a paper about How effective is each edr solution in comparison to all the other edr solutions people can't get enough of it and I think it's because we have this huge need and huge appetite for accountability in our field. 12:28.15 Jonathan Johnson And. 12:39.14 Jared Atkinson And one of the problems that we run into is like I think in the the example that you give the the indicator that you didn't solve the problem is that people continue to get sick but maybe like in infosec we face a situation to where the problem is. 12:49.91 Andy Robbins Sure yeah. 12:58.11 Jared Atkinson not as transparent. like the the manifestation of the problem is not as transparent maybe it's inconspicuous, right? So like if you let's say your you get breached because whatever your preventative or detective controls failed. You don't miss it like it's not obvious that you know that that happened right? and so that that creates ah creates a problem. I think there's like this. There's this weird thing to where everything is like ah everything that we do is a nested abstraction. So this problem that we're like just to kind of I don't know if we could maybe share this luke or if maybe in post-production. We could add this accountability graph that Johnny referenced that Andy put out. Um. But the the way that it looks is that there's kind of 3 nodes with some edges connecting them right? The first node is that there's a problem and the problem informs like some sort of audit control I suppose the audit control then validates this solution. So what is that? What is your like what is your solution to the problem and then the problem hopefully solves the the problem. Or the solution solves the problem. Um, the the weird thing is is that like a problem is like ultimately an abstraction right? And so um, you you like there's this weird thing to where you can try to address the abstraction but you. 13:57.70 Andy Robbins Um, right. 14:13.63 Jared Atkinson It depends on like what layer what level of analysis that you enter into the abstraction and it's not always obvious what level of ah abstraction is most appropriate right? And so okay, cool. Let's do it. 14:15.13 Andy Robbins Totally totally. I got a great example to to piggyback off you all right? So perfect. Perfect example is Ps exec my favorite lateral movement technique just the canonical lateral movement technique that we've been talking about for 20 years plus why have we been talking about it for 20 years plus you know, maybe we can talk about that later but ps exec so abstractions within abstractions I also think about perceptions of that problem. So what's the problem. Well first of all. There's that old saying that a problem well stated is half solved. So I think even before we even talk about what the problem is I think whoever can define that problem. The best is. Probably the one who is primed to solve it the most effectively for everyone Louis Pastor he figures out there's bacteria in the milk if I heat it up. It kills a bacteria the heating up part of that isn't as important as his ability to identify what the actual problem was. 15:32.43 Jonathan Johnson Um. 15:34.50 Andy Robbins Like it wasn't as if this was known by every single person who was trying to deal with this issue so going back to ps-exec I might look at ps exec and say the problem is that the network is flat. And because the network is flat. You can just hit 4 4 5 on every window system from every other windows system. So because I see the problem as a logical network segmentation or architectural problem. My solution is going to be Let's try to get people. To use internal firewalls. Let's get people to use Vlans with Ackles between those vlans let's try to get people to use the windows endpoint firewall I might do that Jared might look at the problem and he might say the problem is it's hard to tell the difference. 16:22.77 Jonathan Johnson Are. 16:32.22 Andy Robbins Between legitimate usage of what Ps Exec does and the illegitimate usage of what Ps Exec does. That's the problem. So my solution is gonna be against that problem. But I think that so like I have my own take on when I think the actual problem is. 16:47.45 Jonathan Johnson Yeah, yeah I was going to say like to that point I think peace I laughed I don't know if you something to laugh begin the piece exact because I just think it's a great example. Um, in general when I think of abstraction here here's what I think. Ah. 16:49.18 Andy Robbins But you know so it looked like Johnny was raising his hand he has something to say. 17:05.32 Jonathan Johnson Part of an issue is with detection as of today. Um, and this can be solved in different ways I haven't necessarily thought of like a really abstracted way to solve this. My my answer is pretty blunt but simply I think like the deeper you get into abstraction of detection. The more complicated. The solution is. And I think there's a threshold by which we pass in terms of complicated solution to where the solution becomes too conceptual and not practical and so we see a lot of times with detection people are just like in general like on say on Twitter they just say like. Why isn't Microsoft doing Xy or Z or why isn't like why don't we just like stop this and to me it's like it's like but yeah, it's yeah yeah, and I think like when I think of. 17:45.39 Andy Robbins Oh yeah, yeah, yeah, any any anytime somebody starts something with the word just like I pretty checked out like just do this like it's not that easy pal. 17:53.30 Jared Atkinson From It's my biggest pet beef. 18:01.31 Jonathan Johnson I'll try say as quick free you so you can get Joe Jared so um and if I'm passing anything that you want to say just stop me and I'll stop. Okay, um so I think while we so let's just like in my head. How I think like abstracting piece Exec if I have an organization comes to me. He's like we wanted to act every thing piece Exec Wise we want we want to see it I said great. Okay, so let's start with like in my head I'd be like let's start with prevention and Jared actually brought up a phenomenal thing about prevention. The last episode of not episode before that um about prevention and it's like the automated detection. It's almost like which is which is a really good way to think about it but I would start off with piece exec with the prevention. Preventative opportunities and so like what's there. Okay, well first off like there. There's w deck policies so like code integrity. Why are we letting anything drop to disk that isn't like necessary signed I think piece exec might be Microsoft signed but there's there's ways to like I'm pretty sure it is. 18:53.52 Andy Robbins Ah I'm not sure if it turned I'm not sure I'm not sure. 18:56.74 Jonathan Johnson Um I Want to say it is I'm not positive but there's ways to exclude those those signing opportunities for me mean dropped or executed on disk right? And so Great. We've Solve. We've solved that piece of the abstraction. The next piece is we can also take Away. We could also do a very like very precise if we time like on The. Detection spectrum right? A very precise detection of any time that that file name is executed started within within the system right? And then we take the abstraction harder now like keep in mind as I go through this abstraction. The solution becomes more difficult. Okay, and so like the next piece is now let's break down what piece Exec does. 19:30.72 Andy Robbins Chairman. 19:34.14 Jonathan Johnson And so then you want to look into okay well Ps Exec is going to drop binary to Disk. It's going to start a service and then it's also going to communicate somehow to the source host. Okay, so now in my head. There's 3 different ways to detect all those things and then we need to bring it together. So As soon as you keep going down and so we make detections for each one of those 3 things. Then it could get abstracted even more Now. It's like okay, well now I just created detection for service creation now I need to test if that's going to work for other service creation attempts and then it's like that's where detection the tree starts the abstraction starts get more difficult difficult I think that's totally fine, but what I'm saying is there are points of where there's a threshold. 20:06.20 Andy Robbins Yeah. 20:13.30 Jonathan Johnson Um, that I think it's hard to justify the practical use case of that detection concept. 20:16.20 Andy Robbins Totally I couldn't agree more I got I got something to say real quick before you go Jared. It doesn't good. So you you brought the point like you look at you look at ps exec and you start going down a tree of. What exactly is it like how does this thing actually work. What is it doing so you said it will create a service on a remote system. It will drop a binary and then use the service to launch that binary and then that binary is creating a reverse shell basically to this the system of origination. But none of it like so that tree. 20:49.60 Jonathan Johnson Um, yep. 20:53.66 Andy Robbins Like already you've got other trees at that same level. So like I don't need to create a new service I could modify an existing service. That's its whole of a tree I don't need to drop binary to disk I could use an existing binary like Powershell that's a hole or a tree I don't need to create a connection back to myself. Maybe I create a new local user. 20:55.48 Jonathan Johnson Um, yep, yep. 21:13.52 Andy Robbins And add it to the local admin screw. But it's a whole other tree. So when you say like following that tree I think I think the result of that effort is you're able to create ah pretty good detections that will Cover. You know the particular procedure very very well regardless of whether I used the Ps Ex Zig binary or if I used Powershell to manipulate the service control manager or whatever right? because like you're you're looking at like fundamentally how do the mechanics of this thing work in each of those. Ah. 21:43.35 Jared Atkinson Not man. 21:47.76 Jonathan Johnson Um, yeah. 21:48.36 Jared Atkinson Now I have 3 things to say by the way. 21:51.62 Andy Robbins Each of those pieces. So here's so here's here's here's okay so here's here's what I want to say here's what I want to say so you know going back to like the problem of ps exec. Okay for me the problem of ps Exec is not. It has nothing to do with ps Exec it has nothing to do with like the service control manager. For me the problem of ps exec is that organizations cannot do least privilege. So ps exec usually you need to be a local admin on the remote system to to do ps exec okay so lots of people in ad have local admin rights everywhere we all, we all know that you know pretty well by now. And controlling who has admin rights anywhere is a really really difficult problem but you know assuming a little bit. The problem is that users can do things that they should not be able to do or you know more specifically a user object has privileges that that human being using the user object doesn't need. In order to do their job but controlling that privilege for the past thirty five years of computing has been impossible for anybody to actually do in real life. So can we solve that problem. I think we can I think that's you know what? our product is trying to help organizations do that's one of the primary things that it does for our customers does that mean that we're gonna totally eliminate privileged access in active directory of course, not. There's no way that you can run ad without giving some people admin rights. You have to be able to do it to patch systems to reboot systems. Whatever to do legitimate admin things. So here's where I think the marriage of the way that I see prevention and you know what you're talking about with you know, creating very high fidelity detections. I think I think this is the exact situation where these things overlap and complement each other because I can't prevent everything but I can reduce the scope of the systems that need to be monitored very closely. To as small of a set as possible hopefully to just like the tier 0 computers like domain controllers and then what you can do is this effort that requires so much technical expertise and so much precision to create the highest fidelity detections. And and triage those detections you can do that against the smaller set of systems that we know are still susceptible to that particular attack. 24:34.58 Jared Atkinson So I have like 2 maybe 3 things to say we'll see kind of where it goes so like give me give me some time but I will pause in between to see like if you guys have something to provide us feedback. So 1 thing going back to what you said Andy about like identification of the problem. There's a really interesting thing that you brought up is this. 24:44.13 Andy Robbins Okay. 24:54.51 Jared Atkinson Perceptual like this perspective difference right? So like it's where are you? where are you viewing the problem from is like ah biases you towards how you how you are able to like manufacture or like state. What the problem is and I I think of this there's this book that I read that's. 25:07.38 Andy Robbins Hundred percent 25:12.90 Jared Atkinson About visual perception. So it's called what is it the ecological approach to visual perception. But I think it's actually like very informative. It's talking about how we view the world around us like literally. But I think it's like it's like and dealing with the environment I think is kind of what. 25:22.48 Andy Robbins Yeah. 25:24.67 Jonathan Johnson Um I want to have to Google what ecological means later because I have no idea. 25:31.21 Andy Robbins John Johnny Johnny pulled that up. 25:32.53 Jared Atkinson Yeah I don't it's some yeah somebody might tell me I'm wrong, but um, but like it. It relates to how we see things with like from the sensors that we have because like I view things like Edr or network sensors as being equivalent to our sensory organs like because like we're within. 25:34.33 Jonathan Johnson Beautiful. 25:50.88 Jared Atkinson We're within the world right? and we are able to perceive the world through our sensory organs right? touch smell sight like vision hearing whatever the fifth one is I can't think of what I forgot but okay anyway, so you have you have 5 strengths? Yeah host-based telemetry right? So yeah, so you have a subset of ah. 26:00.00 Andy Robbins I Don't know I lost track happen. Ah oh it's ah it's a host based telemetry. That's the fifth sense. Yeah. 26:11.10 Jared Atkinson Sensory capability within like how you approach the problem and like 1 of the things that they talk about in this book is like the nesting of things right? So this is abstraction but things are nested and so if I'm standing very far away from a problem I may see a mountain but as I get closer to the mountain I I see trees right. 26:23.87 Andy Robbins Um, yeah. 26:29.40 Jared Atkinson And as I get close to the tree I see branches and as I get closer to the branch I see leaves and as I get closer to the leaves I see cells and there's literally the range or the scope of the problem goes from the cosmic right? which is like unattainably large to the like subatomic right? and so like and where I'm standing if I'm standing you know. 26:40.35 Andy Robbins 1 Yeah yeah, yeah. 26:49.39 Jared Atkinson Ten kilometers away I'm going to have a completely different perspective than somebody that's standing at the foot of the tree right for what the problem is sure. Yeah. 26:53.14 Andy Robbins Can I connect dance can I say something real quick so ah, Carl Sagan said if you want to make an apple pie from scratch you first must invent the universe and I always think about that whenever people are talking on Twitter about like you know, just do this. 27:11.86 Jared Atkinson Um, yeah. 27:12.81 Andy Robbins Do this detection rule or just like key on this event or do this thing like in order to do that You already need to have like a highly robust ah windows event forwarding and aggregation indexing system already set up so like you know to make an apple pie from scratch to do. Detection at that level you first have to invent the universe and you first have to have basically a universe of log and event capability. 27:39.45 Jonathan Johnson Can I just say how good Apple pie is like you lost me at Apple pie because now I can't stop thinking of Apple pie. Ah. 27:41.40 Jared Atkinson Yep. 27:45.35 Jared Atkinson No okay here we go. So so yeah I think it's like I think about this perceptual like the perceptual perspective from the telemetry perspective which is like. 27:56.45 Andy Robbins Okay. 27:58.64 Jared Atkinson Somebody who like the same phenomenon is going to produce different types of telemetry right? So but correct and you you as as as the observer depending on what type of what perspective you're using or you have you're going to potentially. 28:02.30 Andy Robbins Um, yeah, like Network telemetry versus host telemetry like perimeter versus like in the land. Yeah. 28:18.12 Jared Atkinson See the same Phenomena same literal phenomenon differently right? So like if you see it from the network you're going to have some context that the host-based side doesn't have and vice versa right? And so like it's an ah, interesting problem that we face. But I think it also ah you know can be abstracted out to the higher like higher order problem which is like identifying What is what is the problem. So. 28:20.35 Andy Robbins Absolutely. Absolutely. 28:36.29 Andy Robbins Yeah there. 28:37.89 Jared Atkinson That's kind of like thought number one is this idea of how perspective ultimately colors our our view of the problem slash our view of a phenomenon or like some activity that occurs. 28:46.95 Andy Robbins I Think with I think with either of those like a problem that is faced with either Network telemetry or host telemetry or whatever just like trying to do detection at all I think one of the big problems. No matter what is volume like you've got way too many events coming in. So then we have like. 29:01.14 Jared Atkinson Okay, yeah, you just created you you opened up a whole box that now I'm gonna have to go down. But. 29:02.19 Jonathan Johnson Um, yeah I would. Ah yeah. 29:05.84 Andy Robbins Then we then we had like ah sure but like but but like you know to my point earlier is like 1 of the one of the big problems in detection is you got too much volume. You've got alert fatigue. You're you're paying Splunk $37000000000 a year to you know, collect and store all these things. Whatever. So the problem of there being too much volume of there being too much. Ah light I guess like to use the perception analogy like too many objects to analyze visually I think that I think that that's a problem that prevention can help. 29:31.70 Jonathan Johnson Um. 29:33.17 Jared Atkinson Yeah, yeah. 29:44.10 Andy Robbins Is limit the scope of what systems matter for collecting events from a limit like visually like how many objects I care about so instead of instead of seeing you know, just just pull something out just like immediately. So. 29:47.84 Jonathan Johnson Um, yeah. 30:02.51 Andy Robbins Instead of looking at a battlefield and you see like 10000 ah people on the battlefield and having to worry about every single one you know worry about the 5 that are actually like marching towards your position right. 30:13.50 Jonathan Johnson Um, yeah. 30:13.60 Jared Atkinson Yeah, but somebody has to be there so somebody has to be thinking about both right? So there needs to be somebody who's thinking about both. Um, if you think about 1 or the other you're you're going to have ah have a problem right? So that's I mean that's in the military we call that the tactical level is the worry about the 5 30:18.67 Andy Robbins Yeah, oh yeah, oh for sure. Yeah. 30:20.44 Jonathan Johnson Um, yeah, what. 30:25.83 Andy Robbins For sure. 30:31.21 Jared Atkinson The operational level is worry about you know the 50 and then the strategic level is worry about the the overall war or whatever like so it's like the skirmish the battle and the and the war is kind of like the the different perspectives. But if you don't think about all of those things simultaneously then you're going to have a ah major. 30:36.65 Andy Robbins Yeah. 30:48.20 Jonathan Johnson Yeah I want I Want to say I want to take a step back to what you said about um, host and Network telemetry being the volume. You know I think like I wouldn't say that's the biggest issue with that telemetry I would say the biggest issue is it's an issue. Yeah I would say one of an either another. 30:48.99 Jared Atkinson Major issue. 31:00.11 Andy Robbins And issue. 31:07.84 Jonathan Johnson Top tier issue with it if not the issue is the inability. No, it's not the inability because it's definitely Possible. It's the lack of knowing how to bring those telemetry sources together to form like real enrichment. So like I Really do believe there's this. I Really do believe there's something called this telemetry triad and I think this is where like um, enrichment really starts to bloom into something Beautiful. You know and that's when you take Network Telemetry Edr Telemetry and Native telemetry given to us. And you learn how to leverage those not only individually, but together. Um you know and I think like that's when we can really start to apply resources in different manners or at least have insight into different activities I've shown this with multiple notebooks that I've released or you know blogs or whatever it may be um, but that isn't being done. Often you know it's like because oftentimes a lot of analytical platforms out there don't allow it to be done Well um. 32:10.70 Andy Robbins You know that you know that reminds me of what you're saying right now is like back in the 1960 s 1970 s there were serial killers in the United States that would go from 1 state to another and do their thing like ah ah what's his name. Sure. 32:27.38 Jared Atkinson Jeffy Dahmer no I don't know no. 32:29.56 Andy Robbins Or or like Ted Bundy Ted Bundy's I'm I'm thinking of so Ted Bundy started operating if you will around the area that I'm in right now. Yeah, then then like then like he went up in Utah he was in like other states for a while but like 1 of the reasons that it took so long to catch him was that the police departments. 32:35.32 Jonathan Johnson Operating. 32:49.30 Andy Robbins And the you know law enforcement agencies at these different levels and in these different places weren't communicating with each other and they had no way to really easily communicate with each other or to share evidence. So if if a detective in Utah was working on a case and like here's themo and here's what I know about this person. They may have been able to solve the case way faster if they could have looked in a database and and said like oh hey, wait a minute this King County thing has a really similar mo and really similar things. Maybe the King County detective knows something that can help me to catch this guy like that's what make that's what that basic me think of is like you've got all these disparate. Telemetry sources that have different perceptions, different capabilities. Whatever but if they are not being used in orchestra with each other if you're not orchestrating those into. Ah. A better perspective then you're probably not doing well I guess I would say. 33:46.22 Jonathan Johnson Well I wouldn't say doing Well I would say each one of those telemetry sources are meant to be used by themselves like absolutely absolutely as we talk about like abstraction and when to hit those harder. 33:57.74 Andy Robbins There's like there's there's lost opportunity if you're not if you're not using and an orchestration. 34:05.57 Jonathan Johnson Detection strategies like we were talking about earlier when it comes to service creation remote service, creation, etc, etc. You have to start pulling these sources together and in order to actually get your best case scenario. 34:08.96 Andy Robbins Yes, yeah. 34:12.27 Jared Atkinson Um, yeah, there's there's a subset of context that is not obtainable from certain perspectives right? I mean like think about like criminal trials for instance, right? So like it's better to have multiple people that recorded some incident. 34:20.38 Andy Robbins Absolutely yeah. 34:31.30 Jared Atkinson Because then you have different perspectives that show something that potentially the like if you only had 1 perspective you might not have seen. Yeah and like maybe it like literally reveals something that was not obvious from a like so like like when we're talking about network telemetry like 1 of the things that Johnny and I worked on previously was this idea of like. 34:35.80 Andy Robbins You the corroboration. 34:50.00 Jared Atkinson Service creation something that's actually quite interesting from a ps exec perspective is ah so first of all detecting that a service was created is not that obvious of ah of a problem to solve and and so like for like understanding what a service is in the first place is not that obvious of a problem right? because. 35:00.40 Andy Robbins Um, oh yeah, yeah. 35:07.29 Andy Robbins Yeah, yeah. 35:09.67 Jared Atkinson You ask different people. What is a service and they'll give you different answers because there's different levels of analysis that you can analyze that problem that question but 1 of the things that we realized was really important is like services. 1 of the reasons that attackers leverage services is it allows you to execute code on a remote system. 35:26.44 Andy Robbins As the system user. Yeah check. 35:27.36 Jared Atkinson And so one of the things that you might yeah sure. Yeah, but like that's not I guess germane to like my point I guess but yes it is a good. It is a thing but um, one of the things that like 1 of the contextual details that's particularly interesting is whether or not the service was created remotely right? But if i'm. If I only have the edr perspective. Um, there's not an indicator necessarily that tells me explicitly that the service was created remotely. However, if I'm able to correlate that or corroborate that with like a network event something like Rpc monitoring of some sort. 35:55.71 Andy Robbins Right. 36:06.24 Jared Atkinson Might be able to see the literal Rpc request to generate that service and now I have that corroboration that contextual information which allows me to apply a different level of scrutiny to the situation. 36:07.42 Andy Robbins That. 36:15.44 Andy Robbins Yeah I also think about Dns like if you're investigating something and you have an ip address or a host name and the relationship between those 2 things has changed since when the event was created and when you're actually investigating. You might be on a wild goose chase because that association is no longer valid. But if you don't have that historical information or if you didn't enrich that at the time then you you might be on a wild Goose Chase 36:47.92 Jared Atkinson Yeah, yeah, so can I go to the second point that I had or we still okay, so the because I mean this is good but I don't want to so I don't want to change it but I don't want to forget my point either? Um, okay so the second point was this idea of like detecting ps exec and so it's like what is. 36:48.16 Jonathan Johnson Yeah, yes, yeah. 36:50.63 Andy Robbins Um, yeah. 36:57.42 Andy Robbins Get it. 37:06.92 Jared Atkinson What does that mean and 1 of the things that Johnny brought up was like you could almost get to the point to where um so there's like abstraction is the is basically the attempt to create like a universal ah universal truth almost right? Um, and so if you think about that from like a mythological perspective. You have like the hero myth. 37:18.27 Andy Robbins Right? right. 37:25.10 Jared Atkinson Right? So like the hero myth would be embodied through Harry Potter star wars the lion king all those are examples of the hero myth. The lord of the rings. All those types of things and the question is it's like why do we keep telling that same story over and over right? like if it's the same story and like we we only care about. 37:34.23 Andy Robbins Right? there. 37:44.15 Jared Atkinson Hero story why don't we just have 1 story like ah you know St George and who slayed the dragon. For instance, why don't we just tell that story over and over and one of the one of the problems is is that as the story becomes too abstract. It becomes something that like we can't we can't embody or maybe like the cultural context that's relevant that allows us to really understand and like. 37:59.74 Andy Robbins Yeah, yeah. 38:03.94 Jared Atkinson Feel like we can put ourselves in that in that kind of area starts to disintegrate right? and like and so there's like there's a constant tension between the abstract and the particular so the particular is like the actual action that you're going to take and if if something becomes too abstract It's not obvious what the first step of action is right and that that becomes a huge problem and so when we say something like detect ps exec that's an extremely abstract concept I think actually like maybe the best example would be something like red teaming so like what does red teaming mean to you well like red teaming comes from. 38:30.54 Andy Robbins Um, yeah. 38:36.16 Andy Robbins Yeah there. 38:39.21 Jared Atkinson A very specific cultural context which is like military type stuff and at one point that cultural context was understood by the vast majority of people that were performing red teams but over time it became so abstract that people had to kind of interpret it on their own because they didn't they didn't have that cultural context. And so they had to kind of like invent their own representation of what they assumed that it meant and now we have like every time you talk about red teaming purple teaming threat hunting is another example, you have to say what I mean by threat hunting is this and that is indicative of either. We never defined it like in threat. Hunting's case I think we never defined it well in the first place. 39:15.42 Andy Robbins Kind of. 39:16.87 Jared Atkinson But in red team's case I think we've we've it's become so abstract that we're not able to particularize it well right? So so with that being said I think there's this cognitive development theorist psychologist named John Piageget who is a ah so swiss psychologist. And one of the things that he was really interested in with like childhood development was this idea of breaking down abstract problems into like bite-sized chunks right? And so there there's the like the idea of 1 thing that I encounter frequently I'm sure Johnny and Luke have heard this a million times but maybe there's some listeners that think this is interesting is this idea of like ah. If I were to tell my 2 year old hey I want you to clean up this room. The chances that you know he's going to be able to achieve that and perform in the way that I expect him to perform the cleaning of a room is very unlikely. Um and so my job as somebody who's you know trying to guide him through life and help him understand what. 40:05.27 Andy Robbins Yeah. 40:13.50 Jared Atkinson You know things are is to break that down for him and so like I might say okay well you know to clean the room. There's a bunch of legos all over the place. So What we're gonna do is We're gonna put the legos away and like as I watch what his reaction is I might realize oh that's not even specific enough and so I'll say see the Yellow. Lego over there. Can you go pick that up and bring it to me right? and then I'll put it in the bag and then I'll say see the blue one over there. Go pick that up and bring it to me and then we'll put it in the bag and ah, there's this I think his name is it's like Vi vi kosky or something. Let's see. 40:31.92 Andy Robbins Nothing. 40:39.69 Andy Robbins Yeah, yeah. 40:50.31 Jared Atkinson To see if I have vygotsky is another psychologist that's interested in this kind of stuff and one of his ideas was the zone of proximal development which is this idea that there's ah, there's an area in learning to where you could perform the task on your own with no instruction and you've kind of mastered that right. 41:06.33 Andy Robbins Okay. 41:07.87 Jared Atkinson And so like repeating that is not valuable because you're not learning but then there's another zone to where if you try to do it on your own. You're going to fail because like you don't you haven't mastered it and there's an area between those 2 extremes that is with the proper guidance of somebody or with the proper cognitive framework. 41:15.67 Andy Robbins Okay. 41:26.64 Jared Atkinson You're able to actually perform the thing successfully with some sort of maybe some sort of help from a mentor or something along those lines and I think there's like tons of value when we start to break down these problem sets of like trying to break it down a into bite size chunks but also ah like from somebody who like works as a trainer occasionally or as a consultant. Help them ah kind of do it on their own right? And so as you're as you're working you want to keep them in that zone of proximal development to where they are able to actually like with a little bit of help or a little bit of encouragement or a little bit of direct instruction. They're able to kind of develop out. How do you detect? Ps Exec right? So it's like okay well let's break it down into what it actually is and like you guys have done that in this podcast. You talked about like the tree and how you break that down and all that kind of stuff but like that may not be obvious to everybody and so what ends up happening is people um and this is. 42:13.38 Andy Robbins Um, yeah, yeah, sure. 42:22.51 Jared Atkinson Happen to me right? So like you hear this abstract concept and you're trying to solve it at this abstraction layer that doesn't even like there is no particular nature to it and so you can't actually act in the world on trying to accomplish it. 42:33.29 Andy Robbins It's like it's like trying to detect Golden tickets without knowing what Golden tickets actually are right. 42:36.46 Jared Atkinson No boy, you're coming at me I Just did that. Yep been there done that. 42:36.74 Jonathan Johnson Oh man area here. We go hot take I don't think hot take I don't think you can necessarily detect Golden ticket I think you can detect the usage of a forge ticket but necessarily the actual like creation of a ticket but who am I. 42:53.54 Jared Atkinson I Don't think I think that's probably once you understand what a golden ticket is that should be self-evident. Hopefully so not too much of a hot take. 42:55.50 Jonathan Johnson Continue. 43:00.73 Andy Robbins I think I think not not to not to like get too dark with an analogy but like the creation of a golden ticket or forging a kerbarow ticket means that you have the credential material for the Curb Tgt user. 43:18.81 Jared Atkinson Yes. 43:20.72 Andy Robbins At that point from my perspective. Well, that's a big question but but I would say that at that point when you've when you've got the Curb Ttts in T hash. It's basically over for the defender in my perspective like. 43:21.77 Jonathan Johnson How'd you get that. 43:34.78 Jonathan Johnson Um, yeah. 43:39.85 Andy Robbins The the options of extraordinarily deep persistence that you gain once you have control of that account are such that I doubt most organizations on the planet could ever Hope. To positively identify all of those possible persistence options and so to use an analogy like in my mind detecting a golden ticket usage is about as useful as detecting detecting getting shot in the head. 44:16.12 Jared Atkinson Yeah, well I mean of course it's better to know than it is to not know I imagine maybe maybe that's not so that might not be asked like that may not be self-evident right? So like ah the example, the example that I was about to go to go ahead. John. 44:20.14 Andy Robbins Um, of course absolutely that it may not be Yeah yeah, but like like I. 44:20.54 Jonathan Johnson Yeah. 44:26.90 Jonathan Johnson Like is your is your point is your point that there's is is your point that there's a there's a threshold of a taxs to where it doesn't even matter anymore where you're like if they get that they get X after they've gotten X they own the org. So. What's the point of doing all this stuff after that. 44:44.86 Andy Robbins Let me let me go let me go the different analogy That's not as dark so you know even know the. 44:47.65 Jared Atkinson no no I think I think that so let me let me try to take a shot at Johnny's question and you tell me Andy if this is good for you so though like ah we all know the the funnel of fidelity tm you know the but the the general ideas is like detection right? The per. 44:54.22 Andy Robbins Okay, okay. Yeah, yeah. 45:01.70 Jonathan Johnson To you. 45:06.20 Jared Atkinson Detection is as I define it like the production. Ah the production of an alert is not the end of the funnel right? So like the the whole point of producing an alert is hopefully that you if like you then validate that it is in fact, bad or unwanted or undesired and then you remediate it as then you try to return everything back to normal. I think Andy's point is is that if you find a golden ticket the likelihood or your confidence that you can actually return it back to 0 or to normal should be approaching 0 um, and the only the only reason why it would not be approaching 0 is because you're ignorant to the possibilities of what could. 45:34.20 Andy Robbins Yes, that's right, That's right? and. 45:43.95 Jonathan Johnson Yeah I think I think like 1 thing I want to point out that you said Andy that I think super important I think like we picked up on this but I don't think like in general is picked up in general and I said general twice in the same sense and if a second is this is that you're very particular in the. 45:44.16 Andy Robbins Great and. 45:44.89 Jared Atkinson Take place. 46:00.90 Jonathan Johnson In the wording that you use when you describe it in Tech you said the detecting that it like detecting Golden ticket usage not like the Titan Golden ticket and when I think of Golden ticket in my head I'm like okay, let's just the creation of a Ford Cgt and then. But the usage piece is where and that's in um, another abstraction right? is like that's when we can dive In. It's like at what choke point can we start to detect these things and like instead of just being like we can't Detect. We can't detect a golden ticket. It's like well what about they use this can I detect that So It's like you might be late because. 46:23.54 Andy Robbins How are you. 46:38.98 Jonathan Johnson They've already technically performed the attack by creating a ford cgt but hey maybe we can detect some piece of that and I think there's like I think I've mentioned this on the podcast before but there's always like a pre. Well I don't know like pre during and post Jared heard me talk about this in terms of detecting and I think like. Overlapping those detection opportunities are super important like for example, sorrygenerat as I say like like kberrotine ldap queries for example for pre curberosine is you know obviously kerberroing and then post like leveraging that user to login somewhere access some resource right? All that in my head is all part of. Kerberosstein. It's just pre during posts. But then the post often leads into another attack accessing a resource with a valid account. 47:21.65 Andy Robbins Um, yeah, totally. 47:23.30 Jared Atkinson Yeah, so there's like a pre. There's a prerequisites to everything right? and sometimes you get those prerequisites for free as a result of like you happen to fish the right user potentially but like your point I Think in this case is the prerequisite to creating a golden ticket. 47:26.99 Jonathan Johnson Um, yeah. 47:41.29 Jared Atkinson Forging a curb tgt right? um is that you have the krtgt Kr B Tgt Account Hash right and that there's a finite number of ways that you can get that. Um, we may not even know what all of them are but is there are. 47:52.64 Andy Robbins Um, yeah I'm I'm sure we don't yeah. 47:57.50 Jonathan Johnson Um, yeah. 47:58.94 Jared Atkinson There's a finite there is It is true to say that there's a finite number of ways and so like it would be advantageous for us to either prevent. Well especially prevent but definitely detect the attempts to access the Krbtgt Account hash as opposed to. 48:13.41 Andy Robbins Um, yeah. 48:16.55 Jared Atkinson Ah, detecting the usage of a golden ticket for instance because like ah that for those that don't and that maybe aren't super familiar with golden tickets. The the point that Johnny was making about you can't to detect a golden like the creation of a golden ticket is that that can be done completely. It should be done if you're. 48:18.18 Andy Robbins Right. 48:35.38 Jared Atkinson Smart attack or completely offline because it's literally basically just a series of bites that are put together in the right way right? Um, and so you ah you have no, you have no perspective right? So it's the tree that falls in the forest when nobody's there to see it is kind of the the thing. Yeah. 48:39.74 Andy Robbins Yeah, the the attacker can do it on their machine that they control that you have no insight into. 48:45.55 Jonathan Johnson Um, yeah. Um, yeah, now he. 48:50.52 Andy Robbins Yeah. 48:54.50 Jared Atkinson But yeah I think that's that's really interesting. So then like this is where you get into I don't understand this very much but I I know enough to say it and and think that I'm like in the right realm but like the markoff chain type thing which is there's a pre. There's a set of prerequisites to creating a golden ticket which is getting that Caribbetgt account Hash right. 49:11.54 Jonathan Johnson Are. 49:13.87 Jared Atkinson And there's a finite number of ways that you can achieve that and then there's ah a a there's prerequisites to all of those ways to get that and so like the further back in the chain that you move the better better off, you would be conceptually right. 49:27.00 Jonathan Johnson Um, yeah. 49:27.53 Andy Robbins Yeah I think so with what 10 minutes left I don't know much time we have left but with with we have 45 minutes left 49:33.31 Jonathan Johnson Um, know but think I think it's longer than it our yeah and and this you have unless you have something going on. 49:33.75 Jared Atkinson We like have have 45 minutes probably we got as much time as you want I like at least yeah at least we got it. We usually do an hour and a half and we're at like 50 minutes or and. 49:43.94 Andy Robbins Oh okay, okay no I don't know no I just I was just gonna like I was gonna put a bow like on this loop they were in right now. So like sure. Okay. 49:51.81 Jonathan Johnson Um, yeah, because Jared had 3 things right? I think and then we're only on 2 Ah. 49:52.61 Jared Atkinson Put a but a bow on this one and then we can move on to the next may I don't know yeah but that I don't know I forgot what the third one was though. 50:03.20 Andy Robbins Ah, okay, so you know we're talking about the we're talking about Golden Tickets. We're talking about access to the Curb Tgt and T hash. But then what we're also talking about is putting effort into detecting. The usage of a golden ticket or accessing the Curb tgts in T Hash right? So any any C level executive knows that they have finite resources. They have finite money. They have finite labor. They have to pick and choose what are they going to put their effort into and. 50:22.90 Jonathan Johnson Um. 50:37.46 Andy Robbins If you are given the choice between putting your effort into detecting the usage of a golden ticket which I analogize to being shot in the head or if you could put your effort into creating a fortress with very strong walls. 50:47.82 Jonathan Johnson Are. 50:57.41 Jonathan Johnson Um, now. 50:57.45 Andy Robbins And putting guards in strategic places and you know if if you were just to assume that these 2 things were equal in level of effort. Yeah, and they they may not be but like if if these things were equal in cost. You would be a fool not to put your effort into. 51:02.53 Jared Atkinson In cost. Yeah, which they might not be fair to be fair. Yeah. 51:16.59 Andy Robbins Building the fortress placing the guards having access controls etc. Those are very very hard problems to actually solve in Active directory in particular. But I think that's you know my thought. 51:29.78 Jared Atkinson Now like going back to the just to explain kind of the the idea of like you're getting shot in the head if you find that somebody like you're already shot in the head and so it like doesn't really matter that you know about it. Yeah, the like the point is is that there's tons of really great ways that once you have the. 51:39.94 Andy Robbins Um, yeah, it's over. Yeah. 51:48.88 Jared Atkinson Caribbe Tgt account. You could create a bunch of persistence mechanisms that are very subversive and there's like a million ways to do that right? and so like it. 51:54.48 Andy Robbins At that at at that point at that point you own everything in the enterprise you own you own every windows system you own the kernel on every windows system you own the network device infrastructure because it's probably weight is authentic. 51:59.46 Jared Atkinson Um, yeah. 52:09.47 Jared Atkinson And there and there are like you know, recommended remediation steps for when that happens but the chances like the 1 thing that the chances that you yeah exactly like there I'm like that I'm you know there's like this interesting thing to where. 52:18.43 Jonathan Johnson Catches you'll catch the chances. You'll catch everything. 52:27.80 Jared Atkinson You talked about audit in that try like the I forget what you called it. The little 3 nodes it was like problem. Yeah accountability chart or whatever. Um, one of them is audits and like 1 of the interesting things is just because you have a process to do something or you read that this is how you should. 52:34.10 Jonathan Johnson Accountability. 52:34.75 Andy Robbins Yeah, yeah, yeah. 52:46.71 Jared Atkinson You should approach this problem like remediating this problem such as like the thing I'm thinking of is roll the Krbtgt Account Password twice just because you have that process and you executed. It doesn't actually mean it doesn't guarantee and probably unlikely to guarantee that. Um. 52:51.89 Andy Robbins Yeah, ah yeah, yeah. 53:01.41 Andy Robbins So you get a false sense of security If you're doing that and trusting that. Yeah yeah. 53:05.44 Jared Atkinson You actually solved the problem right? and I don't think anybody I don't think anybody actually audits that right? So like how many people during a red team have been like they have ah they have the golden ticket. Let's run through our remediation step which is roll the you know the Careibbetg account password twice. And then let's ask the red team if it actually like you know, kick them out of the network. Not many people have done that right and then just because it worked that one and even if it does work even if it does work just because it worked that one time doesn't mean that it will work the next time. Um, now there. There's like the the same logic of like. 53:26.40 Andy Robbins Um I don't think any and the answer to that is usually go. 53:26.49 Jonathan Johnson Yeah. 53:34.40 Jonathan Johnson Um, yeah. 53:34.41 Andy Robbins Yeah. 53:40.79 Jared Atkinson There's so many different things that they could do that. It's and it's unlikely that you know them all and that if you discover that you have a golden ticket that you would be able to find all like their persistence that that's the limit of that is approaching 0 right? Um I think it's I think it's also. 53:52.16 Jonathan Johnson Um, yeah. 53:52.64 Andy Robbins Yeah. 53:57.97 Jared Atkinson Simultaneously true, but maybe not as transparent or maybe we're more ignorant to it that like when you build the fortress. It's It's not as um, it's not obvious that you've covered every door if that makes sense because not like we don't even know what all the doors are I guess. 54:07.78 Andy Robbins Um, absolutely it totally makes sense. Yeah yeah, hundred percent hundred percent 54:11.40 Jonathan Johnson Yeah, 1 1 thing you brought up Andy that kind of like stuck with me on and resonated I want to talk about his effort. So um, when it comes to detection when it comes to prevention and we say we have some listeners which I hope we do. They're kind of. Wanting to make a difference in their organization and start to do things in a different manner. How do we? How do we say how much effort into putting into a detection whenever you have such a big gap right now. So right? like. Jared like I remember we've talked this with Robby is like he has that blog about eighty twenty right but what's the threshold by which we start to do abstractions on techniques to start to apply more advanced detection strategies. What's that effort level and when do we apply that effort level. 55:00.72 Jared Atkinson Yeah I have ah it's a less so like the 8020 thing is like a super abstraction right? and then like we need to start trying to deconstruct that I suppose. Well how do you know that you hit 80 I guess but the um so like there. 55:09.43 Jonathan Johnson Yep, exactly because you can't expect everybody to start at the very bottom like yeah. 55:16.10 Andy Robbins Um, yeah. 55:20.60 Jared Atkinson You have some finite capacity right? Which is what Andy's talking about right? You only have so many employees that only work for so many hours and are and are only so efficient but you can you can start to understand what that is through like measuring metrics right? So like let's say um, like and there's. That that capacity has to be spread across multiple things which is like implementing patching for instance or creating detection engineering and or triaging alerts right? But let's just like that that makes it really complex. So let's just talk about like triaging alerts. For instance, one of the things that you can do is you have like a relative. 55:42.54 Andy Robbins And. 55:57.75 Jared Atkinson Like a relative proportion. What what is the relative proportion of your capacity that you're willing to give this particular control based on the risk that you perceive that it poses to you right? So um, you shouldn't treat like not all. 56:08.54 Jonathan Johnson Um, yep. 56:16.96 Jared Atkinson Detections pose the same like are controlling for the same amount of risk right? and like how do you quantify that risk good part that's ah, that's a really freaking hard question. Yeah exactly. But um, you like basically the thing that poses. Whatever your analysis process is for determining the risk that an attack poses. Um. 56:19.78 Jonathan Johnson Yeah, not all attacks are created equally. 56:36.27 Jared Atkinson The thing that poses the most risk should probably be given the most capacity if that makes sense. Um, yeah, and so that's kind of like the first thing would be to understand what your capacity is which I don't think a lot of orgs actually do and then the second thing is like to make sure that your allocation of that capacity. 56:38.76 Jonathan Johnson Um, yeah, most attention. 56:45.76 Jonathan Johnson Um, yeah. 56:48.27 Andy Robbins Chair. 56:55.22 Jared Atkinson Actually aligns with your perception of the threats or the risk of each thing that you know you're dealing with that's kind of I mean that's still abstract I think but it's like a little bit less. Maybe it's a framework that people could start to work within. 57:00.36 Jonathan Johnson Um, yeah. 57:08.87 Jonathan Johnson Yeah I I have close thoughts there like the reason why I ask is because I kind of like 1 thing I've been learning while I do research is a scoping and b identifying how much effort I want to put into something right? because let's say like I don't need to put effort so much into. 57:15.49 Andy Robbins 1 up. 57:26.15 Jonathan Johnson Um, carboroine for example because there's so many great resources out there as maybe ah, don't know what's another good example, let's say petit potum that came out this year right? because like there's more known about curboroine than there is petit potumm. So maybe I want to like look into those and I trust. The the researchers that have already put information out about carrosine right? and so like identifying the scope and identifying how much effort I want to put into something so that way I'm not hanging up and um, like I think there's a threshold like obviously you can go as zb as you want with anything and there's always value in that because you learn something but there might be a threshold by which it like. Might not become applicable in your organization. So you might have to like grab that and say how am I working on this my personal time. Okay, let's dive into something else for work and actually there was one client that I that um I had a long long time ago that I think did this really? Well um is basically every week at the beginning of the week they had a meeting and. The manager would ask I forget what they use I'm so say like cycles like um, the terminology but they're like how many cycles do you think it's going to take you to get this done and he at that point he's identified how many cycles he thinks each person gets a week so like he's like I know this person works really hard. He can probably get things done a lot more efficiently. They get 10 cycles. So the person's still learning. They might get 8 cycles and so they're like someone's like oh I'm working on x excuse's like okay how many cycles eating is going to take. They're like 4 okaying now you have 4 of the cycles you can use in a week and as time goes on if you weren't able to complete everything that week you asked okay did you actually apply for cycles or did you use up all your cycles. 58:56.36 Jared Atkinson Or did take more. Yeah yeah. 59:00.16 Jonathan Johnson And start to like ah maneuver and adjust that so you understand what your team is capable of um I thought that was a really cool approach actually. 59:02.47 Andy Robbins Right? Yeah, ah so little little little change of Yearss little little different thought for you. So. 59:06.43 Jared Atkinson I think there's ah I go ahead Andy I just blanked out on what I was gonna say. 59:19.76 Andy Robbins I was talking earlier about how I think there's this like huge appetite for accountability for our ability to embrace accountability and to enforce accountability and I'm reminded by myself of ah. I'm reminded of my own experience as a pen tester and so when I started off in pen testing I was going to banks and credit unions and using things like interpreter response. 59:52.30 Jonathan Johnson How many years ago is this by the way like forty fifty years ago something like that. Yeah okay makes sense. 59:56.63 Andy Robbins Yeah, this is is like the 1960 s yeah, that's right? So ah, actually forty years ago would have been the the 1980 s um just to make you feel a little older. They're adjoured so you know this this would have been like 2010 to like 2015 01:00:06.93 Jonathan Johnson Um, Michigan. 01:00:16.24 Andy Robbins Ish like that range so using things like interpreter metasplo ps exec power view responder all these all these tools and capabilities to turn unauthenticated layer 2 access in a bank. Into domain admin for the bank right? So we would go. We would be there for a week we would use these tools and these techniques to take over the bank write a report and. Usually the customer would say okay, thank you very much. We're going to put your recommendations in place and hopefully you know next year you come back. It'll be harder. Yeah, so next year it wasn't harder ever at any place for years. 01:01:08.25 Jared Atkinson Okay, hold on hold on. so so Joe Vest joe vest once made me think about this from like a training perspective right? So there was like ah how this gonna make me seem like a bad person but there there's like sometimes you have a subsets. 01:01:08.29 Andy Robbins And this isn't this it. No no, no, no, no, no, no, no, no. 01:01:09.60 Jonathan Johnson You're just that good Andy. 01:01:26.71 Jared Atkinson Like you're trying to teach a topic or maybe I'm talking about a topic here on the podcast and like you guys all look at me when I finish talking about it and I'm like oh they they must have been too dumb to understand it which like in in the case of you 3 Maybe it's true right? But but Joe but Joe vest kind of made me think one time I was complaining about something. It's like it seems like. 01:01:37.86 Andy Robbins Right. 01:01:45.45 Jared Atkinson You know, nobody's really picking up what I'm putting down on this and I was attributing that to um, the students being being the student's fault right? and Joe vest was kind of like well like if nobody's picking it up. Maybe the problem you know, maybe the problem is like you like maybe like you're approaching this. 01:02:02.31 Andy Robbins Right? Yeah check. 01:02:05.29 Jared Atkinson Incorrectly and like once once I realize that it's like oh man that's like like that's literally true and every like even that this is the zone of proximal development type thing but it's like and like my point is is that if all of your clients after a pentest aren't getting better. Maybe the pentest. Is the wrong thing. Maybe that's not what they need but like it's so true because like basically what you're doing is you're providing them with something that takes them outside of that zone of proximal development if that makes sense. 01:02:25.99 Andy Robbins That's that's yeah, that's exactly right. 01:02:33.16 Jonathan Johnson Yeah I would say like that makes me think of like you know I used to have this comment I mean I've had it probably with everybody on this call before is the so what factor right is like if you can ramble on for however, long, you want to ramble out about some idea or some solution you have. But what is the problem. 01:02:33.29 Andy Robbins Um, yeah, just. 01:02:52.23 Jonathan Johnson What is your solution and what's the so what? Because like if you cannot hit those 3 things home. 01:02:52.84 Jared Atkinson Yeah, did you say I idea by the way I dear. Okay here you said you said you could ramble on about some idea you have and. 01:03:00.90 dcppodcast Yeah I heard that too. 01:03:00.66 Jonathan Johnson Um I didn't what I say idea about think I said ideal but listen you guys is cornfed internet over there is is making it go bad. 01:03:01.50 Andy Robbins That's what I heard Yeah for sure. 01:03:06.80 Andy Robbins Um, that's that that corned know no I heard some cornfed Missouri accent coming out of you just now who are. 01:03:12.74 Jared Atkinson And Missouri that Missouri is getting you? yeah. 01:03:16.97 Jonathan Johnson Man just because you're hooked up to Microsoft's internet right now doesn't mean you can talk that split. 01:03:21.15 Jared Atkinson Okay, yeah, so okay, so so that I was like sitting here while Andy was talking because I had forgotten what I wanted to say after Joni talked and I've I've thought of it. So ah, one of the things that he talked about was this eighty twenty rule and one of the things that's interesting is ah. 01:03:25.84 Andy Robbins Life. 01:03:38.11 Jared Atkinson When I talked about like the nesting of the mountain versus the tree or like the mountain the Valley the tree the branch the leaf the cells so on and so forth like if your problem isn't at the cellular level. You don't need to be. You don't even need to understand the cellular level and I think this is like ah super. 01:03:42.27 Andy Robbins Yeah, yeah. 01:03:54.86 Jared Atkinson This becomes really obvious when we start talking about something like gravity right? and so ah for years and years and years we we understood gravity to be this constant value this constant force and this is like the newtonian perspective of gravity right? And that's great because ah. From the perspective of terrestrial life. That's what it was right because I mean you know spoiler alert it has to do with like the mass of objects right? And the earth is so much bigger than anything that we were dealing with when we like terrestrially that ah it didn't matter right? like relatively like everything else everything else is gravitational pull doesn't doesn't matter right? But then we started launching launching things into into space right? to where there's other very large bodies that are potentially as close or close enough to the thing that we're launching into space that that nine point eight meters per second squared figure didn't seem to get us to the right conclusion. And so ah and that's because we're we're ultimately trying to solve a different problem that required a different level of analysis and so like I don't know if I actually don't know what the like what the problem set was that caused Einstein to have to figure this out but eventually Einstein realized that there was. 01:04:56.97 Andy Robbins And yet. 01:05:09.29 Jared Atkinson Ah, something to do with like competing bodies in space and how much they weigh and how they have different force and you know there's an infinite number of ah bodies in space and so but some are so far away that they don't matter and there's there's all this kind of stuff but like I just imagine um, somebody like this probably didn't happen. Maybe it did. But somebody launching a rocket like they're trying to send a probe to jupiter and they they calculated it for based off of nine point eight meters per second you know square they launch it out there and then they end up missing by like two light years and that's like that's kind of like and then they're like oh shit looks like we got that wrong, right? and so then like. 01:05:40.72 Andy Robbins Yeah. 01:05:41.78 Jonathan Johnson Um. 01:05:45.57 Jared Atkinson You know that's kind of how I imagine it and so there's this. Um, there's this thing that you will like you will always have ah a lower resolution perspective of the phenomenon than than what is like ground truth right? You'll never under like you you almost never can get to ground truth right? Even if you think that like you understand. Leafs at the subatomic level There may be a level below the subatomic level that we just literally don't and we don't know know about at this point right? But the the important thing is is like do you need to understand it that deeply that's Johnny's point like where do you stop? It's like you almost have to you almost stop at the point at which you can act in the world. So like when something's too abstract. 01:06:08.22 Andy Robbins Yeah. 01:06:22.66 Jared Atkinson You don't have you don't have an idea about what your next step is right? and so like you want to try to understand it to the point to where you get your next step or like what is the first step then you take that step and like ideally before you take that step. You have you have ah an idea of what you expect to happen when you take the step. 01:06:24.92 Andy Robbins Yeah, yeah. 01:06:40.76 Jared Atkinson So like when you perform the action in the world. Hopefully you have an expected output and then what you do is you take the step you measure whether or not the outputs that you expected occurred if it did then you knew enough for it to be good enough if if the expected output does not occur. You got to go back to the drawing board and figure it out again right. 01:06:52.93 Andy Robbins Yeah, yeah. Right. 01:07:00.57 Jared Atkinson Which that would be the equivalent of like I launched a probe at Jupiter and like we we freaking missed right? Um, that means that we didn't understand it enough and we got to we got to dig deeper. 01:07:03.54 Andy Robbins Right? So right? So first of all I hope Luca is able to edit out Johnny's snoring into the microphone that's been happening for the past 2 minutes but secondly I want I want to go I want to go I want I want to go I want to go back real quick to this like. 01:07:15.77 Jonathan Johnson Um, wow. 01:07:22.93 Jared Atkinson Okay, okay, okay. 01:07:23.23 Andy Robbins Pentest problem. Okay so any pen tester knows exactly what I'm talking about you. Do it, you do a pen test and against an organization you give them some recommendations. You come back the next year nothing's changed and sometimes you can even use the exact same attack path to get all the way to da or. 01:07:29.60 Jared Atkinson Yep. 01:07:42.50 Andy Robbins Maybe a slight little variation instead of pivoting to this host you pivot to that host and then that host and then you got da whatever. But if we look back at what I was talking about with accountability in that in that 3 node graph of accountability. Pint testing is not a solution pen testing is an audit mechanism pen testing is auditing red teaming is auditing. 01:08:08.50 Jonathan Johnson Well okay, hear me out hear me out's not the man yet. Okay, okay continue I I think I think like the I think the main purpose of ah I think the main purpose of it is auditing I don't think. 01:08:09.67 Jared Atkinson Um, talk about a hot take man just get it I Actually agree with you but I think a lot of people don't like that. 01:08:14.59 Andy Robbins So yeah. 01:08:24.49 Jonathan Johnson People that are performing pintest view it as that. 01:08:26.27 Andy Robbins They may not view it as that but a smart customer does see it that way So take. 01:08:31.54 Jonathan Johnson But my question but my question if the people performing it do not see it that way is the output Actually its original purpose. 01:08:40.60 Andy Robbins I Don't know if I can I don't know if I could answer that sure sure well and and that is kind of that is related to the point that I'm trying to make is pentestine's been around for a while pen testing. 01:08:41.97 Jared Atkinson It's good question like who who defines it as that. 01:08:44.21 Jonathan Johnson Um, yeah. 01:08:58.47 Andy Robbins You go in you rec shop. You come back the next year nothing's changed pen testing is auditing whether people like it or not so what exactly is it that pen testing is supposed to be auditing. It's supposed to be auditing the effectiveness of an organization's information. Security posture. So that includes a lot of things. Includes patch management. It includes these privilege it includes tiered administration. It includes detection capability and includes all those things so I don't really care to get into the argument about what pen testing is versus what red teaming is but both of them are very closely related and both sure. Yeah, so. 01:09:30.20 Jonathan Johnson Um, that's that's it. That's a Joe Vast episode 01:09:36.47 Andy Robbins Both of both of them are closely related and both of them are audit controls against an organization's security posture. Maybe they target you know determine the effectiveness of different things. But the point is years and years and years and years and years of pen testing and red teaming. 01:09:38.87 Jonathan Johnson Um, yep. 01:09:55.41 Andy Robbins Ah, like they're not. They're not. They're not without value. They're not without merit. They are tools that are used for the wrong job the job that they should be used for is to. 01:10:11.50 Jared Atkinson Yep. 01:10:14.98 Andy Robbins Assess the effectiveness of particular mechanisms in an organization's security posture. They also assess the effectiveness of things like Microsoft's internal sdlc processes by way of you know, taking advantage of ah a. An o day vulnerability for example, so you're not, you're not doing anything to you know test how effective the customer is at dealing with oday you're testing by Proxy Microsoft's internal software development life cycle you know processes. 01:10:45.20 Jared Atkinson This is this is the equivalent to when people say a V is dead and it's like no your expectations of a of what a v should be doing are not aligned with what it was designed to do basically so like when yeah, so like what. 01:10:56.24 Andy Robbins Sure. 01:10:56.33 Jonathan Johnson Um, yeah, and is thereby using the controls that a V provides or potentially provides in their. 01:11:03.30 Jared Atkinson Yeah, well, it's only a subset. It's only addressing a subset of the problem which at one point used to be our full understanding of the problem right? So like a V used to used to cover the entirety of the problem but like the problem has evolved over time and so now it covers a so very small subset of the problem but people still. 01:11:03.37 Andy Robbins Right here's here's here's. 01:11:12.42 Jonathan Johnson Yep. 01:11:16.46 Andy Robbins Um, yeah, yeah, right, right? right. 01:11:22.80 Jared Atkinson Problem is is that people still expect it to cover the whole problem. But it's like no it still it still does what it is what it always did. It's just the problem has expanded if that makes sense. 01:11:28.63 Jonathan Johnson Yeah, like my like I personally believe that like purple team was I think over time as like everything evolves so security and new offerings are created to help like like solve those problems I think 1 problem. That has evolved is the inability for red team pin testing teams to help out blueside to help the org out and so then purple team was created which I think a purple teaming and I'm like wasn't this the original purpose of red and of red in general. 01:11:53.60 Andy Robbins Sure. 01:12:05.91 Jonathan Johnson And then at that point like what's the point purpose of red teaming at that point what is that what tool like what is that solving. 01:12:10.74 Jared Atkinson Yeah, ah like ah this may not be a full perspective of what red teaming does but red team means like ah it provides 1 attack path which then allows you to evaluate what is like how confident. And like I don't even know if you could evaluate this well but how confident should I be that if I were to be attacked. We would we would be able to detect it right? But like there's no there's no like breadth perspective that's being given so it's like yeah we like. Let's say you let's say the red team tries to do kberosting and you detect it did you detect curber roasting because you have a fundamentally comprehensive kber roasting detection or did you detect it because they just happen to use the variant of curber roasting or the procedural approach to kba roasting that you were prepared to detect right. 01:13:01.56 Jonathan Johnson Or were you on high alert because you heard that. Ah yeah, or like sorry didn't mean interrupt you or like um, are you on high alert because you heard a red team is happening. 01:13:03.92 Jared Atkinson Ah, red in doesn't answer doesn't answer that question. 01:13:10.50 Jared Atkinson Yeah, but yeah, there's and so there's like there's numerous. There's like kind of like a people process technology approach to 1 of the one of the big fundamental problems is that the people that are conducting offensive ah kind of like services. Don't understand the defensive process like the detection and response process. Um, and so in my opinion the the offensive services are not aligned to even make sense from the like to be consumable to the defenders right? So there's like potentially a training aspect to it which is like I think of like Jocko willink was talking about 1 of the big differences. Ah, but like between the police and the navy seals. So Jocko willink was like a navy seal officer that led a bunch of teams and stuff and one of the things he talked about was one of the big differences between the navy seals and the police are that the police spend 5 % of their time training and 95% of their time executing. And the navy seals spend 95% of their time training and 5% of their time executing and so like obviously you know just by compound interest. Basically the navy seals are going to be much more effective at what they do and like ah like. 01:14:18.40 Andy Robbins Yeah, yeah. 01:14:23.16 Jared Atkinson Ah, traditionally like 1 of the things was It's like a red team would see the blue team as a training audience and so you would you would provide some training impetus for the for the blue team and that would allow them to work through their processes like let's say let's say that we thought that this worked right and they like when a golden ticket occurs. We're going to roll the Krbtgt Account password well like you don't want the first time that you do that to be after you detect a legit golden ticket you you would ideally do that in some controlled environment to where like somebody you exercise that process and hopefully you would do it numerous times so that when it does actually happen. You're ready to do it. You know with no questions asked but like. How many times have you encountered a customer that ah they they are breached and they have they have an ir plan but nobody knows where to find the ir plan. Nobody's practiced the ir plan and so they just wing it right? That's like literally the story of infosec and that's because we never train where we're the police. Not the navy seals Basically we always are executing. We're not. 01:15:20.94 Andy Robbins Yeah. 01:15:21.63 Jared Atkinson Not training and so that's like 1 aspect of what red teaming could do. You could also do like a validation kind of aspect which Andy I think is kind of getting at which is um like let's let's run this thing through its paces and make sure that it actually achieves the objective that we expect it to. 01:15:27.37 Andy Robbins Get yeah. Oh yeah, so I got another I got another analogy for you. Luke are you able to show that 3 node graph to our viewers. Okay, and for the audio only people I try to I'll try to speak to as well as I can. So if we look at this graph. What do we have at the top. Have a problem the problem informs an audit control and the audit control validates the solution which solves the problem. So I've got an analogy for you. You know I was just talking about how. Red teaming pen testing is an audit control. So it goes in the lower right? And what solution is it validating everything that we mentioned already logging and alerting patch management etc and what are the problems that those solutions you're supposed to be solving is that people are getting access to information illegitimately. Or you know that the organization doesn't want those people to have access to the causing denial of service or whatever, right? All the problems that infosec is supposed to be solving. Let's look at an analogy. So the analogy is ah car safety auto safety. So at the very top the problem. The problem is people get into car wrecks and they are injured or killed. That's a pretty big problem if we chase that to the audit. Mechanism the audit mechanism for this is the national highway safety board or administration or whatever. It's called so. What do they do? they take vehicles from auto manufacturers and they to use your term they put them through their paces. They run them into walls. They flip them over they collide them with other cars. They look at what happens in real life in you know, contrived circumstances but representative of real-life what happens when we test the safety features of these vehicles. So they are validating some kind of solution. What is the solution crumple zones seat belts, airbags etc and those things are meant to solve the problem of people being injured or killed in in car wrecks. 01:17:46.30 Jared Atkinson Airbags. 01:17:56.79 Andy Robbins Now here's here's here's the analogy that I want to make what if for the past twenty years the national highway safety administration you know the crashing cars over and over and over and over running them into each other running them into walls flipping them over putting like setting them on fire. Whatever. And there's some kind of safety issue that just permeates the entire auto industry that no autof manufacturer has a solution for whose fault is that is that the fault of any particular auto manufacturer. Is it the fault of the national highway safety administration is it the fault of the engineers at the auto manufacturers who are supposed to be solving these problems I don't think it's anybody's fault I think it's just it is a responsibility that all of us have to evolve the state of. In the analogy auto safety and so people come along and they develop you know pedestrian awareness auto break assist you know radar to automatically slow down cruise control. Whatever so these new technologies emerge that can solve those big problems. Red teaming and pen testing they are proving over and over and over and over and over that we still have big problems to solve in infosec. 01:19:23.96 Jared Atkinson Um, I think the um, an interesting thing when we look at this accountability graph is um, the resolution at which you ah analyze the problem is extremely important as well. So like you presented the problem as unauthorized access to information. But like if. 01:19:41.56 Andy Robbins We we could we could We could broaden that just like confidentiality integrity availability right. 01:19:43.20 Jared Atkinson Our audit. Yeah, let me let me let me let me work through this real quick. Yeah I think that makes it worse actually but the like um yeah, so like my point is is that if you design your audit to say yes or no they accessed information. They weren't supposed to. It's not obvious what the next. 01:19:49.73 Jonathan Johnson Um, yeah. 01:19:58.78 Andy Robbins Yet. 01:20:02.47 Jared Atkinson Step is to solve that problem and I think that's like ultimately the the issue that we face is that we're we're we're dealing with too abstract of a problem to actually identify where the breakdown actually occurred right? and so I think that what should what eventually ah red teaming will evolve into is something that's more. Greet discreetly focused so that you can solve individual problems right? So It's like um if if you were to if the auto manufacturing industry was like yeah well the problem is is that people are dying when they crash. Um, and that's all they looked into and then it's Like. Yep, they still died when they crashed here like you're not able to evaluate an independent control. So like how do you know that crumple zones are actually a worthwhile control to maintain or Airbags are a worthwhile control or seatbelts are worthwhile control. Well you have to like test them independently to some degree so that you can distinguish between. 01:20:57.30 Andy Robbins Yeah, yeah. 01:21:00.93 Jared Atkinson Like how do you know that a seatbelt works If you also have a crumple zone because maybe the crumple zone's doing all the work and the seatbelt's just there for for no reason right? And so like I think you have to ah if you try to measure it like ah a pin test I think is measuring it at this like super high abstract level and ultimately we need to start bringing it down. 01:21:05.77 Jonathan Johnson Um, yeah. 01:21:20.20 Jared Atkinson To a more tangible like particularized is the word that I would use a particularized perspective which is something like um when we create a detection for Kerber roasting how likely is that detection to actually catch any. Version of like any any implementation of Kerberos in that an attacker chooses to use right? and like so then it's more about breadth as opposed to I don't even I don't even want to say ah a pin test is depth because usually you would compare breath versus depth but a pin test is just like 1 maybe a pin test is actually depth to some degree. 01:21:49.68 Andy Robbins Yeah I think I think most people would say a pen test is Breadth Well a red team is depth I think most people would say that. 01:21:57.71 Jared Atkinson Is breadth. Yeah yeah, but like yeah so a red team is even more just choosing one path and like you kind of go with it and like so like as a red teamer if you are successful at you know, step a. You're not going to then like evaluate whether or not you would be successful if you change something up right? or you're not going to provide feedback I think there's like ah there's like technical controls which is like do we produce an alert right? So like when ah when this like when somebody tries to do Kerboros and do we produce an alert and then there's like kind of like procedural or. Human controls which are when we receive an alert. Do we properly categorize that and I think if you like the human control is actually like a big sticking point. So like don't I'm I'm putting this aside for now just for conversation but like it's a huge problem I think like basically our analysts making the right decisions and I think a lot of times the answers. No um. 01:22:51.26 Andy Robbins Sharing. 01:22:53.20 Jared Atkinson But like you can you can actually conduct a test like fully open so like one of the things that people do when they red team is they're like we want to do this without people knowing like we want this to be a surprise or ah, a black box evaluation. Um, but if you're only testing the technical controls. The technical controls are there like. 01:23:02.24 Andy Robbins Ah, right. 01:23:11.38 Jared Atkinson They don't They don't have consciousness so they don't They're not going to change their how they act based on knowledge that ah that ah they're being evaluated if that makes sense and so like if you could separate out the human aspect and the technical Aspect. You could actually like run it through its paces and it's actually like for instance, advantageous. Give the red teamer the detection logic that's being used so that they can then like try to evaluate the flaws right? and I think like I think as as we evolve, we're going to move away From. Can you get domain admin. Let's just say that That's like the typical kind of use case of a red team to. 01:23:33.43 Andy Robbins Hundred percent 01:23:47.58 Jared Atkinson Hey, we created this detection that looks for this particular attack or maybe you could do it for prevention 2 we have this preventative control that tries to stop this attack. Can you validate that it it. In fact, does what we expected to do and that's going to be. You need to hit this thing from as many different perspectives as possible and understand. Ah, what the assumptions are that are baked into our control and you know challenge those assumptions as much as possible. 01:24:09.97 Jonathan Johnson Instead of having just like a mindful like a mind mind field of controls or solutions and hoping 1 works. 01:24:10.35 Andy Robbins Yeah. 01:24:19.29 Jared Atkinson Yeah, because like let's say um, let's say that the red team is unsuccessful is that indicative does that indicate like and on the siso right? So like I pay a red team to come in and you know get domain admin or whatever, whatever it is try to access some system. 01:24:20.49 Andy Robbins Right. 01:24:36.34 Jared Atkinson If the red team's unsuccessful does that indicate that like we're good or does it indicate that they suck. 01:24:40.17 Jonathan Johnson Okay, yeah, so that's a good. Yeah, so like I was thinking about this earlier I just didn't bring it up thought it sounded dumb. But I'm just going to yolllo it and probably soon to sound dumb. So um, you know I think I think ah with yeah exactly you miss 100% the shots you don't take um. 01:24:48.65 Jared Atkinson Hey Answers always know if you don't try you know what? I mean. 01:24:59.87 Jonathan Johnson So I think how do we validate? you know like Andy you're talking about like um accountability. How do we hold the blue side accountable I think like it's a doubleedged sword. Okay, so like say we're let's take it. 01:25:10.89 Andy Robbins Um, or how do we or how do we hold our vendors accountable. How do we hold the operating system accountable. 01:25:19.13 Jonathan Johnson Let's take a detection engineering team and let's say like it's hard to it's a double edged sword because say if like no alerts fire today does that mean our preventative controls were successful or does that mean we saw enough detections. It's like exactly or does it mean we're not or does it. That's what I mean is like there's. 01:25:29.73 Jared Atkinson Or does it not mean anything like there may be literally nothing. There may be literally nothing to take away from that. 01:25:38.67 Jonathan Johnson Both ends of the spectrum if we're doing our job So good that no detections fire and all the preventative like controls caught everything or and that means we're doing our job or it means we don't have anything in place and everybody got through and. 01:25:52.24 Andy Robbins I'll tell you what it means I'll tell you what it means it means that you don't have the 3 nodes in that accountability graph you have 2 you have a problem and a solution. You don't have an audit mechanism so you can't know you can't know how effective the detection engineering team is. If. You don't have an audit solution. 01:26:09.85 Jonathan Johnson Um, what would you? What would you say in a practical sense. A audit mechanism would be. 01:26:15.67 Andy Robbins I Think most immediately because of mine is a tool like Calberra which is like something that can automatically execute ttps in a controlled environment. So say. For example, you had automatic Caldera tests firing. 01:26:25.18 Jonathan Johnson Um, yeah. 01:26:34.80 Jonathan Johnson Um, yep. 01:26:35.15 Andy Robbins Every hour of every day and maybe you have some kind of key like the host of it comes from or whatever so that you know that it's Called. They're a test What does that give you that gives you the ability to every day know that you're a detection system. Which is going to be a system of systems is functioning Correctly, as far as you can tell So as new as New ttps come about or as detection evasion techniques come about those have to be put in place as well. 01:26:58.97 Jonathan Johnson Um, yeah. 01:27:10.43 Andy Robbins And that strengthens the audit mechanism being the Caldera tests. 01:27:11.79 Jonathan Johnson Yeah, there's there's 2 things I see do you want to go Jared or. 01:27:16.88 Jared Atkinson I was just gonna piggyback on like my point earlier to where caldera would be you acting in the world and the expectation is that it like fires an alert and like the error one of the errors is that it does not fire alert which means that your your baseline assumption of. How your system is functioning is incorrect right? So like that's the thing is like that's I launched the probe to Jupiter and I missed. Well my there was some assumption I made that was incorrect right? same thing happens here on maybe a you know, less large scale miss both. 01:27:36.32 Andy Robbins Yeah, yeah. Um, yeah. 01:27:43.96 Jonathan Johnson Um, yeah. 01:27:48.13 Andy Robbins Yeah, yeah, like take ah the windows of that forwarders crashed for the windows of that crashed whatever and you and you don't know about that. 01:27:48.51 Jonathan Johnson Um, there's there's 2 There's 2 issues I see with the yeah. 01:27:56.35 Jared Atkinson Yeah, but the yeah the error like the cause of the error may not be obvious is one of the problems with broad testing right? So like that's why I like the discrete idea is like you can more easily 0 in on what the cause of the problem is like the causal issue. 01:28:01.10 Andy Robbins Shut up. 01:28:09.43 Jonathan Johnson Yeah, yeah, so there's 2 issues I see with that and um with that auto mechanism 1 being that um I think colddera could definitely and be a solution. Can't like I understand you're just mentioning one solution. But. 01:28:09.83 Andy Robbins Totally totally. 01:28:26.81 Jonathan Johnson Those need to be layered because there's 2 issues to see with that coldea is only going to want run a technique one way. It's not abstracted to run multiple times like in multiple different variations right? So that's an issue. So then we start to get blinded by perspective because we're like okay, well now we have say kerbaroine. Have Cribrosin cover guys. Well, it's like well that's not the only way you can do kberostine you know. So now we're now our perspective is blinded and that's where that's where I think like we need to train our blue teams. But if we train them a certain way or only in one way they're blinded then also if you run like those calera things over and over and over again. You're adding more to your attack surface and I think that's 1 thing that's not thought of when we think of automating these attacks so like you think of like the attack iq framework or you think of called dera and all these other things you have to drop something typically like a lot of times you have turn off defender in order to let it work and then let it do its thing. And then hope a detection fires you almost have to like you almost have to uncheck the preventative boxes in order to allow it to run and then even then to my knowledge caldera doesn't and I know attack iq doesn't at least it didn't a year ago it doesn't clean up its artifacts. So now you have more things sitting on the the host that you didn't know about there's only like. 1 tool out there that I know that actually cleans up its artifacts and so it's like there's that's a hard problem to solve as well. So now we've in order to solve a problem. We've created another problem that we didn't necessarily know about someone might not know about and that's another avenue or like say like let's say I don't know this this case we say. Carter creates a vulnerable service and he just leaves it. There. Well now that vulnerable service still lives there for an attacker to take advantage of so. 01:30:04.96 Andy Robbins Sure sure. 01:30:08.50 Jared Atkinson I think I I know we got a wrap up I think like to kind of piggyback on that I think ultimately whatever that system is Caldera Atomic Red team would be an example I think there's some other commercial solutions for it like you want to continuously build that up and make it more comprehensive over time. 01:30:19.79 Jonathan Johnson Um, yeah. 01:30:27.82 Jared Atkinson But like in order to do that. There's that's where like the value of a red team might come in like ah because you're going to have some subset of the problem solved through your automation and like ideally as you solve solve it in new ways or you address it in new ways you you roll that into the automated solution. 01:30:30.33 Jonathan Johnson Um, yeah. 01:30:42.50 Jonathan Johnson Um, yeah. 01:30:45.93 Jared Atkinson But you also always want somebody coming in with fresh eyes to provide different perspective which then allows you to grow that that audit capability. 01:30:47.18 Andy Robbins Yeah, yeah, well if I were I If if I were the decision maker in an organization I wouldn't want to have to wait that long to get that get that new intelligence or evolve that capability I would expect my commercial vendor. 01:30:47.66 Jonathan Johnson Um, yeah, like I think. 01:30:56.59 Jonathan Johnson Um, yeah. 01:31:05.70 Andy Robbins To be doing the attack research to doing the detection evasion research and automatically making that available to me without me even having to think about it and then of course you know providing some kind of digest like here's what we added did you catch it. Sarah. 01:31:07.78 Jonathan Johnson Um, yeah. Um, yeah. 01:31:16.77 Jared Atkinson Yeah I think there's a I think there's a problem of relying on vendors in that' when you when like you don't want to us. You don't want to assume that the vendors' doing a good job. Yeah. 01:31:17.97 Jonathan Johnson Um I. 01:31:23.38 Andy Robbins Um, you have to hold from accountable or not course not yeah. 01:31:27.80 Jonathan Johnson Yeah I think ah what I think ah so I realized that I I said I only know of one tool and I didn't say it tool I'm going to set the tool name. This is not a marketing scheme whatsoever. It is part of my so there is Atomic test harnesses out there that red canary is put out and the purpose of those. Is to have multiple variations per attack in order to judge the level of depth of your detection and it also cleans up all those artifacts. Um, those are public for people. But again, that's not a marketing and scheme I Just realize I didn't say the name of it. So if people were wondering. That's it. 01:31:53.58 Andy Robbins Thanks. 01:31:59.16 Andy Robbins Sure. 01:32:02.23 Jared Atkinson All right dudes. Well I think that's good I had 1 quote that I thought was appropriate I shared this with the with our team yesterday and nobody nobody responded to me so we'll see if this actually makes sense. But as we start talking about like this idea of how pin testing fails to hit the Mark. I think one of the things that I I think about and like if you listen to this you understand that I'm obsessed with this idea of abstractions and like trying to find the right layer and all that kind of stuff but the danger of abstraction or generalities is that it makes us think we know more than we really do because abstractions inherently include assumptions and if we never test. Those assumptions because we're always dealing in the most abstract sense then we never actually are able to validate that you know we're making the right decision so kind of ah, always be careful about the assumptions you're making and always evaluate whether or not, you're analyzing the problem at the appropriate level of abstract. 01:32:54.78 Jonathan Johnson Mean we would end the podcast on a real fucking good quote like that now I want to talk about that quote. 01:32:59.30 Andy Robbins Let shape let me let me let minute. Let me give a parting thought as well this year in 2022 python responder turns 10 years old 10 years of and incredibly reliable offensive security tool that every pen tester knows about every red teamer knows about the longevity and persistent reliability of responder mimickats ps exec etc is a condemnation on our discipline. 01:33:40.35 Jonathan Johnson I want to say one parting thought here this year Luke turns 10 years old I'm sorry I had to say that sorry Luke love you. 01:33:48.93 Andy Robbins I Love it. 01:33:51.29 dcppodcast Hey Andy we have a opening for a post on the podcast. So just go ahead and consider that you know consider this an offer for. 01:33:56.25 Jared Atkinson No man. 01:33:56.54 Andy Robbins Oh yeah, okay cool pick I do. 01:33:56.66 Jonathan Johnson If. 01:34:03.44 Jared Atkinson All right? Well that was a good way to. 01:34:06.46 Jonathan Johnson Dare You should do it like this so that I just look. 01:34:10.17 Andy Robbins Yeah. 01:34:11.95 dcppodcast I am a. 01:34:12.40 Jared Atkinson Yep yep, you took us from you took us from like Linkedin to and to Tiktok just like that. 01:34:16.60 Andy Robbins Yeah.