00:00.00 Jonathan Johnson Everyone thanks for joining us on I think this is episode 21 of the podcast since we have rebranded and not no longer doing seasons full disclosure I kind of wanted to do a season just because I thought the new intros and all the commercials were cool but you know what I what I say comes less. Anyways. Ah, with us today is ah Anton Anton for those that don't know you would you mind introducing yourself. 00:24.39 Anton Yeah, thanks. Ah, thanks so much for having me first of all, it's crazy that that I'm here among the mix of guests that you guys have had on so far that that's just mind bogging to me so we have big. Thank you? Yeah, my name's ah Anton Rusky Anton Lovesdnb on Twitter I work at lorez. Doing prep all blue team kind of things. But yeah, just super excited to to be here and hang out. Oh thank you that? Thank you so much. Ah, thank you. 00:49.44 Jared Atkinson Yeah man, Well you got you got to give yourself a little bit more credit you put out some good good stuff. So you you're definitely definitely somebody. That's worth talking to right and getting your opinion and thoughts. Yeah. 00:49.61 Jonathan Johnson Um, awesome. Will yeah. Yeah, absolutely, we're honored to have you here. So thanks for joining us today I know ah 1 thing that really stood out to me most recently was your your purple team post or your latin your lateral movement post forgive me. 01:03.80 Anton Oh thank you? Yeah, the the honors online. 01:16.64 Jonathan Johnson Um, and what could you kind of like walk through like what your initial methodology was for that post and like what was the impact you wanted to like push out after posting that. 01:26.72 Anton Yeah that's ah, that's a good question I think ah so like for for my day job. What what? we usually do is like purple team engagements and as part of those we do a lot of like lot of movement stuff techniques ttpss and I kept finding that clients didn't really like. Rasp all the like like the categories of lot of movement and like all the different nuances and I think lateral movement is one of the more interesting categories just because there's like so much thought that goes in to it from both like a threat actor perspective and a defensive perspective. Um, and then I always had that blog post by by jacksontkind of like I read it and I had it in the back of my head and I thought that post would lend itself really well to just like looking at lateral movement and thinking about lateral movement like a little bit differently than than what I usually blog about. Um, so usually like when I do a blog. It's like you know here's a technique here's how you detect it like here's a data source you you need and things like that. But the lot of movement one I thought that there's so much more like nuance and interesting ah like aspects to to chew on for it. So I really wanted to mix Jackson's like mental model and apply it to lot of movement and then kind of like mix in obviously like miter in there just because everyone loves miter and everything has to get mapped to attack and I thought like I just thought it would make an interesting like mix of of aspects that. 02:52.70 Jared Atkinson Um, yeah, yeah. 02:53.35 Jonathan Johnson Um, and. 03:04.11 Anton Would kind of highlight lateral movement as ah as something that's like a little bit more ah like requires a little bit more kind of thought behind it than other miter categories. Maybe. 03:14.21 Jonathan Johnson Okay. 03:15.23 Jared Atkinson When we're working with customers. 1 of the things that we we like to kind of talk one of the difficult things is we have a finite amount of detection engineering resources right? and so in an ideal world. You'd be able to write a detection for every technique sometimes that's not even technically feasible. 03:24.70 Anton Right. 03:34.14 Jared Atkinson In other cases, you just don't have the resources so you have to prioritize in some way and 1 of the reasons that you you alluded to and potentially even said outright that's really nice about lateral movement is that it's kind of like a central a central tactic to an overall attack path right? So it's like. Ah, you, you talked about how like initial access is a prerequisite to lateral movement but then lateral movement will help you do things like ah gain credential access. So like let's say there's some credential that you need access to and it's not available on the machine that you initially are on. You could laterally move to that machine and I think one of the interesting things. Is a lateral movement There's just ah, there's a finite number of lateral movement techniques that are kind of available so like with the exception of maybe exploitation of remote services just because there's a bunch of crappy remote services out there right? but generally speaking like if. 04:24.45 Anton Hit go. 04:28.60 Jared Atkinson If You want like ubiquitous laterteral movement techniques that are going to be available across the board in most environments, there's There's a finite number of ways that attackers can achieve that tactic and so it's it kind of lends itself to and it's almost guaranteed to be leveraged through. During an attack path as to where like 1 of the things that we used to recommend was kind of before edrs became Ubiquitous. We'd tell people if you have something that's maybe doing periodic scans for telemetry or for you know data from the network then maybe you want to look at persistence because persistence. By its nature is persistent. It's going to be there for a long time. It's Non- ephemeral and so that lends itself very well to that approach where it's like you don't You're not collecting near real time type information. The problem is is that like you could almost have persistence without. 05:06.74 Anton Right. 05:19.85 Jared Atkinson Like you'd have implicit persistence just by having multiple footholds for instance, right? And so it's not guaranteed that persistence is going to be used as to where lateral movement is and I think the finite nature of the different ah avenues of lateral movement kind of lends itself. Well for detection efforts. 05:25.55 Anton Right. 05:36.57 Anton Yeah, and it's also like a lot of lile movement techniques like blend in with like regular you know like network operations and stuff. Especially if you have like scanners or or service accounts that that do weird things. So yeah, like the the the whole like concept of like. Baselining and and then and like anomaly detection I think is like super important, especially for like lateral movement because if you look at your like your network and you you see like 1 hose talking to each other on like s and b or winner m you know that might be normal but it but it might not be so that I think that's where like the. The whole like mental model thing and then the the crunching of the miter data comes in probably more you know more handy with with a lot of movement than it would with other miter categories. But yeah, it's ah it's definitely interesting one I find too that the like the data sources required. For for a lot of movements specifically a lot of clients just don't have them. You know they don't they don't have P Gap and the right segment or the net flow in the right segment and I find that ed just generally kind of struggle with these techniques probably because the the techniques used are so similar to. 06:37.25 Jared Atkinson Um, yeah. 06:48.58 Jared Atkinson Um, yeah. 06:48.87 Jonathan Johnson I Think um, what's interesting me when I think about lateral movement is I definitely see almost like there is more data sources that can be applied to lateral movement than like say something like credential dumping on a host you know, credential dumping for living like dumping else as you have like. 06:48.93 Anton Operations that are normal. 07:08.74 Jonathan Johnson Open process right? like that's 1 you might have like a fileul creation if they use like mini dump right? Dump things like that. You're very specific on what you can and use for that. Um specific attack but ladder movement you have a couple different things you have. Um, host-based telemetry in terms of like a process network connection to another host and then you have the actual network traffic and then you have I mean just like with any time you want to access another box. You have um logons that have to happen there so you have that as ah as a data source and then you have like the server-side process. Um, data. That's there too. Obviously the incoming network connection to a process but then a process is going to execute something right? and so like you have all these things that I feel like when lateral movement is looked at um, expanding upon all the different data sources that are available to us is often overlooked. And people get very isolated into one data source like oh I must have network telemetry. It's like ah like network telemetry would be very nice to have absolutely but like it's not the only thing just like someone's like oh I must have host base like well no, not necessarily if you have network you can get you know maybe like 38% you know 45% of the way there. Um. 08:16.93 Anton A. 08:19.63 Jonathan Johnson That's one thing I like about latter movement is you have a lot of options. Um and a different that you could apply different strategies to for detection for that particular tactic. 08:26.87 Anton Yeah, and I think that's actually like a really good use case for for miter for for all the attack data that that a lot of people seem to overlook is like the data source aspect and which data sources apply to which techniques. So like I see a lot of people try to use like miter as like a bingo card right? like checking off like yeah we got coverage for this but like you know that's Fine. There's nothing wrong with that approach is probably like the best way of measuring. You know that kind of thing but ah I think the more tactical use case for it would be to look at like hey I want to I want to wrangle. 08:43.95 Jonathan Johnson Yeah, yeah. 09:01.65 Anton Lateral movement and what data sources do I need to do that and invites a really good resource for that and I and I think like the like you mentioned all the different data sources like stitching those together from like an analyst point of view and and building like a picture I think people kind of struggle with that because there's. 09:17.76 Jonathan Johnson Um, yeah. 09:20.39 Anton You know they pull a data source from here. They pull data source from there. You know the timestamps are off the field names are different like all that little nitty gritty stuff that that you have to like you know like you know boots on the ground kind of thing where you have to actually like do the thing it becomes harder. But I think like. 09:26.38 Jared Atkinson Um, yeah. 09:37.28 Anton These days with like xdr tools and stuff. It's getting a little bit easier. But if you're looking at you know, like Pcap that you're trying to like parse and slice and dice and then correlate those to it's difficult right to to do that kind of thing manually. 09:47.10 Jonathan Johnson Oh yeah. 09:48.63 Jared Atkinson 1 of the one of the things that I struggle with with the data like the data source thing is definitely a worthwhile endeavor and I think it's getting better. So I'll start with that but 1 of the things that I think that's currently missing with the way that data sources are represented is it's ah represented in a quantitative. Fashion but not a qualitative fashion. So like. For instance, if you were to say you know what's the most central from miters perspective. What's the most central data source like what do you think that would be like what's the most frequently used data source yeah process command line parameters right? yeah. 10:17.16 Anton Ah, from the chart. It's all the command line stuff that that process creation. Yeah. 10:24.89 Jared Atkinson Don't know that that's actually what the numbers bear out but I suspect that that's what the numbers bear out. Um, but the the problem is is that you have to like yes could process Command line parameters be useful in almost all use cases. Yes, but does it provide you with sufficient context in order to make a decision in any use case. 10:26.94 Anton Right. 10:44.50 Jared Atkinson Probably not right in a lot. So like you think ah like I think we look at service creation as like our that's kind of our canonical example and it's I think it's germane because that's a lateral movement technique to some degree I don't know if it's classified officially as lateral movement but it it is remote code. It allows remote code execution. Um, and. Like 1 of the things is is like yeah I could observe service creation using process command line parameters if somebody uses like http://sc.exc or you know whatever equivalent type type tool. However, you can certainly make services without using those tools right? or to where from the command line perspective the intent or the like the action that's occurring is. Is opaque or Non-t transparentsparent. Um, and so it's it's an interesting thing to where we need to make sure that we add a qualitative metric to it in order to properly evaluate and not just go with like the hey you know miter says that if we have process sch command light parameters. We'd be able to detect. 250 different techniques. It's like well you'd be able to detect some portion of 250 techniques is the important part. 11:44.10 Jonathan Johnson Yeah I think I think yeah I didn't cut you off did I Jared Okay I think it's interesting when people say you can detect given like this these many attacks given this telemetry. Um. 11:45.17 Anton Right? yeah. 11:49.69 Jared Atkinson No, no, no. 12:03.53 Jonathan Johnson Think there's a lot There's so many more variants in play though. So like let's look at analytic platforms for example, like if if you want to start utilizing multiple data sources and do joins like there's really not a lot of options that are phenomenal out there like I'm a. Like to me I don't like Splunk you know splunk like makes my life a lot harder than it really needs to be um, you know like Kabana is like nice if you're wanting to look for like 1 singular thing. Um I'm a big cousto fan or Jupiter fan. So like. 12:29.39 Jared Atkinson Sure. 12:36.62 Jonathan Johnson I like those capabilities and like then you could start to wrangle in multiple data sources and start to apply them for detections. But I also think it's another variant that I see is like and Jared I think you'll you'll love this conversation. You'll you'll just wait guys watch I'm gonna say this word. He's a bit. Um, exactly. 12:50.71 Jared Atkinson You know my you know my trigger. Basically yeah oh god. 12:55.40 Anton So. 12:55.16 dcppodcast Detectionomics. Oh. 12:56.19 Jonathan Johnson Yeah, ah I think like what is the noise ah like the like ah the signal. Yeah, the signal to noise ratio like see if we had process Command line parameters. Great. We can detect a lot of things. But. 13:01.87 Jared Atkinson The base condition. Yeah. 13:11.70 Jared Atkinson Mm. Okay. 13:12.30 Jonathan Johnson What is a false positive rate. That's going to come along with that if we're only looking at that data source. 13:16.87 Jared Atkinson What's the overall volume of that data source versus what's the useful volume of that data source and like just and and well so that's a qualitative thing as well. So it's like oh man. Okay, yeah, you you got me yet. Well we cut. Okay, so there's like ah there's like a. 13:19.33 Jonathan Johnson Exactly. 13:25.90 Jonathan Johnson Um, totally see yes that I told you but. 13:33.92 Jared Atkinson How many techniques would this thing be useful for right and then it's like what percentage of let's say we let's say we were to say what? what data source if we collected it would give us the highest percentage of coverage right? Because. Like the the reality is you're never going to have 100% coverage of any technique. So Let's say it gives you 25% here 70% here like which one which data source has the biggest bang for like well provides the best coverage but that's not even bang for your buck because what you're saying is like your buck is relative to the overall volume but like let's say this thing gives you. 14:04.21 Jonathan Johnson Exactly. 14:08.20 Jared Atkinson Lots of coverage but you know 99% of the volume that you receive is not part of that coverage. 14:11.45 Jonathan Johnson Like like open process. For example, like is a phenomenal data source in my opinion, especially if you know how to like manipulate like big mass math or whatever in your antelote platform. The issue is volume right? is like. 14:19.60 Jared Atkinson Yeah, well, it's only useful for it's only useful for one I mean maybe this isn't true, but it's basically useful for 1 technique or like 1 subset of techniques right? because like ah like maybe it's maybe it's useful for like injection. It's useful for overpass the hash. It's useful for. 14:29.95 Anton Yeah. 14:30.56 Jonathan Johnson Ju for Quiet it's It's useful for yeah anytime you're accessing a process to do something like it's great, but the the problem is like it's not a data source that you could be like yes, this definitively happened. 14:39.36 Jared Atkinson But credential like credential dumping. 14:48.93 Jared Atkinson Well, there's also there's also a component of like ah what percentage of attacks would use like let's say this thing gives you 90% coverage but it also only 1% of the total logs that come from that data source are useful right? So now you have. 14:49.33 Jonathan Johnson Once I saw this and also the volume is so loud that it's. 15:08.83 Jared Atkinson 99% but it gives you 90% coverage so you're like that's pretty good but then like you analyze and I don't know how you actually achieve this but conceptually you analyze and you say this attack technique that it gives me 90% coverage for is only used in 1% of all known you know attack pass. Well, it's like okay that's like I don't know when you start doing like. 15:21.22 Jonathan Johnson Um, yeah. 15:28.74 Jared Atkinson Percentages and probability you have to start multiplying those numbers against each other and so then it you know gets really small, really quick. Yeah. 15:31.66 Jonathan Johnson Yeah, get scary. Yeah, and also to me like this kind of goes back to our conversation that we had the other day Jared about like taking conceptual ideas and making them practical for an impact and this is one of those words like I look at like the data sources given to us by miter and like given that. X data source gives me say 95% coverage of the mida miter mita miter framework in my head and my head analysi paras guys. Don't forget um in my in my head like I think well with more coverage more volume of noise is going to be. 15:54.95 Jared Atkinson Might have. 16:09.43 Jonathan Johnson Applied to it as well. So like conceptually sure like great I have 95% coverage of the miter attack framework practicality of that is. 16:14.63 Jared Atkinson Um, yeah, Well I think that there's like ah so I get where you're going I think there's like a weird thing to where it's like ah often you hear something along the lines of and I'm paraphrasing. But it's like. The more data I collect the like harder it is for me to find the thing that I'm interested in which I don't know that that's actually true right? because like I could collect Net flow information and then I could limit my search to only look at process access events right? And so like the fact that I have net flow information doesn't matter in the context of how like difficult. 16:47.60 Jonathan Johnson Yeah, yeah. 16:48.24 Jared Atkinson That query is necessarily um, but you do have a finite capacity for collecting logs and so like the more volume you collect in 1 place the less volume you could collect in some other place right? And so like that's that's where it matters but like once you've collected it I don't know. Like it probably does like there's going to be some you know sim engineer that's going to be like yeah it does actually have a impact but like the the query you could write the query in such a way that you can minimize the impact pretty pretty well in most modern like query languages because they're smart enough to kind of limit the subset of data that you're interacting with. 17:23.15 Jonathan Johnson Yeah, so no I was going to ask you a question if that's cool. Go go ahead I like I want to own air truck. 17:23.18 Anton I Find a oh search engine I was just going to say like I think like and maybe you guys have more insight into this but from from what I've seen a lot of clients. Don't think about this kind of stuff or or discuss it especially if they do discuss it. 17:26.25 Jared Atkinson No go ahead. 17:42.87 Anton It's probably not in the context of their like threat model and and their like business objectives right? I mean usually it just it's like Mitr and like that's it. But I find that there's very little like strategic and like tactical discussion about like hey what data do we actually need. 17:56.58 Jared Atkinson yeah yeah I like to I like to kind of compare. So I've been working on this kind of comparison to the human perceptual system. So like your telemetry collection. You use sensors right? So like Edr is an example of a sensor. 18:01.88 Anton What are we collecting it for like what are we going to do with it. A lot of it I think yeah. 18:16.28 Jared Atkinson Ah, your netflow sensor is an example of a sensor those are equivalent to our sensory Organs right? So like human beings have eyes which are you know,? let's say the edr of our of our you know environment like of our system I guess you have ears which might be the net flow right? and like so on and so Forth. You have all these but you have a finite number of of of. Senses and there's potentially things to be sensed that you're not collecting right? So There's something outside the capability of Edr So currently edr only collects a subset but then there's like ah when you use your eyes you actually are your retinas are receiving more light so more photons than your brain. Presents like is presented with right and there's like ah there's actually a process.. There's a part of your brain that processes and says this is more important than this other thing and that just happens implicitly we don't understand how it works right? but like basically we actually in the terms of Cyber Security. We have the ability in some sense to. Decide What we think is more important right? And so like we're functioning I think it's called the the thalamus is the part of the brain that's doing this in the visual system but it's ah we're like functioning as the thalamus in like our logging policy to some degree and so it's like we only know about that which we see but how do we know that. 19:15.19 Anton Right. 19:33.47 Jared Atkinson That which we see is that which we should see so like in human in the human Sensory system. There's like an evolutionary process to where if you didn't see the things that you needed to see you were eaten or killed or whatever and like I don't think that we have We don't have like an equivalent. Ah we don't have as closed of a loop. Ah feed ah feedback loop like maybe the maybe the ah comparison is like a red team exercise you run that and that's the feed. That's the equivalent of being killed or like a ransomware attack like if you're ransomware. Then obviously you weren't looking at the right things? Um, but like it's not as I guess like ah. 20:04.92 Anton I. 20:11.96 Jared Atkinson Human like human the human species has more of a collective type approach to it as to where we're learning it individually and the problem is is that by the time you learn the lesson. You're already dead so it doesn't like actually benefit you going forward I don't know I like I haven't quite figured out how to present that but that's kind of like my analogy that I'm I'm starting to. 20:28.69 Anton It makes a ton of sense and especially like that just like you was right like every corporation every every business is different with like different networks and stuff So the the priority on which data or what they need to be seeing and what they need to be like how that cycle completes of like. 20:31.31 Jared Atkinson Like. 20:47.44 Anton Visual and getting into the brain. You know that they the equivalent on the logging side of that that always is going to be different for for each organization. So see. Yeah yeah, and like every environment has its own like subal limitations and and skill sets that that are available to it and and things like that. So yeah, the. 20:51.82 Jared Atkinson Um, yeah, yeah, and you're a product of your environment to some degree. 21:06.47 Anton But I just don't see like that discussion happening like when I bring it up to clients. They're just like I We never really you know thought about this kind of stuff before. So yeah. 21:06.92 Jared Atkinson Um, yeah, yeah, we'll say you know you know what it kind of turns into is like ah so like yeah, everybody has different environments and they they have different needs based on that environment. But there's like some metaset of rules right? So There's like. Meta set which applies to everybody and then there's the individualized you know particularized per your environment and the meta set should be generally speaking represented by vendors right? They should be solving the things that are universally applicable to everybody right? and so but like ah maybe. 21:40.39 Anton Him. 21:44.12 Jared Atkinson Maybe what you're experiencing and I like I don't disagree with the sentiment that you're that you're discussing but like maybe what you're experiencing is people are assuming that the vendor has solved both the universal and the particularized for them as opposed to saying. Okay, we have the universal and like there's a question of whether or not. 21:57.45 Anton Yeah. 22:03.84 Jared Atkinson The vendors have properly solved the universal that's that's a whole different problem right? But like let's say they did you still you still have to solve the particularized which is like the or the idiosyncratic aspect of it and like it's not realistic to expect that the vendor is going to solve it. But I think a lot of people do. 22:21.80 Anton Yeah I think a lot of people like get the you know, not nod to pick on vendors too much but I feel like a lot of clients. Don't hold the vendor's feet to the you know proverbial proverbial fire enough like if the detection is missed. They're always like hesitant to work with the vendor or something like that and. 22:31.14 Jared Atkinson Um, yeah, yeah. 22:36.32 Jonathan Johnson Um, yeah. 22:39.70 Anton I Think that's one part of it and I think the other part is because they don't think of that defensive orientation. Strategically they don't pick the vendor strategically either like I always see like a mismatch of ah like the vendor does certain things well but the client already has other things that do the same. 22:49.59 Jared Atkinson Yep. 22:58.30 Anton Process well as well. So nobody really thought like you know they they pick the vendor does a thing but whether that thing is actually important to the organization or whether actually providing new value. There's a lot of like it's super complex right? If you think about like an organization how to go through that whole process like all the people involved the consensus. 23:04.57 Jared Atkinson Um, yeah. 23:10.30 Jonathan Johnson Um, yeah. 23:11.92 Jared Atkinson Yep yep, yep, there's there's yeah. 23:17.47 Anton But has to take place. Ah yeah. 23:17.69 Jonathan Johnson There's always these bakeoffs. There's always these bakeoffs right? and I've always been curious about the bakeoffs because like the question is like like what is the metric that you want the vendor to hit and it's like okay coverage of miter but the issue is is like in my head a attacks are expanding past miter like if you look. 23:33.73 Jared Atkinson Um, yeah, well. 23:34.61 Jonathan Johnson Some attacks like to come out like they might not even map directly to miter so again so miter might be a great starting point. But it's not where we want to end and so like to that? Yeah, and so like that being said when we do these bake offs. Um, you know. 23:43.11 Anton Pp. 23:44.29 Jared Atkinson Miters trailing trailing reality. Yeah. 23:53.25 Jonathan Johnson You see customers like saying like they do this thing really well. But this person's do this. It's like well what is your org need like what other like tooling. Do you have in play like do you need a vendor that does like that has like mixed in network things really? well if you already have a network solution. Probably not, you might want to look into. 23:58.20 Anton The head. 24:12.31 Jonathan Johnson Does that vendor hold like log on and identity information more than that right? So it's like picking and choosing what you want so that like it's not what does this vendor solve today for our problem like how do they solve our problem today. But how do they solve a problem today and any. Like problems that we might have in the future that we don't know is a problem yet and like and I think when it comes to vendors to for term about Edr is like I was just having this conversation with somebody though day is like I think like most sensors. 24:32.35 Jared Atkinson I Think it. 24:33.80 Anton Yep. 24:47.75 Jonathan Johnson You have to think about what type of data you want and what type is given to you like. For example, like you have like process bound telemetry which is like the basic of all Edr Ven is going to give you but then you might have like some like. You might have some like mde that are starting to put like a little bit of pizzazz in their information like I just saw this past week like well messing with my mde that they're inputting like net like network share tables in there's no data in there yet. But they're starting to input those things so they're starting to move past the process and to me that's additional. 25:16.87 Jared Atkinson Um, yeah. 25:20.79 Jonathan Johnson Information that we can start to leverage in terms of detection right? Um, and so. 25:23.52 Jared Atkinson I think there's this I think there's this weird thing to that point Johnny to where Edr vendors are actually like they're ah they're the transport mechanism of the genome and the evolutionary kind of example to where it's like. Our feedback to them should inform them and they're the thing that maintains the the genes as it progresses over time because like you know, conceptually we're we're finite and we're gonna die. But the the vendor lives on yeah I don't know if that's true, but there there's this weird thing to where um like we're. 25:47.66 Jonathan Johnson Yeah, this on forever. 25:55.73 Jared Atkinson We're all ignorant to different degrees right? That's kind of like a fundamental truth of reality right? um and like conceptually a like a lot of times the customer is demanding something from the vendor that the vendor knows is not actually the best you know course of action going forward, but they're incentivized based on you know money. To do what the customer asks otherwise like it's great if you like how many times have you seen like the the vendor that has the best solution doesn't actually win because kind of like the market's not ready for them if that makes sense. Um and so like basically what ended up happening is all these vendors. 26:25.62 Anton Yeah, if. 26:32.57 Jared Atkinson Saw, let's say I don't know who who it was but let's say it was carbon black because I kind of perceived them to be 1 of the first like true kind of edrs in the in the sense that we think of it now and it's like everybody saw what they were collecting and then it's like well we have to at least collect that right and then like and that kind of set the the flag in the ground I guess that everybody had to meet on. And then like to Johnny's point Md is kind of like going forward and and they they have like better control for this type of stuff I think but so maybe it's easier I don't I don't actually know but conceptually it should be should be easier for them to expand this and they have like full control of the whole stack. But um, like they're they're able to kind of. Go out and say okay well what if we collect this thing and just kind of see what happens but that's that's the exploration that I think is extremely valuable is like like how do you know like so just because everybody is collecting process events. How do we know that? That's the most important thing to be collecting right? So that's like 1 of my fundamental. 27:26.90 Anton Um, yeah. 27:30.71 Jared Atkinson Issues with like Anomaly detection is there's There's ah, there's an infinite number of dimensions of analysis right? So There's a there's an infinite number of things you could look at and an anomaly can manifest itself in any ah along any of those dimensions right? But you're you're only analyzing. That behavior along a finite set of dimensions. So What it like how are you? So sure that like when you're looking for anomalies like I think it's probably true that bad behavior like ah like malicious activity is anomalous. The question is is how are you sure that the anomaly will Manifest. So in the place that you're looking for the anomaly especially given that the attackers know what your what your perceptual capability is right? So They know what event you're looking at and they're like we saw this this used to be a thing to where you would like make your malware and then you would run it against virus total and you'd be like okay look it doesn't. Doesn't pop up on anything I'm good to go right? that that was them taking advantage of the known battle space or the known perceptual capability of their target and then going from there. So it's It's I think that there's some some really bad assumptions in the Anomaly kind of approach in general. 28:43.15 Anton Yeah, and I think no it it makes sense because I think that dynamic that you described is is only exacerbated by a non-s strategic purchase and placement and deployment of the edr right? So it it makes sense especially like. 28:45.28 Jared Atkinson I like completely went in a different direction. Johnny. 28:45.86 Jonathan Johnson No I. 28:54.89 Jared Atkinson Um, yeah, yeah. 29:01.80 Anton Even something is basically do you want the edr to be more hands off or or do you have skilled people like go in like Md is a good example like do you want it to do like auto remeddiious stuff or do you want to really take advantage of that advanced hunting capability like do you have the people who know like kql to know what they're looking for that kind of thing and if you do. 29:17.71 Jonathan Johnson Um, yeah I think. 29:18.60 Jared Atkinson Yep yep. 29:20.83 Anton That might be a great purchase right? then but that's a great idea to to get but it depends. 29:23.20 Jonathan Johnson I I think also like man I just thought of this like okay so we had a conversation I don't know if it was with Andy the last time or maybe somebody before that. But I remembering this up like and Jart's her being like harp on this a thousand times so he's gonna probably roll his eyes. You know he's from like not this again, but like but like the data triad really. 29:37.78 Jared Atkinson It's okay I'm gonna go with it. 29:42.50 Jonathan Johnson Like I talk about this single called the data tryout and really what I think is like you have like 3 really like sources of data and that's network you have native ah telemetry so like in windows let's do windows security events and then you have the edr and like I really think dependent on what your detection strategy is. If you want to start to really loop all those in like you really get your best bang for the buck if you start to like do really advanced detection strategies and start to apply those like I did that with some Rpc stuff. Okay, so like the issue though that I see is like how many this is a great question for you Anton how many. Customers have you seen that are actually collecting collecting and utilizing window security events. Yeah, so like that's where I that's why I think like but mde might be starting to add these things like network share into their Edr platform. 30:23.80 Anton Yeah, and not not as many as I'd like to see on. Yeah, right? Yeah right. 30:29.77 Jared Atkinson Considering it's free. 30:35.31 Anton Thing right. 30:38.72 Jonathan Johnson Because they're like hey like we need these events in order to like do better analysis for Detections I know and like again like they have access to it because it already just comes from etw provider. Let's just pull this data because it's the same. It's I haven't like there is no data in the table yet. But it's probably not the same. Or it's probably going to be the same fields as like the windows security event I think it's like 51 45 is going to hold. Um, but I think that is where like some vendors are going towards and so that they start to actually like close that gaplic gate now you guys don't even need the window security events. We'll take care of like. 31:03.39 Anton Yeah. 31:16.96 Jared Atkinson Well yeah, one 1 question that I have about the the what like what's the distinction or what's the purpose of the distinction between native events and edr because like if you're looking at it from a functional perspective. They're telling you the same same thing conceptually right? But like if. 31:17.19 Anton Right. 31:17.32 Jonathan Johnson 2 thirds of this triad piece for you. 31:33.89 Anton I Think oh sorry. 31:36.71 Jared Atkinson The value is is that like 1 is just there I guess is the is the distinction. But but like they're they're overlapped for sure. 31:39.30 Jonathan Johnson No, no, no, no like I think like I think there's a there. There's some overlap but not like a hundred percent and so like um. 31:48.25 Jared Atkinson There could be 100% overlap though conceptually like if you have ah if you have a device driver installed on the system as part of your Edr you could replicate. 31:54.40 Jonathan Johnson Conception conception. 31:58.41 Anton Right. 31:58.63 Jonathan Johnson Yeah I mean conceptually yeah if like you're ingesting all like the wind security events. Yeah, absolutely which I think is what Mde might be like planning to do as they expand their product as like we see more tables being added but like right now the reality practically like no one's doing that like. 32:14.22 Anton Yeah, yeah. 32:16.12 Jonathan Johnson Um, at least not to the full capacity. So like when I say native events like these are events that are that do not cost us to quote unquote collect now we might need to store it somewhere that's going to cost us money but I'm not having to pay for that right now that's just like the operating system. 32:22.92 Jared Atkinson Um, yeah. 32:32.47 Jonathan Johnson Went ahead and built that in and like I can just go ahead start to collect that you know we have security application system Logs All those nice things. Um that we can leverage Those are what I consider native and like they hold different telemetry perspective than like say typical I would say process bound edr. 32:49.85 Anton Yeah. 32:51.48 Jonathan Johnson So like for example, like 50 1 45 the network share that I'm talking about. Um if I mean that event could be used for so many different things if like you have like a piece exec and you'll see like the Ibc dollars sign like network share access through a specific name pipe. For that great or also someone's just like you have a user accessing and a network share to access a file. Great. You'll see that um is that technically process bound on the backend sure but like it's not like it's not explicit to x process did. 33:22.84 Jared Atkinson Um, yeah I guess I guess like my. 33:29.93 Jared Atkinson Um, I guess my perspective is like you You could very well look at the windows event log as an edr like they're functionally equivalent and so like I I don't know that there's like I. 33:30.82 Jonathan Johnson Did why thing. 33:37.00 Jonathan Johnson Yeah. 33:44.50 Jared Atkinson Like for. Ah 1 example I don't know if they still do this but 1 of the data sources that might or used to list was um was ah the windows event log as a data source and like you like there is um. 33:53.60 Jonathan Johnson Um, yeah. 33:58.48 Jared Atkinson An obvious issue when you have like process monitoring as a data source and windows event logs as a data source because you're looking, you're using 1 category to describe 2 things that are like 2 things that are at different levels of analysis right? that like if you're going to categorize things together. They have to be applied at the same level of analysis I guess and so like because process monitoring. 34:14.52 Jonathan Johnson Yeah, so. 34:18.36 Jared Atkinson Is in windows event logging if that makes sense but it's also in Edr. 34:19.69 Jonathan Johnson Yeah, so so could you use when a security events as your like main you like event source for all things you could I guess. 34:28.73 Jared Atkinson Ah, yeah, I'm not I'm not arguing with your with your main point I'm just curious about the logic ah like the the the data data collection triad it it seems to me that there's kind of really 2 categories networking and and maybe there's like network. Edr and application potentially but like well yeah, but I think I think there's yeah yeah I think there I think there's 2 different layers of abstraction. 34:49.50 Jonathan Johnson It was. It's all abstraction Jared so like if we look at the host base side if we look at the host base I'm just laughing because that's what we we brought up abstraction in the Robbie everything like is how deep of the abstraction. Do you really want to go? Yeah, there's different layers here. So it's like. 34:49.86 Anton Here. 35:05.92 Jonathan Johnson If we okay, we look at the host base. That's an overarching like category. But then you start to like dive deeper into it and like you have to separate the 2 um collection opportunities because Edr was built in to start to collect the things that people thought the native windows events weren't giving them. 35:22.64 Jared Atkinson Yeah, yeah, yeah, but but that my point is is that you have network eat host base. Let's say and native right? But you're set you're telling me native is a subset of host based but then like. 35:24.41 Jonathan Johnson And they wanted those things and like and so like. 35:38.49 Jared Atkinson So we're we're talking about 2 things at the same level and then we're talking about 1 thing broken out at a lower level but then like on the network level you could say pcap you could say net flow. You could say the the contents of the message. Oh okay, yeah yeah, but like both both are a set. 35:41.98 Jonathan Johnson Yeah, what? So so what is peak out. Give me that Net flow doesn't yeah I don't know I've never really dealt with network data. Yeah something that was a legit question not me trying to. 35:58.16 Jared Atkinson Okay, yeah, okay, fair enough. Okay, fair enough. Yeah, yeah, yeah, so I think I think what? my ah my criticism I guess not I'm not trying to be a jerk or anything but my criticism of the triad I know. 35:58.72 Jonathan Johnson Trump like just Stump the jump there like I I don't know like that's the reason why I just left network there like I expanded upon the two things that I knew. 36:11.84 Jonathan Johnson Um, what it? what? Ah what an asshole and I'm Skitin I just. 36:15.85 Jared Atkinson Is that is that you have 2 supersets host base and network and then for one of those supersets you chose to broke it out break it out at a lower level of analysis and like you have a functional level of analysis which is an edr could technically have feature parodity with the windows event log maybe not literally. 36:17.98 Anton Yet. 36:25.84 Jonathan Johnson Yep. 36:31.42 Jonathan Johnson And vice versa. Yeah. 36:34.17 Jared Atkinson Maybe not literally. But yes vice vice versa and like the edr it's like yeah they they decided they wanted to capture processes because there were subsets. There was a subset of context that the built-in process monitoring did not collect but could could have collected and in fact, they've like expanded that over time that the edr sought were critical. 36:44.40 Jonathan Johnson Um, man. 36:53.22 Jared Atkinson Right? to detection and so then the edr is just like well we don't have control over what windows collect. So we're just going to add something. Um. 36:57.47 Jonathan Johnson Yeah I think that's very yeah, that's a valid point I mean like I I purposely didn't expand network stuff because like I haven't necessarily like dogan into that too much but like yeah so fair criticism there like it's I Just like I utilize that as an example because like I think of you know, what's kind of sad too is like. 37:02.97 Jared Atkinson Yeah, fair enough? Yeah, okay. 37:15.86 Jonathan Johnson Feel like there's this war going on between like network data versus host based data. It's like yo dogs can we just use both but like yeah exact exactly right? and so like um yeah. 37:20.19 Jared Atkinson No yeah, can I give it can I give an example of that. Yeah, so okay, so one of the I think Johnny you might have been involved in this. This may have been before you joined Spector ops at the time but um will came out with rubus right? which was. 37:20.57 Anton You have to have to? yeah yeah, it. 37:39.52 Jared Atkinson Ah, his solution to overpass the hash right? So overpass the hash allows you to take a credential for a non- logged in user inject that into lsac's memory and then you you would be able to authenticate as that user basically requests a kboos ticket. Without like literally logging that user on and there's you know some practical reasons why you might want to do that but we'll realize that um in doing overpass the hash you actually made yourself detectable along the same route as like Mimi traditional mime cats credential dumping. Um, and if there's 1 thing that edr vendors should be able to detect. It's like traditional. Memecast credential dumping because that's like the thing that everybody stake their claim on and so he's like well I probably don't want to accidentally you know run into that that would be kind of dumb and so he created ah ruby rubius which was built off of this kikkiio tool that delpe the delpi had written and the idea was we could just make a raw kerboos request. Ah. The domain controller instead of injecting and then allowing the system to do what it does and requests make the kerb ro request. Why don't we just make the kbo ros request ourselves right? and that was great except the the problem was now you have a process that's not ls aka the rpc server right. 38:50.39 Jonathan Johnson Poor 88 38:52.10 Jared Atkinson Making making this kerbooss request right? Which is then detectable in its own right? Um, the problem is is from the host base perspective. All you can get is Kerberos made a port 88 connection to this ip right? That's that's what you get from the network connection you you get this like let's say you have a pcap. This particular type of Kerberos request so like a tgt rec request a tgt request was made from this system to the domain controller with these details. Let's say um, but then you can't go back to the process and so there's like a missing without either. You're missing critical context because now like. From the host based perspective now I have to look at every every kboos request of which I'm only actually interested in a subset I'm only interested in Tgt Rex but there's you know, let's say 7 different kbaros requests that could be made. There's there may be a lot more but 7 that I'm aware of um. 39:36.44 Jonathan Johnson Um, yeah. 39:47.63 Jared Atkinson But from the network you can't figure out the process and so how do you How do you distinguish that this is a you know abnormal Kerbaros request because you only have the perspective of the request which doesn't have that context and that's why like having both gives you enough information to actually discern. Ah maybe maybe not actually enough. But it gives you more of more context to be able to make a decision with. 40:09.63 Anton Yeah, we run into that with like Kb roasting is a good example too like sometimes the product will detect like rubus itself but not the kb roasting activity at ah and in clients get tripped up on that like hey I thought the edr was supposed to detect kb roasting. But. 40:12.62 Jared Atkinson Um, yeah. 40:23.37 Jared Atkinson Um, yeah, and if if your detection is. 40:25.52 Anton That I think that distinction is is like really important. 40:26.80 Jonathan Johnson I Think the edr is like meant to like cover the base like the very like low precise level of detection Strats and then like it's up to whoever the detection team is to expand upon that. 40:40.31 Jared Atkinson Yeah, it yet. 40:41.87 Anton Yeah, and that's why it's so important to like figure out what you're doing with this eer like what what you want it to do for you like where where do you want that lift happening and then what like how thedr fits in your like your strategy and. 40:46.28 Jonathan Johnson Um, yeah precisely. 40:52.51 Jared Atkinson I Think this is the the purple team value proposition right? because you you could look at it from a low resolution perspective and say oh we detected this activity right? But you could also look at at a higher resolution perspective and Say. We detected this because they happened to use the known the most commonly known tool to perform this activity and like if you're detecting Rubius You detected it by the skin of your teeth is basically what you should take away from that right? Um, as a yeah, that's like. 41:13.22 Anton Friend. 41:18.55 Anton Yeah, yeah, like you detected like the Curb roasting string on the command line not the actual yet right. 41:20.53 Jonathan Johnson Yeah, yeah. 41:26.85 Jared Atkinson That's like you shouldn't You shouldn't take too much pride like it's good that you did that right? It's it's that's a good outcome right? and like great but like you shouldn't be prideful that you caught it that way you should You should then ask some very serious questions of like why didn't we catch this at a more fundamental level. 41:29.70 Jonathan Johnson Um, yeah man at. 41:31.50 Anton Yeah. 41:41.36 Jonathan Johnson Yeah I ah yeah, so this is yeah so spicy conversation I think like yeah, well actually have 2 points I want to bring up so I have to this this round. Um. 41:41.44 Anton Right? exactly. 41:49.52 Jared Atkinson Okay, here we go Johnny's good for one of these a podcast. 41:58.60 Jared Atkinson Yep, you got it. You got it? ip. 42:00.92 Jonathan Johnson Yeah, back back before we move on back to your thing like I agree like I think like what the pull point of like the triad thing was really to bring the importance that we shouldn't be using one end of each spectrum so we shouldn't be using only host space for so we only use a network we got to meet in the middle somewhere. Um. 42:18.58 Jared Atkinson Yep I agree with that. 42:19.33 Jonathan Johnson And I'm just ignorant to the fact of all that I've just used zeke my whole life. So like that's just like like the Rpc remote service creation stuff Jared like I have a Poc Anton I think it's public that like how do you detect like going through like the base condition and going through like remote service creation and like at each like at the triage level. You start to dive into like the network along with hosts and then start to bring it all together in a jupyter notebook and then you can be more definitive I like hey like this is the client process and this is what they performed this was the middle piece of the network through zike and this was the server process and what they performed and now we can start to look at this like. 42:41.17 Anton Yeah. 42:54.36 Jared Atkinson Well yeah, Anton actually has a really good sorry I'm I'm trying to I'm trying to let you get to your points. But I think this is germane the Anton in your in your lateral movement blog post you talk about this same phenomenon with scheduled tasks right? So it's like when you laterally like if if what you're concerned about. 42:56.28 Jonathan Johnson Like this. 43:06.91 Anton Yeah. 43:13.37 Jared Atkinson Regarding services is lateral movement. There's other reasons why so an attacker might use services and it may be beneficial to detect services generally, but sometimes it's beneficially beneficial to reduce the scope to just like service creation for lateral movement because that gives you more context but like for instance Johnny found. 43:27.10 Anton Right. 43:31.98 Jared Atkinson That you cannot like just using purely host based means it's basically I don't know if it's impossible but it's very difficult. We'll say to identify that the service was created you like from a remote system right? and that like ah. 43:45.21 Anton Right. 43:49.41 Jared Atkinson I Don't know that this is true but our perception like our our thought process is that the idea that it was created from a remote system is like a very important contextual piece of information. Um for determining whether or not something is strange because like remotely created services in order to laterally move. You've. 43:56.86 Anton Yeah. 44:06.76 Jared Atkinson Think it's true that you must create the service remotely. Although maybe maybe you create it remotely via means that like we don't expect like Powershell remoting or something like that. Um, but yeah I think it's like that's an important piece and you can't get that unless you correlate it with some sort of network. 44:23.77 Anton Yeah, and when people think of like scheduled tasks. They think of like SScH tasks on the command line right? Or if you do it remotely like that's just it's not there. So and you don't even read about it now that I'm thinking about it more like you don't read about it in like federal Intel report. So I can't recall a time that you've read you know. 44:24.68 Jared Atkinson Data source. Um, yeah, yeah. 44:37.88 Jared Atkinson Um, yeah. 44:42.10 Anton Hey as service was done remotely and I think that goes back to your earlier point Jared about like what we see and whether we're actually seeing the things that we need to be seeing and maybe maybe in this case, we're not right kind of an existential question. 44:42.13 Jonathan Johnson Um, yeah. 44:48.82 Jonathan Johnson Um, yeah, ah I think like yeah, so like I have an abstraction I have an abstraction mass or abstraction mass. Um, abstraction map out there I did a blog with Matt Hand on schedule tasks and like and abstraction map and I think like what. 44:49.50 Jared Atkinson Um, but well the the question is is what is a scheduled task. 45:06.70 Jonathan Johnson 1 thing I liked about your laterteral movement post was I felt like it touched on multiple different layers of the abstraction dependent on where someone wants to like start to leverage detection and so like I actually really liked that about the post like a lot um is like you. You could go like when the abstraction you start to like I look at the layers in the abstraction map as different like different levels of strategy from a detection perspective that someone could leverage in their environment depends where you want to go? Do you have the technology to go lower because the lower you go. Potential of noise and volume of noises could be potentially there but also the value of context is also there as well. So you have to kind of pick and choose your battles there. Um and that was one thing I realized because with tasks was like hey you can do these things remotely as well. But like just like services like. 45:51.79 Jared Atkinson Yeah, yeah. 46:02.12 Jonathan Johnson It's like I kind of like always thought of like services and skills of tasks even though tech not like technically they're different. They kind of in my head are very similar. Um, and I realized that it's very difficult to see like hey someone did the schedule task remotely because like. 46:06.20 Jared Atkinson Um, they're functionally equivalent. Yeah yeah. 46:08.61 Anton Think correct. 46:18.46 Jonathan Johnson Surely these Rpc calls. But there's also different avenues of people can do like I think there's actually I can't remember what I put in the direction map now. But I think there might be a w my method as well. Which yeah, which leverages something different and so it's like now precisely. Yeah, right? And so like and just like for example, windows said that. 46:24.60 Jared Atkinson There is and scheduled jobs for sure. There's a w my method. Yeah. 46:37.21 Jonathan Johnson Xe is deprecated. It's not you just have to change the value in the registry key and so like then you have to not only do like layers of detection strategies from a depth perspective but also from like I don't know what you call it like a horizontal perspective because like that. 46:48.62 Jared Atkinson Breath. Yeah. 46:52.60 Jonathan Johnson That's just the way it might go and I think for a lot of the times when defenders are looking at detection strategies. It's like okay well now for this 1 thing say schedule tasks. We now have I mean we're a hundred percent covered but we have like 6 different detections is that feasible to do for everything? um. Which I think is often a common struggle right. 47:12.48 Jared Atkinson Yeah to to that point. It's like the the abstraction map isn't saying that you must create your detection at the bottom which is your this is your point Johnny. It's saying if you don't create your detection at the bottom then you have a gap and you should be aware of that gap explicitly right? And so like Anton you talked about okay well you could use a. Like is a 46 97 event to detect schedule task creation but like 1 of the fundamental questions that we should be asking as detection engineers is and maybe this is where purple teams come in is um, is it possible to create a scheduled task in a way that this event does not get generated and if so that's a. 47:33.42 Anton Right. 47:51.47 Jared Atkinson That's a pretty big blind spot right? and we should be aware of it. 47:53.20 Anton Right? Yeah, and and then I keep I keep reviewing this but is it's so true to dynamic that I keep it those like like very few people are actually like thinking about that and and especially in a way that's like that's their job that that. 48:01.87 Jared Atkinson Um, yeah. 48:06.21 Jonathan Johnson Um, yeah. 48:08.35 Jared Atkinson There's there's this ah kind of set. There's a set So detection is a complex system right? you you made reference reference to that earlier and one of the there's this idea called Better wrong than vague and the idea is that in any complex. Are you familiar with that. Okay, yeah, so they. 48:11.88 Anton Right. 48:19.89 Anton No, but I love it already just whether yeah. 48:21.41 Jonathan Johnson Um. 48:24.73 Jared Atkinson So The the general ideas is that we we cannot understand complex systems because there's too many variables. There's no way for us to even understand what the variables are let alone understand how the variables impact the system right? And so one of the big distinctions is there's complicated systems which are things that ah. Maybe technically difficult to solve. But once you solve it. It's repeatable, right? And then there's complex which is multivariate and we can't like the economy is a complex system for instance and like the best you could do with complex systems is you can manage the Problem. You can't solve the problem you can manage the problem right? and you could manage it better or Worse. Um, and the the idea of. 48:44.56 Anton Then. 49:01.29 Jared Atkinson Better wrong than vague is that in any complex system. You must It's a necessarily necessary prerequisite to to have assumptions right? because you don't understand that you don't understand the system fully um and our job as you know critical thinkers is to identify. Those assumptions such that we make them explicitly as opposed to implicitly and what you're what you're saying is like there's too many people that are making implicit assumptions and the problem with implicit assumptions is nobody can challenge them right? So like I may make an an assumption but to Johnny that particular assumption. Isn't an assumption because he might have some knowledge that I don't have and so he could come in and say hey like I noticed that you made this assumption but that that assumption is completely wrong, right? and like by making the assumption explicitly I can avoid catastrophe or like you you move towards avoiding a catastrophe because you have other people that can criticize or analyze the assumptions. 49:44.50 Jonathan Johnson Um, yeah. 49:55.19 Jonathan Johnson 1 thing I want one thing I want to yeah that trusts me they're locked in. Um I want to normalize 2 things too is like just because um, you either. 49:57.35 Jared Atkinson Sorry you still have two thanks by the way Joni that I hopefully didn't forget. Okay. 50:10.20 Jonathan Johnson I mean just because you were wrong on a specific subject matter or your perspective was off a little bit or it changes given someone brings light to a new perspective like 1 thing I really really would like to normalize is like you weren't. Dumb for being quote unquote wrong, you just happen to be incorrect about that one instance and it might have be because you didn't have this technology. You didn't have the perspective or you didn't have the background knowledge and that's one hundred percent okay like but the the sharing of knowledge needs to be normalized minus the I'm better than you because I knew this because like there's something someone else knows to like. 50:24.22 Jared Atkinson Um, yeah. 50:37.42 Jared Atkinson Um, yeah. 50:42.63 Jonathan Johnson And another subject that I'm ignorant about like for example, like I I kind of purposefully choose to stay away from kerbro at all cost. You know, like if I can if I can if I can stay away from it typically I attempt to try to so like I leave that to the experts. You know what? I mean like I just like let them do that I'll I'll learn from it great but like um. I think that's 1 thing in the industry I would just love to see change is the transfer of knowledge and collaboration minus the ego like yeah. 51:08.80 Jared Atkinson Well, it's It's better to be enlightened than it is to be ignorant right? And like I use ignorant in a technical sense not in a like a pejorative sense. It's ah like we are We are ignorant as I mentioned and when you find out that you made a bad assumption. This is in life. This is ah a technical matter. When you find out, you made a bad Assumption. You are being enlightened by definition right? and it's better to be more enlightened than it is to be more ignorant like you don't want to bask in your ignorance. That's. 51:32.46 Jonathan Johnson Um, yeah now. 51:34.29 Anton No I always tell my wife like 1 of my biggest fears in life is being like ignorant I that the whole dynamic of just being like I don't want to know maybe maybe I get like the specific things like fibrose like I'm I'm python allergic I can't i. 51:49.96 Jared Atkinson Um, yeah. 51:50.60 Jonathan Johnson Yeah, but. 51:53.22 Anton Like the whole jupyter thing I get it. It's wonderful I can't like I've tried so many times to to to wrang it I doesn't click so like not in that sense of ignorance. But the whole like quest to become like less ignorant I think is is super like important to me like personally and and professionally but like. 51:54.78 Jonathan Johnson Um, yep. 52:02.51 Jonathan Johnson Um, yeah. 52:09.46 Anton You'll never get there right? There's no such thing as like an all-knowing like being right? That's that's impossible. But yeah. 52:10.49 Jared Atkinson Yeah, that's correct and what what happens is people look at the problem from different abstraction layers right? and so like ah and then we talk. Ah, we often say things like on Twitter it's common for somebody to say something and be like oh well like everybody agrees this and it's like well. 52:17.64 Anton Have. 52:29.46 Jared Atkinson Obviously not because we're disagreeing about it right now in this conversation right? And so like it's not. There's there's a lot of times that people will like take something for granted and that's because they're analyzing it at this like superficial level of analysis and like that's why people feel like oh well I have a stupid question. It's like no, it's not stupid. 52:45.46 Anton The Vietnam. 52:47.24 Jared Atkinson Like you might actually be thinking about this problem at a completely different level than what other people are considering. 52:50.97 Jonathan Johnson Yeah I think I think also like what separates people is like okay like I just got a new perspective like on something today and I was ignorant to it yesterday. Great now I want to do my due diligence and figure out more about it but also look back at any solutions that I put on. 52:51.10 Anton Right. 53:10.18 Jonathan Johnson Put out previously on my previous knowledge and fix those like that is what separates the masses is that integrity that integrity of like saying like hey I know I touched this before but let me go let me just go back and check if this applies still and just like move forward. 53:23.59 Jared Atkinson I Think that that actually psychologically is one of the reasons why people are resistant to ah to basically throwing away their assumptions because it's It's not always clear how how deeply that assumption is rooted in like everything that you do right? like you might be married to the person like. Is psychological but like you may be married to the person based on some assumption that somebody is just going to come destroy or like you built your life around it in some way and it's like if I let that go then I like necessarily have to let all these other things go and there's lots of resistance to it. 53:53.47 Jonathan Johnson Yeah, like for example, like I gave some like you know research to some people a little while ago and then like last night we'll playing around with some of the research like I realized like my understanding my breadth of understanding. 54:07.26 Jared Atkinson Was incorrect. Sure. 54:11.71 Jonathan Johnson Wasn't at the level particularly what I thought and I was actually up till 2 am this morning making sure like going back through all my previous research to see if like the suggestions I you know I gave to them still applied and what needed to be changed so that whenever Eight o'clock snapped open this morning at work I could toss it over to them to make sure that they're good to go. And I don't say that to like build myself up. But I think like it's the enthusiasm and caring like that of not wanting to be ignorant but also making sure that impact is still apply because like the non- ignoranceance man this huge the non- ignorancerance is probably because I'm jacked up on c 4 right now energy drink by the way. 54:35.83 Jared Atkinson Um, yep. 54:47.79 Jared Atkinson Yeah, a font rest. Ah. 54:50.28 Anton I. 54:50.61 Jonathan Johnson Please sponsor us. Ah, Skittle's flavor. Um, the non ignorance doesn't only impact me so like if I hold that if I hold that new knowledge to Myself. What is that impacting what is that doing great my own ego Cool. What is that like that does it does nothing for me but I need to Impact. My workflow and my customers and all the partners that I have and we have and so I need to like spread that knowledge out to them. That's where the true impact comes from and if I was wrong on something I have to have the integrity and say hey yo like I was ignorant of this. 55:16.92 Jared Atkinson Um, yeah. You you. 55:25.14 Jonathan Johnson I Just need to fix it now and this is the actual information and here's the research. Why cool. 55:27.71 Jared Atkinson You have to feed your adaptation back into the genome back to the evolutionary an analogy. Oh man. Okay. 55:31.79 Anton Yeah, it's worth yeah like I come from ah like before I got an indoor into infosec I that min major like youed. But so I come from like an academic background and the whole like concept of building on knowledge. 55:34.80 Jonathan Johnson Whoa trademark that Jared but trademark that. 55:42.34 Jared Atkinson Um, hey nice dude. Yeah yeah, yep. 55:51.12 Anton Is like you know, obviously like I think I carried that forward to to the info that career. But I think people assume that like that that knowledge building you know like I'm picturing like a snowball rolling down a hill and like getting bigger and bigger like it doesn't happen super fast like a lot of people think like when you do like a Ph D It's like a completely different like. 56:09.34 Jared Atkinson Um, well it goes like this. Yeah. 56:11.90 Anton World changing like yeah like a paradigm changing thing like paradigm changing research that it's like paradigm like it changes like your whole like that's not always the case right? You just have to push that needle just a little bit further with your with your work and research. So yeah, that's definitely like ah important. 56:14.35 Jonathan Johnson Um, paradigm we. 56:22.39 Jonathan Johnson Um, yeah. 56:26.89 Jared Atkinson You you just gave me a new analogy potentially to try to investigate for the abstraction which is like we all understand what world War Ii was but like you could understand it so much more in depth as you go in like I mean maybe you could use any example right? But like we generally know that you know like ah. You know the nazis invaded Poland or something like that. But you could go so much deeper into why you know what? you know where when all that stuff. So. 56:53.70 Anton Yeah, there's a reason why the world war two scholarship is still taking place. You know, seventy years later so yeah there's there's definitely work to be done there. 56:57.31 Jared Atkinson Um, yeah, yeah, yep, yeah for sure Johnny hit your hit your 2 points I keep I feel like I keep derailing you. 57:03.29 Jonathan Johnson Yeah, um, ah. Are um Anton first question for you. So like I we had this conversation with Andy last podcast and I basically challenge the purpose of red teaming like. 57:13.11 Anton Get. 57:25.15 Jonathan Johnson If we have purple team now why does red teaming exist but I want to flip that on Ted real fast before we talk about that in your instances of purple team. How often do you come across detection engineering teams or socks in general that are like we got them. We collect them. They're super loud. We got them and they're like yeah but you didn't catch this like does it matter we got them here. 57:42.58 Anton Yeah, ah, it's hard to say like some of our purple teams follow on from a red team or penta some of them don't but the ones that do for for clients who have like a mature sock they they manage to. 57:44.60 Jonathan Johnson How often have you ran across that. 58:01.78 Anton To get like certain pieces but but not the full chain I think that's where the the kind of like the rubber hits the road and where clients have a little bit more trouble and where Propaltey could help in taking like a whole bunch of. Atomic things that you might have found like you might have found a fish or you might have found like a weird powershell command line. But how they link together and and thinking about like okay well what if the threat actor or red team or did something a little bit differently or what if you know one host wasn't sending the right logs or something like that. Think that's where the the kind of like the value add in purple is I know purple team like there's a whole bunch of like different like everything in infosec right? like red t and pen test like people have different definitions of it and I think another differentiator is whether you're doing purple team has like. 58:40.46 Jonathan Johnson Um. 58:49.87 Jonathan Johnson Um, yeah. 58:56.62 Anton Ah, third -party consultant or like internal purple team because my sense is that internal purple teams focus more on like validation and like atomic testing and and that's definitely part of purple team but from a consultant point of view like if I'm with a client for a week like I'm not just going to run you know like red canary atomic red team for them. Like they could kind of do that themselves. So so the value is more of that like educational pieces that kind of don't fit in between red and blue and that's that's where the purple part of it comes in hopefully that answered your question I I might have missed the market. 59:29.24 Jonathan Johnson Yeah, yeah, no so I I ask because like I've seen this and then like it made me kind of shift my thought process on that conversation we had with Andy then and the reason why I say that is like I've often seen. Some organizations if like a red team is going on and a defender gets an alert and they see like that specific activity happen and they can like walk it back. Walk it forward a little bit and then they hit you with that. Oh I thought they'd be less loud smiley face and it's like. To me. It's like well first off to me that's ego I'm like okay, let's take a step back here like did the red teamer want to kick off that alert and start to see like your timeliness of on like actually like going through the triage and investigation process because that is something that needs to be tested during. 01:00:07.20 Anton And. 01:00:23.75 Jonathan Johnson Red and purple teams as well. Not just doesn't alert fire. What is your response time because I really do believe like just like for example, like if someone goes to a batting bat ah batting cage the more repetitions they put in the better they are and that is like something that we have to like move forward and normalize in our industry is like the more times that you look into something. More efficient. You're going to do it. Let's just hope like the the way you're learning. It is the correct way to do it but like efficiency is going to be is going to be there. Um, but like I was thinking about like because I bring up the conversation with Andy I was like hey like with red teams. Do they just pass off a report. And then at that point they're like cool we're done or do they actually like what's the debrief come to and then like I was talking to some other friends that do red teaming like sometimes like apparently like some defenders don't even show up to the meanings for debriefs in some instances and the thing is like why like my my questions started to become like well well why aren't they and I asked myself like. 01:01:11.41 Jared Atkinson Um, yeah. 01:01:20.28 Jonathan Johnson Because like some point during the red team. There were like they caught something so they're really proud or they didn't like base. Essentially they didn't want to get shit on during the debrief but like the reality is like you have to drop that ego because it's not about you. It's about your organization and so like the question then becomes Like. What is the value of red teaming in General. What's the value of red teaming when defenders won't even like listen and so like how do we fix both those issues. 01:01:45.43 Anton Yeah, yeah I think it's so dependent on the client and this is from you know a consult world and I think what? what? what? I see like missing in conversations is that layer between like the client and the Pm team of the. 01:01:47.55 Jared Atkinson Um, yeah. 01:02:04.90 Jonathan Johnson Um, yeah. 01:02:04.93 Anton Of the company that's doing the testing I think they're kind of like the unsung Heroes of this like the the project managers and the salespeople because they're the ones that set up the engagement with the client and then set the expectations So on our end for the purple Team. We always try to have like a kickoff call or 2 or 3 Maybe. Just because because communication I think is like so critical in all this because like a red team can mean so many different things to so many different clients and it might be the case that the client doesn't even need it or want a red team. They're just calling it a red team but they were what they really want is like a purple or a pentest or something. So. Like that's the I think that's the crux of it like that communication and and the the second point of the communication is like validation of the communication like do we understand each other like here's what here's the service that we're doing here's what we're expecting as a deliverable here's the people that we want involved. And I feel like when that communication falls Apart. That's when you run into the issues where the client's like not getting what they expect or the the right people aren showing up and then there's like communication between like the client and the tester. But then there's also like internal communication within the client environment. 01:03:07.30 Jonathan Johnson Um, yeah. 01:03:19.64 Anton And I find that even there that breaks down sometimes too and that's why you don't have like the right people showing up to the right calls because the right pm on their side wasn't involved and like something got missed and the I think like that. Ah, that's all part of it. So yeah, like I think we in infosec have all these definitions in our head and everyone knows. 01:03:26.79 Jonathan Johnson Um, yeah. 01:03:38.30 Anton What what their own definition of purple is but until you get what the client figure out what it is exactly they're looking for what it is. They're expecting as a deliverable for this because yeah I've done purples where it it is just like atomic testing where I run ttps I market you know, detected not detected and then we move on. 01:03:54.79 Jonathan Johnson Um, yeah. 01:03:54.92 Jared Atkinson Um, yeah. 01:03:57.10 Anton And then because that's what the client wanted. That's what their. That's what their expectation was and there's others that are more hands-on more involved some clients want more focused on the defensive side of things where we don't really run ttps that much we could we dig into their sim and do all that stuff. So and that all falls under under the umbrella of purple. It's just the. 01:04:10.80 Jared Atkinson Um, and I think there's this, there's this problem and like ah we have lots of these terms red teaming purple teaming threat hunting detection engineering the reason why we use those words is like they're. 01:04:16.23 Anton The the communication piece important. 01:04:29.16 Jared Atkinson Purposely abstract right? The whole idea is that they're abstract and the the general reason why we use abstract abstract words is because we don't want to explain the details of them every single time but like that approach is predicated on the idea that we all understand what red teaming means in a consistent fashion which we we obviously don't. Um, and so there's this big problem but like and like maybe it doesn't matter until like the rubber hits the road potentially which is on on an engagement and it's so important to understand like what are you trying to achieve from this because like ah ah the way that I view a traditional red team is like you have a single attack path that you go through and you have some end objective right? that you're trying to achieve. 01:04:58.92 Anton Yeah. 01:05:08.54 Jared Atkinson Um, and there's there's way more to it. But that's like generally how I see them most frequently implemented and the question that I have is like what is the value proposition of that like because the value proposition is almost It seems like we need to prove that we're vulnerable. So that we could get the funding to do the other stuff because like you have such a Luke's going to laugh at me this is like a camera reference but you have different apertures right? and I don't know whether high aperture is like a broad perspective or vice versa Luke. 01:05:38.30 dcppodcast So ah, the wider the aperture the narrower the perspective the less is in focus. 01:05:42.85 Jared Atkinson Okay, so we have a okay, whatever we got. We have a broad perspective. Um, that than't help me at all. Okay, you have a very small aperture which means you have a very broad perspective. So like let's say let's say you do detect the red team. 01:05:49.30 Jonathan Johnson I. 01:05:50.49 dcppodcast So yeah, you have a so you have us so you have a small aperture. 01:05:51.40 Anton I Kind of get it I got to get it. 01:05:59.84 Jonathan Johnson Um, yeah. 01:06:02.60 Jared Atkinson What does that tell you? what do you take away from that. Well what that tells you is they happen to do something in a way that you happen to be able to detect but like you can't take that as a like any indication of what will happen in the future right? because the next attacker might not use that thing and you haven't tested literally anything else and then if you don't detect. 01:06:03.10 Anton Yeah. 01:06:12.90 Jonathan Johnson Um, we win. Yeah. 01:06:21.91 Jared Atkinson Them you like you. You also can't evaluate where the problem happened because the problem could be. We didn't collect the right telemetry. The problem could be our detection was not robust enough the problem could be our tier 1 analyst received the alert. So our detection worked but they marked it as a false positive. The the problem could be. We detected it. We marked it as an incident and then we ran a remediation step and the remediation step didn't do what we thought it would do right? There's there's so many different ways that that process could break down that I I think that you almost have to like the value proposition almost has to be at a more ah atomic. Perspective because otherwise you're just looking at and like I'm happy to be in line like this is something I think I'm probably somewhat ignorant on right? but and I'm sure that there's tons of people that that have opinions about what I'm saying and like I just asked that like you know actually consider the answer before you come yell at me about. Why a red team is valuable. Um, you know because if if you say something dumb then you know it is what it is but um, but like yeah I think I think like as a defender the value to me is like I want to know whether or not how I approach this specific thing actually worked um, not not. 01:07:23.41 Jonathan Johnson Um. 01:07:34.29 Anton Yeah. 01:07:38.88 Jared Atkinson You know we caught you? Okay, cool, great. You caught us but like that doesn't that literally has no, you could make no assumptions about the future based off of a successful detection in ah in a red team. 01:07:39.86 Jonathan Johnson Um, yeah. 01:07:47.13 Jonathan Johnson Um, it might and for all the listeners I'm not saying red teaming isn't useful. So don't don't cancel me but what I'm saying is I do get confused between the use cases. 01:07:54.36 Jared Atkinson Um, yeah. 01:08:03.46 Jonathan Johnson Or the the problems that red teaming solve and the overlap with what purple team solves as well. 01:08:04.98 Jared Atkinson Yeah, this this goes into the av like when people say av is dead. The problem isn't that a like a V hasn't changed a v still does the thing that it always did maybe it does more than what it used to the thing that. 01:08:20.27 Jonathan Johnson Your perspective on what a V should do is yeah. 01:08:24.26 Jared Atkinson That's correct. Yep your expectations of a V have changed and so like you think that Av is worthless but like a V is the thing that's allowing you to focus on the things that you focus on because if a V wasn't there. You'd be a shit show man. 01:08:35.34 Anton Yeah I remember back ah like before I joined Flores I was ah just working at a small insurance company locally and I was in charge of like the same an environment was like I don't know 1200 systems so pretty manageable right? then that had like sysmon everywhere. 01:08:36.16 Jonathan Johnson Yep, hundred percent 01:08:53.20 Anton Ah, pcap and all this good stuff and I wanted a red team done because I wanted to go beyond like an atomic test I wanted someone who's like does this professionally and has little tricks for like password spraying maybe involve like a physical element to it. You know, like something. 01:09:09.60 Jonathan Johnson You wanted your perspective to be expanded. 01:09:11.35 Jared Atkinson Um, yeah. 01:09:12.80 Anton Yeah, like longer term like an engagement with like you know proper C two that I can't like easily replicate myself or that you know commodity malware if book he doesn't doesn't do like that kind of stuff so I wanted to see like was I was I missing anything Huge was I like making wrong assumptions that kind of stuff. So I think like that's where. 01:09:17.68 Jared Atkinson Um, yeah, sure. 01:09:31.88 Anton Value of a red team is because um, like I had pretty minimal exposure to like professional pen testers and hackers right before I joined Las and now that I see what what our red team and pentest folks get up to it adds a ton of value. Especially they they find like little quark. 01:09:37.64 Jared Atkinson Um, sure sure. 01:09:49.85 Anton And and authentication systems that like um little tweaks to certain things just stuff that like a human has to do in a human with like the right experience for it. You know like that that they have like years under their belt like they're grizzled that kind of thing right? whereas I think the purple is more for like a. 01:10:01.66 Jared Atkinson Yeah I guess ah in in my opinion Atomic doesn't necessarily mean automated I Guess Yeah, so but yeah, that's that's a good perspective I think is that there's value in both approaches. But if you're. 01:10:08.19 Anton Yeah, oh sorry. Spread Yeah to. 01:10:21.27 Jared Atkinson Expectation is tell me how robust my detection capability is like ah like a generic red team might not be the best tool to solve that problem. Yeah. 01:10:28.73 Anton No, probably not yet and that's again, why the whole communication thing and understanding like as cheesy as it sounds like we understand your needs but like it's just so important to understand like what it is that the client is trying to do and what stack they have right? because if they have you know. Just like 1 poor av hanging out on their systems like a red team that won't add much value for sure. Yeah. 01:10:52.10 Jared Atkinson Yeah, sometimes we have customers that like they're trying to justify budget and so they're like yeah we kind of need you to show impact like at a broad like this is at the you know broad scale. Um, and that's like a value proposition for sure. But it's like then it's like okay, well now we need to actually fix the things that we're showing impact on and like I think you have to. 01:10:56.85 Anton Yeah, yeah. 01:11:11.60 Anton Yeah. 01:11:11.98 Jared Atkinson Reduce The scope in order to have meaningful change and then maybe you you could evaluate that to some degree and like you could you cage the techniques that are used in the in the red team for instance to things that you like if you've been working on something over a long period of time. You cage the capabilities of the red team to some subset of techniques or something like that. But like yeah I think any like no no holds barred I think is ah, there's a very specific use case for that and it's probably not applicable in the vast vast majority of cases. 01:11:45.50 Anton Definitely yeah I've seen like Cyber insurance drive a lot of engagement to lately so dad. There's it keeps coming back to the the communication piece that you just have to you have to get on the same wavelength as a client and then understand what they want for sure. 01:11:46.93 Jared Atkinson Okay. 01:11:57.90 Jonathan Johnson I think also with the communication piece too like those those clients that might have like an Mdr vendor as well like communicating with them and being open like have open calms there as well because like it's hard enough to like if you have a red team. Um. Make sure you're communicating well or like a purple team communicating well with whatever consultant company is doing that but also like it's almost like a game of telephone where it's like okay now you have to like go to the mdr and be like a like what's this this and this and then like because the goal should be in my head is like a like let's like test the Mdr make sure that they're doing a good job. B like help the mdr like you've paid all this money for a read red team like have that conversation to like try to expand their detection capabilities too because it's going to help you I mean like it's. 01:12:33.78 Jared Atkinson Um, yeah. 01:12:34.10 Anton Then. 01:12:44.89 Jared Atkinson That's an interesting point because I think a lot of companies view mdrs as like a broad thing and like they view the value proposition of the Md I'm not saying that the this is what the mdr says they're doing because I actually don't know like I don't I don't talk to the sales portion of mdrs at all. So I don't know really what they're. 01:13:00.70 Jonathan Johnson Yeah, neither do I. 01:13:01.71 Jared Atkinson But they're promising or whatever. Okay, fair enough. Um, but like maybe there's like a implicit or possibly explicit thing of like we don't detect everything but we will detect something in every attack and so then like a red team is a valid test of that proposition right? because like if you don't detect. If. You don't notify me that there's a red team then like you failed you failed to meet the criteria that you so that you set upon yourself, but like yeah if you're doing it yourself if if like you don't have an Mdr and you're doing it yourself like that value proposition doesn't really exist I guess is kind of the point. 01:13:34.25 Jonathan Johnson Um, yeah I mean like no go ahead. Go ahead. God. 01:13:35.18 Anton Yeah, some of them Ohs are I I was was going to say some of the most like engaging and fun prepps that that I've done has been with the Msp or the Mdr vendor like collaboratively going through stuff and that's ah the client just always gets like a ton of value out of that and and same with the the third party too because they get. 01:13:43.54 Jared Atkinson Um, okay yeah. 01:13:54.71 Anton Validate their detections in a manner. That's a little bit more open than a red team would and you know we could replay stuff. Try variations see if they're missing data sources stuff like that. So yeah, it's ah it's always better the the more the merrier especially for the purple side. 01:14:04.46 Jared Atkinson Um, yeah. 01:14:06.50 Jonathan Johnson Yeah I think it's intriguing too because like I'm very hesitant I'm very hesitant like hesitant and this was when I worked at spectter and even now working recinaily in general saying like this technique is covered. 01:14:22.91 Jared Atkinson Oh yeah, you can't you you You can't actually say that in any full way. Yeah. 01:14:25.40 Jonathan Johnson Like I've always been very hesitant. You can't do that. Really yeah and so like when I hear people when I when I hear people say like we got that covered I'm like you know like. 01:14:26.10 Anton Yeah, on the yeah. 01:14:35.37 Anton Do you hear. 01:14:36.49 Jared Atkinson Well, it's a superficial. It's like a abstract way of saying and we have the most common ways that that gets used covered. Yeah. 01:14:42.17 Jonathan Johnson Yeah, like I'm I'm not good at sales you know like so like given that might be why I might have that reaction but like in general I think like my tech like when I abstract things in my mind I'm like yeah like that is definitely a. 01:14:42.89 Anton Right. 01:14:59.79 Jonathan Johnson Way that we can detect that right? but there could be something or tomorrow or some ransomware group that's doing this like doing a completely different variant of this attack that like we just haven't seen and so it like goes away with that conversation of like unknown unknowns like we just like we don't know what we don't know. And like due to that like I am never okay with saying like a hundred percent we got this like. 01:15:20.29 Jared Atkinson Okay, let me let me throw this at you I know you still have one point and we only have about 15 minutes so we'll try to do this fast? Okay, oh cool. Okay, so um, what what? you're talking about is what I call micro detectionomics which is if I choose to detect a specific technique. 01:15:25.12 Jonathan Johnson No no that my points are done. Yeah. 01:15:34.93 Jonathan Johnson Oh big words dude I'm from Missouri man. Okay, so like you have to like e l I five you know I'm saying like these big words. Okay, thanks. 01:15:38.50 Jared Atkinson And Luke come on dude it's. Yeah, that's what I'm trying to do? Yeah yeah so micro detectionomics is the category that says if I choose to detect a given technique say w my lateral movement. How certain am I that my solution or solutions will actually detect when that's when that technique is used right and like there's. Like like you said you could never be fully certain but you could get you know, generally some good ideas right? And that's where abstraction mapping comes into play. But then there's macro detectionomics which is this idea of like what techniques should I focus my finite detection engineering resources towards solving and I think there's this interesting conversation around. Um, while it is impossible and this is the point of red teaming right of my red team criticism I suppose but the ah while it's impossible for us to predict what techniques will be used in a given attack path I just call them a attack pass because that's kind of what it is right um. There are like not all techniques are used equally. So there's a disproportionate relationship between the frequency of different techniques say like credential dumping using like traditional mimiatts type methods is probably like 1 of the higher or like certain lateral movement and so there there probably is some subsets of. Possible attack techniques that at least one of which is guaranteed to be present in any attack path. So like you, you might not be able to say this this technique will be used in every like I I know for a fact, you can't say that right? So you can't say this technique will be used in ah and this technique will be used in everything but maybe there's like. 10 techniques that if you have a you know, above average solution for these 10 techniques then you're likely to encounter that technique in any arbitrary attack path that occurs because there's you know some subset of attack techniques that are used way like we think about like Apple music listens right? So like when you think about like. Beyonce gets significantly more listens than you know some guy that's just starting out or podcasts right? Joe Rogan gets way more listens than detection challenging paradigms right? or Michael Jordan scored more points in the Mba than the average Mba player right? So that like there's always this disproportional. Ah. 01:17:44.13 Jonathan Johnson Which is which is a shan them I. 01:17:54.26 Jared Atkinson Exponential type relationship between between these things right? and so there's there's that that relationship almost certainly exists within techniques right? So like you know there's certain techniques that we know about but they're almost never used and there's other techniques that we you know you kind of assume are used in almost every case and so it's like at what point will we have. Like what's the smallest possible set of techniques that if we can reliably detect those and obviously not like with certainty but like pretty reliably if we could detect those we will reliably detect because you only have to the other presupposition is that you only have to detect one technique within an attack path. And if you could do that successfully and then respond to it Properly. You're good to go right?? um. 01:18:34.39 Jonathan Johnson Yeah, respond to it properly in the sense of like you have the technology ability to walk that attack path backwards. Even if you didn't have the detections for that yet. 01:18:40.24 Jared Atkinson Sure, yeah, yeah, so like yeah once you know what you're looking at. It's a completely different problem set than like detection and investigation are completely different problem sets and you have different tools that you're you know available to you and so yeah, that's the presupposition is you only have to detect. 1 one technique to I to then be able to investigate an attack path. Um, and so like what's the minimum set like is it 10 is it 20 is it 50 I don't know is it 200 it could be. But yeah. 01:19:09.64 Jonathan Johnson Or what or what techniques like which tactic should you like focus more heavily on like. 01:19:16.44 Jared Atkinson Ah, yeah, well yeah, that's a that's a question as well is like is it more tactic focus like if you detect that the eights lateral movement techniques that Anton laid out like is that enough or is there like you know potentially could somebody evade those and still be successful. 01:19:24.31 Jonathan Johnson Yeah. Because I have a feeling like for example, like it's not 100% that someone's going to D Cync or dump lss I do think and given that's set of technique level but like I do think it is 100%. Someone's going to learn laterally move. 01:19:45.13 Jared Atkinson Yeah, well like in theory in Theory you could technically fish the you like obviously this if if this is the case then like that that environment's screwed anyway. But technically you could you could happen to fish the person that you needed access to to achieve whatever your objective was. 01:19:47.80 Jonathan Johnson In a word. Um, yeah. 01:20:02.83 Jonathan Johnson Yep. 01:20:03.88 Jared Atkinson And so like then you technically don't have to lotter really move I guess but um, but yeah I don't know it's it'd be an interesting thing That's like um, you know Ga I don't know if you guys know gabe the engineer. He just released this like this thing that I I forget what it's called like some sort of flow type thing that talks about it. 01:20:20.19 Anton The graphing thing. Yeah where I saw I. 01:20:22.91 Jared Atkinson Yeah, the graphing thing of like how how everything how like an attack path gets laid out in the graphs and like what entities are being. You know, attacked or whatever using what techniques and I think that with that type of information you could. You could start to build up enough of a corpus of attack path graphs. Start understanding like what are the? What's the most central technique because yeah, yeah, Basically that's the and then it's like okay well that's probably where we need to focus our resources. 01:20:48.57 Anton Yeah I don't know but you guys but I always get asked or like we get asked like what are the like top 5 like hardening things that we could do or like what? what's the like. It's so hard to answer that right? because like yeah. 01:20:53.49 Jared Atkinson Um, yeah, um. 01:20:55.50 Jonathan Johnson And. 01:20:58.27 Jared Atkinson Yeah, right now it's and anecdotal but it'd be really nice if we could do something like I think that there needs to be a more academic element to cyber security and like Chris Sanders I think is probably the person that's pushing that the most um and like you know that's a. 01:21:05.18 Anton Um. 01:21:10.46 Anton Yeah. 01:21:15.37 Jared Atkinson That's a question that obviously there's the like kind of there's it's bias. There's a selection bias because you only know about the attack pass that you've detected right? So there's some subset of attacks that presumably you haven't detected. However, for what you have detected you could and you could actually answer that presumably I Just don't know that we have the data. 01:21:33.89 Jonathan Johnson Um, yeah. 01:21:34.41 Jared Atkinson Be able to answer it properly or like nobody's done the work to provide that answer right now. It's just like I know what happens to seem popular amongst the Twitter crowd if that makes sense. 01:21:42.75 Jonathan Johnson You know, academically like you had to get like a lot of buy in and then you're like oh crap like you have a lot of people accessing data and then you have to go through like yeah I would like to hit to it. Just yeah. 01:21:54.55 Anton I Think like no matter like that that initiative that that we just thought of like fantastic I think the from the client point of view the more that they're able to collect wrangle and use like tactfully like actually like because some clients like have data but they don't They don't. Don't do anything with it. So the more like even though you might be missing detections. You know you might be. You might have gaps but the more you're able to like collect operationalize and get comfortable with like the more of a fighting chance you have to detect the the stuff that you might not have you know like a built-in detection for but just generally spot like weird. Stuff like that like back of my old job when I had like really good visibility with sua stuff I found all kinds of like crazy crazy things that I wasn't specifically looking for but they just kind of like bubbled up. Yeah and like a lot of them were like necessarily malicious but like you know app crashing or. 01:22:43.83 Jonathan Johnson Popped up. Yeah. 01:22:50.59 Anton Map writing like a random like text file 3000000 times a second or some weird stuff like that. So I think like I think the concept of like ownership of your network is something that I see like lacking a little bit and I think like a lot of the the risk is kind of like you know like the ci ah the csp like. 01:22:58.12 Jared Atkinson Um, yeah. 01:23:08.33 Anton When you do with risk you can either accept the like transfer or or whatever I see a lot of like transferring of risk without actually like understanding what it is. They're transferring and then how you like? yeah yeah, what's that yeah. 01:23:09.40 Jared Atkinson Um, yeah. 01:23:11.38 Jonathan Johnson Yeah, yeah. 01:23:15.74 Jared Atkinson Um, yeah. 01:23:17.58 Jonathan Johnson Yeah, like as fed Asset Inventory right? like I like how many people do you see? really do yeah how many people like do you know really have access to your resources or still have a computer like at like some work You don't like know. 01:23:32.73 Anton Now. 01:23:32.77 Jared Atkinson A good thing that you just touched on is and like 1 of the things that we battle is there's a finite number of alerts that we can produce before we start to affect people's ability to deal with the next alert or maybe you even have a negative impact to wear it the more alerts you produce the less. You're able to cover because of alert fatigue. But I think. 1 of the things you touched on was ah this idea that our job isn't necessarily to detect malicious stuff right? That's like the low resolution way to look at it. It's to explain the previously unexplained behavior if that makes sense so like because like 1 of the i. I don't know again. This is an academic question of like what leads to alert fatigue and like I suspect that one of the things that leads to alert fatigue is the less frequently. You find things the more quickly you're going to like find things that you're you know are malicious the more frequently or the. 01:24:13.48 Jonathan Johnson Um, any. 01:24:27.53 Jared Atkinson More quickly you're going to become alert fatigued if that makes sense Yeah knows. 01:24:27.62 Jonathan Johnson Do do you think like sorry I mean interrupt there I'm just curious like do you think that's like a alert to analyst ratio potentially like that's one like that's 1 thing and then also maybe like. 01:24:37.71 Jared Atkinson Well yeah, so I yeah. 01:24:43.75 Jonathan Johnson That ratio might change dependent on the skill level of that analyst. Okay, okay, okay, sorry yeah. 01:24:45.52 Jared Atkinson I Yeah have a whole thought process on this but let me let me finish. Yeah, so but I think there's like ah if you could change the paradigm of what people think they're doing so like if you change it from you're looking for malicious because that's what all this like sayso asked like did you find anything you know? Um, but what if you changed it to It's like. 01:25:00.74 Anton No. 01:25:05.90 Jared Atkinson Your job is to discover things that are currently unexplained discover behavior that's currently unexplained and explain it regardless of whether it's malicious or not and that's considered success I think that completely I think that that could have a positive impact to your point Johnny I think that there's like 1 of the things that we ask about is like you have a. 01:25:14.16 Anton Yeah. 01:25:25.40 Jared Atkinson Finite capacity for alert generation right? and I think the variables and I'm happy to have this conversation and if anybody has variables that they think of that I don't list I'm happy to hear about it. But I think the variables are the number of analysts you have the amount of hours that they work. On alerts right? So like not necessarily like you work 40 hours a week so you get 40 hours per an analyst because you can't work people full time on alert processing. So it's like maybe 75% 30 hours a week or I don't know I don't know how you derive that that number but whatever that number is and then it's like there's some efficiency factors like for instance. 01:25:56.10 Jonathan Johnson A. 01:26:02.33 Jared Atkinson I might be able to like process alerts faster than you can. So maybe I could do 10 alerts per hour and you could do eight alerts per hour right? Well you you average that out and then you come up with this is our capacity for alerts and ideally in an ideal world. You are producing alerts at that capacity. Right? Because then you're not wasting resources to some degree. Ideally, you have like you know improvements and things worked into that 10 hours or whatever that you're not that people aren't processing alerts you've kind of like figured that out. Um, but the question is is like how do we increase our alert capacity. Well you could hire more people you could work people longer or you can make them more efficient. 01:26:27.57 Jonathan Johnson Yeah, yeah. 01:26:38.72 Jared Atkinson And like you make them more efficient through like training is one way I guess or automation or documentation Potentially and I think it's like generally easier for companies like most companies are not at like their analysts aren't working at their peak efficiency right? So there's like a. 01:26:54.67 Anton Yeah. 01:26:57.56 Jared Atkinson A lot of people like the easy button is hire more people conceptually. But I think like you would be. It's a lot easier to just get people to be more efficient invest in your people make them more efficient. 01:26:58.21 Jonathan Johnson Um, yeah. 01:27:07.56 Jonathan Johnson Um, yeah. 01:27:08.25 Anton I Think like from from what I've seen the whole other fatigue and this might be like a controversial opinion I worry a little bit that the whole other critique thing has become like a buggy man thing where. 01:27:18.72 Jared Atkinson Um, yeah. 01:27:22.47 Anton People inclined to like afraid to onboard data and to get more data because they're like oh what about alert for fatigue where they have like 0 alerts. You know what? I mean like they haven't even like you haven't even you haven't even tasted it yet and I think like from an analysts point of view like if you had like a really good like you mentioned documentation like if you knew. 01:27:29.32 Jared Atkinson Yeah, well that yeah. 01:27:39.61 Anton Had a good inventory. You had a good sense of like whats server does what like it should be fairly quick to dismiss an alert that's benign right? and I think like Expel that Io like that that they do a really good job of like articulating how they would prioritize and how they like manage their data and the metrics they take and I think like. From what I've seen Anyways, the analysts don't really get that level of support. So yeah, just I I think like other fatigue Definitely valid, but like you have to like experience it first before it actually like stops you from doing things and I think that Dynamic is a little bit reversed from what from what I've seen. 01:28:13.85 Jared Atkinson Yeah, and I think I think ah alerts so false positives are a necessary precondition to reducing false negatives right? because you have no way to measure false negatives and the way that you do that is by being by allowing false positives because like let's say you have 0 alerts. 01:28:31.95 Anton Then. 01:28:33.33 Jared Atkinson Well like if you have 0 alerts then ah, you're assuming that what like the alerts that are produced are the only types of malicious activity that could occur which is a seriously flawed presupposition I think and so then it's like okay well we have to open the scope like when you talk about. 01:28:45.92 Anton Um, and. 01:28:52.62 Jared Atkinson Classification you talk about sensitivity and specificity and sensitivity is like what is the likelihood that if something is bad I will I will consider it to be bad and the way that you increase specificity which is you it allow more false positives right? You you reduce the threshold by which you. 01:29:10.75 Jonathan Johnson Um, yeah. 01:29:11.19 Jared Atkinson Produce an alert and that's like basically that's the only way to that's the only way that we could do it because we don't have a way to measure the other side. Ideally, you do it in a smart way. Obviously right like you don't like a way to not have any false negatives is just to alert on every event that you ever collect I mean you you still actually would have false negatives because. 01:29:25.53 Anton Strength. 01:29:30.19 Jared Atkinson There's potentially bad stuff that doesn't manifest itself in the data that you collect but like you would reduce your false negatives by a lot. It just would be unmanageable so we need to do it. We need to do that in a smart way but reducing all false positives to 0 is a fail a failing strategy for sure. 01:29:44.66 Jonathan Johnson Um, I think that I think the one way to drive that point home is to like have a set of analysts sit down understand what their detections are like purposely fire those detections and then do like 3 or 4 variants that don't and then be like yeah. 01:29:45.31 Anton Friend. 01:29:58.87 Jared Atkinson Yeah, well, that's the purple that's like a purple team strategy I Guess yeah. 01:30:02.77 Anton A part of it. Yeah. 01:30:04.37 Jonathan Johnson But think like that's where I see like a huge impact is because like conceptually like that all makes sense right? Jared like they're like yeah like absolutely. But it's like in the actual practical use case of like their day-to-day job I think it's easier to be like yeah but like we got 4 out of 8 covered we like I'm okay to accept the risk like are you though. 01:30:19.71 Jared Atkinson Well yeah. 01:30:19.84 Anton Yeah. 01:30:21.50 Jonathan Johnson So like let's let's sit down and actually make you sit through these gaps and then like how hot do you like I would argue the hotter you feel in that seat the more like the less you are willing to actually accept that risk. 01:30:36.20 Jared Atkinson Yeah, and I I could go on this topic for another half hour so I don't know if you guys have time but I I'm happy to Johnny Luke 01:30:39.58 Anton I Do yeah for sure. Yeah I Guess the only thing I just worry about the fact that like people don't because they're so scared of the they they're fatigueing all that that they don't try you know and I mean like like they fail before they start pen of thing. That's the. 01:30:50.30 Jared Atkinson Yeah, yeah, but then if but then if you don't try you might as well not be doing it like you're wasting money right? like So my I guess my point is is that it's okay to not try. Maybe but you might as well not have a detection and response team If you're not going to allow false positives because you. 01:30:51.28 Jonathan Johnson Um, yeah. 01:30:56.63 Jonathan Johnson Um, yeah. 01:30:57.50 Anton Yeah. Maybe yeah. 01:31:09.57 Jared Atkinson Like if you don't allow false positives then you you can't do the job that you're hiring people to do. 01:31:13.80 Jonathan Johnson But well before you go well before you go on this like false positive train that I know you really want to go on. 01:31:13.12 Anton Right. 01:31:18.85 Jared Atkinson Do we have do we actually have more time Luke are you good. It's like launch time for you good. Okay. 01:31:23.69 dcppodcast I probably have like another 15 or 20 01:31:26.62 Jonathan Johnson Okay, so um, before you go on that like let me ask this and I want ask this terbye in the room even even Luke over there should an analyst whenever an alert fires and they go to investigate that alert should they be trying to prove. That it is malicious or should they go in there proving that it's non malicious because I feel like whichever way they do so and like I really don't think there's a case or somebody goes in there blindly and's like okay I'm just going to look at this look at this and just like whatever like I think as time goes on and as you start to become more efficient your practice you choose 1 of those 2 things. 01:32:01.31 Jared Atkinson Um, yeah. 01:32:03.68 Jonathan Johnson And the way you look and investigate that data are going to happen differently. So what are your guys' thoughts and opinions like how should an analyst look at the data if Alert fires should they look at it like I'm trying to prove this is malicious or I'm trying to prove. This is not malicious. 01:32:14.50 Jared Atkinson Yeah I think there's um, for me I think it's not quite that black and white. Potentially so like I look at it from like the perspective of the funnel and the goal at each phase until remediation basically is to say should I allocate more resources towards this event right. And so like the job like the job of detection in my opinion isn't to I isn't like it shouldn't be to detect malicious behavior. It should be due to to detect events that deserve more resources right? because like well I can. 01:32:46.25 Jonathan Johnson Okay, well I'm not talking about detection like I'm talking about like I Yeah well like let let's say you are at whatever point your organization has built out for an analyst to say malicious or non. 01:32:51.76 Jared Atkinson And Triage you do the same thing I Guess it's like. 01:33:04.51 Jonathan Johnson You've already gone through this contextual piece of applying that data back because of the reality is each organization's going to do differently some people like like merge triage and investigation together some people merge the detection triage together. Not everybody has it split out like the funnel. So let's say like. 01:33:18.72 Jared Atkinson Um, yeah I. 01:33:21.46 Jonathan Johnson You're at whatever point you see fit within the funnel to actually perform the investigation and you have you analyst. They have all the data they want. 01:33:27.27 Jared Atkinson I I think eventually you have to get to the point of saying is this something that I want to be occurring in my network. So I think that eventually because you can never be certain whether it's malicious or benign like I don't think you could be very very close to certainty. But I don't think you could be certain. 01:33:39.39 Jonathan Johnson Um, yeah I don't think there's a hundred percent either 01:33:46.19 Jared Atkinson And so there's a threshold at which you like I think we we tend to think of everything as a binary comparison of like a single parameter right? So we say oh well if if it meets this criteria then we alert on it and it's like you know the service was created by a process other than services ec. But like that's a lowresolution way to look at it because really what we're doing is we're establishing. A threshold at which if something's above that threshold you alert on it right? But if you only look at one one parameter or 1 variable then you only have scores of 0 and 100 and so your threshold is 100 but like I think in real life your threshold is like 70 or something like you know you could adjust that threshold depending on your risk acceptance or risk tolerance. But. 01:34:08.10 Jonathan Johnson Um, yeah. 01:34:25.31 Jared Atkinson I Think you basically are saying you're I think you're ultimately determining whether or not it's malicious or not or it's at least past the threshold of being considered to be undesirable and maybe there's like a there's an alternative thing of like if I kill this thing will it impact business because like. 01:34:35.59 Anton M. 01:34:42.43 Anton 10 01:34:45.90 Jared Atkinson Yeah, you might as well just kill everything that doesn't impact business to to be honest. 01:34:47.49 dcppodcast I Think that's what actually puts people on the back foot when they're in larger larger organizations and they're triaging these kinds of things I think they are almost forced to look at it from a innocent until proven guilty perspective because it takes too much time to look at it. 01:34:48.83 Jonathan Johnson Yep. 01:35:06.78 dcppodcast At a guilty until proven innocent perspective. It takes too much time to do that to every single alert. So I think when you're in a large organization that has like a bunch of these and who like falsely puts metrics around alert closure. You have to look at it as benign if nothing stands out you close it. 01:35:22.87 Jared Atkinson Yeah, that's it interesting. Okay, go go ahead. Anton I got follow up. But. 01:35:23.35 Anton Yeah, that is an interesting point. Yeah I would I would love us it knows I's just going to say like I would love a study about the bureaucracy of an organization how that affects like the cyber response and the cyber capabilities. Yeah right. 01:35:25.50 Jonathan Johnson Um, yeah. 01:35:36.64 Jared Atkinson The threshold. Yeah, basically the threshold. Well. 01:35:37.89 Jonathan Johnson Um, yeah. 01:35:41.31 Anton Because I've had so much friction with like cab and like high till and like getting changes through and like tickets and by the time you get like 1 little you want to do like one little change and it takes like months. So I think like that that that dynamic is interesting but in terms of like other malicious benign like. Every time I get another like it's never enough right? like if you just get like 1 piece of like I always want more like I and I'm always interested in like I want to build like a narrative you know, like where did this come from what happened like where did it go and then at that point I think the benign and malicious but hopefully be at least closer to self-evident. 01:35:59.10 Jared Atkinson Yeah, correct. 01:36:00.37 Jonathan Johnson Um, yeah. 01:36:15.57 Jonathan Johnson Um, yeah. 01:36:15.72 Jared Atkinson Um, yeah, well if you if you think about like the english common law system and like how we apply law at least in America and I'm I'm sure in Canada it's the same right? But um, it's innocent until proven guilty and the reason like. 01:36:17.10 Anton And it was with that like 1 piece of data. But yeah, it's ah definitely a struggle. 01:36:33.25 Jared Atkinson This is a binary classification problem because the options are so that Okay, there's the reality. The person is innocent or they're guilty but we don't know the reality because like you can't just assume that the person's telling you the truth because like you know Surprise Surprise guilty people will say they're innocent and so. Um, and sometimes actually innocent people will say they're guilty. That's like way less frequent but that actually there are documented cases of that right? And so what we're doing is we're viewing the evidence to make a declaration of what we think is the case and sometimes those things don't match up the truth and our classification don't match up and that that would be false positives which is where. 01:36:59.59 Jonathan Johnson Um, yet. 01:37:08.28 Jared Atkinson We put somebody innocent away or a false negative which is where we let somebody guilty, go right? and both of those are errors which represent a an issue or like ah ah that our that our perspective of the problem was too low resolution to make the proper decision. Um, now the the problem. Comparing in my opinion like a computer scenario like a detection scenario to the legal system is like we our legal system is built on this idea that the individual is sovereign right and almost like that. There's like ah you know whether you believe in religion or not. It's like there's like the like a spark of divinity is within each person. So. Each person is has some like resemblance to God Whatever that may mean um and so the the idea that you would accidentally throw somebody's life away um is a huge risk and the general like not saying that we don't mess this up but the general idea of the the legal system is that. That's too big of a price to pay to be wrong, right? And so we error on the side of like let's not mess that up. Um I don't think that that analogy applies to like a process running on a computer system because like what's the worst case scenario. Well you lose a billion dollars well like a life is maybe you could make an actuarial. 01:38:07.44 Anton That. 01:38:25.83 Jared Atkinson Argument that maybe a life is worth less than a billion dollars. But you know conceptually a billion dollar like I I tend to think that like there's almost nothing you could do on a computer system that would be equivalent to accidentally locking locking an innocent person up I guess. 01:38:39.56 Jonathan Johnson Yeah I ask because like when we talk about like alert in an Analysi fatigue. Um, like I think whichever way an analyst sits down they look at they have all the data they want. They've gone and dug in now they're willing to now they're at the point where they want to make the decision. I think there's biases biases apply to both sides so like um, one that I can see if you're going in there trying to prove that it's not malicious like if you're just like okay like I want to prove that this fits in my organization. All these different things. 01:39:00.81 Jared Atkinson Yeah, that's true. Yeah. 01:39:15.66 Jonathan Johnson A while back we like at the beginning and we talked about ignorance right? and like the willingness to accept ignorance and I can definitely see in some instances where there might be times where people are like I'm not sure I see like this being used on 2 workstations. The prevalence of this is like not super high. I'm kind of ignorant for what it's being used for but because of this I'm willing to close it out and that that could be that could hold a lot of harm for your organization on the other side if you're in there trying to prove that it's malicious that takes a lot of time in the more time you spend on an alert. 01:39:47.49 Jared Atkinson Um, ah okay. 01:39:51.40 Jonathan Johnson Lower the threshold of alert per analyst happens. So that Alert fatigue can happen quicker to see what it doesn't make sense when I'm where I'm like rounding back in. 01:39:53.90 Jared Atkinson Um, yeah, yeah, Ah, but you okay so you you may have kind of changed my perspective a little bit or at least let me articulate it better. But Okay, so if if your goal is to prove that something's benign. In order to close it out then that means that you're going to inevitably kill things that were benign but you just couldn't prove that they're benign right? So that that means you're going to be false false positive prone right? Um, if your goal is to only kill things once you. Prove that they're bad then that means inevitably there's going to be bad things that that don't get killed but you know don't get remediated. We'll say um, which means you're false negative prone now like 1 of the I think that there's already a substantial bias towards false positives. In the way that like our feedback mechanism is set up and so I would argue that you should ah be on like you should error on the side that makes up for your blind spot if that makes sense which means that you should probably error on the side of we need to prove that this thing is benign. Wait in order to like we need to prove that it's benign in order to remove it from the pipeline if that makes sense. Um because I Also think that there's ah you know that exponential thing I talked about about how like they call it. The pareto principle is generally the idea but it's like um, the. 01:41:14.72 Jonathan Johnson Yep, absolutely. 01:41:27.76 Jared Atkinson Some techniques are used exponentially more than other techniques I think that's true of the impact of false negatives versus false positives. So the impact of a false negative is like in Maersk the shipping company's example, it's like a billion dollars a day I'm making that number up, but it's something substantial like that. Ah, billion dollars a day because of a ransomware attack or Saudi Ramco is another example, right? and so there's there's no cap to the amount of impact that a single false negative can can have but the the cost of a false positive is the equivalent of the amount of time that an analyst spends looking into it and so there's a finite. 01:41:56.11 Jonathan Johnson Yep. 01:42:04.57 Jared Atkinson There There is a cap. There's like it. There's There's a finite amount of cost to false positives now the the I guess criticism of that approach is that ah the occurrence of false Negatives. So like the idea is is false negatives. The impact is exponential and the impact of false positives is linear but the. Criticism is that the occurrence of false negatives is linear and the occurrence of false positives is exponential. So It's like they they flip when you talk about how frequently they occur. But I I still think I think that like you have to be we have to be extremely sensitive to the threat of false negatives. 01:42:42.30 Anton Yeah, and I think like that if the response to a false positive is to like decrease the amount of data that you're looking at or to like suppress something like maybe that's not the right decision because maybe with more data you'll be able to. 01:42:43.17 Jared Atkinson And my opinion. 01:43:01.94 Anton Get rid of that false positive. So I like I just worry about that like knee-jerk reaction of false positive to be like hey we have a false positive so they had this data is like all noisy. We got to throw it away where it could be the case that if you add an additional data source this false positive. You have more information about it so you can just close it quicker or. 01:43:09.21 Jared Atkinson Um, yeah. 01:43:19.40 Anton Build like a correlation so it won't fire anymore. So yeah I think like I think these are things that are really important to think about. But I don't see a lot of people thinking about them. Unfortunately yeah so I wish it. 01:43:25.56 Jonathan Johnson Yeah I think like I think part of the issue to that and like this goes back to a conversation Jared and I had the other day. It's like there's a lot of these like because this conversation right here is like relatively conceptual right? Um, hard to put it into practicality. 01:43:36.22 Jared Atkinson Um, yeah, it's hard to. 01:43:36.33 Anton Yeah, yeah. 01:43:42.89 Jonathan Johnson Into a practice where like it's being redundant. It's being like performer like redundant exactly and like I think like that's like the struggle that I see because it's like you know there's a lot of great conceptual ideas for detection engineering and like the whole pipeline in general but the question is like. 01:43:45.92 Jared Atkinson Especially with how secretive everybody kind of is and how they yeah yeah. 01:43:48.83 Anton Damn. 01:44:00.97 Jonathan Johnson Are people practically implementing those if so are they even willing to come out and say how they're performing these practices because like that could help another organization's pipeline in general right. 01:44:11.21 Jared Atkinson Yeah I think this is it reminds me of like the sins of commission versus sins of O mission type thing to where like you know when I let's say when I was a kid if I did something bad and I didn't tell my mom that I that I had done it and then she found out later. She'd be like why'd you lie to me I'm like oh I didn't lie to you. She's like well you didn't tell me that's the same thing as line right? That's basically like um false negatives are like the sin of omission. It's it's this invisible thing that we don't even realize like we we don't even realize is happening Um, but that doesn't mean that it's less bad if that makes sense cool. Well I think. 01:44:44.55 Jonathan Johnson Yeah, no I agree. 01:44:49.46 Jared Atkinson Yeah, this is our longest podcast episode ever. So Anton appreciates you hanging out with us. Dude yeah yeah, we appreciate it man. Yeah for sure. Yeah. 01:44:51.76 Anton Okay, sweet all right I win the prize. Yeah no my pleasure. Thank you guys. 01:44:55.31 Jonathan Johnson Yeah man, thanks for taking the time and and on this week to hang out with us. It's also great to meet you in person I see all the cool work you do and I'm like I've never actually got to like meet you yet. So. 01:44:59.33 Anton Yeah, likewise likewise like I said my pleasure. Oh thank you I know hopefully after Covid I could like get out to like a con or something see you guys a person. Nice. 01:45:09.80 Jared Atkinson Yeah, yeah for sure Hopefully black hat everybody I live in Vegas So I'd be happy to see people here. Yeah, we could do that we get. We gotta figure out how to like do it inside the casino I feel like that might not be possible I don't. 01:45:12.96 Jonathan Johnson Dude Yeah, that'd be awesome. 01:45:17.42 dcppodcast Dude Jared are we thinking some ah some more live and vegas episodes for like it. 01:45:26.98 Jonathan Johnson That'd be pretty sick and I can just put on my old like specter op shirt and just slide under the radar and be like yeah I'm an employee let me just like let one of the rooms and then. 01:45:28.12 Jared Atkinson I Don't know if any of us have poll like that. So. 01:45:33.50 Jared Atkinson There you go? Yeah cool all right guys? Well yeah, appreciate you and this was a great great Convo I Really enjoyed it. So thank you and we'll talk to you soon here's. 01:45:35.80 Anton Thought Likewise Yeah, thank you I did to Cheers guys. Thank you fight. 01:45:43.27 Jonathan Johnson Thanks.