EP39 From False Positives to Karl Popper: Rationalizing Cloud Threat Detection

Guest: Jared Atkinson, Adversary Detection Technical Director at SpecterOps Topics: What are bad/good/great detections? Is this all about the Bianco's pyramid? Is high good and low bad? How should we judge the quality of detections? Can there be a quality framework? Is that judgment going to be site specific? What should we do to build more good directions? Is this all about reducing false positives? Can we really measure false negatives? How can we approach this? How can we test for detection goodness in the real world? What are the methods that work? It can’t be just about paper ATT&CK coverage, right? What are your top 3 tips for improving the detection practice at an organization? Resources: “The Pyramid of Pain” post by David Bianco “On Threat Detection Uncertainty” “Detection Coverage and Detection-in-Depth”  “Detection in Depth” by SpecterOps “Philosophy of Science: Rationality Without Foundations" by Karl Popper (yes, really) Red Canary “2021 Threat Detection Report”  "The Black Swan: The Impact of the Highly Improbable" by Nassim Nicholas Taleb John Piaget's theory of cognitive development