Skip to content

DEF CON 34 talks are here: 354 talks, summarized and searchable.

Browse DEF CON 34

The Folly of Rewarding A While Hoping for B

Casey Beaumont

PolicyTalks51 minBeginnerOpen on YouTube ↗

Where

Talks
CactusCon 14

Duration

51 min

Explains how security metrics can reward noise and activity, and how to measure real risk reduction instead.

Summary

The talk opens with a common frustration: security teams often report numbers that are easy to count, such as blocked events, ticket volume, or vulnerability totals, without showing whether exposure actually went down. Casey Beaumont draws on a 1975 management paper about rewarding one behavior while hoping for another, using examples from politics and academia to show how measurement systems steer what people do.

She then walks through how this plays out across functions. Incident response can end up rewarding heroics and fear of being made redundant, vulnerability programs chase inflated counts and arbitrary deadlines, SOC teams are judged on throughput that fuels alert fatigue and burnout, and GRC reporting can favor paperwork over real risk decisions. For each area she suggests alternatives: severity breakdowns with tags, known-exploited prioritization, outcome-focused threat hunting and red team stories, and insider risk programs built on trust rather than surveillance.

To redesign metrics, she proposes asking whether a number would prompt celebration or worry, what behavior it encourages on a bad day, and whether it supports a decision. Her four pillars are risk outcomes, learning, operational sustainability, and trusted enablement. She recommends pairing numbers with before-and-after narratives, applying double-loop reflection, and auditing each metric against organizational goals. The core message is that metrics teach an organization what security means, so they should teach the lesson leaders intend.

Related talks