I’m on my way to Stuttgart, to speak at IT-DEFENSE, 2019 on the topic of “metrics, the quest for meaning.” Things may be spotty here for a week; it depends on my energy level.
Security metrics is a topic I have been very interested in for a long time: how do we show that what we are doing, works? The answer, naturally, is “it’s harder than it looks.” Since I started teaching clients about metrics – about 15 years ago – relatively few (practically nobody, really) have been able to do much more than tactical tracking of “speeds and feeds” and a few “gee, wow, numbers.” This is a big problem for security because, in order to be taken seriously, we need to show effectiveness – and most of the measures regarding security are bad ones: “credit cards leaked” or “terabytes of secret NSA data stolen.” Those don’t actually tell us anything useful, either, but they are what the media talk about.
Consequently, most of the figures you’ll hear in security (“80% of attacks come from the inside” or “antivirus is 15% effective”) are made up. And don’t get me started on the various surveys-as-marketing we have to deal with; it’s a gigantic pile of self-selected samples and the survey questions are badly designed.
Off to the airport! This is probably the last time I will do a talk on metrics, so I’ll give it my best.