I’m on my way to Stuttgart, to speak at IT-DEFENSE, 2019 on the topic of “metrics, the quest for meaning.” Things may be spotty here for a week; it depends on my energy level.
Security metrics is a topic I have been very interested in for a long time: how do we show that what we are doing, works? The answer, naturally, is “it’s harder than it looks.” Since I started teaching clients about metrics – about 15 years ago – relatively few (practically nobody, really) have been able to do much more than tactical tracking of “speeds and feeds” and a few “gee, wow, numbers.” This is a big problem for security because, in order to be taken seriously, we need to show effectiveness – and most of the measures regarding security are bad ones: “credit cards leaked” or “terabytes of secret NSA data stolen.” Those don’t actually tell us anything useful, either, but they are what the media talk about.
Consequently, most of the figures you’ll hear in security (“80% of attacks come from the inside” or “antivirus is 15% effective”) are made up. And don’t get me started on the various surveys-as-marketing we have to deal with; it’s a gigantic pile of self-selected samples and the survey questions are badly designed.
Off to the airport! This is probably the last time I will do a talk on metrics, so I’ll give it my best.
Safe travels.
Happy New Year.
Fly safe!
Depending on where you land, you might well fly over me.
That topic really sounds interesting and open to all kind of bullshittery.
I hope you’ve safely arrived by now and are happily partaking of some good German food and a decent beer.
Do you have notes or slides or will a video of the presentation be posted?
avalus@#3:
There’s room for lots of BS in the topic, but it’s a serious problem that security practitioners need to understand better. There are constantly people saying “security practitioners need to know how to talk to businesspeople” and being able to do that entails an understanding of the business’ problems. Far too often, security people are left saying “the sky is falling!” but we’re unable to quantify the impact.
My favourite local prohibited weapon thing is the ban on “long” single edged implements. What counts as “long”? No one knows, not even the cops (I asked)
dangerousbeans@#7:
My favourite local prohibited weapon thing is the ban on “long” single edged implements.
Massachusetts used (maybe still does?) to regulate swords under “long” – a reenactor friend of mine wound up with a criminal record for bringing a viking sword into the state and running into the wrong cop.
What gets bizzare is collectible items – for example, cops might be harassing someone for owning a “dagger” but as a collector I appear to be OK owning a katana. Generally, as long as you keep your claymore at home you won’t have a problem but if you carry a “hunting knife” you’re alright but don’t carry a “dagger” or “trench knife”; none of it makes sense to me. Especially given the equally absurd gun laws. If they weren’t so damn expensive I’d pack a Stryker bone saw, MBS style.