Terrorism, the TSA, and the false positive problem

According to Alexander Furnas writing at The Atlantic, our ever-inventive friends at the Department of Homeland Security and the TSA are ever on the trail of new and more intrusive ways to make travel even more unpleasant. They have developed a ‘pre-crime’ detection program (reminiscent of the film Minority Report) known as FAST (Future Attribute Screening Technology), because all bad programs need a catchy acronym to help gain acceptance.

FAST will remotely monitor physiological and behavioral cues, like elevated heart rate, eye movement, body temperature, facial patterns, and body language, and analyze these cues algorithmically for statistical aberrance in an attempt to identify people with nefarious intentions.

This all sounds super-scientific and precise.The problem is that whenever you are trying to determine the likelihood of a low-probability event with anything other than a 100% accurate predictive test, you encounter the inevitable problem of false positives.

I have written before about the false-positive problem as applied to medical diagnostic tests that can cause unnecessary fear and alarm to patients who are unaware of how to properly interpret statistical data. This same false-positive problem will inevitably overwhelm the extremely intrusive and privacy invading FAST program. Furnas runs some numbers:

Here is why: let’s assume for a moment that 1 in 1,000,000 people is a terrorist about to commit a crime. Terrorists are actually probably much much more rare, or we would have a whole lot more acts of terrorism, given the daily throughput of the global transportation system. Now lets imagine the FAST algorithm correctly classifies 99.99 percent of observations — an incredibly high rate of accuracy for any big data-based predictive model. Even with this unbelievable level of accuracy, the system would still falsely accuse 99 people of being terrorists for every one terrorist it finds. Given that none of these people would have actually committed a terrorist act yet distinguishing the innocent false positives from the guilty might be a non-trivial, and invasive task.

This is why I think that once we have locked cockpit doors, installed metal detectors, and passengers feel free to intervene to overpower aggressive fellow passengers, we have reached the level of reasonable precautions, in that there is little or no chance of an airplane being used as a weapon and the chances of dying in a hijacked plane are far less than in everyday life. We are now well past the stage where diminishing returns have set in and all we are doing is making air travel infuriating and discriminatory.

I wrote before about the debate between Sam Harris and Bruce Schneier, where the latter challenged the former’s support for the profiling of Muslims. The entire exchange can be read here. After giving all the technical reasons as to why the absurdly expensive and intrusive methods adopted by the TSA and the Department of Homeland Security are undesirable, Schneier’s conclusion is worth repeating:

But perhaps most importantly, we should refuse to be terrorized. Terrorism isn’t really a crime against people or property; it’s a crime against our minds. If we are terrorized, then the terrorists win even if their plots fail. If we refuse to be terrorized, then the terrorists lose even if their plots succeed.

We are at a small risk every day from death or injury due to criminals, maniacs, accidents, and defective products. We have learned to live with the fact that we cannot have perfect protection from such threats without making life intolerable. Similarly there is no question that there will be a small risk of terrorist attacks of varying levels of effectiveness as long as we continue to invade other countries and kill their people using drones and other means. Why cannot we live with the same tradeoffs of safety versus liberty when the source of danger is terrorism?

Stephen M. Walt recently attended a conference in Turkey and describes his own traveling experience:

Finally, I flew here on Turkish Airlines via John F. Kennedy Airport in New York. The flight was fine, but the on-the-ground experience in JFK was one of the more miserable I’ve had in the past decade. And I couldn’t help but wonder — and not for the first time — how this affects how non-Americans view the U.S. when they arrive here. So I have the following modest proposal to offer: Every U.S. congressperson should be forced to fly through JFK on their own (i.e., with no staff to help), and to go through the normal TSA procedure (no VIP lines). And then they should be flown to a really first class airport in some foreign country (say, in Singapore, or Munich), so that they can see just how decrepit U.S. transportation infrastructure has become. And a few hours interacting with the Keystone Cops at JFK’s TSA checkpoints would be instructive for them too.

I think Americans have no idea how rapidly this country’s public services and infrastructure, once the envy of the world, is deteriorating as massive amounts of resources are diverted to defense and tax cuts for the wealthy. Add to that the needless aggravation caused by the never-ending paranoia generated by the so-called ‘war on terror’ and you have a country that people will be reluctant to come to unless they have to.


  1. Coragyps says

    What astounds me most about the whole “airport security” mess is that, from the terrorist point of view, hijacking or blowing up airplanes has already been done! It’s now “last decade” stuff. Sure, it could happen again, but a huge part of the impact of terroristic acts is their surprise value. 9/11/01 was unexpected, and that’s a huge part of why it was so dramatic and frightening.
    Why don’t we see armed escorts with every gasoline and propane delivery truck in the country? One of either driven into a shopping mall on a mid-December Saturday would be simpler and just as horrifying as taking a plane into a skyscraper. Odd.

  2. Paul Jarc says

    Why cannot we live with the same tradeoffs of safety versus liberty when the source of danger is terrorism?

    I think dying in a terrorist attack seems worse than dying from an accident or a health problem because there is an adversary who actively wants to cause the death. Winning out over an adversary is more gratifying than avoiding an accident. Dying in a terrorist attack also seems worse than dying in a criminal attack because the targets are not just the actual victims, but the entire tribe that the victims belong to.

  3. Mano Singham says

    I am not so sure about this. Having lived through a time and a country where terrorist acts were commonplace, I found that people treated it like any other human-instigated risk factor. I don’t know if people in other countries where terrorism is endemic think similarly but I find it hard to imagine that it reaches the level of paranoia that exists in the US, where I believe it is caused by deliberate hyping of this particular threat.

    In other countries, governments seem to try to calm the people by minimizing the threat. Here the government tries to maximize the threat and scare you to death. Both may have ulterior motives for their actions but I definitely prefer the former.

  4. Dunc says

    And I couldn’t help but wonder — and not for the first time — how this affects how non-Americans view the U.S. when they arrive here.

    Very, very badly indeed. And compared to the rest of the world, landing in a US airport was hellish even before all this post-9/11 crap.

  5. bcoppola says

    Hell, trying to get back in my own damn country from Canada by just driving across the fucking border that’s only 45 minutes from home is bad enough these days.

  6. Dianne says

    FAST (Future Attribute Screening Technology)

    If we can’t stop this from happening, the next best thing would be to make sure it is implemented by FAST Action Response Teams. And that their caps had the acronym on them.

  7. smrnda says

    I actually don’t think the first batch of language sounded precise or scientific at all, but having been an editor and having edited academic journals it may be that my mind screens for BS. I mean, the things they cataloged are about the same as things used in polygraphs (‘lie detectors’) which are not known to be very accurate.

    As for safety and liberty, you have to start with some estimation of how likely an event actually *is* and then decide what dangers you really need to protect yourself from. I think certain types of harmful or hostile acts – like crime or terrorism – people’s minds just pick out more since you’ve got this direct ‘bad guy’ you can picture. It’s hard to point to more likely but more mundane factors; I mean, lots of people die in work related accidents but that seems to be something that people seem unable to get worked up about, probably because the conscious actions that to into them – low levels of workplace safety or weak enforcement of existing rules – are harder to picture than a guy shooting someone with a gun.

  8. amhovgaard says

    Judging by the list of cues, this technology is going to catch a lot of people who have a fear of flying…

Leave a Reply

Your email address will not be published. Required fields are marked *