No system of detection is perfect, whether it be in medicine or police work. There is always the chance of false positives and the wider you make your dragnet, the larger the number of false positives that you are going to get. While the massive secret databases of the NSA are touted as an efficient means of detecting patterns to thwart terrorist attacks, it is simply a statistical fact that any pattern matching software will throw up false positives.
With the revelation of the NSA’s massive surveillance program that targets pretty much everyone, it is guaranteed that many wholly innocent people will find themselves in the crosshairs of the government as potential terrorists and will suffer harassment even though they have done nothing to warrant such suspicion. But what makes this worse is that since the program is kept secret and people do not have the chance to clear their names in open court, they will have little or no mean of proving their innocence. We have already seen many cases of people being put on no-fly lists (created using name-matching software) and prevented from boarding planes with no reasons given, so they have no means of clearing their names.
But it can and does get even worse. Gail Collins reminds us of the story of Brandon Mayfield as a cautionary tale of how the national security state can get things horribly wrong and ruin the lives of people. He was picked up as a suspect in the 2004 bombing of a commuter train in Spain as a result of a faulty match in a fingerprint database. What was even worse, he was married to an Egyptian and his daughter was named (gasp!) Sharia! What more evidence of terrorist sympathies do you need? You have to read Collins’s account to truly appreciate the hell that he and his family went through (arrests, secret searches of their home and offices, etc.) to appreciate how bad it can get.
Fortunately the Spanish authorities felt, reasonably enough, that a crime on its soil was unlikely to have been committed by someone who had never been to that country and so they continued their search and found the person whose fingerprint truly matched. That was what resulted in Mayfield being ‘cleared’ though you can be sure that he remains on the watch list
Cory Doctorow describes a more recent case of a faulty match of patterns.
You should care about privacy because if the data says you’ve done something wrong, then the person reading the data will interpret everything else you do through that light. Naked Citizens, a short, free documentary, documents several horrifying cases of police being told by computers that someone might be up to something suspicious, and thereafter interpreting everything they learn about that suspect as evidence of wrongdoing. For example, when a computer programmer named David Mery entered a tube station wearing a jacket in warm weather, an algorithm monitoring the CCTV brought him to the attention of a human operator as someone suspicious. When Mery let a train go by without boarding, the operator decided it was alarming behaviour. The police arrested him, searched him, asked him to explain every scrap of paper in his flat. A doodle consisting of random scribbles was characterised as a map of the tube station. Though he was never convicted of a crime, Mery is still on file as a potential terrorist eight years later, and can’t get a visa to travel abroad. Once a computer ascribes suspiciousness to someone, everything else in that person’s life becomes sinister and inexplicable.
Of course, these cases will not convince those who have been conditioned to think that they are in imminent danger of a terrorist threat and that we need to give the government all these secret powers to keep them safe. Most people also have an unreasonably high opinion of computers. They think that the chances of them being mistakenly identified as a potential terrorist is small. And they are correct. But for those few who do happen to be out of luck, it can destroy their lives.
People seem surprisingly willing to give up their freedoms as long as someone else is likely to pay the price.