Corporatism favors the rights of corporations ahead of those of their customers; right now we’re in the middle of a complex shouting-match regarding what companies like Facebook can decide to provide to marketing partners; are we trusting them too much with our data? Are they going to handle it responsibly? What makes people imagine that companies are not going to immediately have a strategy meeting and ask, “what is the worst thing we can do with our customer data? Because: let’s do that!”
The worst of it gets done under the rubric of service-level agreements or “shrink wrap” licenses – some document that says (down in the fine print) “use of our system is taken as consent for us to provide data to partners, as we see fit.” As much as I loathe “targeted advertisement” systems, they’re not really the problem. The problem is: what happens when the data you thought was private gets fed into the retro-scope?
Once it’s in there, it’s never coming out – even if it’s wrong or partial. One of the big concerns some of us have, is that the information is going to wind up in some great big data-dump somewhere, and matching algorithms will trawl through it, looking for new relationships that can be constructed, then added back into the data-pile (as possibly inaccurate “conclusions”) – it’s bad enough when it’s done in the name of marketing, but what if some corporate executive just decides to give the data away to be helpful? Imagine the terms of service say “we will never sell your data!” (that’s right: we’ll give it away) and “we are not a medical service provider” (so we are not covered by medical privacy regulations like HIPAA or HITECH). Once a copy of the data has been shared, it’s now part of someone else’s data pile and they will never delete it, because they don’t have to.
Some organizations have particularly nasty habits regarding data deletion and aggregation. Not surprisingly, those are the same organizations that build retro-scopes – the FBI, the NSA, and the rest of the world’s equivalents. But Amazon.com isn’t so great, either – their Rekognition system has a huge, unregulated, backing store of images to match against. So does the FBI’s face recognition database; thanks to “voluntary sharing” between state motor vehicle databases, they have at least everyone’s driver’s license photo(s) going back years. Amazon’s got a gigantic data set, too. [rekognition]
Amazon Rekognition makes it easy to add image and video analysis to your applications. You just provide an image or video to the Rekognition API, and the service can identify the objects, people, text, scenes, and activities, as well as detect any inappropriate content. Amazon Rekognition also provides highly accurate facial analysis and facial recognition on images and video that you provide. You can detect, analyze, and compare faces for a wide variety of user verification, people counting, and public safety use cases.
Amazon Rekognition is based on the same proven, highly scalable, deep learning technology developed by Amazon’s computer vision scientists to analyze billions of images and videos daily, and requires no machine learning expertise to use. Amazon Rekognition is a simple and easy to use API that can quickly analyze any image or video file stored in Amazon S3. Amazon Rekognition is always learning from new data, and we are continually adding new labels and facial recognition features to the service.
The “adding new labels and features” part is what ought to worry you. These systems, like the FBI’s face recognition database, are being built with inadequate regulation: the algorithms get changed and they may affect you and you’ll never know.
That all sets the stage for why this worries me: [reg]
Some would argue he has broken every ethical and moral rule of his in his profession, but genealogist Bennett Greenspan prefers to see himself as a crime-fighter.
“I spent many, many nights and many, many weekends thinking of what privacy and confidentiality would mean to a genealogist such as me,” the founder and president of FamilyTreeDNA says in a video that appeared online yesterday.
He continues: “I would never do anything to betray the trust of my customers and at the same time I felt it important to enable my customers to crowd source the catching of criminals.”
His “customers” are not the people who are interested in catching criminals. His customers use his service to see who they are related to, or whether their DNA contains the markers for possible disorders. Perhaps they are racists, who want to discover their ancestry is human. Almost certainly his “customers” are not the FBI.
And so, by crowd sourcing, what Greenspan means is that he has reached an agreement with the FBI to allow the agency to create new profiles on his system using DNA collected from, say, corpses, crime scenes, and suspects. These can then be compared with genetic profiles in the company’s database to locate and track down relatives of suspects and victims, if not the suspects and victims themselves.
I understand and appreciate the idea, but not the implementation. The FBI is not trustworthy. There are well-documented incidents in which the FBI has used DNA evidence to convict, while ignoring DNA evidence that exonerates. Allowing them to do this sort of thing in secret is allowing them to control the knowledge about their searches; in other words they might automatically check to see if you’re implicated as a serial killer. There’s an asymmetry of knowledge problem if the FBI is checking to see if you’re a serial killer, but they are not checking to see if everyone who is in prison as a serial killer may actually be exonerated by DNA evidence. Since this was all happening in semi-secret, nobody was able to question the FBI’s approach.
I’m also curious as to whether or not the database was protected against large-scale scraping. The FBI generally isn’t technically sophisticated enough, but if you gave the NSA access to a database like that, they’d slurp the whole thing down into some classified database someplace else, and probably eventually leak it to every other intelligence agency in the world. Does this matter? I don’t know. It ought to be a matter of public policy that is examined a bit more closely before someone decides to “just do it.”
“In order for the FBI to obtain any additional information, they would have to provide a valid court-order such as a subpoena or search warrant.”
The problem is that the FBI won’t need a search warrant against a particular database – all they need to know is that somewhere, out there, is a DNA sample that matches someone. The rest of the data they want is in the NSA’s database; they just need to know where to look.
What are the public policy problems with this? I’m not asking rhetorically – I have no idea. Off the top of my head, I think it’s not unreasonable to let the FBI discover if there may be a DNA record that points to a serial killer, rapist, or whatever. But can we/should we also require that the FBI check for exonerating DNA information for any prisoner that requests it? The scenario as it’s presented appears to be: someone gets a knock on the door, answers, and is handcuffed and arrested because new DNA evidence pinned a murder on them. Shouldn’t that scenario be counter-balanced by a prisoner being informed “we’re releasing you because we just found out that you really didn’t do that crime you’ve been saying you didn’t do.” What if you have someone on death row for a crime they say they didn’t commit, and the FBI uses a DNA database to discover that, in fact, someone else did the crime? Obviously, the guy on death row is exonerated and the state owes them a huge apology for convicting them on false evidence. Does that sound incredible? It’s not. In the US, racist policing practices and selective prosecution have resulted in exactly that sort of situation in the past. It seems to me that someone needs to be telling the FBI, “OK, so now you’ve got access to DNA databases: we want you to spend as much time trying to exonerate people as you spend trying to convict people.” How would that work?
Summary: perhaps we are trusting corporations too much, and perhaps we are trusting the FBI too much.