In an earlier post, I linked to a test to see how good we are at recognizing faces. There is also software that takes measurements of parts of the face that are supposedly highly distinctive (such as the shape of the ear) and uses that data to find points of similarity to give estimates of the likelihood of an exact match.
But Ava Kofman writes that this technology is far from foolproof and can lead to errors that can lead to the ruin of innocent people, using the example of one man whose life has been upended because of faulty matches.
No threshold currently exists for the number of points of similarity necessary to constitute a match. Even when agencies like the FBI do institute classification guidelines, subjective comparisons have been shown to differ greatly from examiner to examiner. And the appearance of differences, or similarities, between faces can often depend on photographic conditions outside of the examiner’s control, such as perspective, lighting, image quality, and camera angle. Given these contingencies, most analysts do not ultimately provide a judgment as to the identity of the face in question, only as to whether the features that appear to be present are actually there.
“As an analytical scientist, whenever someone gives me absolute certainty, my red flag goes up,” said Jason Latham, who worked as a biochemist prior to becoming a forensic scientist and certified video examiner. “When I came from analytical sciences to forensic sciences, I was like some of these guys are not scientists. They are voodoo witchcraft.”
In 2009, following the National Academy of Sciences’ call for stricter scientific standards to underpin forensic techniques, the FBI formed the Facial Identification Scientific Working Group to recommend uniform standards and best practices for the subjective practice of facial comparison. But the working group’s mission soon ran up against an objective difficulty: Like some other forensic sciences, facial comparison lacks a statistical basis from which its conclusions may be drawn.
This is, in part, because no one knows the probability of a given feature’s distinctiveness. As a FAVIAU slide on the “Individualization of People from Images” explained, “Lack of statistics means: conclusions are ultimately opinion-based.”
We tend to be too easily impressed by analyses that assign numbers to estimates, giving them a heft that may not be warranted.