I’ve decided to celebrate June 10 as Facial Recognition Day. On this day, every year, I will try to think of some way to interfere with facial recognition.
Happy “facial recognition day”
I know, of course, that you’re not prepared for facial recognition day this year, but the best thing (really) is to celebrate it on some random day, in some random place, in a manner of your own choosing. I am going to post this on Facebook and maybe a couple of other places that do facial recognition, and see if people will take the time to mark it as me. Perhaps the algorithms will spin and whirr and eventually anyone who looks like this (Sorry, John Oliver!) may get tagged as being me. Then, of course, I’ll do something terrible and anyone who has a big nose and glasses on will get busted. Or something. It’ll be tough for Sam Harris and Jerry Seinfeld, too.
[source]
Back in April, it emerged that South Wales Police planned to scan the faces “of people at strategic locations in and around the city centre” ahead of the UEFA Champions League final, which was played at the Millennium Stadium in Cardiff on June 3.
On May 31, though, a man was arrested via AFR. “It was a local man and unconnected to the Champions League,” a South Wales Police spokesperson told Ars. It’s not clear whether this was due to the technology being tested ahead of the match.
We’re told that there was a warrant for the man’s arrest, but the spokesperson declined to provide any further details about the suspect. We know from the request for tender published by the South Wales Police, however, that the man’s face was probably included in the force’s “Niche Record Management system,” which contains “500,000 custody images.” [ars]
Before you ask (as I did) the “ADNABOD WYNEBAU WEDI EI GOSOD” on the side of the van is not a prayer to Cthulhu, it’s Welsh. When I originally saw this piece of news, I thought it was New South Wales, and the writing on the side made me think it was a “COVEFE” van, or something.
As I’ve mentioned elsewhere [stderr] facial recognition is already widely deployed – they just aren’t telling us. I’m sure they’re going to wait until there is an appropriate mass casualty incident, and they’ll start using it openly to round up people who wear “big nose and glasses” kits. The information available is sketchy – deliberately so – and I believe it’s very inaccurate. I saw a police van with a camera just like the one in Wales parked at the oncoming lane at Monroeville Mall. I probably only noticed it because I’m a Lieutenant Major Colonel in the Tin Foil Hat Brigade and we’re trained to look for stuff like this.
Meanwhile, per The Guardian: [guard]
Approximately half of adult Americans’ photographs are stored in facial recognition databases that can be accessed by the FBI, without their knowledge or consent, in the hunt for suspected criminals. About 80% of photos in the FBI’s network are non-criminal entries, including pictures from driver’s licenses and passports. The algorithms used to identify matches are inaccurate about 15% of the time, and are more likely to misidentify black people than white people.
If it’s wrong 15% of the time, and it’s more likely to misidentify black people, it’s pretty much garbage. I wonder what they’re using in Wales.
What can we do to make their algorithms wronger? We need to come up with a way of making the system blossom with a million false positives. It’s only going to get better, so the time to start corrupting its neural networks is now.
Face Time
Groucho Ranum
This reminds me of an incident about a hundred years ago, in SLC. I was walking home, and there was a cop car pulled into the street. I was walking past it, when the cop yells at me to ‘come here’. I walked over, he pulls a photo clipped to his dashboard, holds it out to me, and asks “Is that you?” I just stared at him for a few, then said no. He asks if I’m sure it isn’t me. I respond with “it’s obviously not me.” (I didn’t say “you don’t think I’d say yes if it was, do you, stupid?”)
Half of them can’t figure out recognition armed with a photo. The whole facial recognition business reminds me of that cop.
A while back, I think on Mano’s blog, there was a link to a site that tested facial recognition by humans. As I recall, I did about average (and maybe a little worse, because I was guessing on the harder ones).
So some people are going to do worse at facial recognition than average, and some people have actual prosopagnosia, and can’t even recognize their own face in a mirror.
Many men can confound facial recognition by growing or removing facial hair.
(preview worked!)
Ah, here’s the post I was remembering
https://freethoughtblogs.com/singham/2016/08/28/how-good-are-you-at-recognizing-faces/
I may have some suggestions: Facebook is bound to end up in that database – if it hasn’t already. So you could use an algorithm which iteratively changes a picture and posts each iteration on facebook along with random tags or whatever is needed. Thus you can quickly generate thousands of images that look kind of the person in the initial image, but have increasingly different features.
For more sophistication, you could gradually morph a starting image into the image of another person and post all the intermediates. Repeat that with a large and diverse enough population and you’ll end up with just that: a population of fictitious people, which are bound to show up whenever someone from the originals (or their relatives) walks past a camera.
You could also just randomly mix and match facials features from a set of images. You may need to smooth out the edges a bit but I suspect whatever algorithms harvest images for those databases probably won’t look much further than the basic markers for recognition anyway.
For my final suggestions, consider using pictures of the Trump family and the current cabinet in the intial sample set. As unlikely as it is to work, it would be worth seeing the results if something did stick. On second thought, given the number of scandals that administration is producing*, noone would take any notice if some White House official is accidentally arrested in Washington for drunk driving in Alaska earlier that day, because some computer said it was him.
And I’m pretty people working in the wonderful field of image processing have done most of the work already.
A disclaimer:
Actually I think noone should do anything of the sort because it would mostly affect the people who don’t have the means to fight back against these false positives.
It may be useless but most of the victims probably can’t raise enough of a fuss to make it worth fixing. In an aggressive police state the unrealiability could be considered a feature, not a bug. The system advertises ‘safety and protection’ while giving cops excuses to do unlawful things. “Facial recognition said she was Jack the Ripper, so we clapped her in irons and threw her into the Thames. How were we supposed to know it wasn’t her? Him, sorry.”
*We need a unit for this. I’d like to propose Waterhertz, [WHz] or 1 scandal / day, just to remain consistent with SI units.
This has solved one mystery, though: https://twitter.com/ramenfuneral/status/871248973729849344
The misidentification of dark-skinned people at a greater frequency is par for the course with these things. Apparently Microsoft’s Xbox Kinect had the same problem for ages – it was much better at recognising pale faces than dark ones. In fact it picked up crude pictures of faces drawn on white paper better than actual black people’s faces.
Owlmirror@#3:
So some people are going to do worse at facial recognition than average, and some people have actual prosopagnosia, and can’t even recognize their own face in a mirror.
I’ve always been fascinated by that. It points to an evolved in brain-location that does face recognition. So, the evolutionary psychologists can say they are right about something.
There are some really interesting cracks in the long-solid facade of Ekman’s work on face recognition. I probably should do a post about it but I’ve been kind of holding back from attacking psychology too much.
chigau@#4:
Many men can confound facial recognition by growing or removing facial hair.
Yup!
Back in 2001 I went on a trip involving snorkeling, so I shaved my mustache in order to make the seal on the mask better. Hey, it’ll grow back, right? A week or so later I went to speak at a conference and the program committee didn’t recognize me at first – I wandered around for a while until finally someone did a double-take, “oh it’s you!” I was surprised, too, after seeing this particular configuration of mustache it pretty much defined me to many people. I looked funny to myself the whole time. (the mustache has been there for a long time, to hide a scar from getting shield-bashed in the face in a reenactment fight in 1986)
@Marcus Ranum, #9
What? Does facial recognition, in and of itself, really have anything to do with evopsych?
It’s more of an example of the modularity of the senses than of anything to do with behavior, I would think.
Speaking of recognition, I see you have a new Gravatar icon. That threw me for the a bit, the first time I saw it…
Concerning prosopagnosia: I’ve read of a case where a sheep herder developed prosopagnosia, and lost the ability to recognize his sheep as individuals. So whatever piece of the brain that’s affected, it’s not limited to the recognition of conspecifics.
Owlmirror@#11:
What? Does facial recognition, in and of itself, really have anything to do with evopsych?
It’s a social activity and, if it’s localized in a brain region, like language recognition and production appear to be, then I’d say evolutionary psychology have another behavior that’s in the genome, that they can point to. Isn’t one of the premises of evolutionary psychology that there are evolved-in cognitive processes that are not entirely learned? That appears to me to be at the core of the great nature/nurture debate, which in turn is at the core of the great “is evo psych BS?” debate.
It’s more of an example of the modularity of the senses than of anything to do with behavior, I would think.
Welllllllll maybe, if we want to say that facial recognition is a “sense” that is separate from vision.
It seems to be a situation that’s similar to speech production: we see that speech production and comprehension are done in different parts of the brain, and involve memory, but since brain damage can alter either one of those capabilities, we appear to be able to say it’s localized in the Wernicke’s and Broca’s regions. Since it seems to always be in those regions I think that argues there’s a component of speech production that’s not completely “nurture” (that seems to be a safe conclusion, I think, at this time) If there’s a particular brain region that is associated with face recognition (which presumably also ties to memory) that is distinct from the apparatus of vision, that sounds a lot to me like a component that is not completely “nurture”
Disclaimer: I am not pushing a view here; my opinion about the nature/nurture question is that we’ll probably find that different capabilities are a bit of one, a bit of the other, or sometimes almost all of one and only a bit of the other. I don’t expect we will ever discover that there is a simplistic answer to a “nature or nurture” question, it’s going to always be a fractally complicated stew of “it depends” but that’s just my random prediction/opinon based on how I think brains seem to work. I believe the nature AND nurture extremes are going to be disappointed in the long run.
Speaking of recognition, I see you have a new Gravatar icon. That threw me for the a bit, the first time I saw it…
Different but still a badger!
mck9@#13:
I’ve read of a case where a sheep herder developed prosopagnosia, and lost the ability to recognize his sheep as individuals. So whatever piece of the brain that’s affected, it’s not limited to the recognition of conspecifics.
That’s super interesting! I’m trying to figure the implications of that – does that argue that it’s more of a learned behavior or that it’s an innate behavior with very broad parameters for what constitutes a “face”?
I’m sad that I won’t be around long enough to see science figure some of this stuff out. Just because it’s cool.
@Marcus Ranum, #14:
This is terribly confused. Facial recognition is certainly used in social activity, but it is a fundamentally neural activity.
Here’s the page on prosopagnosia, which traces the syndrome to problems in the fusiform gyrus. It’s in the brain.
I am not expert on the claims of evopsych, but if it is, I am pretty sure that’s something that critics of evopsych have no problems with, per se.
Heck, here’s some search links: Pharyngula for evo-psych; Pharyngula for evo psych; Pharyngula for evopsych (PZ isn’t always consistent in how he spells the abbreviated phrase). I defy you to find anything that suggest that he denies that there are aspects of the brain that aren’t learned. Indeed, he seems to be saying that that’s a strawman of the criticisms.
I’m pretty sure that that’s not it at all. Rather, the complaint seems to be more that evopsych tries to tie modern behaviors as the ultimate products of behaviors that evolved during the Pleistocene, using just-so stories to make claims of there being specific similarities.
That’s not what I am trying to say at all. Facial recognition is highly dependent on vision! But facial recognition is a function of part of the brain; something that I think can be reasonably called a module. Indeed, all of vision turns out to be highly modular [visual cortex], with parts just recognizing (or “tuned for”, to use the language they use) vertical or horizontal lines; others color; still others, motion.
While I am referencing the Wikipedia pages, I really got the sense of how these things work from the popular works of the late Oliver Sacks: The Man Who Mistook His Wife for a Hat ; An Anthropologist on Mars.
Have you really not read them?
Owlmirror@#16:
I need to review. FWIW I’ve read old Luria and mid-range Sacks(pre-“leg to stand on”) Let me caffeinate and catch up and (no joking) make sure Alzheimers’ isn’t getting me.
“[D]oes that argue that it’s more of a learned behavior or that it’s an innate behavior with very broad parameters for what constitutes a ‘face’?”
My guess: it’s an innate cognitive ability to distinguish among similar things according to sometimes subtle and complex visual cues. It may not be limited to faces at all, though face recognition would be obviously be an important application. For example: if a grocer developed prosopagnosia, would he have trouble distinguishing among different varieties of apple? Or different individual apples? How about a Toyota dealer recognizing different model years of Camry? I don’t know.
I vaguely remember that even very young infants respond differently to face-like images compared to the same elements arranged in a non-facelike way. So there is likely some built-in machinery for recognizing faces as such. However distinguishing one face from another is another matter.
Experience also plays a role. I’m better at distinguishing among the faces of white people than among the faces of black people. That may reflect a degree of racism on my part, but I think it mostly reflects the fact that most of the people in my world are white. In order to distinguish them, I need to pay attention to other traits. If someone is black, that in itself goes a long way toward identifying them, and I can use fewer bits to encode the rest. I suspect that the same is true in reverse where white people are a minority.
Owlmirror@#16:
Facial recognition is certainly used in social activity, but it is a fundamentally neural activity.
OK. I was using terminology wrong – that’s what I meant: there’s some kind of neural (which I assume means: evolved-in) capability, which is used socially. That supports the claim that not all behaviors are completely learned, unless we keep redefining “behavior” to mean “learned” in which case there can be no discussion about what is learned and what is evolved-in because we’ve placed them into inviolably separated domains.
I think we’re in agreement, I simply used words in a way you don’t agree with, but I think I’m talking about the same thing with the same intent. My terminology about this stuff is based on an early 1980s psychology education, from a time right before neuroscientists really started getting traction picking this stuff apart. So I may be abusing language – if I am being confusing, it’s not deliberate and I apologize.
Here’s the page on prosopagnosia, which traces the syndrome to problems in the fusiform gyrus. It’s in the brain.
When I was an undergrad, I believe we called this “anomia” – or, the consequences of it were “anomia” – I don’t recall the term “prosopagnosia” ever cropping up, and I don’t recall that anything had been localized to a particular brain region. That’s super cool – thank you for the link. When the term first cropped up here I looked up a definition but didn’t read deeply enough.
If it’s in the brain and is in a particular place in the brain, would we be able to say it’s an evolved-in function? Clearly there are going to be other behaviors built on top of it (the basic facial recognition is going to, presumably, activate memories and that’s how we get from ‘those face-shapes remind me of Don’s…” to “Oh, it’s Don!”
I defy you to find anything that suggest that he denies that there are aspects of the brain that aren’t learned. Indeed, he seems to be saying that that’s a strawman of the criticisms.
I may be mis-speaking again. I agree with PZ about this topic. Let me try to clarify:
There is a continuum of arguments between “the blank slate” which argues that most or nearly all behaviors are learned, and “human nature” which argues that there are some behaviors in humans that are built in. Between those two positions are a gigantic tangled thicket of “yeah, but…” and my personal view is that it’s “yeah, but” in excruciating detail about everything: there is neither a “blank slate” or “human nature” I think that’s pretty much PZ’s position, too, as I understand him.
If I were to try to summarize my view, it’s that pretty much any behavior is going to be influenced by learning, so it’s going to be very very difficult to point and say “that behavior is innate, due to evolution!” without going to a very low-level behavior, such as the “bottom half” of face recognition (the part that deconstructs faces into patterns).
With regard to strawmen and criticisms, I think I am also agreeing with PZ: there are some people like Steven Pinker (in his book “the blank slate”) who appear to be arguing that there’s a “human nature” of some kind or another, therefore evolution affects our behaviors, but it’s complicated except that feminists get it wrong – OK that last bit may be unfair but Pinker spends a suspicious amount of time trying to poke holes in certain ideologies. Other strawmen that get erected around this issue are the many attempts at scientific racism, or scientific gender supremacy – the argument for those (in this topic) usually breaks along the lines that “certain people have a genetic predisposition for X” If you accept the “tangled thicket” view, then those discussions are not worth having yet because we haven’t isolated genetic components to behaviors sufficiently to make any kind of pronouncements on what’s built in versus learned.
I think that’s all a fair summary. If that last bit is close to true, then much of evolutionary psychology would be putting the cart before the horse, because it’s trying to claim that certain things (which may be behaviors, or may be built in, to some unknown degree) are a consequence of social selection, or that the breakdown in degree can be assessed using the methods of psychological surveys of college undergrads.
Rather, the complaint seems to be more that evopsych tries to tie modern behaviors as the ultimate products of behaviors that evolved during the Pleistocene, using just-so stories to make claims of there being specific similarities.
Yes, that’s what I meant. I didn’t put the long form in my comment because it would have turned into a whole posting (as this has) The “just so” stories are claims that behaviors are the result of evolution, not learning. By “learning” I mean “culture” – learning about the civilization you grow up in.
The whole discussion gets complicated by that continuum I referred to earlier: depending on where you want to argue you are on the continuum the evolutionary claim is stronger or weaker. When I said “at the core of the debate” that’s what I meant: the debate centers around where the argument falls on the “blank slate” versus “human nature” continuum – a continuum I think is likely to be very difficult to tease apart in terms of what had more effect and when and how (and that’s without even thinking about epigenetics)
. Facial recognition is highly dependent on vision! But facial recognition is a function of part of the brain; something that I think can be reasonably called a module. Indeed, all of vision turns out to be highly modular [visual cortex], with parts just recognizing (or “tuned for”, to use the language they use) vertical or horizontal lines; others color; still others, motion.
I agree.
So, we have modules, that are used in ways that are strongly influenced by learning.
The evolutionary psychologist who wanted to argue that Ancient Babylonians probably had special facial recognition capabilities for detecting ringletted beards – that it was not merely a learned behavior – would have to somehow tease apart the functioning of the module, from how we learn to use it, and then argue that functioning a) exists b) would be something we could have evolved during the time period when Babylonian beards were the rage.
Is that a fair summary?
I seem to be not communicating clearly on this topic, and I apologize; it’s not intentional. Damn it, I pride myself on communicating clearly, so when I clearly miss the mark it’s distressing to me.
Have you really not read them?
I may have my Sacks books out of order. I stopped reading after I read A Leg to Stand On, but I have read Awakenings, A Leg to Stand On, Musicophilia and The Man Who Mistook His Wife For a Hat. Fascinating stuff. I’ve also read Pinker, Chomsky, Luria and Ramachandran and some long-forgotten textbooks in the 80s and probably other stuff that’s just blended into that garbage-pit I call “my memory.”
mck9@#18:
My guess: it’s an innate cognitive ability to distinguish among similar things according to sometimes subtle and complex visual cues. It may not be limited to faces at all, though face recognition would be obviously be an important application. For example: if a grocer developed prosopagnosia, would he have trouble distinguishing among different varieties of apple? Or different individual apples? How about a Toyota dealer recognizing different model years of Camry? I don’t know.
Well, that’s basically what they are doing with neural networks: you do a variety of contrast transforms, edge detections, line detection, and see what pops out, then send that to a matching engine.
I have read somewhere that frogs have “fly detector” neurons. Are they in the fusiform gyrus? There is some stuff here: [wikipedia] that is pretty interesting. (it probably explains why cats go ape shit at lasers)
I vaguely remember that even very young infants respond differently to face-like images compared to the same elements arranged in a non-facelike way. So there is likely some built-in machinery for recognizing faces as such. However distinguishing one face from another is another matter.
Was that Piaget? I’ve run across a few things like that, including some stuff I’d consider pretty sketchy. (I refer to Harlow’s monkey abuse experiments) (PS: !*#$& psychologists!)
I’m better at distinguishing among the faces of white people than among the faces of black people. That may reflect a degree of racism on my part, but I think it mostly reflects the fact that most of the people in my world are white. In order to distinguish them, I need to pay attention to other traits.
It may also be the contrast range of our optical systems. In the case of the automated computer face-recognition systems, I think it’s mostly that (they probably should be looking at ultraviolet and infrared spectrum, too!) Melanin doesn’t reflect ultraviolet, it absorbs it. Anyhow, if we accept that a great deal of face recognition involves memory, then you’d almost certainly perform better on what you’d trained against – just like any neural network.
Marcus Ranum @ 20:
Frogs don’t have a fusiform gyrus because they don’t have a neocortex at all. Their fly detectors are in the retina, and the task is computationally fairly simple: detecting small dark spots moving across the visual field. Higher-order visual processing happens mostly in a part of the midbrain called the optic tectum. Mammals also have an optic tectum, and it’s also involved in visual processing, except that we call it the superior colliculus. The good stuff, though, happens in the neocortex, starting with Brodmann’s area 17 at the back of the head.
It is said that non-human primates don’t have a fusiform gyrus. However I suspect that that observation is based on morphology; monkeys probably have corresponding brain bits but they aren’t folded the same way.
Facial recognition is a much more abstract task than fly detection. We have to construct a three-dimensional representation of the face and do pattern matching on the result. However we’re not as good as frogs at catching flies, with the occasional exception like Willie Mays.
I don’t know who did the scrambled face experiments. I don’t think it’s the sort of thing that Piaget did, but I’ve been surprised before.
Another thing I don’t know is whether the scrambled face experiments controlled for symmetry. The babies might have been responding to the symmetry of a face-like arrangement versus the messiness of having the mouth off to one side, the nose in a corner, and two dots here and there.
I think I don’t really have more to add here, besides mentioning that there is some data that other animals can recognize distinct human faces (and as a probable implication, distinguish between distinct members of their own species, at least). Let me see…
I know that crows have been shown to have long-term facial memory, and I see that magpies and pigeons have some of that ability as well.
I recall seeing anecdotes of other animals being able to recognize specific humans, possibly by face alone. I think elephants are an obvious example, and also octopodes (one octopus would always squirt water at an individual it took a dislike to), and probably other animals as well. But I can’t find specific examples in the literature of tests of such facial recognition abilities.
Finally, I also recall seeing that some wasps also recognize nestmates by facial differences. Dunno how they would do with recognizing humans, but there it is.
I think my point here is that the ability to recognize faces has probably evolved multiple times in different lineages.
Oh, here’s an obvious one : facial recognition of humans (and dogs) in dogs.
And speaking of sheep (#11/#15) above:
Sheep don’t forget a face and Prosopagnosia: A face-specific disorder, which has an abstrct stating:
(emph mine)(don’t know what to make of that!)
(sorry, no obvious full text access)
You have very likely found the case that I had remembered, or rather mis-remembered. I don’t know what to make of it either.
A few more studies, on primates:
• Orangutans (Pongo pygmaeus) Remember Old Acquaintances
• Face recognition in primates: a cross-species study (cross-species meaning humans and macaques )
• Human and Chimpanzee Face Recognition in Chimpanzees