Muddling the Dunning-Kruger Effect

The Dunning-Kruger Effect states that people with the lowest competence tend to overrate their competence, but people with the highest competence tend to underrate themselves. This was shown in 1999 paper by Dunning and Kruger.1 Here’s one of the figures from the paper:

A graph showing people's self-assessed ability, and actual test score. The bottom quartile gives themselves a rating in the 60 percentile, and the top quartile gives themselves a rating in the 75 percentile.

This figure shows results from a test on humor. People are scored based on how well their answers agree with those of professional comedians, and then they are asked to assess their own performance. There were similar results for tests on grammar and logic.

The Dunning-Kruger effect has entered popular wisdom, and is frequently brought up whenever people feel like they’re dealing with someone too stupid to know how stupid they are.  But does the research actually mean what people think it means?

Before reading into this subject, I must admit that I had a major misconception.  I thought that people’s self-assessment was actually anti-correlated with their competence.  I thought someone who knew nothing would actually be more confident than someone who knew a lot.  (This leads to an amusing dilemma: Should I choose to give myself a lower rating, because it would that increase posterior probability that I’m more competent?2)

But it is not true.  People who know nothing are less confident than people who know a lot.  People who know nothing are overconfident relative to their actual ability, but they are still not as confident as people who have high ability.

[Read more…]

Attraction and emotional granularity

This article was written for the Carnival of Aces themed on “Nuance & Complexity“. It is being cross-posted to my other blog, The Asexual Agenda.

Asexuality is chiefly about noticing a distinction between the emotions you perceive in other people, and the emotions you perceive in yourself. We give a name to this distinction, for example by saying some people experience sexual attraction and some people do not. And we discuss appropriate responses to our emotions, for example by saying that some emotions mean we want to have sex, and other emotions do not.

Within ace communities, we often discuss further distinctions in emotions. Again, we give names to these distinctions, for example by talking about romantic attraction, platonic attraction, aesthetic attraction, sensual attraction, and so forth. And we discuss appropriate responses to these emotions, for example by describing what kinds of relationships might satisfy our emotions, or if a particular emotion only makes us want to look at a person.

The ability to distinguish different emotions is a nascent research topic in psychology. And while you shouldn’t let psychology research dictate how you live, looking into the research may give us insight into a common topic.

[Read more…]

Public ignorance/awareness of science

Have you ever read one of those news stories reporting that Americans are shockingly ignorant of science? For example, here’s an article saying that a quarter of Americans think the Sun orbits the Earth.

And if you found that shocking, brace yourself for the next one. According to this article, a quarter of Americans think the Sun orbits the Earth!

Okay, so both of the news articles are saying the same thing. But one of the articles is from 2014, and the other is from 1988. They’re both reporting on an NSF study, which has been repeated every couple years for three decades. They always ask whether the Sun orbits the Earth or the Earth orbits the Sun, and they consistently find that about a quarter of USians don’t know or get it wrong. They also ask if electrons are smaller than atoms, if lasers focus sound waves, and if antibiotics kill viruses.* News sources like to put the Sun/Earth statistic in their headlines, because it sounds the most shocking to readers.

You might guess from my tone that I’m a bit more apathetic about the whole thing. Yeah, it’s bad that USians are ignorant of elementary astronomy. But science is not a collection of factoids, and factoids are not the most important component of scientific literacy. As far as facts go, there are way too many for anyone to know all of them, and it’s difficult to judge which facts are more or less important for people to know.  If a fact is “basic” and “obvious”, that might make it socially unacceptable to be ignorant of it, but it also might make it less important to know, given how easy it is to look up the answer.  In my opinion, it’s far more important for people to understand scientific reasoning, like how experiments are designed, and how to read graphs.

[Read more…]

The ethics of educational experiments

Rebecca Watson had an interesting article/video about the ethics of A/B testing. A/B testing is a type of experiment often performed by tech companies on their users. The companies split users into two groups, and show two different versions of their software/website to each group, and measure the results. The problem is that when scientists perform experiments on human subjects, there’s a formal ethical review process. Should tech companies have an ethical review process too?

Of course, this question is being raised as a result of a specific experiment performed by a specific company. Pearson produces educational software, and performed an A/B test where some students were shown motivational messages. They presented results at a conference, and part of their conclusion was that this was a promising methodology for future research. But is it really, if they didn’t comply with the ethical standards in science? They certainly didn’t get consent from all those human test subjects.

Watson also brought up another case from 2014, when Facebook performed an experiment that changed the amount of positive/negative posts people saw in their news feeds. They published a study, and it was called “Experimental evidence of massive-scale emotional contagion through social networks”. Sounds pretty bad, eh?

Watson seems to conclude that A/B tests should get consent, at least in the case of Pearson. But I think this is going too far. The thing is, A/B testing is absolutely ubiquitous. Watson says, “having worked in marketing and seen A/B tests, it’s just a normal thing that companies do,” but I think this understates it. My fiance and I were trying to figure out how many A/B tests Google has running at any time, and we thought it might be one per employee, implying tens of thousands of experiments. And most of them are for boring things like changing fonts or increasing the number of pixels of white space. If we judge A/B tests on the basis of just two tests that appear in the news, “cherry picking” doesn’t even begin to describe it.

[Read more…]

The construction of emotions

I recently read How Emotions are Made, by Lisa Feldman Barrett. It explains the theory of constructed emotion and its implications. This is the best book of nonfiction I have ever read. Repeatedly throughout the book, I had to put it down because I was so blown away that I needed a moment to think through the implications. And since I’m a blogger, my thoughts would often drift towards how I might write about these ideas and share them. This post will be a bit of an introduction explaining the basic concepts as I understand them, and I hope to write more in the future.

Regular readers know that I believe in nominalism–I think there is a meaningful sense in which everything is socially constructed. I understand that a lot of readers disagree with this, and we may never persuade one another. But fortunately this is irrelevant. When we speak of the theory of constructed emotions, it isn’t a broad philosophical claim, it’s an empirical claim that is specific to human emotions.

When psychologists study emotions, they can record a number of objective measurements, such as facial configurations, positive/negative valence, high/low arousal, and activity in different parts of the brain. However, these objective measurements do not match up to emotional categories. A single emotional category could correspond to many different facial configurations, while a single facial configuration could correspond to any number of different emotions. Yes, there are many qualitatively distinct feelings we can feel. However, when we give a name to those feelings, and place those feelings in an emotional category, this categorization process is not purely based on the feelings themselves. It’s based on the emotional concepts that are available to us, it’s based on the context in which we have those feelings, and it’s based on what we think the purpose of those feelings are.

[Read more…]

What is an apology?

This is a repost of an article I wrote in 2014.

There are countless cases in the news where a public figure does something wrong, and we all collectively ask, “Why don’t they just apologize?” or “Why don’t they apologize the right way?”  In the mean time I’ve often thought, “Why does anyone apologize ever?  What is an apology aside from a collection of emotions with no rational analogue?”

An apology is a sort of script.  Alice wrongs Bob.  Bob demands an apology from Alice.  Alice apologizes.  Bob forgives Alice.

OR

Alice refuses to apologize.  Bob is angered and seeks other means to punish Alice.  He could deny her trust, deny her social status, or even punish through legal means.

But what’s in it for Alice?  What’s in it for Bob?  As far as Alice is concerned, the outcome of apologizing is clearly better than that of refusing to apologize.  As far as Bob is concerned, punishment may provide either a psychological or game-theoretic value–why should any of that change just because Alice arranges some words in a particular way?

[Read more…]

The Second Law and its misuses

An OrbitCon session brought to my attention to the fact that Steven Pinker spouts a lot of bullshit about the Second Law of Thermodynamics. (The Second Law is, “entropy cannot decrease over time in a closed system.”) In Pinker’s book, Enlightenment Now, he begins by refuting the creationist argument that the Second Law contradicts the theory of evolution. This is easy to do, you just say that the earth isn’t a closed system, and dramatically point at the sun. But Pinker then proceeds to forget about the sun, and argues that the Second Law of Thermodynamics explains poverty. I don’t have the book available, but Pinker has written an essay along similar lines:

Poverty, too, needs no explanation. In a world governed by entropy and evolution, it is the default state of humankind. Matter does not just arrange itself into shelter or clothing, and living things do everything they can not to become our food. What needs to be explained is wealth.

Here’s the thing: creationists are really really wrong about the Second Law. There’s plenty of room to be less wrong than creationists, but still really really wrong. For those of us who have taken an interest in fighting creationism, we know we can just point at the sun and be done with it. But just because you’re familiar with this argument, please don’t mistake that for an understanding of the Second Law. Don’t be like Steven Pinker.

Here I will state and explain a few basic principles about entropy, with the goal of going beyond a mere refutation of creationist arguments.

[Read more…]