Ugh, Not Again

P-values are back in the news. Nature published an article, signed by 800 scientists, calling for an end to the concept of “statistical significance.” It ruffled my feathers, even though I agreed with its central thesis.

The trouble is human and cognitive more than it is statistical: bucketing results into ‘statistically significant’ and ‘statistically non-significant’ makes people think that the items assigned in that way are categorically different. The same problems are likely to arise under any proposed statistical alternative that involves dichotomization, whether frequentist, Bayesian or otherwise.

Unfortunately, the false belief that crossing the threshold of statistical significance is enough to show that a result is ‘real’ has led scientists and journal editors to privilege such results, thereby distorting the literature. Statistically significant estimates are biased upwards in magnitude and potentially to a large degree, whereas statistically non-significant estimates are biased downwards in magnitude. Consequently, any discussion that focuses on estimates chosen for their significance will be biased. On top of this, the rigid focus on statistical significance encourages researchers to choose data and methods that yield statistical significance for some desired (or simply publishable) result, or that yield statistical non-significance for an undesired result, such as potential side effects of drugs — thereby invalidating conclusions.

Nothing wrong there. While I’ve mentioned some Bayesian buckets, I tucked away a one-sentence counter-argument in an aside over here. Any artificial significant/non-significant boundary is going to promote the distortions they mention here. What got me writing this post was their recommendations.

What will retiring statistical significance look like? We hope that methods sections and data tabulation will be more detailed and nuanced. Authors will emphasize their estimates and the uncertainty in them — for example, by explicitly discussing the lower and upper limits of their intervals. They will not rely on significance tests. When P values are reported, they will be given with sensible precision (for example, P = 0.021 or P = 0.13) — without adornments such as stars or letters to denote statistical significance and not as binary inequalities (P  < 0.05 or P > 0.05). Decisions to interpret or to publish results will not be based on statistical thresholds. People will spend less time with statistical software, and more time thinking.

This basically amounts to nothing. Journal editors still have to decide what to print, and if there is no strong alternative they’ll switch from an arbitrary cutoff of p < 0.05 to an ad-hoc arbitrary cutoff. In the meantime, they’re leaving flawed statistical procedures in place. P-values exaggerate the strength of the evidence, as I and others have argued. Confidence intervals are not an improvement, either. As I put it:

For one thing, if you’re a frequentist it’s a category error to state the odds of a hypothesis being true, or that some data makes a hypothesis more likely, or even that you’re testing the truth-hood of a hypothesis. […]

How does this intersect with confidence intervals? If it’s an invalid move to hypothesise[sic] “the population mean is Y,” it must also be invalid to say “there’s a 95% chance the population mean is between X and Z.” That’s attaching a probability to a hypothesis, and therefore a no-no! Instead, what a frequentist confidence interval is really telling you is “assuming this data is a representative sample, if I repeat my experimental procedure an infinite number of times then I’ll calculate a sample mean between X and Z 95% of the time.” A confidence interval says nothing about the test statistic, at least not directly.

In frequentism, the parameter is fixed and the data varies. It doesn’t make sense to consider other parameters, that’s a Bayesian move. And yet the authors propose exactly that!

We must learn to embrace uncertainty. One practical way to do so is to rename confidence intervals as ‘compatibility intervals’ and interpret them in a way that avoids overconfidence. Specifically, we recommend that authors describe the practical implications of all values inside the interval, especially the observed effect (or point estimate) and the limits. In doing so, they should remember that all the values between the interval’s limits are reasonably compatible with the data, given the statistical assumptions used to compute the interval. Therefore, singling out one particular value (such as the null value) in the interval as ‘shown’ makes no sense.

Much of what the authors proposed would be fixed by switching to Bayesian statistics. Their own suggestions invoke Bayesian ideas without realizing it. Yet they go out of their way to say nothing’s wrong with p-values or confidence intervals, despite evidence to the contrary. Their proposal is destined to fail, yet it got more support than the arguably-superior p < 0.005 proposal.

Maddening. Maybe it’s time I got out my poison pen and added my two cents to the scientific record.

Happy Emmy Noether Day!

Whenever anyone asks me for my favorite scientist, her name comes first.

At a time when women were considered intellectually inferior to men, Noether (pronounced NUR-ter) won the admiration of her male colleagues. She resolved a nagging puzzle in Albert Einstein’s newfound theory of gravity, the general theory of relativity. And in the process, she proved a revolutionary mathematical theorem that changed the way physicists study the universe.

It’s been a century since the July 23, 1918, unveiling of Noether’s famous theorem. Yet its importance persists today. “That theorem has been a guiding star to 20th and 21st century physics,” says theoretical physicist Frank Wilczek of MIT. […]

Although most people have never heard of Noether, physicists sing her theorem’s praises. The theorem is “pervasive in everything we do,” says theoretical physicist Ruth Gregory of Durham University in England. Gregory, who has lectured on the importance of Noether’s work, studies gravity, a field in which Noether’s legacy looms large.

And as luck would have it, today was the day she was born. So read up on why she’s such a critical figure, and use it as an excuse to remember other important women in science.

Rapid Onset Gender Dysphoria

Remember that old thing? No? OK, quick summary:

Parental reports (on social media) of friend clusters exhibiting signs of gender dysphoria and increased exposure to social media/internet preceding a child’s announcement of a transgender identity raise the possibility of social and peer influences.

Littman L (2018) Parent reports of adolescents and young adults perceived to show signs of a rapid onset of gender dysphoria. PLoS ONE 13(8): e0202330.

In short, maybe social media is making the kids transgender? This seems like something someone should study, and someone did!

Poorly. [Read more…]

Some Much-Needed Follow-up

You can almost watch my opinion flip in real time.

September 25, 2014 at 11:57 am

If Benson made a habit of linking to TERF materials, even though she knew where they came from and had plenty of alternatives, I wouldn’t be so quick to defend her. But this is a single cartoon that is only problematic because of its source, and even then you had to either know TERF lingo or read carefully to discover the source was problematic. It should be entirely forgivable, at minimum, especially if Benson made it clear she didn’t endorse trans exclusion once she knew of the source. Which she did.

That some people aren’t willing to forgive this no matter what Benson does outs them as demanding perfection from imperfect beings. Only the most fanatic religious fundamentalists agree to that.

=====

September 25, 2014 at 9:38 pm

Not only does coded language allow you to get away with saying racist/sexist/classist things, you might trick non-racist/sexist/classist people into supporting you. I myself was thinking of sharing an image elsewhere, until I saw octopod raise the TERF flag and went “hmmm, I might be missing something here.” On and off over several hours, I scratched my head trying to work out what that was. “I suppose that one comic ‘reinderdijkhuis’ linked to made it explicit, but I didn’t spot anything else as bad. Though, now that I think of it, that rainbow comic looked like a coded message. And it was weird the masthead used the word ‘cotton’ but I’m HOLY SHIT HOW COULD I BE THAT BLIND….” […]

In that moment, a page I originally thought contained a mix of funny but heavily obscure comics was revealed to be a vicious cacophony of sexist dog whistles. EVERY comic was dripping with hate, but in some of them it was so carefully hidden that it looked like feminist commentary. Those could easily float around Facebook, with only a select few snickering over the true message being passed around. Imagine sharing an image that mocked Obama for being a warmonger, following the link to the source, and stumbling across a white supremacist website. If you were black, that would be horrific.

Hopefully that should explain why the image had to go, and why I was wrong to edge towards the “devil’s advocate” chair. My apologies for taking so long to clue in.

I was a latecomer to Ophelia Benson’s transphobia, other people had been aware of it for at least a year before my flip began. The whisper network had started talking, and I decided to listen. I owe a debt of gratitude to the people who helped me move from clueless to slightly-less-so, people like abbeycadabra, Janine, Xanthë, and Jason Thibeault. That also means I should take critiques from them seriously, as my understanding isn’t as far along as theirs.

Frankly, for a trans person, there’s something surreal and erasing in seeing cis people feuding with cis people over whether we exist. I mean, I am grateful that there are cis people being allies for us and pushing back against the transphobes (and homophobes and every other kind of -phobe.) But the fact that people have to come up with logical arguments and “evidence” that our transness is “real,” thus keeping the question alive of whether we do, in fact, exist, keeps giving me the creepy feeling that maybe I’m just a figment of my own imagination. I think the technical term is “depersonalization.”

It’s like when people run around “proving” that 1 = 0 — nobody sees any real need to “disprove” it, because it’s obvious that such a proof is BS. (It’s a reductio ad absurdum on the face of it.) But it seems like even those who believe in our existence feel the need to prove it. I was just reading HJ Hornbeck’s post about trans athletes, which has all kinds of “scientific,” “objective” evidence that gender dysphoria, gender identity, etc. are real. The problem with going down that path is not only that it concedes the possibility that it could be “disproven,” but also that trans people who don’t fit into the definitions and criteria in those “proofs” are then implicitly left out of the category “real trans.”

I was originally going to type up something in response, but after re-reading this comment that instinct feels mistaken. I agree with all of it, anything I add would just be restating something they said, and that would promote the idea that trans people’s opinions only have weight if cis people agree with them. So I’ll give Allison the final word.

This is BTW why I don’t like the idea of medical tests for transness, or proofs that trans people’s brains are observably different from cis people’s. Ultimately, being trans lies in one’s own understanding of oneself, gained through hard and painful experience. If I know based on my own experience of myself that “trans” best describes me, and some brain scan “proves” that I’m not, which am I to believe? (“Who are you gonna believe? Me? Or your own eyes?”) I spent most of my life ignoring my experience of myself and trying to live the way society told me I should, and it damned near killed me, and I think most trans people (at least we older trans people) have had the same experience.

Not Well-Meaning

I’ve dealt with Darryl Bem’s work a few times, and my general impression was that he was a well-meaning kook: yes, he’s believed PSI was real for decades, but I got the impression that he was willing to listen to his critics and incorporate their feedback.

[Dr. Kenneth] Zucker referred to another article in Clinical Practice in Pediatric Psychology that he claimed implies that “the first line of treatment should be a gender social transition.” Dr. Diane Chen, one of the authors of that paper, told Rewire.News that was incorrect. “I would not agree with that,” wrote Chen. “As you’ll see from the ‘ongoing controversies’ section for pre-pubertal youth, we discuss the relative harm of encouraging social transition.” The paper recommends instead that parents of children considering or undergoing social transition keep their statements to their children open-ended with respect to their eventual adolescence and adulthood.

I don’t get that impression with Dr. Zucker. Siobhan managed to contact the man after his name came up in the news again, and he still seems to be spreading misinformation and common myths.

A common myth about the type of service described by the AAP, and the service Dr. Janssen offers, is that youth are rushed into referrals for hard-to-reverse or irreversible transition procedures. “If someone comes in and their child is saying, ‘I’ve been thinking about this for the last two weeks,’ it’s not like anybody’s going to make a recommendation that child goes on a some sort of irreversible intervention,” Janssen told us. “It’s more like: Let’s understand this and let’s see how this develops over time.” Any biomedical intervention for an adolescent would only be recommended after they meet the criteria for gender dysphoria in the current version of the Diagnostic and Statistical Manual of Mental Disorders (DSM), which requires a strong desire to be another sex that persists for at least six months.

Here he is, in print, repeating TERF misinformation about gender dysphoria treatment under the guise of it being a political disagreement. But while I’m willing to give Dr. Harriet Hall some benefit of the doubt on the topic of gender dysphoria, as she demonstrates no expertise, Dr. Zucker quite literally wrote the definition. He cannot invoke ignorance as a defense. Worse, he may not have changed his approach to gender dysphoria treatment, despite the evidence suggesting he should.

The [Centre for Addiction and Mental Health]’s report stopped shy of characterizing Dr. Kenneth Zucker’s practice as conversion therapy, but it did conclude his methods were “out of step” with the latest research findings and that they warranted sweeping reforms. Zucker’s clinic, which was housed inside CAMH but operated largely independently, closed later that year;  […]

Zucker confirmed with Rewire.News that he still offers services similar to his CAMH clinic at his private practice.

If Darryl Bem is a well-meaning kook, Dr. Zucker is a dangerous one. He appears immune to outside criticism, yet comes across as an authority to a lay person. Siobhan’s article lays this out quite nicely; despite being a news report, she has no problem poking giant holes in his assertions. I recommend giving it a read.

When The Joke Is On You

I had no idea.

We have Charles’ five assertions. We now conduct an empirical investigation, examining all the individuals in the universe. We might suppose that Charles intends the word “Caesar” to signify or designate Prasutagus (who, as every schoolboy knows, is the husband of Boadicea). On this supposition (5) could be called true and all the rest would have to be called false. Or we might suppose that “Caesar” signifies the historical Julius Caesar, in which case (l)-(4) could be called true and (5) would have to be called false. There do not seem to be any other candidates since any number of persons must have conquered Gaul and/or crossed the Rubicon and /or used the ablative absolute to excess. And so we act on what might be called the Principle of Charity. We select as designatum that individual which will make the largest possible number of Charles’ statements true.

Wilson, N. L. “Substances without Substrata.” The Review of Metaphysics 12, no. 4 (1959): 521–39.

Apparently, the “Principle of Charity” was never named until the second half of the 20th century! My philosophy classes made it obvious that the concept existed well before then, yet apparently no philosopher had valued it enough attach a name. For those in the dark, the “Principle of Charity” is that when critiquing an argument, you should consider the most rational variation of it. You might know this better as “steel-personing.”

Most opposition to abortion relies on the premise that the fetus is a human being, a person, from the moment of conception. The premise is argued for, but, as I think, not well. Take, for example, the most common argument. We are asked to notice that the development of a human being from conception through birth into childhood is continuous; then it is said that to draw a line, to choose a point in this development and say “before this point the thing is not a person, after this point it is a person” is to make an arbitrary choice, a choice for which in the nature of things no good reason can be given. It is concluded that the fetus is. or anyway that we had better say it is, a person from the moment of conception. But this conclusion does not follow. Similar things might be said about the development of an acorn into an oak trees, and it does not follow that acorns are oak trees, or that we had better say they are.

Thomson, Judith Jarvis. “A defense of abortion.” Biomedical ethics and the law. Springer, Boston, MA, 1976. 39-54.

The Principle creates a distinct pattern: describe your opponent’s view as strongly as possible, then poke holes in it. Thomson does the entire arc in her opening paragraph, and quite a few afterward, but her entire defense of abortion is one long version of this. She makes it clear that she doesn’t think a fetus should immediately be granted full personhood, and all the human rights associated with that, but nonetheless grants it full rights. Thomson proceeds to defend abortion anyway, on the grounds that we value personal property more highly than the right to life. I definitely recommend reading her paper, as (if successful) it renders the primary argument of anti-choicers irrelevant.

This article will argue that humor, in particular irony and satire, when used in the service of criticizing oppressive power structures and especially by members of marginalized groups, is a potentially powerful tool for increasing receptivity and recognition of other ways of knowing and experiencing society. […] However, when these same ironic, satirical, double-voiced tools of humor are used by members of dominant groups to disparage, mock, or discredit marginalized groups or social justice scholarship that seeks to make oppression visible, they serve no such purpose but rather perpetuate dominant epistemologies and power structures.

Baldwin, Richard. “When the Joke Is on You: A Feminist Perspective on How Positionality Influences Satire (RETRACTED).” Hypatia. pg. 2

Which brings us to another “hoax” paper of PB&J. There’s two main points on offer here, and both of them are quite plausible. [Read more…]

Anthropologists on Race

When’s the last time you held a scientific journal? Probably never, I bet. In the age of digital publishing, distinct “volumes” are mostly a nod to tradition instead of something curated, during those rare times where you can access them at all.

This virtual issue, organized by contributing editor Channah Leff and managing editor Sean Mallin, brings together articles published in American Anthropologist around race and biology, focusing on genetics as one way to understand–and counter misunderstandings about–human difference. From early work on immigration and evolution to more recent work on epigenetics, anthropologists have been at the forefront of conversations about what race is–and what it isn’t.

Which makes this virtual edition of American Anthropologist quite a treat. It isn’t often you get to hear scientists break down the concept of race, and rarer still to realize how long they’ve been questioning it for.

With what we know now, two conclusions are quite inescapable. First, human races – like higher taxonomic units – are subject to evolutionary change. Second, the particular traits by which races distinguish themselves are subject to natural selection, and therefore do not have eternal taxonomic value. I n retrospect, all of the characters used in constructing a classification of man must have been grist in the evolutionary mill.

Now we cannot have change and no change simultaneously. Present frequencies of blood groups or of morphological traits are, at best, interim reports of present conditions. They need not be identical to frequencies in the recent or remote past, and they need not predict gene or trait frequencies in the future. […] As a consequence, the search for ancestors becomes far more difficult than it once seemed. … As soon as we accept changes in gene frequencies, we can no longer employ present frequencies as certain indications of past events.

While this obvious corollary admittedly pulls the rug from beneath our more cherished reconstructions, evidence for changing race may free us from the burden of prefabricated and hypothetical ancestors.

Garn, Stanley M. “Race and Evolution.” American Anthropologist 59, no. 2 (April 1, 1957): 218–24. https://doi.org/10.1525/aa.1957.59.2.02a00030.

Nor is that even the oldest paper in the collection; one from 1912 found large physical changes in the children of immigrants which brought them in line with the locals, demonstrating a plasticity that contradicts to the rigidity demanded by biological race. If you’d rather have a broad overview of the subject,

We present a review of the history of scientific inquiry into modern human origins, focusing on the role of the American Anthropologist. We begin during the mid–20th century, at the time when the problem of modern human origins was first presented in the American Anthropologist and could first be distinguished from more general questions about human and hominid origins. Next, we discuss the effects of the modern evolutionary synthesis on biological anthropology and paleoanthropology in particular, and its role in the origin of anthropological genetics. The rise of human genetics is discussed along two tracks, which have taken starkly different approaches to the historical interpretation of recent human diversity.

Hawks, John, and Milford H. Wolpoff. “Sixty Years of Modern Human Origins in the American Anthropological Association.” American Anthropologist 105, no. 1 (March 1, 2003): 89–100. https://doi.org/10.1525/aa.2003.105.1.89.

For those who are about to wail about them taking a constructivist approach which denies genetics, you’re in for a bit of a shock.

Indeed, multiple studies in 2017 have dramatically expanded our knowledge of genomic variation involving hundreds of ancient and present‐day peoples from across the globe (Marciniak and Perry 2017; Nielsen et al. 2017). Maybe not surprisingly, the results of these studies have empirically confirmed that our understanding of human genetic variation was incomplete, flawed, and biased (Martin, Gignoux, et al. 2017). More relevant to this review, these studies, in addition to the massive amount of data that they produced, have also added dozens of new twists to how we perceive human variation. […]

We have known for some time that contemporary genetic variation is best explained by “geography.” In other words, the closer two humans are geographically, the less their genetic variation to each other is expected to be (Novembre et al. 2008)—mostly independent of ethnicity, religion, or any other group identities. Now our field is at a stage to move beyond simple geographic distance and take the topographic features (e.g., mountains, deserts, seas, etc.) into account to visualize and understand the paths and barriers to contemporary genetic variation (Peter, Petkova, and Novembre 2017). Ancient genomics has now added a chronological twist to it. It turns out that genetic continuity in a given region across time is often an exception rather than the rule (Kılınç et al. 2016; Lazaridis et al. 2016; Skoglund et al. 2017; but see Yang et al. 2017). People move, interact with their neighbors, and create ever‐changing gradients of genetic variation across time and geography.

Gokcumen, Omer. “The Year In Genetic Anthropology: New Lands, New Technologies, New Questions.” American Anthropologist 120, no. 2 (June 1, 2018): 266–77. https://doi.org/10.1111/aman.13032.

A number of anthropologists embrace genetic testing, and find that it also discounts lay views of race. For instance, genetic testing has found there are distinct lineages, which is what biological race would predict, but was able to trace some of them back to admixture from Neanderthals and Denisovians. In other words, the racial categories we’ve settled on today don’t map to the lineages we find in our genes.

There’s even some general-purpose awesomeness in this treasure trove.

The value ladenness of this science allows us to identify an important popular fallacy—that a primary axis of modern society is science versus nonscience. Yet no one is really “anti‐science”; such a person is a product of scientistic paranoia. We all make decisions about what science to accept, what science to reject, and what science to ignore. … After all, biological anthropology is obliged to navigate between the creationists, on the one hand, who don’t take evolution seriously enough, and enthusiasts of fads like eugenics in the 1920s or “The Paleo Diet” today, on the other hand, who take evolution too seriously. So, who is worse: the citizen who rejects evolution or the citizen who uses evolution to rationalize a program of genocide? Both are out there and are actively constructing, imposing, and utilizing different meanings on the science; whether or not either of them accepts the descent with modification of species—and is thus “pro‐science”—may be a trivial question.

Marks, Jonathan. “Commentary: Toward an Anthropology of Genetics.” American Anthropologist 116, no. 4 (December 1, 2014): 749–51. https://doi.org/10.1111/aman.12153.

As if you didn’t need enough reason to dig in, this is a limited-time offer: these papers will slip back behind the paywall at the end of 2018. So go on, feast your eyes and feed your brain.