The “Summary” That Wasn’t

Remember that letter from eight days ago? Emphasis mine:

On Friday, the Special Counsel submitted to me a “confidential report explaining the prosecution or declination decisions” he has reached, as required by 28 C.F.R. $ 600.8(c). This report is entitled “Report on the Investigation into Russian Interference in the 2016 Presidential Election.” Although my review is ongoing, I believe that it is in the public interest to describe the report and to summarize the principal conclusions reached by the Special Counsel and the results of his investigation.

This prompted a lot of discussion of Barr’s memo; follow that New York Times link, and the headline declares it a summary. CNN called it a summary too, as did the Washington Post, Vox, The Atlantic, Business Insider, the CBC, and so on. Three days ago, or five days after he released his first memo, Barr released a second.

Also, I am aware of some media reports and other public statements mischaracterizing my March 24, 2019 supplemental notification as a “summary” of the Special Counsel’s investigation and report. For example, Chairman Nadler’s March 25 letter refers to my supplemental notification as a “four-page summary of the Special Counsel’s review.” My March 24 letter was not, and did not purport to be, an exhaustive recounting of the Special Counsel’s investigation or report. … I do not believe it would be in the public’s interest for me to attempt to summarize the full report or to release it in serial or piecemeal fashion.

Wait, so that original memo wasn’t a summary? Then what was it?

… an exhaustive recounting of the Special Counsel’s investigation or report. As my letter made clear, my notification to Congress and the public provided, pending release of the report, a summary of its “principal conclusions”—that is, its bottom line.

Any reasonable person would treat “summary” and “summarize its principle conclusions” as synonymous, and conclude Barr was releasing a summary. Barr is trying to pull a Bill Clinton and push a specific interpretation of specific words that’s at odds with their general understanding. As a lawyer, he almost certainly chose those words deliberately and with that intent.

That’s barely the start of what was wrong with Barr’s original memo.

Former federal prosecutor Renato Mariotti observes that “he is likely pushing back because calling it a ‘summary’ suggests that the letter accurately summarizes the entire report, and it does not do so.” Moreover, by hiding even the length of the report in the first letter, Barr helped President Trump perpetuate the assertion that Mueller hadn’t found much of anything. If it took almost 400 pages to lay out his findings, we can bet there’s plenty of interest to the American people.

Other Justice Department veterans agree that Barr is playing defense. “I think he’s clearly a bit stung by the criticism he’s gotten this week, and this letter was his attempt to look like he is committed to transparency without actually making any new commitments,” says former Justice Department spokesman Matt Miller.

Note the timing as well: Barr’s first memo was released two days after he announced he had the SCO report, when the media was desperate for any scraps and would eagerly blast them to the public. His second memo was released on a Friday night, when the media was less likely to notice and report on it, and long after everyone had already called the first memo a summary.

Then there’s the issue of redactions: Barr identified two types of information he’d like to redact in his first memo, info related to ongoing investigations and “matters occurring before a grand jury.” In the second memo two more categories pop up, “material … potentially compromising sensitive sources and methods” plus “information that would unduly infringe on the personal privacy and reputational interests of peripheral third parties.”

All but the first of these four categories are problematic. […]

“This is not how things are meant to happen,” said Professor Neil Katyal, Professor of National Security Law at Georgetown University Law Center and former Acting Solicitor General of the United States, who drafted the special counsel regulations, on MSNBC on March 29.

In connection with Barr’s unwillingness to release the unredacted report to Congress, Katyal said: “The fact that he won’t do that is really suspicious and tells me that there is information in the Mueller Report that Barr doesn’t want to come out. I don’t think it’s for up-and-up reasons. I think it’s because it’s embarrassing to the president.”

As that Forbes editorial points out, this wasn’t a problem with past Special Council reports. Ken Starr finished his report on a Wednesday, and Congress was given a full, unredacted version of it the same day. It too contained grand jury material, but Starr merely had to consult with a judge to get that released to Congress. The public themselves got restricted access two days later via the internet. The turnaround was so rapid because, as Special Council, Starr knew his report had to be delivered to Congress and the public. He’d done the hard work of working out the redactions while drafting the report, so the publication would proceed as rapidly as possible.

Yet Barr is implying Mueller had no idea he’d be submitting his report to Congress or the public, and offloaded that work to Barr. Tack on the fact that Barr’s job application included an unsolicited memo which claimed the President was immune from prosecution, and his past work was stopping the Iran-Contra investigation by pardoning the key players, and this stinks of a deliberate cover-up. No wonder the House Judiciary committee is preparing the subpoena cannon.

 

“You might think that’s OK”

If you’ve read my blog for a while, you’ve probably noticed that I treat US Republican politicians as if they were a hive mind. That’s obviously false, but when they act as a unit to continue family separation policies or put partisan hacks on the Supreme Court then their differences are small enough to safely ignore.

Today, we got another example of that. The Intelligence Committee within the US House of Representatives held a hearing on Russian interference. Rather than contribute towards that, however, every Republican on the committee used their time to demand the head of the committee step down. Why? According to a letter they released,

Despite these findings [of the Special Council report], you continue to proclaim in the media that there is “significant evidence of collusion.” You further have stated you “will continue to investigate the counterintelligence issues. That is, is the president or people around him compromised in any way to a hostile foreign power?” Your willingness to continue to promote a demonstrably false narrative is alarming.

Either Adam Schiff knew this was coming, or he’s damn quick on his feet, because he shot back with this. Forgive the length of this quote, but it’s worth absorbing in full. [Read more…]

Ugh, Not Again

P-values are back in the news. Nature published an article, signed by 800 scientists, calling for an end to the concept of “statistical significance.” It ruffled my feathers, even though I agreed with its central thesis.

The trouble is human and cognitive more than it is statistical: bucketing results into ‘statistically significant’ and ‘statistically non-significant’ makes people think that the items assigned in that way are categorically different. The same problems are likely to arise under any proposed statistical alternative that involves dichotomization, whether frequentist, Bayesian or otherwise.

Unfortunately, the false belief that crossing the threshold of statistical significance is enough to show that a result is ‘real’ has led scientists and journal editors to privilege such results, thereby distorting the literature. Statistically significant estimates are biased upwards in magnitude and potentially to a large degree, whereas statistically non-significant estimates are biased downwards in magnitude. Consequently, any discussion that focuses on estimates chosen for their significance will be biased. On top of this, the rigid focus on statistical significance encourages researchers to choose data and methods that yield statistical significance for some desired (or simply publishable) result, or that yield statistical non-significance for an undesired result, such as potential side effects of drugs — thereby invalidating conclusions.

Nothing wrong there. While I’ve mentioned some Bayesian buckets, I tucked away a one-sentence counter-argument in an aside over here. Any artificial significant/non-significant boundary is going to promote the distortions they mention here. What got me writing this post was their recommendations.

What will retiring statistical significance look like? We hope that methods sections and data tabulation will be more detailed and nuanced. Authors will emphasize their estimates and the uncertainty in them — for example, by explicitly discussing the lower and upper limits of their intervals. They will not rely on significance tests. When P values are reported, they will be given with sensible precision (for example, P = 0.021 or P = 0.13) — without adornments such as stars or letters to denote statistical significance and not as binary inequalities (P  < 0.05 or P > 0.05). Decisions to interpret or to publish results will not be based on statistical thresholds. People will spend less time with statistical software, and more time thinking.

This basically amounts to nothing. Journal editors still have to decide what to print, and if there is no strong alternative they’ll switch from an arbitrary cutoff of p < 0.05 to an ad-hoc arbitrary cutoff. In the meantime, they’re leaving flawed statistical procedures in place. P-values exaggerate the strength of the evidence, as I and others have argued. Confidence intervals are not an improvement, either. As I put it:

For one thing, if you’re a frequentist it’s a category error to state the odds of a hypothesis being true, or that some data makes a hypothesis more likely, or even that you’re testing the truth-hood of a hypothesis. […]

How does this intersect with confidence intervals? If it’s an invalid move to hypothesise[sic] “the population mean is Y,” it must also be invalid to say “there’s a 95% chance the population mean is between X and Z.” That’s attaching a probability to a hypothesis, and therefore a no-no! Instead, what a frequentist confidence interval is really telling you is “assuming this data is a representative sample, if I repeat my experimental procedure an infinite number of times then I’ll calculate a sample mean between X and Z 95% of the time.” A confidence interval says nothing about the test statistic, at least not directly.

In frequentism, the parameter is fixed and the data varies. It doesn’t make sense to consider other parameters, that’s a Bayesian move. And yet the authors propose exactly that!

We must learn to embrace uncertainty. One practical way to do so is to rename confidence intervals as ‘compatibility intervals’ and interpret them in a way that avoids overconfidence. Specifically, we recommend that authors describe the practical implications of all values inside the interval, especially the observed effect (or point estimate) and the limits. In doing so, they should remember that all the values between the interval’s limits are reasonably compatible with the data, given the statistical assumptions used to compute the interval. Therefore, singling out one particular value (such as the null value) in the interval as ‘shown’ makes no sense.

Much of what the authors proposed would be fixed by switching to Bayesian statistics. Their own suggestions invoke Bayesian ideas without realizing it. Yet they go out of their way to say nothing’s wrong with p-values or confidence intervals, despite evidence to the contrary. Their proposal is destined to fail, yet it got more support than the arguably-superior p < 0.005 proposal.

Maddening. Maybe it’s time I got out my poison pen and added my two cents to the scientific record.

Happy Emmy Noether Day!

Whenever anyone asks me for my favorite scientist, her name comes first.

At a time when women were considered intellectually inferior to men, Noether (pronounced NUR-ter) won the admiration of her male colleagues. She resolved a nagging puzzle in Albert Einstein’s newfound theory of gravity, the general theory of relativity. And in the process, she proved a revolutionary mathematical theorem that changed the way physicists study the universe.

It’s been a century since the July 23, 1918, unveiling of Noether’s famous theorem. Yet its importance persists today. “That theorem has been a guiding star to 20th and 21st century physics,” says theoretical physicist Frank Wilczek of MIT. […]

Although most people have never heard of Noether, physicists sing her theorem’s praises. The theorem is “pervasive in everything we do,” says theoretical physicist Ruth Gregory of Durham University in England. Gregory, who has lectured on the importance of Noether’s work, studies gravity, a field in which Noether’s legacy looms large.

And as luck would have it, today was the day she was born. So read up on why she’s such a critical figure, and use it as an excuse to remember other important women in science.

Rapid Onset Gender Dysphoria

Remember that old thing? No? OK, quick summary:

Parental reports (on social media) of friend clusters exhibiting signs of gender dysphoria and increased exposure to social media/internet preceding a child’s announcement of a transgender identity raise the possibility of social and peer influences.

Littman L (2018) Parent reports of adolescents and young adults perceived to show signs of a rapid onset of gender dysphoria. PLoS ONE 13(8): e0202330.

In short, maybe social media is making the kids transgender? This seems like something someone should study, and someone did!

Poorly. [Read more…]

I Think I Get It

We seem to be in a cycle. Every time PZ Myers posts something about transgender people, the comment thread floods with transphobes. Given the names involved, I suspect this is due to Ophelia Benson’s effect on the atheio/skeptic sphere.

Regardless, there may be another pattern in play. The go-to argument of these transphobes was transgender athletes, with the old bathroom line showing up late in the thread. I had a boo at GenderCritical on Reddit, to assess if this was just a local thing, and noticed there were more stories about athletics than bathrooms over there. Even one of the bigots thought this was new. Has there been a shift of rhetoric among transphobes?

If so, I think I understand why.

[Read more…]

Some Much-Needed Follow-up

You can almost watch my opinion flip in real time.

September 25, 2014 at 11:57 am

If Benson made a habit of linking to TERF materials, even though she knew where they came from and had plenty of alternatives, I wouldn’t be so quick to defend her. But this is a single cartoon that is only problematic because of its source, and even then you had to either know TERF lingo or read carefully to discover the source was problematic. It should be entirely forgivable, at minimum, especially if Benson made it clear she didn’t endorse trans exclusion once she knew of the source. Which she did.

That some people aren’t willing to forgive this no matter what Benson does outs them as demanding perfection from imperfect beings. Only the most fanatic religious fundamentalists agree to that.

=====

September 25, 2014 at 9:38 pm

Not only does coded language allow you to get away with saying racist/sexist/classist things, you might trick non-racist/sexist/classist people into supporting you. I myself was thinking of sharing an image elsewhere, until I saw octopod raise the TERF flag and went “hmmm, I might be missing something here.” On and off over several hours, I scratched my head trying to work out what that was. “I suppose that one comic ‘reinderdijkhuis’ linked to made it explicit, but I didn’t spot anything else as bad. Though, now that I think of it, that rainbow comic looked like a coded message. And it was weird the masthead used the word ‘cotton’ but I’m HOLY SHIT HOW COULD I BE THAT BLIND….” […]

In that moment, a page I originally thought contained a mix of funny but heavily obscure comics was revealed to be a vicious cacophony of sexist dog whistles. EVERY comic was dripping with hate, but in some of them it was so carefully hidden that it looked like feminist commentary. Those could easily float around Facebook, with only a select few snickering over the true message being passed around. Imagine sharing an image that mocked Obama for being a warmonger, following the link to the source, and stumbling across a white supremacist website. If you were black, that would be horrific.

Hopefully that should explain why the image had to go, and why I was wrong to edge towards the “devil’s advocate” chair. My apologies for taking so long to clue in.

I was a latecomer to Ophelia Benson’s transphobia, other people had been aware of it for at least a year before my flip began. The whisper network had started talking, and I decided to listen. I owe a debt of gratitude to the people who helped me move from clueless to slightly-less-so, people like abbeycadabra, Janine, Xanthë, and Jason Thibeault. That also means I should take critiques from them seriously, as my understanding isn’t as far along as theirs.

Frankly, for a trans person, there’s something surreal and erasing in seeing cis people feuding with cis people over whether we exist. I mean, I am grateful that there are cis people being allies for us and pushing back against the transphobes (and homophobes and every other kind of -phobe.) But the fact that people have to come up with logical arguments and “evidence” that our transness is “real,” thus keeping the question alive of whether we do, in fact, exist, keeps giving me the creepy feeling that maybe I’m just a figment of my own imagination. I think the technical term is “depersonalization.”

It’s like when people run around “proving” that 1 = 0 — nobody sees any real need to “disprove” it, because it’s obvious that such a proof is BS. (It’s a reductio ad absurdum on the face of it.) But it seems like even those who believe in our existence feel the need to prove it. I was just reading HJ Hornbeck’s post about trans athletes, which has all kinds of “scientific,” “objective” evidence that gender dysphoria, gender identity, etc. are real. The problem with going down that path is not only that it concedes the possibility that it could be “disproven,” but also that trans people who don’t fit into the definitions and criteria in those “proofs” are then implicitly left out of the category “real trans.”

I was originally going to type up something in response, but after re-reading this comment that instinct feels mistaken. I agree with all of it, anything I add would just be restating something they said, and that would promote the idea that trans people’s opinions only have weight if cis people agree with them. So I’ll give Allison the final word.

This is BTW why I don’t like the idea of medical tests for transness, or proofs that trans people’s brains are observably different from cis people’s. Ultimately, being trans lies in one’s own understanding of oneself, gained through hard and painful experience. If I know based on my own experience of myself that “trans” best describes me, and some brain scan “proves” that I’m not, which am I to believe? (“Who are you gonna believe? Me? Or your own eyes?”) I spent most of my life ignoring my experience of myself and trying to live the way society told me I should, and it damned near killed me, and I think most trans people (at least we older trans people) have had the same experience.

… and Ophelia Benson

That small thing? I saw a referer pop up from Butterflies and Wheels, when one of Ophelia Benson’s commenters linked to me as an example of outrageous behaviour. Whenever that happens, I refer back to the rule I established three years ago.

For my part, I wrote myself into a corner with that last post. “Ophelia Benson is transphobic” became a “dog bites man” story, there wasn’t anything new or notable about it. The best evidence was on the table, people had entrenched in their opinions, and there seemed little point in flogging that horse further. So I hate-read Benson for a few weeks or so, then got bored and stopped caring. Maybe twice in that time she’s been mentioned in my circles, I checked back in, asked myself and others “does this qualify as noteworthy?,” then after some deliberation decided it wasn’t.

This time, it was. So I did my homework, typed up the first of a two-part post, and promptly got distracted. I promised to return to it during Trans Awareness Week, then broke that promise as academics and life caught up to me. PZ’s post landed just as I was clawing back towards a more stable spot, so I dusted off those old drafts.

[Read more…]