OHJ: The Covington Review (Parts 4 & 5) »« Take My Course on Boghossian’s Making Atheists!

A Bayesian Brief on Comments at TAM

I was asked about remarks made by Chris Guest (President of the Australian Skeptics, Victorian Branch) at this year’s TAM. He gave a quick twenty minute talk on Bayesian reasoning and its abuses, with which I entirely concur. (This talk begins with Guest’s introduction at timestamp 48:30.) He criticizes my work briefly at the end, but understanding his remarks there require understanding his remarks throughout the talk. His only mistake is that when he gets to my work, he makes one crucial mathematical error that invalidates his entire critique…

The Overall

His overall thesis is correct: just as you can lie (or self-deceive) with logic and statistics, so you can lie (or self-deceive) with Bayes’ Theorem. Thus, we need to know how to tell the difference, and police ourselves to make sure we’re using them correctly. As he notes, that it can be abused in no way discredits Bayesian reasoning, any more than the misuse of logic discredits logic.

His point that Bayesian conclusions are often counter-intuitive yet correct is also apt. That means the mere fact that they are counter-intuitive cannot be used as a reason to reject them. Another lesson many critics refuse to learn. If your intuition contradicts a sound application of Bayes’ Theorem, it’s your intuition that needs to be corrected, not Bayes’ Theorem that needs to be abandoned.

His example of HIV testing depending on which population you test is also an argument from prior probability, which I use in On the Historicity of Jesus (ch. 6) to argue the same thing he does: the prior probability that Jesus existed depends on which population he comes from. For example, is it that of all religious leaders, or that of all heavily mythicized savior deities? Jesus obviously comes from the latter. And so, just like a high-risk population for HIV, his non-existence is more likely, than it is for religious leaders generally.

Them

Guest then briefly addresses one example of Christian apologetic abuses of Bayes’ theorem (at timestamp 57:20), aptly showing the errors of the McGrews in attempting to use Bayesian reasoning to prove Jesus was miraculously resurrected from the dead. Importantly, he says that they are on the right track with using Bayes’ Theorem to test their claim, and that they are right to admit their claim is extraordinary, and to have accurately mathematically represented what that means. He takes no issue there.

Where Guest sees the McGrews going wrong (and he’s right) is that they embarrassingly fabricate their likelihoods. “I think the problem is that they have set their likelihood terms at impossibly high values,” and “claim that all their facts are conditionally independent” (1:01:20), when in fact they are often dependent (e.g. social pressure and collusion among “the disciples” is more likely than the baseline for the general population), and he mentions that they fail to properly calculate the effect of such alternative hypotheses as “hallucination.”

I should also add that, though Guest might not know this, most of the McGrews’ factual claims are also straightforwardly false, e.g. we have no credible record of even one disciple dying to avoid renouncing their belief in the resurrection, and not only is the claim that women were cited as witnesses false—not a single Gospel says they got any information from any women—but their claim that women weren’t trusted as witnesses in antiquity is also false—women were widely just as trusted as men as witnesses of fact (see Not the Impossible Faith, chapter 11). Obviously, if you are putting false facts into your Bayesian equation, you will be getting false results as well.

But back to Guest’s commentary. On his own last point Guest doesn’t elaborate, but for example, when the likelihood “Paul saw Jesus” is placed in ratio for the competing hypotheses “Jesus rose from the dead” and “Paul had a hallucination,” you are at best looking at 1:1, not what they calculate, which is an absurd 1000:1. Indeed, most arguably it’s even reversed, since the facts of Paul’s vision and testimony (if we take them as rote) have more in common with hallucinations than with meeting revived people (TCD, pp. 304-07), so it should probably even be 1:1000, given that out of every thousand times you meet someone who has died and was restored to life, they don’t appear to you mystically inside your mind—at least not more than once surely. (And this we can document, since meeting the revived dead happens quite a lot now, thanks to modern medicine; but the same holds even for modern claims of miraculous resurrections.)

But let’s set that aside, and assume it’s 1:1. The apologist would claim, maybe, “but you are presuming a hallucination, and Paul was too hostile to the gospel to have one of those,” but that’s an argument regarding the priors. How often do people hostile to a religion “have a vision” that convinces them they’re wrong? Since I’ve heard this from countless preachers (who insist they were enemies of god and self-serving atheists reveling in sin until Jesus “spoke to them” or whatever…some converts to Islam have said similar things), it’s clearly far more frequently the case than “miracles,” which Guest says the McGrews place odds against of 1 in 10 to the power of 40. Yet the number of people hostile to a faith who “experience a conversation” with god or something akin that convinces them to convert is far, far more than that. If there were even one such person in the world today (with a population over six billion, at least one billion of whom hostile to some religion or other that some of them will end up nevertheless converting to) its frequency would be something like one in a billion, or 1 in 10 to the power of 9. In other words, billions and billions of times more likely than a miracle.

So we can fully accept that such “unexpected conversions” are extremely rare and still it will be billions of times more likely that that’s what happened to Paul than a visit from a resurrected corpse. Worse, as I point out in my chapter on the resurrection in The Christian Delusion (which is my definitive take-down of all McGrew-style arguments), “Though such a conjunction of causes [inspiring a hater to reverse course] would be uncommon, Paul’s conversion was uncommon, thus confirming an ordinary explanation. Had the extraordinary been at work, Paul would not have been alone” (pp. 307-08). In other words, “not a real resurrection” predicts few persons hostile to the gospel would convert. That prediction is exactly confirmed. Whereas only a real resurrection would have converted most or all hostiles, or so many as to defy natural probability.

Thus “but you are presuming a hallucination, and Paul was too hostile to the gospel to have one of those” is invalid Bayesian reasoning. We are not “presuming” hallucination, we are validly including the possibility of it in the math (which the Christian apologist, invalidly, is not). And “Paul was too hostile” does not make the observations improbable. To the contrary, it exactly predicts what we observe: virtually no one “too hostile” converted (we have on reliable record only one such person, out of the hundreds or thousands opposing the movement throughout the first half of the first century). Thus, the likelihoods do not favor resurrection. The McGrews are just lying with statistics. Like anyone else who misuses math to trick people.

Me

Guest only gets to me at the end of his talk (at timestamp 102:05). As he humorously leads into that, he says he’s “all in favor of teaching the controversy,” hence he means to present an atheist example of doing what the McGrews did. He immediately says he hasn’t read my book (On the Historicity of Jesus) and is only going on what he’s “been told” about it, that “there’s some quality scholarship in there,” but he’s not here criticizing that. He is instead interested in my paper on Tacitus: Richard Carrier, ‘The Prospect of a Christian Interpolation in Tacitus, Annals 15.44’, Vigiliae Christianae 68 (2014), pp. 1-20. Which was just recently published.

This I have reproduced in Hitler Homer Bible Christ (chapter 20). And I should note that VC actually objected to my including a proper mathematical analysis in the paper because they didn’t like math and insisted I remove it. So I had to use colloquial ways of presenting the same information—not my choice. I restored the removed section in HHBC (pp. 392-94). And it is this that Guest is responding to. As there explained, the section in question is not in the article in VC. Which means his criticism doesn’t apply to the peer reviewed article. At best, it would only call for my revising the appendix to it that I included in HHBC.

Guest is first bothered by not knowing where I get my estimates from. But as stated in the section, they are just measures of what I mean by “unlikely,” “very unlikely,” and similar judgments. My argument is that “assigning higher likelihoods to any of these would be defying all objective reason” (p. 393), which is a challenge to anyone who would provide an objective reason to believe them more likely. In other words, when historians ask how much this evidence weighs against authenticity, they have to do something like this. And whether they do it using cheat words like “it’s very unlikely that” or numbers that can be more astutely questioned makes no difference. The cheats just conceal the numbers anyway (e.g., no one says “it’s very unlikely that” and means the odds are 1:1). So an honest historian should pop the hood and let you see what she means.

The logic and justification for doing this I extensively lay out in Proving History (which was peer reviewed by a mathematics professor). It does not appear Guest has read that or is aware of the case made in it for the method used in my Tacitus paper. Even though the section he is responding to says in its first paragraph that “I justify and explain the method here employed in” Proving History. Crucially, because the numbers expose what I really mean when I say something is, for example, “very unlikely,” a critic should therefore address the numbers. That’s what they are there for. Guest does not do this.

He says this is just like the McGrews case, except it isn’t, even by his own account. Unlike the McGrews I do not “set [my] likelihood terms at impossibly high values.” And Guest notably does not say that I do. I also fairly account for alternative hypotheses of how the enumerated facts could come to be (as articulated in the body of the paper). As one should. So what analogous objection does Guest have then?

He claims my four facts (A, B, C, D) are not independent. He gives no argument for this; he just insists they could be. That’s how not to use Bayes’ Theorem.

But that’s not the problem. I could fully grant these are dependent probabilities. Where Guest goes wrong is that he incorrectly says the argument I present is invalid if any of the probabilities I present are dependent probabilities. But that’s not true. For example, I give the probability of A as 1 in 20 and of B as 1 in 5. If the probability of B given A is 1 in 5, then it is both the case that B is a dependent probability and my math is entirely correct. Guest makes no accounting for this. He doesn’t even examine the argument or the evidence to check for this.

How Dependent Probabilities Actually Work

The general fact to be explained is “It is not believable that Tacitus would know of such an enormous persecution event, but all subsequent Christians have no knowledge of it for over two hundred years” (p. 385). I broke this down into three different portions of the evidence for which it is true: A, B, and C.

The fourth fact, D, that the passage internally sounds like it’s about something else, is not an argument from the silence of later Christian documents and thus has no causal relationship to A through C, nor vice versa, except on the interpolation hypothesis, which entails all these facts are 100% expected, so their being dependent probabilities on that thesis is irrelevant; e.g. if D entails A through C, such that A through C have a probability of 100% if D, then when D has a probability of 100%, so does A through D. Exactly as I assign. So Guest can’t be claiming dependence invalidates my math here. It does not.

So he must mean that, for example, on the hypothesis of authenticity, A entails B and C, such that whatever probability A has, A+B+C has the same probability, not a compound probability. But even if B and C are dependent on A, it does not follow that their dependent probability is 100%. Therefore, it does not follow that A+B+C has the same probability as A.

I identified as A the actual Christian stories we do have of Nero’s persecution of Christians. None of which mention the Tacitean fire and purge. This is all but impossible if that event happened, and though we might ask how likely it is that Tacitus made the event up, we know it is to some extent improbable that even if he did still no Christian would ever have heard of it for centuries, despite Tacitus being widely read. That’s the relevant matter here: if A is the case, then we can expect that the probabilities of B (no other Christian literature mentions the fact) and C (no Christian author who we know would have read the works of Tacitus mentions the fact) will go up. But they do not go up to 100%.

Because it’s entirely possible (and non-negligibly probable) that Christian authors composing the Neronian persecution tales we have would not know of Tacitus or the events he relates (just because either happened does not mean later Christian martyrologies will have known about it). But that not only they but also all other Christian authors would not know of it is even less likely. How much less? I weigh it as 5:1 against.

As I even explain, “One could swap the odds between (A) and (B), since realistically I find both very improbable, but I consider the silence of stories we actually have to be more improbable than the absence of stories we should expect to have, even though the latter is also quite improbable in this case” (p. 393). In other words, that the existing stories would be ignorant of an event is improbable. That there aren’t even more stories than those is even more improbable still. For example, we could have had the Neronian tales we have, which are ignorant of this event, and other Neronian tales that aren’t ignorant of this event. That the Neronian tales we have are ignorant of this event, I assign a probability of 1 chance in 20. That there would also not be any other tales that aren’t ignorant of this event, I assign a probability of 1 chance in 5. And I’m willing to imagine swapping these for each other, since the order of dependency is not important to the overall conclusion that for both to be the case is extremely unlikely. I am thus saying the odds of both being the case is 100:1 against. Because I do not think it could plausibly be more likely than that.

Meanwhile we have one more unusual detail: that even authors who knew Tacitus don’t mention the event. This is not logically necessary. We could explain all the other documentary silence (A+B) by positing the ad hoc supposition that no extant Christian authors read Tacitus and thus his information didn’t get passed on and by also positing (yet another ad hoc assumption, thus further reducing the likelihood) that he was by far the most famous author to have mentioned it (so that by not reading him, you would probably not know of it) and by also positing (yet another ad hoc assumption, thus further reducing the likelihood) that those who read it in Tacitus or elsewhere would not think to tell any Christians about it (even when it was Christians reading Tacitus) and by also positing (yet another ad hoc assumption, thus further reducing the likelihood) that Christians themselves, though the very victims of such an incredible event, never passed any knowledge of it on among their ranks (such that even oral knowledge of it died out quickly even within the movement). If we posit all of that, we could then say that not learning it from Tacitus would probably mean not learning it at all. Hence my previous estimates are really, in effect, my estimate of how likely all that is. Which is 100:1 against at best. I welcome any attempt to argue it surely must have been more likely.

So now we have one more fact to consider: C, that even Christian authors who knew the works of Tacitus did not know of the event or tell any other Christians about it. Because this cannot be subsumed under A+B, since A+B are only possible on a hypothesis of authenticity if no Christian read Tacitus and no Christian passed knowledge of the event on through any other channel. But here, we have a group of authors who actually read Tacitus. It is again possible that they would mention the Neronian event and no one else would. Thus, it was possible that if ~C (Tacitus readers did mention the event, or no Christian authors could be established as Tacitus readers), then still A & B. Therefore, A & B do not entail C. Nor does C entail A or B (as just noted, it’s possible only Tacitus readers knew the passage, and that no one read what they wrote about it). So how likely is it, even given A and B, that also Christian authors who knew the works of Tacitus would not mention the event? I say 5:1 against. That is a dependent probability. Yet still an entirely valid probability.

This is called a cumulative case argument. That A+B+C is even less likely than just A or B or C alone. I find it’s 500:1 against that all three of those facts would be found together. Thus adding B and C to A reduces the likelihood of the hypothesis. Dependence of the probabilities is mathematically irrelevant to this fact. As long as your probabilities account for dependence, you are good to go. So unless Guest can argue that 500:1 is the wrong odds for A+B+C, he has no valid criticism of my analysis or my use of Bayes’ Theorem.

Conclusion

I could have added some pages explaining all this. But in my brevity I assumed it would be worked out by anyone with an eye to challenging it. It would be a valid rebuttal to prove that if A, then certainly B and C, for example. But Guest proves no such thing. It is not a valid rebuttal to say that if A, then the probability of B and C are different than they would be when ~A, “therefore my analysis is invalid.” That is a mathematical non sequitur. That the probability of B and C would be different than they would be when ~A says nothing about whether they would be different than the probabilities I assign them.

Comments

  1. Ben Wright says

    “I give the probability of A as 1 in 20 and of B as 1 in 5. If the probability of B given A is 1 in 5, then it is both the case that B is a dependent probability and my math is entirely correct.”

    Actually, you’re wrong here. If P(B|A) = P(B), then you can show that P(A|B) = P(A) and A and B are independent by definition. A criticism of assumed independence is valid not because of a counter-assumption of dependence, but because independence is a strong claim that is usually invoked to simplify the equations. I’ve not got your working to hand, so I don’t know how the missing dependence term would affect things, but just about any construction that uses two or more of A, B, C and D should include AnB, AnC etc. or provide a robust justification for assuming independence.

    • says

      I didn’t say P(B|A) = P(B). So I don’t know what you mean to say here.

      If P(AB) = 1/100, then it is consistent with that that P(A|B) = 1/20 and P(B|A) = 1/5 or P(A|B) = 1/5 and P(B|A) = 1/20. And that is the statement I made.

      I didn’t use this notation in the appendix at all, so no assumptions could have been made about what notation I was using (and one can use the notation with implied givens omitted, e.g. mathematicians frequently give the dependent probability P(h|e.b) as P(h|e), allowing the dependence of h on b to be understood). But I do agree if one only glanced at my appendix and didn’t think about the possibilities, then they could assume I was talking about completely independent variables by bringing that presumption to the text, i.e. if he was looking for an example of this, he would “see” it wherever it wasn’t explicitly told him it wasn’t there. So that error is explicable. And easily corrected.

      The problem is not that he mistook what I meant. The problem is that he made the blanket statement that any use of dependent probabilities in a Bayesian equation invalidates the equation. That is false. For exactly the reasons I explain: as long as you multiply the dependent probabilities as dependent probabilities, there is no invalidity. So you can easily use dependent probabilities in Bayes’ Theorem like this. Guest tells the audience, in effect, the opposite. That’s the mistake.

  2. Ben Wright says

    That clears it up.

    “For example, I give the probability of A as 1 in 20 and of B as 1 in 5. If the probability of B given A is 1 in 5,”

    could probably be better worded though, to more explicitly state that the first probability for B is not necessarily independent. Unless, of course, I’m misreading things again. It’s been a day of having to very precise and clear about the meaning of quantities and my brain has turned to soup.

    • says

      Oh yes. I think I could have added a few sentences to improve the clarity.

      Which is why I have less of a problem with Guest mistaking what I was saying, than with his issuing a blanket assertion that you can’t do Bayes’ Theorem with dependent probabilities.

      But then, reverse engineer: had he realized he should have qualified by explaining that you can do Bayes’ Theorem with dependent probabilities, the very fact of his realizing he needed to make that point might have inspired him to stop and think, wait, maybe that’s what Carrier was doing? Because then he’d realize he had to make sure I wasn’t, before assuming it; because then he’d remember there is a possible way to do this that is valid.

      So the general mistake is itself part cause of the particular mistake. The general mistake is still the more serious. Because it miseducated the audience. The particular mistake is less serious, because it merely misjudged something.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>