So far I know of only two critiques of my argument in Proving History that actually exhibit signs of having read the book (all other critiques can be rebutted with three words: read the book; although in all honesty, even the two critiques that engage the book can be refuted with five words: read the book more carefully).
As to the first of those two, I have already shown why the criticisms of James McGrath are off the mark (in McGrath on Proving History), but they at least engage with some of the content of my book and are thus helpful to address. I was then directed to a series of posts at Irreducible Complexity, a blog written by an atheist and evolutionary scientist named Ian who specializes in applying mathematical analyses to evolution, but who also has a background and avid interest in New Testament studies.
Ian’s critiques have been summarized and critiqued in turn by MalcolmS in comments on my reply to McGrath, an effort I appreciate greatly. I have added my own observations to those in that same thread. All of that is a bit clunky and out of order, however, so I will here replicate it all in a more linear way. (If anyone knows of any other critiques of Proving History besides these two, which actually engage the content of the book, please post links in comments here. But only articles and blog posts. I haven’t time to wade through remarks buried in comment threads; although you are welcome to pose questions here, which may be inspired by comments elsewhere.)
Ian’s posts (there are now two, A Mathematical Review of “Proving History” by Richard Carrier and An Introduction to Probability Theory and Why Bayes’s Theorem is Unhelpful in History; he has promised a third) are useful at least in covering a lot of the underlying basics of probability theory, although in terms that might lose a humanities major. But when he gets to discussing the argument of my book, he ignores key sections of Proving History where I actually already refute his arguments (since they aren’t original; I was already well aware of these kinds of arguments and addressed them in the book).
When Ian isn’t ignoring the refutations of his own arguments in the very book he’s critiquing, he is ignoring how applications of Bayes’ Theorem in the humanities must necessarily differ from applications in science (again for reasons I explain in the book), or he is being pointlessly pedantic and ignoring the fact that humanities majors need a more colloquial instruction and much simpler techniques than, for instance, a mathematical evolutionist employs.
To illustrate these points I will reproduce in bold the observations of MalcolmS on what Ian argues (which also does a fine job of summarizing Ian’s substantive points; my thanks to him for all of it), and then follow with my own remarks, which I have also expanded upon here (saying a bit more than I did in the original comments).
1. Your form of Bayes’s theorem is “confusing and unnecessarily complex.” His preferred form of BT is P(H|E) = P(E|H)P(H)/P(E).
He has 2 objections to your form of the formula: (1) The denominator has been expanded, which he feels is unnecessary. [I pointed out to him that most textbooks actually state BT with the expanded denominator and that most applications of the theorem that one encounters use it in that form as well, but he replied that in his own field (AI) he only needs P(E), so maybe this is just his personal preference from his own experience. Moreover, you give both forms of BT in the appendix.]
And (2) adding the background [i.e., the term b] explicitly is “highly idiosyncratic,” “condescending,” and “irksome,” reminiscent of William Lane Craig. [I agree that it is unusual and unnecessary, but it is not wrong, as even he acknowledges. Moreover, it is easier to transition to the form of the equation that you actually are using, whereby some of the evidence is incorporated into the background, but you never make this explicit.]
2. He also criticizes you for failing to explain the derivation of BT or discussing the definition of conditional probability generally: “While Carrier devotes a lot of ink to describing the terms of his long-form BT, he nowhere attempts to describe what Bayes’s Theorem is doing. Why are we dividing probabilities? What does his long denominator represent?” Consequently BT becomes “a kind a magic black box.”
He then states cryptically: “In this Carrier allows himself to sidestep the question whether these necessarily true conclusions are meaningful in a particular domain. A discussion both awkward for his approach, and one surely that would have been more conspicuously missing if he’d have described why BT is the way it is.”
[I'm not sure what point Ian is making here, but I think he is alluding to the difficulty of calculating P(E), which he discusses in his 2nd post. As I'll point out later, his criticism is based on a misunderstanding of how you are applying BT.]
MalcolmS finds these two objections to be too trivial. I think they are outright pedantic and belie an abandonment of pedagogical goals and therefore are not really a worthwhile criticism. For example, a humanities major is not going to understand what P(E) is or how it derives from the sum of all probabilities; they are also going to have a much easier time estimating P(E|H) and P(E|~H) because estimating those probabilities is something they have already been doing their whole lives–they just didn’t know that that’s what they are doing. Much of my book is about pointing this out.
This is also why I keep the term b [for background knowledge] in all equations (as I even explain in an endnote, which clearly Ian did not read: see note 10, p. 301): so that laymen won’t lose sight of its role at every stage. Mathematicians like Ian don’t need it there. But historians are not mathematicians like Ian. This is also why I don’t waste the reader’s time by explaining how Bayes’ Theorem (or BT) was derived or proved; instead, I refer interested readers to the literature that does that (e.g., note 9, pp. 300-01). That’s how progress works: you don’t repeat, but build on existing work. I don’t have to prove BT or explain how it was derived; that’s already been done. I just have to reference that work and then show how BT can be applied to my field (history).
3. His next criticism is one that I partially share: “Carrier correctly states that he is allowed to divide content between evidence and background knowledge any way he chooses, provided he is consistent. But then fails to do so throughout the book.” He cites as an example, p. 51, where the prior is defined to explicitly include evidence in it. [The prior should be the probability that the hypothesis is true before any consideration of the evidence.] He continues with this quote from your book [which I also find objectionable]: “For example, if someone claims they were struck by lightening five times … the prior probabilty they are telling the truth is not the probability of being struck by lightening five times, but the probability that someone in general who claims such a thing would be telling the truth.”
This is his response: “This is not wrong, per se, but highly bizarre. One can certainly bundle the claim and the event like that, but if you do so Bayes’s Theorem cannot be used to calculate the probability that the claim is true based on that evidence. The quote is valid, but highly misleading in a book which is seeking to examine the historicity of documentary claims.”
Ian’s point is that BT is defined as P(H|E) = P(E|H)P(H)/P(E), with the prior by definition being P(H), i.e., without any conditioning on the evidence. In the example of the claim of being struck by lightning 5 times, the hypothesis H would normally be “someone was struck by lightning 5 times” and the evidence E would be “he claims to have been struck by lightning 5 times.” Then the prior would indeed be the probability of being struck by lightning 5 times. You instead have as your prior the conditional probability that someone is telling the truth when he claims “in general such a thing,” which would be (part of) the evidence for the claim.
[Effectively what you are doing is treating the evidence E ("someone claims to have been struck by lightning 5 times") as if it were an intersection of E with a larger set, F ("someone claims such a thing"), that is more general, and then absorbing F into B. The form of BT that you are using is then: P(H|EB) = P(E|HFB)P(H|FB)/P(E|FB).
Now, this trick may be useful for actually calculating P(H|BE), since then you avoid having to calculate P(H|B) or P(E|B), but you haven't been entirely upfront with the reader about what you are doing.] Moreover, as he points out in the quote above, you still are left with P(H|FB), which is very similar to P(H|EB), so you haven’t really used BT to solve the problem.
[In the actual applications in your book, what you generally do is use BT in this way to reduce the problem to conditional probabilities with more general evidence, and then use an empirical frequency to estimate them. This avoids the problem that he raises about having to estimate, say, the probability of the NT existing.]
Indeed. And this likewise ignores the fact that historians need to do different things than scientists. Thus the way I demarcate b from e is what is most useful to historians–and again, I even explain this explicitly in an endnote (note 10, p. 301) and discuss it several other times in the book (see “background knowledge vs. evidence” in the index, p. 333). Ian is simply ignoring all of that, and thus not responding to what my book actually argues.
Historians are testing two competing hypotheses: that a claim is true vs. the claim is fabricated (or in error etc.), but to a historian that means the actual hypotheses being tested are “the event happened vs. a mistake/fabrication happened,” which gives us the causal model “the claim exists because the event happened vs. the claim exists because a mistake/fabrication happened.” In this model, b contains the background evidence relating to context (who is making this claim, where, to what end, what kind of claim is it, etc.), which gives us a reference class that gives us a ratio of how often such claims typically turn out to be true, vs. fabricated (etc.), which historians can better estimate because they’ve been dealing with this kind of data for years. We can then introduce additional indicators that distinguish this claim from those others, to update our priors. And we can do that anywhere in the chain of indicators. So you can start with a really general reference class, or a really narrow one–and which you should prefer depends on the best data you have for building a prior, which historians rarely have any control over, so they need more flexibility in deciding that (I discuss this extensively in chapter 6, pp. 229-56).
You could, if you wanted, build out the whole Bayesian chain (e.g. see endnote 11, page 301), all the way from raw data, but why should historians trouble themselves with that? They already have context-determined estimates of the global reliability of statements based on their experience. If they get into an argument over conflicting estimates there, then they can dig into the underlying assumptions and build out the whole Bayesian case from raw data, or at least from further down the chain of underlying assumptions. But it’s a massively inefficient waste of their time to ask them to do that all the time, or even a lot of the time.
Ultimately, all Bayesian arguments start in the middle somewhere. If they didn’t, they’d all have priors of 0.5 (or whatever equally ramifies the spread of all possible hypotheses). Ian might prefer to start somewhere past the assembly of raw sensory data and toward the frequency-sets of basic event-occurrences (so maybe he would try to answer the question “Did Winston Churchill exist?” by starting with questions like “What is the physical probability that a man named Winston Churchill would be born in pre-WWII England?,” which would be a ridiculous way to proceed). But even that is doing what I am doing (he, too, is skipping a step: in this case, how we know the frequency data about names is correct, given a certain body of sensations Ian experiences, and so on). Historians usually skip all the science steps. Because they’re doing history. Not science (in the narrow sense; I discuss the relation between history and science on pp. 45-49). But one can always go back in and check those steps. If you had to for some reason.
In short, historians need to be more flexible in how they model questions in Bayes’ Theorem. Ian’s pedantry wouldn’t help them at all. Because it really doesn’t matter how you build the model–as long as you can articulate what you are doing, and it’s correct (as I explain in chapter six, especially). Because then it can be vetted and critiqued. Which is all we want. And that is all my book arms the historian to do. And that’s all she needs in order to get started.
Indeed, having these conversations (about what models to use and what frequencies fall out of them, and thus how to define h and demarcate e from b in any given case) is precisely what historians need to be doing. My book gives them the starting point for doing that. Because otherwise it won’t be the same for every question, because the data-availability differs for each case and thus historians have to demarcate differently in different cases. Scientists don’t face this problem, because they always have old data (which entails a prior) and new data (which gives likelihoods) and then only address problems that have tons of precise data to work from. Historians can almost never do any of those things. They have to adapt their application of Bayesian reasoning to the conditions they are actually in. Proving History explains why. A lot. Ian, apparently, just ignores that.
MalcolmS also observes how pedantic and insubstantial these criticisms are…
So far all of his criticisms have been stylistic (either about how equations were expressed or how they were explained), rather than truly mathematical. The rest of his post is no different.
4. He takes you to task for using Bayes’s formula as a synonym for Bayesian reasoning. In particular, he ridicules this quote from your book: “any historical reasoning that cannot be validly described by Bayes’s Theorem is itself invalid.” His objection seems to be that, while BT is of course true, there are other equations that one could derive from the definition of conditional probability that couldn’t be derived from BT.
[Actually, one could derive the formula for conditional probability from BT if one had some sort of definition of conditional probability that implied P(AB|B) = P(A|B) and P(A|AB) = 1: If one assumes BT (i.e., P(H|E) = P(E|H)P(H)/P(E)) then P(H|E) = P(HE|E) = P(E|EH)P(HE)/P(E) = P(HE)/P(E). But this is besides the point, since you only limited your claim to "historical reasoning," a term which you unfortunately didn't define.]
He further states your attempt to prove this “laughable” assertion is not credible, but gives no other reason than what I just stated above.
Which is an example of a non-critique critique: saying something is wrong, but giving no reason why, nor even interacting with the argument you are gainsaying at all. Ian’s overall claim is that Bayes’ Theorem can’t be used to reach historical conclusions because the probabilities are all unknown. But if that’s true for BT, it’s true for all probabilistic reasoning about history, which means all reasoning about history whatever.
I demonstrate (with a formal deductive syllogism even: pp. 106-14; supporting the informal arguments on pp. 62-65 and 81-93) that all historical arguments are fundamentally Bayesian (whether historians realize this or not), so if Ian were correct that no conclusions about history can be reached by Bayesian reasoning, then he is saying no conclusions about history can be reached. Period. Such radical skepticism about history I have refuted before (in Rosenberg on History, where I also show how, if that were true, science is also impossible, as it depends on historical facts, i.e. data and reports about things that happened or were observed, so if you can’t do history, you can’t do science).
That Ian totally ignores this, and doesn’t address my syllogistic argument at all, makes his critique here useless. Indeed, that is the point of my formalizing an argument: so critics will be able to identify any errors that invalidate the conclusion. If he’s not even going to do that, then he isn’t taking the book’s argument seriously. And so neither should we take his critique seriously.
One wonders what method he thinks would replace Bayes’ Theorem, that historians can use. Since all historical arguments consist of deriving conclusions from statements of probability, is there any logically valid way for them to derive those conclusions other than Bayes’ Theorem? (Or anything that reduces to it? See pp. 92-93; and again, pp. 106-14.) If you want to be a useful critic, you have to answer that question. I suspect any sincere effort to do so will result in realizing the answer is no.
5. He then goes on to discuss your “cheeky” proposal to unify frequentist and Bayesian interpretations of probability. His criticisms here are that your proposal is “unnuanced” and presented as if were original, when it is not. (Not that he is accusing you of taking credit for others’ ideas but rather of being possibly unaware of previous work in the field.) He also states that this “hubris” is typical of “a tone of arrogance and condescension that I consistently perceived throughout the book.”
Which is just ad hominem. I’m quite sure I don’t know all the arguments published on the debate between Frequentists and Bayesians (it must be in the thousands, counting books and articles), as I’m sure neither does Ian. Or any living person probably. But certainly, if anyone has articulated the same conclusion as mine before, I’d love to accumulate those references (it seems Ian claims they exist, but then fails to adduce a single example). So by all means, if anyone knows, post them in comments here.
That is neither here nor there. The real issue is whether my resolution of that debate is correct. Whether Ian dislikes my tone or thinks it’s arrogant or condescending is not a valid rebuttal to whether it is correct. I also don’t think he’s making an objective assessment, since I am responding to the debate as framed in recent literature by leading professors of mathematics (some of which I actually cite in the book), so he is here actually critiquing them for not knowing the solution I propose. After all, if even they don’t know about this supposedly condescendingly unoriginal argument of mine (and if they did, they’d have resolved the debate with it in the literature I cite), then why is it condescending for anyone like me to suggest it?
6. As a final comment on the mathematics he raises 2 issues but doesn’t elaborate on either: “I felt there were severe errors with his arguments a fortiori…and his set-theoretic treatment of reference classes was likewise muddled (though in the latter case it coincidentally did not seem to result in incorrect conclusions).” This is the entire extent of his discussion of these, from his perspective, problems.
I readily concede that my colloquial discourse will lead to ambiguities that chafe at mathematicians; but this is precisely the kind of shit they need to get over, because they are simply not going to be able to communicate with people in the humanities if they don’t learn how to strategically use ambiguity to increase the intelligibility of the concepts they want to relate.
It’s like Heisenberg’s Uncertainty Principle: you can have precision with unintelligibility to almost everyone but extremely erudite specialists, or you can have ambiguity but with intelligibility to everyone else. The more ambiguity, the greater the clarity, but the lower the precision. This is a fundamental principle of all nonfiction literature, especially any popularization of scientific or mathematical concepts to a nonscientific, nonmathematical public.
I have a particular audience. I am writing for them. And they are not mathematicians or scientists. However, I always think there are several points where I could be a better writer. Because I always know there is room for improvement. It would be more helpful to see someone articulate a point I make in my book better than I did. I would love that. And if anyone points me to any examples of that, I’ll definitely blog about it.
7. In his conclusion he has some positive things to say about the book:
“Outside the chapters on the mathematics, I enjoyed the book, and found it entertaining to consider some of the historical content in mathematical terms….History and biblical criticism would be better if historians had a better understanding of probability….
I am also rather sympathetic to many of Carrier’s opinions, and therefore predisposed towards his conclusions. So while I consistently despaired of his claims to have shown his results mathematically, I agree with some of the conclusions, and I think that gestalts in favour of those conclusions can be supported by probability theory.”
But here is his final critique:
“But ultimately I think the book is disingenuous. It doesn’t read as a mathematical treatment of the subject, and I can’t help but think that Carrier is using Bayes’s Theorem in much the same way that apologists such as William Lane Craig use it: to give their arguments a veneer of scientific rigour that they hope cannot be challenged by their generally more math-phobic peers.”
As you can see, he hasn’t presented any concrete objection to the mathematics in the book – just the way the mathematics was presented and explained and the overall tone of the book.
So it would seem. That’s not really a substantive critique.
Moreover, the difference between me and W.L. Craig is revealed by all of Ian’s qualifying remarks–like “coincidentally did not seem to result in incorrect conclusions,” a backhanded way to admit I’m actually using it correctly, unlike Craig, thus negating his analogy. The whole point of my book is to prevent Craig-style abuses by making it clear how to use BT correctly and how to spot its being used incorrectly. And indeed, I repeatedly emphasize that anyone who wants to use it needs to be clear in how they are using it so it can be vetted and critiqued, thus avoiding the “dazzling with numbers” tactic by arming the reader with the ability to see through it. (Hence I somehow managed to “psychically” refute Ian’s argument in note 33, p. 305, before he had even made it; likewise my remarks on pp. 90-92. In other words, Ian didn’t really read the book very carefully, as he is clearly unaware of my rebuttals to his own arguments.)
From the whole of his initial critique, Ian doesn’t seem to have as much experience as I do in trying to explain Bayesian reasoning to nonmathematicians. Much of my book was formed in response to the difficulties I faced when doing that. Things Ian thinks would be a better way to proceed, I have discovered first-hand are often the worst way to proceed. In communicating ideas to humanities majors especially, I have learned you have to approach explanations in very different ways than trained mathematicians do; and that often, mathematicians do not understand this.
Now I’ll turn my attention to Ian’s 2nd post…
Most of his post is taken up with an explanation of conditional probability and Bayes’s theorem, which is actually pretty good; it would’ve been a good idea to devote a few pages in your book to something like this. But so far there’s no criticism of your book, or even really much mention of it. I’ll start the numbering over again from 1 to list his criticisms, which eventual start to appear.
1. For historical questions, there usually is no easy way to calculate, or even estimate in many cases, P(E). He says, “I’ve never seen any credible way of doing so. What would it mean to find the probability of the New Testament, say?…I’m not sure I can imagine a way of calculating either P(H∩E) or P(E|H) for a historical event. How would we credibly calculate the probability of the New Testament, given the Historical Jesus? Or the probably of having both New Testament and Historical Jesus in some universe of possibilities?”
[He writes as if he didn't actually read your book, although I know that he has, because he is going by his own knowledge of using Bayes's theorem and not looking at the examples where you apply it. As I pointed out in a previous post, you get around the issue of estimating P(E) and P(H) by conditioning on general statements of the evidence, so that you're calculating P(H|F) and P(E|(~H)&F). These need to be estimated somehow, but may be easier since the evidence F is more commonly encountered. It's funny that he observed you doing this in his first post but then never thought through the implications for the rest of his posts. I suggest that if you explained what you were doing mathematically and how it differed from the way scientists usually state and apply Bayes's theorem there would not be so much confusion.]
MalcolmS is right: my book is actually articulating ways to get around the very problem Ian’s talking about (which I certainly acknowledge: note, for example, my discussion of it on pp. 110-14). Where we can’t, we can’t claim to know anything (Axiom 3, page 23; also Axiom 5, page 28). His question about how we derive “the probability of the New Testament” is unclear (what exactly does he mean?), but I address something quite close to it on pp. 77-79, using the Gospel of Mark rather than the whole NT (and I get even more specific in my discussion of emulation criteria later on: pp. 192ff.), which appears to completely answer his question. So why, then, does he not know that I answered his question? If he is ignoring my answer, then he is not critiquing my book, but some straw man of it.
In any event, the problem he is talking about (and that I also talk about in the book) is addressed by (a) being less ambitious in what you will attempt to prove (a lesson historians often need to learn anyway) and (b) being more clear and precise in laying out what evidence it is that you think produces a differential in the consequent probabilities (most evidence simply will not, and therefore can be ignored). Thus “the whole NT” is irrelevant to historicity; likewise even “the exact content of Mark.” We will need to get much more specific than that. What is it in Mark that makes any significant difference? And why? And how much? Whatever your answers to those questions are–literally, whatever they are–we can model your answer using BT. And in my next book I do that.
Ironically, as I noted before, Ian is committing the very mistake here that I warn against in the book: if we cannot estimate P(E|H), then historical knowledge is simply impossible. Because all historical conclusions implicitly rely on estimates of P(E|H) (and/or P(E|~H)), or their differential (using the “Odds Form” of BT: see “Bayes’ Theorem, odds form” in the index, p. 333). That’s all historical conclusions ever reached before now, and all that will ever be reached by anyone ever. Thus, if BT can’t solve this problem, no method can. And if Ian thinks otherwise, it’s his task to produce that method, a method by which (a) a historian can get a conclusion about history without (b) ever relying on any implicit assumption about any P(E|H) or P(E|~H); or, for that matter, P(H). Good luck with that. Because it can’t be done. If he’d tried it, he’d know.
2. He then asks whether using the expanded-denominator version of the formula [which he again annoyingly attributes to you and Craig as if you are the only ones who write it this way, and he's not talking about the inclusion of the background here either] could ameliorate this problem with estimating P(E):
“This is just a further manipulation. The bottom of this equation is still just P(E), we’ve just come up with a different way to calculate it, one involving more terms. We’d be justified in doing so, only if these terms were obviously easier to calculate, or could be calculated with significantly lower error than P(E).”
[He doesn't seem to think that P(E|~H) is often easier to calculate than P(E), nor does he notice the advantage of using a variable, P(E|~H), that is independent of the other 2, P(H) and P(E|H), unlike P(E), which isn't. Perhaps if he looked at your examples or even the standard example of medical testing he'd see why the expanded form often works better.]
Here MalcolmS has already effectively rebutted this point. Ian just fails to grasp the way BT has to be employed in the humanities, and what it takes to translate how people in the humanities already reason into terms definable within BT. The utility of knowing the whole structure of Bayes’ Theorem is so we can understand the logical structure of our own thought–and thus make better arguments, and better identify the flaws in others.
3. His next criticism is a bit bizarre, as he complains about having to use estimates: “If these terms are estimates, then we’re just using more estimates that we haven’t justified. We’re still having to calculate P(E|H), and now P(E|~H) too. I cannot conceive of a way to do this that isn’t just unredeemable guesswork. And it is telling nobody I’ve seen advocate Bayes’s Theorem in history has actually worked through such a process with anything but estimates.”
[Of course you use estimates - even in the sciences one does. Unless you're doing a problem with dice or cards, the numbers one plugs in are always estimates. And you present several examples where you attempt to justify your estimates. It would help if he actually addressed one to show why it was nothing but "unredeemable [sic] guesswork.”]
He sums up in a similar vein: “So ultimately we end up with this situation. Bayes’s Theorem is used in these kind of historical debates to feed in random guesses and pretend the output is meaningful.”
Here MalcolmS has already effectively rebutted this point, too. Ian seems to be conflating “not knowing x precisely” with “not knowing x at all.” I explicitly address this fallacy several times in the book (early in chapter six, and in my discussion of arguing a fortiori: pp. 85ff.). In short, historians don’t need the kind of precision Ian seems to want. In fact, as I explain in the book, that they can’t get it even if they wanted it is precisely what demarcates history from science (pp. 45-49).
4. As a teaser for what he intends to write about in a later post, he states why he thinks a fortiori reasoning doesn’t work:
“But, you might say, in Carrier’s book he pretty much admits that numerical values are unreliable, and suggests that we can make broad estimates, erring on the side of caution and do what he calls an a fortiori argument – if a result comes from putting in unrealistically conservative estimates, then that result can only get stronger if we make the estimates more accurate. This isn’t true, unfortunately, but for that, we’ll have to delve into the way these formulas impact errors in the estimates. We can calculate the accuracy of the output, given the accuracy of each input, and it isn’t very helpful for a fortiori reasoning.”
[His characterization of your a fortiori argument - "if a result comes from putting in unrealistically conservative estimates, then that result can only get stronger if we make the estimates more accurate" - is easily demonstrably true: P(H|E) is monotonically increasing in P(H) and P(E|H) and decreasing in P(E|~H), so it follows immediately that if one takes a maximum estimate for P(H) and P(E|H) and a minimum one for P(E|~H) then the estimate for P(H|E) using Bayes's theorem is a maximum, and similarly one derives a minimum for P(H|E). Furthermore, tightening the possible ranges for these variables yields a tighter range for P(H|E), so the a fortiori argument is in fact valid.
I think what he is intending to say is that while one may in principle get a possible range for P(H|E) from possible ranges for the other probabilities, in practice, the range turns out to be too wide to be useful. Whether that turns out to be the case will be seen when you actually try to apply it.]
His final comment supports my reading of him: “It doesn’t take much uncertainty on the input before you loose any plausibility for your output.”
[If this were true it would true for all uses of BT, so how does he account for its use in science? It's not like those estimates for false positives in DNA testing (in his example) lack errors.]
Here MalcolmS has already effectively rebutted this point as well. Like he says, we’ll have to see what Ian comes up with. But I suspect he is ignoring everything I explained in the book about the fact that historians have to live with some measure of ambiguity and uncertainty, far beyond what scientists deal with. That’s what makes history different from science (again: pp. 45-49; and again, Axioms 3 and 5, per above). Ian seems either to want history to be physics, or to think I want history to be physics. The one is foolish; the other betrays a failure to really read my book (e.g. pp. 60-67).
5. He hints at another problem here but says he’ll explain some other time:
“[I]n subjective historical work, sets that seem not to overlap can be imagined to overlap in some situations. This is another problem for historical use of probability theory, but to do it justice we’ll need to talk about philosophical vagueness and how we deal with that in mathematics.”
As MalcolmS says, Ian’s criticism on this point is too vague to even know what he means. At present no response is required.
6. There are 4 footnotes to the post, only the last of which could be taken as a criticism of using BT with uncertainty (specifically the form of BT with the denominator expanded). Here it is in full:
“You’ll notice, however, that P(E|H)P(H) is on both the top and the bottom of the fraction now. So it may seem that we’re using the same estimate twice, cutting down the number of things to find. This is only partially helpful, though. If I write a follow up post on errors and accuracy, I’ll show that errors on top and bottom pull in different directions, and so while you have fewer numbers to estimate, any errors in those estimates are compounded.”
[Since P(E|H)P(H) appears in both the numerator and denominator, an increase in it would increase both the numerator and denominator, so the effects from each would offset, not compound! But P(H) appears in the second term in the denominator, so it is not quite that simple. Changes in P(E|H) would partially cancel in the numerator and denominator, making P(H|E) less sensitive to changes in it, but for P(H) depending on the ratio of P(E|H) and P(E|~H), the effect of changes in the prior on the top and bottom of the fraction can indeed compound, but not for the reason he stated.]
That is his whole criticism. Basically he doubts that BT can be practically used in BT because he feels the inputs are too subjective and have uncertainties that are too wide. We’ll have to wait for your next book to see if you can pull it off.
The quoted argument doesn’t seem mathematically informed, unless he means errors in P(E|H) and P(E|~H) can pull in different directions; otherwise, P(H) and P(~H) always sum to 1, so they actually consist of a single estimate, not two, so they can’t pull against each other. If you estimate P(H) against yourself, you have also estimated P(~H) against yourself, by definition. And if you do the same with both P(E|H) and P(E|~H), they can’t ever pull in opposite directions, either. The compounded error between them will then only make your argument more a fortiori. So there isn’t any discernible criticism here that I can make out. (You can see how I already address this whole issue on pp. 85-93.)
Overall Conclusion: So far I am not impressed. It doesn’t look like Ian’s taking any argument in the book seriously. His critiques are almost like some Cato Institute effort at refuting a policy that they don’t really have any valid argument against but they have to refute it anyway so they come up with whatever trivia and handwaving they can think of. The fact that almost all of Ian’s critique is already rebutted in the book itself (often directly and explicitly) only strengthens the analogy.