(See part 1, part 2, part 3, and part 4.)
In the part 4previous post I stated that peer-reviewed papers have prima facie credibility and if you want to challenge their veracity, the burden shifts to you to do so. If you want to discredit it, you have to produce contrary data or detect a serious flaw in the methodology, or show that there has been an error in the calculation.

None of these things has been done, at least as far as I have seen. All that the people condemning the study have said is that they do not believe it. I wonder if they have even read the study before condemning it. Take for example, this report from Norman Solomon, about how the media and pundits respond to such estimates. He points out that the present large numbers of casualties were predicted by reputable groups before the war but were dismissed by the media.

While we stare at numbers that do nothing to convey the suffering and anguish of the war in Iraq, we might want to ask: How could we correlate the horrific realities with the evasive discussions that proliferated in U.S. news media during the lead-up to the invasion?

In mid-November 2002 – four months before the invasion began – a report surfaced from health professionals with the Medact organization and International Physicians for the Prevention of Nuclear War. “The avowed U.S. aim of regime change means any new conflict will be much more intense and destructive than the [1991] Gulf War,” they warned, “and will involve more deadly weapons developed in the interim.”

At the time, journalists routinely gave short shrift to that report – treating it as alarmist and unworthy of much attention. The report found that “credible estimates of the total possible deaths on all sides during the conflict and the following three months range from 48,000 to over 260,000. Civil war within Iraq could add another 20,000 deaths. Additional later deaths from postwar adverse health effects would reach 200,000. … In all scenarios the majority of casualties will be civilians.”

During a live TV debate on Dec. 3, 2002, I cited the report’s estimates of the bloodshed ahead and then asked: “What kind of message is that from the Bush administration against terrorism and against violence for political ends?”

CNN anchor Wolf Blitzer turned to the other guest: “Jonah Goldberg, do you accept that assumption in that report on these huge casualties, including a lot of children, if there were an effort to go forward with so-called regime change in Baghdad?”

Goldberg, a pundit with National Review Online, replied: “Frankly, I don’t. I mean, I haven’t looked at the exact report, and I think that there are a lot of groups out there that inflate a lot of these numbers precisely because they’re against the war no matter what.”

Notice that Goldberg had not even read the report, or shown any indication that he had at least read the critiques of knowledgeable people. This kind of behavior is typical for these people. All they do is speculate based on political biases. For Goldberg, the report numbers are too large for him to stomach, so the authors must be having a political ax to grind.

In showing such a cavalier disregard for actual reading documents or citing sources, Goldberg is following in the footsteps of his hero George Bush. Bush said he did not believe the numbers and cited General Casey, the top US commander in Iraq, as supporting him. It is true that Casey said “That 650,000 number seems way, way beyond any number that I have seen. I’ve not seen a number higher than 50,000. And so I don’t give it that much credibility at all.” But where did Casey get his own number? According to an AFP report, when questioned as to his source: “Casey said he did not know where he had seen the estimate of 50,000 or whether it was produced by the military.” In other words, he simply pulled it out of the air. It is this kind of flim-flam that is practiced by these people, hoping that the public will not notice that they have not provided any substantive critique of the 655,000 figure.

Other people have challenged the latest Lancet as “obviously” political because it was released just before the 2006 elections, and the 2004 study was also released before the elections that year.

I find this a curious argument. The Iraq war is perhaps the biggest issue of the day. Surely the voting public should have the best information on it when they vote for their leaders? It is in fact an obligation of the authors of such studies to try to release it in time for voters to evaluate the numbers and make decisions. The assertion that facts about the war and its consequences should not be given to voters is a bizarre idea. It has appeal only to those who genuflect at the thrones of power, who feel that the “leaders” are all wise and knowing and we, the public have no right to the facts, but must simply defer to their judgments.

The claim of unfair bias can only be justifiably leveled if the authors had (say) obtained very low numbers of deaths (which would have pleased the Bush administration) and deliberately withheld it until after the election. Or if they had cut corners in their data collection and analysis and rushed to print with a flawed paper purely in order to embarrass the administration. But such arguments have not been made by anyone. Instead the critics point to the timing of the release as if that were a sufficient argument against it.

A final point. While a lot of the focus has been on the number of violent deaths, I was disturbed to read in the report that about 53,000 deaths were “due to non-violent causes were estimated to have occurred above the pre-invasion mortality rate, most of them in recent months, suggesting a worsening of health status and access to health care.” These deaths rose above the pre-war levels only in 2006. This is a very disturbing but predictable sign. Wars are not only violent, they also let loose pernicious silent killers. They destroy water and sewage systems, they disrupt farming and agriculture, food distribution networks break down, medicines become scarce, hospitals suffer from lack of supplies and electricity, and people cannot earn enough to get food or medicines. All these things lead to serious health problems which last for a long time and whose effects are hard to reverse.

There will come a time when active warfare comes to an end in Iraq. People’s attention will shift away. But the breakdown of the health, sanitation, and food networks will remain, becoming a silent killer that will enact its cruel will on the Iraqi people for a long time to come.

(See part 1 and part 2 and part 3.)
The critics of the Lancet study have had just one main argument against it: incredulity. They are like the intelligent design creationists who, because they cannot imagine that life as we know it could have evolved, simply assume that a creator must exist without even looking at the evidence.

Some try and make the case that if the level of deaths are really so high, the media would have reported it. The authors of the study are not idiots. They have considered this question in the light of what we know from other conflicts.

[The figure of 655,000] is far greater than reported by various media accounts and morgue tallies. This is not surprising, as reporting of events from incomplete sources cannot, in any statistically meaningful way, be converted into national death rates. Other than Bosnia, we are unable to find any major historical instances where passive surveillance methods (such as morgue and media reports) identify more than 20% of the deaths which were found through population-based survey methods.

Richard Horton, editor of Lancet writes:

[T]he reason for the discrepancy between these lower estimates and the new figure of 650,000 deaths lies in the way the number is sought. Passive surveillance, the most common method used to estimate numbers of civilian deaths, will always underestimate the total number of casualties. We know this from past wars and conflict zones, where the estimates have been too low by a factor of 10 or even 20.

In the comments to yesterday’s post, Eldan Goldenberg refers to a critique of the Lancet study put out in a press release by IraqBodyCount. The main thrust of their comments is that for such a large number of deaths to go under the radar implies massive breakdown of the system. But the Lancet editor seems to be saying that that kind of official undercounting is the norm is such situations, not the exception. It is just that this feature is not highlighted in other situations. Again, the IraqBodyCount critique is not of the study itself but based on the feeling that the figure is ‘unreasonable.’ But with research, we cannot adjust figures based on the reasonableness. All that surprising results require of their authors is careful scrutiny of the methodology to see if systematic errors have distorted the results. Researchers do not have the luxury of adjusting figures

However, the authors of the Lancet study, like any serious scholars, realize that their study has limitations and reflect on them and their possible effects.

Any collection of information is open to potential bias, and has limitations. All efforts were made to randomly select the households to be included in this survey, but it may have been that households with more deaths or households with fewer deaths were over represented in this survey. The finding that the 2006 results are very close to the 2004 household results suggests this did not occur. As in all surveys, a larger sample would have likely have produced a result with greater precision, although this would have exposed the survey teams to higher risk. In the future, when safety has improved, a large survey will be needed to determine in detail the total implications of the conflict for the people of Iraq.

The households were selected for this survey according to population size we obtained from the Ministry of Planning, but this may not have fully reflected migration within or outside the country. However, it is unlikely that this would have occurred at a scale necessary to affect findings.

Perhaps the greatest potential limitation to this type of survey is the problem people have recalling the date of specific events, especially over several years. Again, the close similarities between the 2004 and the 2006 data suggest this was not a major problem. Households could have concealed deaths from the interviewers, though by promising anonymity to households we tried to minimize this risk. We are certain that households did not report deaths which did not occur, as 92% of households had death certificates for deaths they reported.

Another reason to have confidence in this study lies in the very process of peer review. When papers are submitted to scientific journals, the referees cannot and do not verify the actual data. What they look to see is whether the study has followed good methods and the authors have explored all reasonable alternative explanations before reaching their conclusions. This is especially done when the results are so surprising, as in this case. The editors of Lancet, clearly mindful of the explosive political nature of this paper, sent it to four referees and you can be sure that those referees checked to make sure proper procedures were followed. This does not mean that the results could not be wrong. Peer review has failed in the past to detect errors and is not designed to detect outright fraud. But it does mean that peer-reviewed papers have prima facie credibility and if you want to challenge their veracity, the burden shifts to you to do so. If you want to discredit it, you have to produce contrary data or detect a serious flaw in the methodology, or show that there has been an error in the calculation. I have not seen any criticism along these lines as yet.

The authors of the study also describe how the data was collected.

The two survey teams consisted of two females and two males each with one male supervisor. All were medical doctors with previous survey and community medicine experience and were fluent in English and Arabic. All were Iraqis.

Those of us who wonder how such studies based on surveys can be carried out within a war zone have to give credit to the courage and dedication of the people who did this. I know that people who try to collect accurate information in war zones run great personal risks because what warring factions want to avoid most is any accountability and they resist efforts by people to collect data. This is why fact-finders in conflict zone deserve our greatest respect and admiration. These ten brave Iraqis did not want their names revealed for fear of retribution. The fact that the ten Iraqi doctors were willing to risk their lives to try and get information about their ravaged country did not want to be identified testifies to the dangerous situation they were in and I for one share the sentiments of the study authors when they write: “We express our deepest admiration for the dedicated Iraqi data collectors.”

POST SCRIPT: Dawkins and Colbert-What could be better?

Watch a highly entertaining interview of Richard Dawkins by Stephen Colbert about Dawkins’ new book The God Delusion

(See part 1 and part 2.)

When I looked at the Lancet study, saw who had done it, how it had been done, and where it had been published, I quickly gained confidence in their number of 655,000 excess deaths since the invasion of Iraq..

The study was based on a survey done between May and July 2006 by a joint team of people at the Johns Hopkins School of Public Health and the School of Medicine of Al Mustansiria University in Baghdad using standard methods. These excerpts from their paper details how they collected the raw data and their analyses.

In this survey, sites were collected according to the population size and the geographic distribution in Iraq. The survey included 16 of the 18 governates in Iraq, with larger population areas having more sample sites. The sites were selected entirely at random, so all households had an equal chance of being included. The survey used a standard cluster survey method, which is a recommended method for measuring deaths in conflict situations. The survey team visited 50 randomly selected sites in Iraq, and at each site interviewed 40 households about deaths which had occurred from January 1, 2002, until the date of the interview in July 2006. We selected this time frame to compare results with our previous survey, which covered the period between January 2002 and September 2004. In all, information was collected from 1,849 households completing the survey, containing 12,801 persons. This sample size was selected to be able to statistically detect death rates with 95% probability of obtaining the correct result. When the preliminary results were reviewed, it was apparent three clusters were misattributed. These were dropped from the data for analysis, giving a final total of 47 clusters, which are the basis of this study. (my emphasis)

The designers of the study seemed to have gone to some lengths to make sure that they had a truly random sample.

A series of completely random choices were made. First the location of each of the 50 clusters was chosen according the geographic distribution of the population in Iraq. This is known as the first stage of sampling in which the governates (provinces) where the survey would be conducted were selected. This sampling process went on randomly to select the town (or section of the town), the neighborhood, and then the actual house where the survey would start. This was all done using random numbers. Once the start house was selected, an interview was conducted there and then in the next 39 nearest houses.

In order to determine trends in the death rate , they split the time up into three periods and the results they obtained were as follows:

For the purpose of analysis, the 40 months of survey data were divided into three equal periods—March 2003 to April 2004; May 2004 to May 2005, and June 2005 to June 2006.

Following the invasion the death rate rose each year.

• Pre-invasion: 5.5 deaths/1000/year
• March 2003-April 2004: 7.5 deaths/1000/year
• May 2004-May 2005: 10.9 deaths/1000/year
• June 2005-June 2006: 19.8 deaths/1000/year
• Overall post-invasion: 13.2 deaths/1000/year
. . .
The [pre-invasion] rate of 5.5 deaths/1000/year will be considered as the “baseline” crude death rate, making the assumption that without conflict this rate would have continued at this level up to the present time, or even dropped somewhat (most likely).

The post-invasion excess death rate was:

• March 2003-April 2004: 2.6 deaths/1000/year
• May 2004-May 2005: 5.6 deaths/1000/year
• June 2005-June 2006: 14.2 deaths/1000/year
• Overall post-invasion: 7.8 deaths/1000/year

As there were few violent deaths in the survey population prior to the invasion, all violent deaths can be considered “violent excess deaths.”

The post-invasion violent death rate was:

• March 2003-April 2004: 3.2 deaths/1000/year
• May 2004-May 2005: 6.6 deaths/1000/year
• June 2005-June 2006: 12.0 deaths/1000/year
• Overall post-invasion: 7.2 deaths/1000/year

Percentage of all (not just excess) deaths due to coalition forces:

• March 2003-April 2004: 14%
• May 2004-May 2005: 21%
• June 2005-June 2006: 16%
• Overall post-invasion: 7.2 deaths/1000/year

Deaths due to unknown forces:

• March 2003-April 2004: 3.2 deaths/1000/year
• May 2004-May 2005: 6.6 deaths/1000/year
• June 2005-June 2006: 12.0 deaths/1000/year
• Overall post-invasion: 7.2 deaths/1000/year

Conclusion

While the actual value may be somewhat higher or lower than this number, the precision of these results is adequate to conclude that loss of life in this conflict has been substantial. (my emphasis)

The authors compare with other major conflicts and rightly conclude that the final figure of over 600,000 excess deaths puts the Iraq war right up there with the others in the scale of violence.

As with other recent conflicts, the civilians of Iraq bear the consequence of warfare. In the Vietnam War, 3 million civilian died; in the Congo, armed conflict has been responsible for 3.8 million deaths; in East Timor, an estimated 200,000 out of a population of 800,000 died in conflict. Recent estimates are that 200,000 have died in Darfur over the past 31 months. Our data, which estimate that 654,965 or 2.5% of the Iraqi population has died in this, the largest major international conflict of the 21st century, should be of grave concern to everyone.

What should be of especial concern is that the number of people killed by the actions of the US and coalition forces is so high. Overall 31% or 186,300 of the violent deaths were attributed to their actions and 13% or 78,130 of the violent deaths were due to air strikes which are still going on at a high rate. For those people who still cling to the fond hope that modern armaments, “smart bombs”, and “surgical” strikes have minimized the deaths of civilians, these appalling numbers should make for sober reading. We cannot blithely dismiss this level of death as “collateral damage”, the unfortunate and accidental by-product of a well meaning invading force.

But don’t hold your breath that the media is going to give the Lancet study the attention it deserves. As Norman Solomon points out:

American news outlets tend to be rather cavalier about the suffering at the other end of the Pentagon’s missiles, bombs and bullets. And there’s a strong tendency to brand documented concerns as unfounded speculation – a media reflex that suits war-crazed presidents just fine.

(See part 1)

Let us take one by one the “criticisms” that are being made against the Johns Hopkins study about the levels of deaths in Iraq. I put the word in ironic quotes because these are more accurately labeled as attacks, since the word criticism implies a certain level of considered and thoughtful response, which has been totally lacking so far. (The actual paper can be read here (.pdf).)
[Read more…]

I don’t think anyone other than the standard issue Bush cultist will deny that the Iraq invasion has been a disaster on many levels. It has alienated the world, it has enraged Muslim sensibilities, and it has strained the American military to the breaking point. And the worst part is that the administration has nothing to offer other than to “stay the course.” It seems clear to me that even Bush and Cheney must have realized that they have no options left, failure stares them in the face, that there is nothing they can do to succeed in Iraq (whatever “success” might mean) and it seems like their only goal is to bluff and try and wait it out until they leave office so that someone else will have to make an ignominious retreat out of that country. This is, in essence, what their “stay the course” policy implies. They can then try and blame their “cut and run” successor for “losing” Iraq. The fact that this policy will result in numerous more pointless deaths mean nothing to such cynical people.
[Read more…]