Monday, October 16, 2006

previous entry | main | next entry | TrackBack (0)


The Lancet study -- the sequel

I've been traveling quite a bit recently, so I'm quite late to the party on the eight page study published in The Lancet which concludes the following:

Pre-invasion mortality rates were 5·5 per 1000 people per year (95% CI 4·3–7·1), compared with 13·3 per 1000 people per year (10·9–16·1) in the 40 months post-invasion. We estimate that as of July, 2006, there have been 654 965 (392 979–942 636) excess Iraqi deaths as a consequence of the war, which corresponds to 2·5% of the population in the study area. Of post-invasion deaths, 601 027 (426 369–793 663) were due to violence, the most common cause being gunfire.
This is a follow-up to a 2004 study that raised a small ruckus prior to the presidential election claiming that the post-war mortality rate in Iraq was higher than the pre-war rate.

The boys at Crooked Timber, as well as Tim Lambert, have been vigorously defending the study against conservative critics. Megan McArdle is more skeptical, has a raft of posts that critique the study.

This post by Echidne of the Snakes is sympathetic to the study but also cognizant of its flaws, and is worth quoting on two points:

Nobody is happy about the study findings, of course. Let me repeat that: Nobody is happy about the study findings; nobody wants to imagine that many horrible deaths and the suffering that goes along with those or the effect on the survivors....

These point estimates are not as "respectable" as showing them in cold numbers might suggest to some. This is because they are based on sample data and sample data derived from a modified form of random sampling. The confidence intervals that are given in the summary above reflect the added uncertainty caused by this.

I have only one observation at this juncture. The problem with journalistic coverage of statistical analyses is that they tend to focus on the "headline number," ascribing a weight to it that it sometimes does not deserve. In this study, the 655,000 figure is much less important than the fact that the authors can claim with 95% certainty that at least 392,000 people have died in Iraq since the war started. That's the sobering fact.

Readers are hereby invited to comment.

UPDATE: Tyler Cowen posts on The Lancet study as well -- and highlights another important fact that explains a large part of my disenchantment with the Bush administration:

[T]he sheer number of deaths is being overdebated. Steve Sailer notes: "The violent death toll in the third year of the war is more than triple what it was in the first year." That to me is the more telling estimate.

A very high deaths total, taken alone, suggests (but does not prove) that the Iraqis were ready to start killing each other in great numbers the minute Saddam went away. The stronger that propensity, the less contingent it was upon the U.S. invasion, and the more likely it would have happened anyway, sooner or later. In that scenario the war greatly accelerated deaths. But short of giving Iraq an eternal dictator, that genie was already in the bottle.

If the deaths are low at first but rising over time, it is more likely that a peaceful transition might have been possible, either through better postwar planning or by leaving Saddam in power and letting Iraqi events take some other course. That could make Bush policies look worse, not better. (emphasis added)

ANOTHER UPDATE: The folks at Iraq Body Count are skeptical.

posted by Dan on 10.16.06 at 12:02 AM




Comments:

92% of the sample respondents had actual death certificates, too. So, the government should be able to corroborate. But they say it's not the case.

Personally, I think something happened with the sample data, and then the extrapolation looks huge. (And if your sample data is off, then 95% CI doesn't mean much.)

posted by: Aaron on 10.16.06 at 12:02 AM [permalink]



Technically, they claim with 97.5% certainty that it's over 392k (that's a one-tailed comparison), and it's subject to the assumption of having a random sample, something violated in practice in every sample survey. Given the rather unusual conditions under which this study was conducted, I think the random sampling assumption is a very tough barrier to overcome in determining the credibility of the estimates.

As for the broader (political?) conclusions of the study, I think it's basically apples and oranges--if 392k people died because of the war, is that better or worse than 18m people continuing to live under a totalitarian state?

And, as Tyler Cowen (I think) at MR asks, how many of the 392k deaths were avoidable after any regime change? Clearly, Shiites would have been spoiling for revenge against Sunnis regardless of how the Saddam Hussein regime ended and who caused it, although I'm certain that there are probably tactics that in hindsight could have reduced the death count--most notably, coopting rather than disbanding most of the former regime's domestic security apparatus.

posted by: Chris Lawrence on 10.16.06 at 12:02 AM [permalink]



Really the Wilful ignorance offered as comment here would be laughable if it was not such a serious topic,
Oh it's the sample data,
Sadam is a bad bad man,
next thing will be “it’s all Bill Clintons fault”

Is there some law of physics that I'm not aware of that makes Republicans unable to accept the truth without spinning

posted by: baba on 10.16.06 at 12:02 AM [permalink]



92% of the sample respondents had actual death certificates, too. So, the government should be able to corroborate. But they say it's not the case.

It's amazing how many lies have started circulating about this study. At some point, the volume of lies starts to overwhelm the ability of people who have actually read hte study to knock them down. But here goes one, at least:

As argued at crooked timber, there is not nearly the sort of central governmental infrastructure in Iraq to collect copies of hte death certificates from around the country, and collate and organize them in Baghdad. The central government has no idea what's happening around the country. Further, they have no interest in collecting death certificates - knowing how many people are dying in Iraq, ostensibly under their watch, makes them look really bad.

Death certificates are issued in Iraq, but the central government does not keep records or statistics on this process.

posted by: DivGuy on 10.16.06 at 12:02 AM [permalink]




Well there are a lot of meta questions to ask, chief among them is the reliability of the sample. I would gather that the reliability of the sample is the single most important criterion which would narrow the variance of the projection. It just seems to me that if the margin of error is so high one has to ask if this is the best way to ascertain such information, or if it was just the best way possible under the circumstances.

Simply because they are scientific doesn't mean they've applied the best tools for the job. What lingers in my mind is, of course, the figures upheld by common wisdom. Why were they so wrong by comparison?

Then I ask whether or not this method is widely accepted as the best method to ascertain battlefield casualties, and if so where has it been used before, or is this simply a novel approach? If novel is it breakthrough, giving us a vast improvement over prior methods?

Then of course there are tactical questions. How can one talk about the lethality of the post-invasion period in the proper context? If this is a civil war, is it particularly bloody by civil war standards?

But I'll tell you what bothers me most about these figures. They are completely out of context with that of the Iran Iraq War. That war raged for 8 years and according to the best figures anyone has put together, killed roughly about 1 million people. How do half a million people get killed in 1/3 the time when American soldiers have only sustained about 2700 deaths?

posted by: Cobb on 10.16.06 at 12:02 AM [permalink]



Aaron, are you sure 92 percent had death certificates? I thought 80 percent of the sample was asked for a DC and 92 percent of that group produced a DC. This could maybe undermine some aspect of the study, I guess, but it could also help explain some of whatever discrepancies there might be between what this study found and what can be corroborated by the government. Plus what DivGuy said.

posted by: Dave on 10.16.06 at 12:02 AM [permalink]



Another thing that should be kept in mind is that the Iraqi Ministrry of Health is totally corrupt and unreliable in its statistics -- in fact, since December it has been quite openly under the control of one of al-Sadr's followers, with the result that Iraqi hositals are now routinely used as operational bases, torture centers and execution centers by the Shiite militia. So asking the Ministry for confirmation of the number of death certificates issued is an exercise in futility:
http://www.cbsnews.com/stories/2006/10/04/eveningnews/main2064668.shtml

http://www.washingtonpost.com/wp-dyn/content/article/2006/09/07/AR2006090700768_pf.html

Meanwhile, if any significant fraction of those death certificates shown to the researchers by the families are fakes, there must be some outfit in Iraq churning out fake death certificates in huge numbers -- for no convincing reason.

posted by: Bruce Moomaw on 10.16.06 at 12:02 AM [permalink]



I'm reminded of the phrase in use on the anti-this-war-now camp before the war began: A steel beam under compression

Changing the regime is not the biggest problem. It's what happens afterwards � you're dealing with an uncontrollable event ... the physical analogy to Saddam Hussein's regime is a steel beam in compression. This is an extremely repressive regime. Even to say those words doesn't do it justice. When it breaks ... it'll give off absolutely no sign at all that it's about to fail ... [and then] Ka-Wammo! And it just goes crazy. That what's gonna happen here. You may have control over how the things start ... There are a variety of ways to do [it] ... You may have a horse you're going in with. But that guy isn't gonna survive first contact.
But I guess they were all just rabid anti war idiots and not foreign policy experts.

Send more troops! That's the ticket.

posted by: Azael on 10.16.06 at 12:02 AM [permalink]



The death rate in the European Union is 10.1. In France it is 9.14, Germany 10.62, Hungary 13.11. (source: index mundi).

So, in Sadam's paradisical Iraq (where the U.N. estimated 2000 to 3000 deaths per month due to the "Food for Oil" sanctions) had a death rate of half that of the EU (5.5 vs. 10.1)???

And after the invasion, in the chaos generated by the 'insurgents' it became only slightly higher than that of Hungary (13.3 vs. 13.11)???

This does not pass the simplest test of reasonableness.

Tell the left what they want to hear and they will accept it unquestionably.

posted by: Wayne on 10.16.06 at 12:02 AM [permalink]



although I'm certain that there are probably tactics that in hindsight could have reduced the death count--most notably, coopting rather than disbanding most of the former regime's domestic security apparatus.


...in which case all we would have accomplished is replacing a brutal dictator with a brutal dictator who would be acutely aware that pissing off people in DC is an excellent way to become an ex-brutal dictator.

Maybe that's all we should have tried to accomplish. The Realpolitik crowd would have been quite content with that result, and a case can be made that dealing with Iran would be simpler in such a situation.

Is it time to give up on the 'promoting democracy' thing and put the SOBs back in charge?

posted by: rosignol on 10.16.06 at 12:02 AM [permalink]



ps: the turingbot seems to have stopped caring about the letters (or lack therof) in the box.

posted by: rosignol on 10.16.06 at 12:02 AM [permalink]



The death rate in the European Union is 10.1. In France it is 9.14, Germany 10.62, Hungary 13.11. (source: index mundi).

So, in Sadam's paradisical Iraq (where the U.N. estimated 2000 to 3000 deaths per month due to the "Food for Oil" sanctions) had a death rate of half that of the EU (5.5 vs. 10.1)???

This does not pass the simplest test of reasonableness.

Just in the unlikely event that this is a real misconception you are laboring under, the crude death rate in countries with a lot of relatively old people tends to be higher than that in countries with a lot of relatively young people. The median age of the Hungarian population is 39 and fifteen percent of the population is over 65. The median age of the Iraqi population is just under 20 and three percent of the population is over 65. So why don't you go reconsider your comment.

posted by: Kieran on 10.16.06 at 12:02 AM [permalink]



Stupid comment parser. Obviously the quotation from the begninng of my previous comment up until the word "reasonableness" should be in italics.

posted by: Kieran on 10.16.06 at 12:02 AM [permalink]



Adding to Bruce's comments: IIRC, the Health Ministry building, like most other ministries, was looted and burned in spring 2003. Presumably that trashed a lot of records. There's a USA Today article from Dec 2003, claiming that the CPA forbade the tallying of deaths. It would be far from surprising if the Iraqi government, 2004-present, was downplaying the death rates; they'd hate to admit how bad things had become.

Megan's criticism of the study, in the one thread that I read, started with 'the number's too big', with absolutely no justification, and descended downward from there.

posted by: Barry on 10.16.06 at 12:02 AM [permalink]



As for the Iraq Body Count people, it's clear that they are suffering from at least two delusions: first, that their tally of media-reported deaths is anything other than an extreme lower bound; second, that the Iraqi national government is efficiently and honestly tracking deaths.

posted by: Barry on 10.16.06 at 12:02 AM [permalink]



I recommend an editorial in the current WSJ (available on line) regarding the methodological flaws in the study. The authors not only used an extremely small sample, but did not collect the additional data that would have validated (or invalidated) the results.

posted by: Wayne on 10.16.06 at 12:02 AM [permalink]



gtaxen zmub cxnye yhtivxfa oyxpq agqnzob tdoejqhk

posted by: ndxayk fjkdn on 10.16.06 at 12:02 AM [permalink]






Post a Comment:

Name:


Email Address:


URL:




Comments:


Remember your info?