by hilzoy
As many of you probably know the Lancet has come out with a study which found this:
“We estimate that as of July, 2006, there have been 654 965 (392 979–942 636) excess Iraqi deaths as a consequence of the war, which corresponds to 2·5% of the population in the study area. Of post-invasion deaths, 601 027 (426 369–793 663) were due to violence, the most common cause being gunfire.”
This is, of course, horrible. Unfortunately, I do not have time to write a long post about this. However, having skimmed some of the reaction to this study, a few notes.
First, it’s worth bearing in mind that very few bloggers are statisticians. Some are, and their views of the previous Lancet study are worth rereading. I have taken courses in biostatistics and epidemiology, and the studies do not seem to me to have any major problems beyond those noted in the article itself, nor do those problems invalidate the results.
The people who did this study, however, are public health professionals at one of the best schools of public health in the country. Statistical studies of population health and mortality are one of the main things that public health researchers do. Biostatistics is not ancillary to their field, as it is to mine; it is a central part of it. These people are very serious experts.
This does not mean, of course, that their work cannot be challenged. Experts are not infallible. It does mean, however, that one should be aware that one is challenging the work of people who are very, very good at what they do. (Note: I do not know any of the people who wrote the study, though I have met one of the statisticians they consulted. He’s very good.)
Second, as people kept saying over and over in response to the last study, the fact that these results are higher than those provided by Iraq Body Count is no surprise. The two use completely different methods. Iraq Body Count relies on “online media reports from recognized sources.” Are all deaths reported by “online media reports from recognized sources”? Of course not. This study, by contrast, visits a representative sample of households and asks about deaths. They asked for death certificates 87% of the time; when they asked, they saw death certificates in 92% of the cases.
When you go out and look for deaths, you are likely to find more of them than when you wait for people to report them and then count the reports. Specifically, you are likely to catch all the unreported deaths. (In this case, “unreported” means “unreported by the media.”)
Anyone who says that the discrepancy between the Lancet study’s figures and any figures derived from IBC or any other group that relies on reported deaths shows, by that very fact, that they have no clue what they are talking about.
Third, about the idea that the timing of this is politically motivated: that in no way impugns the accuracy if the study. Moreover, if I had done a study like this, I would want to get it out before the election too. Information like this, if valid, is exactly the sort of information that people should have in order to make an informed decision about who to vote for. If anyone wants to show that the study’s conclusions are false, they should do so. But saying that the fact that it was released now shows that they aren’t valid is just wrong.
UPDATE: Here’s a good explanation of the methodology, from Cervantes. H/t Majikthise.
Thanks for mentioning IBC’s methods. The Post’s article on the study kept mentioning IBC’s numbers as if they’d done a similar study using similar methodology and come up with wildly different results. The two groups aren’t even counting the same thing.
(just so i can beat Ugh to it)
earlier Lancet study “thoroughly debunked by a wide variety of experts”
now that’s some tribalism!
I’ve spent much more time ranting about this topic (the earlier Lancet paper) than I should. But a little more won’t hurt.
First, Iraq Body Count’s methodology doesn’t seem to pick up very many people killed by the US, once you get past the first two months of the invasion. For instance, in the third year of the occupation they could definitively count 370 civilians as killed by the US. (Many deaths were from unknown sources, but 370 is the number they could clearly identify as caused by the coalition.) It’s hard to get bodycounts of any sort from the US military, but from what I’ve read in a few places (no links handy), we were killing a 1000 insurgents per month in 2004-2005. Hard to believe we killed many thousands of insurgents in a given year and only 370 civilians. That’s not the usual ratio in any guerilla war I’ve read about. My guess is that we’re killing comparable numbers of civilians and insurgents, whatever the true number happens to be.
However, with respect to this paper I’m a little surprised about all these death certificates–it suggests that the Iraqi government bureaucrats are issuing death certificates, but doing a rather poor job counting them when they give out death toll statistics. Or else there’s fraud somewhere in this paper. I think it’s likely the government is either lying or incompetent, but if I were going to question the reality of this estimate that’s the point I’d focus on.
You’re suggesting (and I don’t know this to be false, certainly, I just mean that you didn’t say it squarely) that you have a source showing that the Iraqi government’s centralized numbers for the number of death certificates they’ve issued don’t match up with this study. Is that the case?
This from the National Review. Talk about misreading (or misleading) the statistics to suit yourself! They conveniently ignore the fact that the Johns Hopkins study describes additional deaths, all war-related, on top of normal mortality.
“The US death rate is 8 per 1,000 population (see here). The UK is 10 per 1,000. According to the study, Iraq’s post-invasion death rate is 13.2 per 1,000. A whole host of countries have death rates way higher than that – see here. Now a lot of this is to do with age of population, but given that the death rate of, say, Cameroon, which has a similar birth rate to Iraq is also around 13, this suggests that the humanitarian crisis in Iraq is no worse than that of many other countries. Different in character, certainly, but it suggests that the epidemic of violence in Iraq is less debilitating to that country than the AIDS epidemic is to Botswana, for instance.”
http://corner.nationalreview.com/
The number does seem wildly high. Those are Rwanda-class numbers.
I haven’t seen anything that suggests the country has been depopulated to that extent.
600,000 out of a population of 24 million is hardly depopulation, in the sense that you’d expect to see empty streets.
The number does seem wildly high.
What, it’s only the equivalent of every man, woman and child in Austin, TX or Baltimore, MD.
US cities
“600,000 out of a population of 24 million is hardly depopulation, in the sense that you’d expect to see empty streets.”
That depends on the distribution, doesn’t it? If randomly killing every 50th spherical iraqi of uniform density, you’d probably not notice.
But that’s not how it happens in real life. It’s more likely that the dead will be clustered around the locations of insurgent activity, firefights, and ethnic or sectarian enclaves targeted for purging by their enemies.
Which, incidentally, could lead to an exaggerated total dead. If you did a similar survey in the US in the 1860s, and wound up surveying people near civil war battlefields, you might well end up with an exaggerated total when extrapolating to the entire US population.
earlier Lancet study “thoroughly debunked by a wide variety of experts”
I don’t have the psychic strength to click on that at the mo’
I’m sure it’s filled with the reasonable, fair-minded analysis we’ve come to expect from the responsible denizens of Redstate.
Sure. I’m not a statistician, but the people who did the study are — isn’t the risk of oversampling supposed to be accounted for in the confidence interval?
I’m not a statistician either, but I remember they specifically excluded Fallujah from the previous study.
FWIW
Did they control for the size of the family their respondents were referring to? If the respondents were talking about the deaths of cousins, it’s entirely possible that one person’s death could be mentioned by multiple respondents.
There’s also the possibility that some respondents thought claiming a death in the family could result in some material benefit.
Just to be clear, I’m not trying to dismiss the idea that lots of civilians have died. I just doubt that the death toll exceeds 200,000.
Jon H, you could always read the study and see the answers to these questions.
I just doubt that the death toll exceeds 200,000.
You have evidence to support your skepticism, I’m sure.
The actual study is hilzoy’s first link.
The President reassures us the study is, for reasons unstated, “not credible”. Let us breath a sigh of relief…..
Right — the study defines a ‘household’ as a group of people who live and eat together, and interviewed households about deaths in that household in the years before and since our invasion. While I suppose it’s possible that someone could be a member of two ‘households’ simultaneously so as to get double-counted, it’s hard to imagine that that could happen frequently enough to skew the results.
It would be interesting to ask someone who still enthusiastically supports the Iraq debacle whether it is worth it if it’s true that 500,000 Iraqis have died violent deaths since the invasion, and see how high the death toll gets before they reconsider. Their frantic need to “debunk” the study suggests 500,000 may be too high, but I doubt it.
(just so i can beat Ugh to it)
Dang.
I’m interested in the death certificate issue too – seems like a lot. Assume a flat distribution of age and a life expectancy of 50 and you get 2% deaths/year, so 2-5%/3 years means a big increase in that office.
Say you get shot for having the wrong name, and your family buries you in the backyard because the morgue is run by the other guys – how do you get a death certificate? Are there good reasons for living people to get death certificates?
I just doubt that the death toll exceeds 200,000.
or it could be just shy of 800,000.
Times online
And speaking of Bizarro World cleek, this is just too funny, their contest winner in:
Why People Should Vote Republican in 2006
…
2. I want a freer America, where my rights– to own a gun, to determine my own healthcare arrangements– are not infringed in the pursuit of an abstract common good.
Yes, vote Republican for a freer America. The mind boggles.
Christmas: “You have evidence to support your skepticism, I’m sure.”
Being snide won’t help your argument.
I haven’t read the new Lancet study yet, but if they’re using the same methods they used last time, the numbers they get are still almost certainly an undercount. And remember these are excess deaths, on top of the high death rate that Iraqis already had from living under Hussein and sanctions.
And on another subject altogether, would people please stop running planes into buildings around here? It’s disconcerting.
Steward Beta writes: “or it could be just shy of 800,000.”
Uh, yeah, that’s exactly the report we’re already talking about. If I’m skeptical of their 600,000 estimate, I certainly don’t give much credence to the 800,000 number, even if it is somehow justifiable by some quirk of statistics based on their data. If their data is questionable, the statistics are useless.
Being snide won’t help your argument.
However, a request for evidentiary support should always be welcome in a debate.
I’m not at all sure why people (like Jon H above) have a gut feel that 600k excess deaths (or between 2 and 5% of the population) in 3.5 years of war, civil war, general chaos and banditry is too high.
Doing the very crude arithmentic brings up a total of about 475 violent deaths per day on average across the country. Does this seem so far out there? It doesn’t to me given the daily reports in the media. Intuitively, the media has got to be heavily undereporting the numbers killed each day and yet news reports of scores of bodies are daily routine in the western press.
“However, a request for evidentiary support should always be welcome in a debate.”
Certainly. As I understand it Jon H clearly stated his concern about clumping. He argues that there are probably areas where obvious depopulation has occurred if the study is correct. I wouldn’t be surprised if such areas have been reported, and people wanting to argue with him could simply cite to such reports.
Anyway, I’m sure the study design considers issues of clumping – the last one explicitly took Fallujah into account as I recall. And probably does something sophisticated in case households are unreachable after say ethnic cleansing.
If their data is questionable, the statistics are useless.
Do you have grounds, rather than just suspicions, to think their data is questionable? Please refer to the actual study.
“Please refer to the actual study.”
Please refer to his actual argument.
“I’m not at all sure why people (like Jon H above) have a gut feel that 600k excess deaths (or between 2 and 5% of the population) in 3.5 years of war, civil war, general chaos and banditry is too high.”
Someone remind me what the #s were in the US civil war, or in the breakup of Yugoslavia, or whatever. Some level is going to be too high – would you blink at 1k/day? Anyway, it’s not crazy to have a feel for the numbers, and it’s not unusual for such a feel to be off by x2 in a complex underreported situation.
As I understand it Jon H clearly stated his concern about clumping.
Jon H has stated a concern that has not so far been backed up by anything concrete. The onus really in on him to show if this concern is worth our attention.
Please refer to his actual argument.
If he makes one. So far, it’s just musing.
“Doing the very crude arithmentic brings up a total of about 475 violent deaths per day on average across the country. Does this seem so far out there?”
Excess deaths, right? 475 violent deaths in excess of what was happening before. The reason I find it a little hard to believe is that the rate of civilian deaths in Germany during WWII was 20 in 1,000 measured over the whole war. The rate implied by the Lancet report is 27 in 1,000 thusfar.
Sebastian – the Lancet study counts all excess deaths, not just civilians, so the Germany comparison doesn’t work. There has been a fair amount of media misreporting on this point.
As I understand it Jon H clearly stated his concern about clumping. He argues that there are probably areas where obvious depopulation has occurred if the study is correct. I wouldn’t be surprised if such areas have been reported, and people wanting to argue with him could simply cite to such reports.
I’m not sure that’s a strong argument on two counts. One, finding such depopulation is dependent on accurate reporting through media sources, which is one thing not available right now. Two, these “criticisms” are detached from actual methodology–I’d find them more convincing if they were connected to actual mechanisms used in the study. Hence the request for evidentiary support that would be welcome.
There’s a Bizarro World cleek?
For those interested, a quick list of places to look on issues relative to the Lancet Study:
Two other important studies are “Annual Mortality Rates and Excess Deaths of Children Under Five in Iraq, 1991-1998” by Mohamed Ali, and “Morbidity and Mortality among Iraqi Children from 1990 through 1998: Assessing the Impact of the Gulf War and Economic Sanctions” by Richard Garfield with George Lopez and David Cortright. These can be found via the first several links below.
Joan B. Kroc Institute for International Peace Studies http://kroc.nd.edu/ocpapers/abs_16_3.shtml
CASI [check sources at bottom of page] http://www.casi.org.uk/
Center for Population Studies (University of London) http://www.lshtm.ac.uk/cps/public/index.html
Fourth Freedom Forum http://www.fourthfreedom.org/Applications/cms.php?page_id=7
WHO sources http://www.emro.who.int/iraq/Information_Resources.htm
“Mortality Before and After the 2003 Invasion of Iraq” Les Roberts, et.al.http://www.zmag.org/lancet.pdf#search=%22iraq%20%22morbidity%20and%20mortality%22%22
“Effect of the Gulf War on Infant and Child Mortality in Iraq” Alberto Ascherio, et.al. http://www.scn.org/ccpi/NEJM-24sep92.html
Mortality sources http://www.gulflink.osd.mil/gwv_bib/mortality.html
Harvard School of Public Health http://www.hsph.harvard.edu/press/releases/press06292005.html
“A Hard Look at Iraqi Sanctions” http://www.thenation.com/doc/20011203/cortright
“The Return of the Body Count” http://www.tomdispatch.com/index.mhtml?pid=2709
“Sanctions Have an Impact on All of Us” Denis Halliday speech http://www.accuracy.org/article.php?articleId=46
General:
Global Policy Forum http://www.globalpolicy.org
Global Security http://www.globalsecurity.org/
PBS Frontline http://www.pbs.org/frontlineworld/stories/iraq/
The two sites below have other relevant articles:
PubMed Central http://www.pubmedcentral.nih.gov/
Findarticles.com http://www.findarticles.com/
‘Please refer to his actual argument.’
“If he makes one. So far, it’s just musing.”
He made a simple, clear argument. If you can’t refute it, you’re just carping, perhaps because you like the study’s results, perhaps because you’re not willing to read the damn thing and see what they have to say about his point – because as hilzoy‘s post makes clear, they can’t be so dumb as to have not dealt with such an obvious problem (and the problem I noted about ethnic cleansing).
gwangung: ‘Two, these “criticisms” are detached from actual methodology’
Adding scare quotes doesn’t strengthen your argument, which is either wrong or beyond my comprehension. He said their methodology has a clumping bias – if you want to argue with him, cite the study, or cite basic epidemiology.
I don’t his argument holds up, but sheesh. Read SH and togolosh‘s exchange above for how to proceed.
There’s a Bizarro World cleek?
she’s a portly little Chinese woman who loves to sing and dance. she makes her living preparing dog meat for the market. she’s never seen a computer.
i am none of those things.
otto, that’s a post’s worth of material there, but I suggest you do a bit of summarizing and linkifying.
“Sebastian – the Lancet study counts all excess deaths, not just civilians, so the Germany comparison doesn’t work. There has been a fair amount of media misreporting on this point.”
I don’t understand how that changes things. The Lancet report pretty much just calls everyone a civilian.
Sebastian:
togolosh:
Sebastian:
If wikipedia is to be believed, the comparable figure for German total deaths during WW II is 108 in 1000.
Rilke: [Jon H] said their methodology has a clumping bias
Yes. And though he has been asked to do so several times, he has not yet cited the part of the report which led him to think this. Without that citation, Jon doesn’t have an argument.
SH — No, if you read it they make it explicit that they are not attempting to distinguish between civilians and non-civilians.
Rilkefan —
He said their methodology has a clumping bias – if you want to argue with him, cite the study, or cite basic epidemiology.
The thing is, that he raised that argument without citing it to anything. I don’t know from serious statistics, but the sort of clumping bias he’s talking about, if a genuine worry, would mean that the methodology was fundamentally flawed. Given that the study was peer reviewed and published in a reputable journal, I don’t see any reason to engage with arguments of the form “Maybe the methodology doesn’t work at all, and the whole study is nonsense,” unless they’re made by someone who at least purports to have a full understanding of the methodology and what’s wrong with it.
Like many people, I started wondering how tall Bush’s “pyramid of skulls” would be. The answer is taller than the White House.
Jes: “he has not yet cited the part of the report which led him to think this”
Ok, it’s page 1. Now refute the argument.
LB, as stated above I entirely agree with you, except that in my field at least if someone says “We measured x using instrument I and got result r+/-error”, and someone replies “Instrument I is subject to bias b”, people don’t say “The paper’s authors are from the UofC”, they say, “See section 4 where the bias is controlled for using technique T”. By the local standards it’s fine to make your argument, as I do above – but the people saying “Cite the paper or you have no argument” above are as far as I can tell not willing to confront the issue for whatever reason.
The Lancet report pretty much just calls everyone a civilian.
Considering that Bremer disbanded the Iraqi Army, pretty much everyone is either a civilian or a traitor.
Like many people, I started wondering how tall Bush’s “pyramid of skulls” would be. The answer is taller than the White House.
Don’t tell Bush that, he’s shooting for the Washington Monument.
SH – the point is that the number you cite is more restrictive – actual civilians.
I think it’s probably fair to say that Iraq was likely going to have a civil war sooner or later, and so many if not all of those 600k +/- 200k deaths were just attributable to the untenable pre-war situation.
Ok, rilkefan I see what you are saying.
Rilkefan-
But what Jon H said wasn’t comparable to “Instrument I is subject to bias b.” It was much more along the lines of “I, someone with no claim to know anything about epidemeology or stats [I don’t know what Jon H’s credentials are, but he hasn’t made any such claim or an argument in any level of detail that would suggest he knows what he’s talking about], look at this peer reviewed paper in a reputable journal and say that the validity of the methodology isn’t obvious to me, so I doubt the result.” Maybe he’s right–I don’t have the statistical knowledge sufficient to know other than by trust in the reputation of the Lancet’s process that the study was validly done.
But the way Jon H raised the issue isn’t worth more than a “Whatever”. If the methodology is fundamentally flawed like that, someone who actually understands it will point it out now that the study is public. There’s no point whatsoever in having an argument here about a critique that neither the maker nor any other participant in the thread is competent to analyze.
Regarding questions about the figures in the recent Lancet report, do not rely solely on this study. Use it along with the other studies that I mentioned above (as well as others located elsewhere in the links provided). From looking into this over the last decade, my general conclusions are (approx. numbers): 1) A conservative estimate of Iraqi civilian casualties from sanctions and coalition bullets and bombs: 3/4 million (1 million is not out of the range of probability). 2) Total civilian and military deaths directly related to the war: 1 1/3 to 1 1/2 million. 3) Some estimates run as high as 2+ million, but I find these figures difficult to support based on the info I have seen.
Note:
Much work has been done on infant mortality since 1990. Less has been done regarding war-related deaths among the elderly, but this is also a group highly prone to suffer from the impact of war (if you are a sixty-five year old diabetic and you can’t get insulin due to sanctions, your days are numbered).
As a percentage of the population, an equivalent number of deaths in the US over the 16 years of the Gulf War would be around 15 million.
Do not take my word on this issue. Check out the sources yourselves.
Estimates for total number of civilian casualties killed by Saddam during the entire period of his leadership, that I have seen, run from 1/4 to 1/3 million (these are not conservative estimates). We must destroy the village to save it . . .
so many if not all of those 600k +/- 200k deaths were just attributable to the untenable pre-war situation
rilke, i have never read anything from you that I either disagree more or believe to be factually false.
First, it assumes that there was no possible occupation plan that would have reduced the level of violence.
Second, it appears to wash the hands of the US from any responsibility for all the accidental / collateral deaths it has caused, not to mention the deliberate deaths of Iraqis who would have never fought each other but were willing to die fighting an occupier.
Third, it treats the actual victims of US mistakes as fungible units compared to some hypothetical civil war.
“They would have died anyway after the fall of Saddam” is a really rotten thing to say. I’m actually shocked.
He made a simple, clear argument. If you can’t refute it, you’re just carping, perhaps because you like the study’s results
Um, no. I can’t refute it because it was done in a vacuum. It’s a simple, clear argument which means nothing if it’s not attached to the research in question.
Moreover, if it’s not attached to the research in question, the carping seems to be more on HIS part. It’s not a knowledgeable criticism. If he has a problem with the methodology, then it’s HIS job to be specific in his criticism.
I see this all the time in dealing with creationists. They’ll bring up all sorts of arguments that sound valid on their faces, but are not applicable because they aren’t relevant—they’re criticisms with no teeth. If it’s a bad tactic for creationists, then it should be a bad tactic for everyone (not that I haven’t been guilty of it occasionally, but if it’s a bad tactic, it’s a bad tactic).
“There’s no point whatsoever in having an argument here about a critique that neither the maker nor any other participant in the thread is competent to analyze.”
There goes the blogosphere.
Jon H argued the following:
That depends on the distribution, doesn’t it? If randomly killing every 50th spherical iraqi of uniform density, you’d probably not notice.
But that’s not how it happens in real life. It’s more likely that the dead will be clustered around the locations of insurgent activity, firefights, and ethnic or sectarian enclaves targeted for purging by their enemies.
Which, incidentally, could lead to an exaggerated total dead. If you did a similar survey in the US in the 1860s, and wound up surveying people near civil war battlefields, you might well end up with an exaggerated total when extrapolating to the entire US population.
which is what I assume subsequent posters refer to as “clumping”.
This is indeed a concern, but I don’t think Jon H has thought it through carefully. Yes, because the study samples from particular blocks, the responses are likely to be slightly correlated with each other if excess deaths are also geographically clustered. But this does not mean the estimates in the study are over-estimates.
Indeed, they are more likely to be underestimates, since it is more likely the sampled blocks will miss the highest concentrations of excess deaths, rather than include a disproportionate number of such blocks.
An analogy may help. Suppose I have a robot that randomly throws darts at a dartboard—randomly meaning the robot always hits the dartboard, but is not aiming for a specific point on the board. Suppose that I run one trial, in which the robot throws 10 darts.
It is unlikely the robot will get any bullseyes, and suppose that on this first trial, the robot gets none. If I total up the points won by the robot, I will thus get a relatively low score. But if I run the trial over and over, eventually, some of the trials will include bullseyes, and the robot’s average score will rise.
In this example, because of the clustering of points around the bullseye, and the geographic sampling of points conducted by the robot, the first trial turned out to be an underestimate of the average points one would receive from random dart throwing.
It is unlikely, but possible, that the robot will get a bulleye in one of its first 10 random throws. In that case, I would overestimate the average points on the board by a substantial amount.
Is this sampling scheme biased? Not in the formal sense, but from a small number of trials, we are very likely to underestimate the average score a little, while there is a small chance that we will overestimate the average score significantly. Increasing our sampling size, and excluding outliers from the analysis will both reduce the risk of an overestimate.
The first Hopkins study excluded a large outlier on exactly this reasoning. The second study doubled, in effect, the sample size, and the results did not change.
I think we can reasonable assume that clumping is not causing substantial bias in these casualty estimates. And even if we are still worried, we should recognize that this concern does not suggest the Lancet figure are too high: the reverse is more likely.
LB, from my perspective, if people are happy saying “It’s in the Lancet and based on the statistical science of epidemiology, I trust it”, that’s fine – my position is about that. But if all we’re going to do here is note expert opinion and nod our heads… And I suspect just reading the study will provide a refutation of the clumping argument – either it’s accounted for in the central value or (more likely) the uncertainty, or there’s discussion (as with the last study) of how the methodology takes the issue into account.
I can certainly imagine the following: the uncertainty cited depends upon clumping model M, which is based on some smaller data sample in Iraq and broad experience elsewhere. Typically people are conservative in estimating uncertainty, but who knows, maybe model M has a real problem in being applied here and the range should really be +/- 300k. And then Jon might be satisfied with the lower range, which (if we’re talking one sigma) will be _too high_ 16% of the time. The fact that the above discussion has proceeded without clarifying that simple point (unless I missed it) is sad.
And I suspect just reading the study will provide a refutation of the clumping argument
It’s on page 1.
And then Jon might be satisfied with the lower range, which (if we’re talking one sigma) will be _too high_ 16%
Have you read the study? The range is a 95% confidence interval. So 2.5%, not 16%.
“First, it assumes that there was no possible occupation plan that would have reduced the level of violence.”
Well, I think I would have opposed nearly any occupation plan, and I’ve not given Bush any credit for actual humanitarian motives, so that doesn’t help me from a moral perspective.
As to the rest, it wouldn’t be the first time I was wrong, and I’m the first to admit my grasp of moral reasoning is worse than tenuous (is it ok to kill someone to save two or n people? hilzoy says [I think] “no way” – I dunno.)
If I understand the clumping argument correctly, the comment by mss takes care of it. With cluster sampling, the chances are that you will miss the hotspots where mortality is heaviest. But if you do hit one, you get a wildly anomolous cluster – that’s what seems to have happened in the first study, where the Fallujah cluster was so untypical that it was simply excluded from the headline figure of 100,000 excess deaths.
The fact that the above discussion has proceeded without clarifying that simple point (unless I missed it) is sad.
Well, my point is that it’s more on Jon to be providing evidentiary support than for others to be trying to defend the study (along the lines of, if you make the argument, you should be prepared to support it). Debate on issues like this is cleaner and more focussed if all the arguments are attached to something real and concrete.
Man I hate statistics.
Last a response to Francis above.
LB – no, not yet, supposed to be getting some work done.
spartikus: “It’s on page 1.”
No, it’s not. One would expect to find it on page 3 I think, but they seem to have gone for the mumble-mumble approach there (and they bootstrapped, which maybe is ok for professional statisticians but makes me leery from my attempts at it).
There’s just one small part of this I don’t understand: if in 92% of 87% of cases, they got death certificates, surely there are many missing persons for whom death certificates could not be obtained. But, then again, if that were so, interviewees would know about them and report them.
And how on earth, going door to door, did they actually see death certificates 80% of the time??
Who is issuing these death certificates? Why can’t we count their numbers?
but they seem to have gone for the mumble-mumble approach there
This is really uncalled for, isn’t it? You appear to be characterizing the paper as evasive because it doesn’t explicitly address a critique made in blog-comments after its publication. If mss’s comment is accurate, the ‘clumping’ critique is weak enough that there is no reason to expect that it would have been explicitly addressed.
“And how on earth, going door to door, did they actually see death certificates 80% of the time??”
I assume there’s some incentive for having a death certificate – the family is entitled to tax breaks, or exempt from jury duty, or … Probably the only way to turn off a person’s food ration cards (are there still such?) and so forth is to register him as dead, and that’s probably a legal requirement to avoid fraud. Anyway, it brings me back to my original question about incentives for live people to pose as dead.
No, it’s not. One would expect to find it on page 3 I think
That depends on whether we’re still talking about the point Jon H raised, I guess.
Which, incidentally, could lead to an exaggerated total dead. If you did a similar survey in the US in the 1860s, and wound up surveying people near civil war battlefields, you might well end up with an exaggerated total when extrapolating to the entire US population.
Ara: There’s just one small part of this I don’t understand: if in 92% of 87% of cases, they got death certificates, surely there are many missing persons for whom death certificates could not be obtained. But, then again, if that were so, interviewees would know about them and report them.
No doubt. But both this report and the previous report were asking for confirmed deaths. A person who was missing would not be a confirmed death.
And how on earth, going door to door, did they actually see death certificates 80% of the time??
According to the first report, they simply asked to see them, after they asked the household to report the deaths of the household. In some cases they were refused, of course.
LB: “You appear to be characterizing the paper as evasive because it doesn’t explicitly address a critique made in blog-comments after its publication”
No, I just read the stat.an. section, and didn’t get a clear sense of what they did leading to what sort of variation. Did you get a clear sense? Maybe that’s just me coming from a quite different field and being in a hurry – maybe there’s another paper coming out somewhere with a fuller explication, or one is just supposed to ask the authors – but the above is independent of the clumping/outlier question.
preview – esp. when fielding phone calls:
LB: “You appear to be characterizing the paper as evasive because it doesn’t explicitly address a critique made in blog-comments after its publication”
No, I just read the stat.an. section, and didn’t get a clear sense of what they did leading to what sort of variation. Did you get a clear sense? Maybe that’s just me coming from a quite different field and being in a hurry – maybe there’s another paper coming out somewhere with a fuller explication, or one is just supposed to ask the authors – but the above is independent of the clumping/outlier question.
rilkefan & spartikus:
The clumping argument is not addressed in the paper because the same authors discussed it in their 2004 paper,and journals are touching about babbling about things you’ve already said.
From page 6 of the 2004 paper:
Second, as Spiegel and colleagues documented in Kosovo,21 there can be a dramatic clustering of deaths in wars where many die from bombings. The cluster survey methodology we used may have, by chance, missed small areas where a disproportionate number of deaths occurred, or conversely, selected a neighbourhood that was so severely affected by the war that it represents virtually none of the population and thus has skewed the mortality estimate too high. The results from Falluja merit extra consideration in this regard.
rilkefan:
they seem to have gone for the mumble-mumble approach there (and they bootstrapped, which maybe is ok for professional statisticians but makes me leery from my attempts at it).
The tone of this comment really irks me. This is a scientific article and is perfectly comprehensible by people with appropriate training. I would have liked a bit more detail, but it would have been of the “equations and more jargon” sort. This paper is unfortunately not the place for a long discussion of methods starting from the ground up.
I have not idea what your beef with bootstrapping is (for others, bootstrapping is a perfectly legitmate, widely used technique for calculating standard errors when analytical techniques are unavailable or involve unwanted assumptions). And the authors of this study used it as a non-parametric robustness check of an assumption of proportional mortality rates across samples, so you could ignore it altogether (though I don’t see the original CIs anywhere in the article).
My point about the death certificates was that on the one hand, 92 percent of the time when the survey team asked for a death certificate as confirmation they got one, but on the other hand if you go by the official death toll statistics that are put out from time to time, the violent death toll in Iraq is probably in the tens of thousands. Which leads me to think somebody is lying. Who is lying I don’t know. Maybe lots of people. I take for granted that the US and Iraqi governments lie constantly on this subject–the issue here is whether they are lying by a factor of ten or so on the level of violence in Iraq. Bush more or less endorsed the Iraq Body Count figure some months ago (when it was around 30,000), which is strong evidence that the figure is much too low, but leaves open the question of whether the true figure is more than ten times higher.
Anyway, it’s outrageous that we don’t know. There’s no reason why the US government couldn’t undertake a serious effort to find out how many Iraqis are dying and publish it and since they don’t, the press should be constantly hounding them on this issue. Oddly enough, the press doesn’t. I have to say I wouldn’t believe any study conducted by our government anyway–I think we’d probably be told that in the three years of occupation US forces had accidentally killed about five civilians and run over someone’s goat. But other groups could run the study and the US could provide them with protection.
If this study is right, then the US itself has killed about 30 percent of those 600,000 people. 180,000 deaths, coincidentally, is about the same as the maximum estimate for Saddam’s genocidal campaign against the Kurds. (I have no idea how we “know” any of the statistics for Saddam’s atrocities, btw.)
I think it’s probably fair to say that Iraq was likely going to have a civil war sooner or later, and so many if not all of those 600k +/- 200k deaths were just attributable to the untenable pre-war situation.
Who are you? And what have you done with Rilkefan?
How about this question: Assuming the lowest casualties from the Lancet report (or split the difference between the IBC figures and Lancet’s lowest number, or any other similar method to approach conservative estimates), what do these deaths mean, what is their impact on the war, public (Iraqi, US, world) opinion, the future of Iraq, etc.?
The clumping argument is not addressed in the paper
I’m probably a victim of my own brevity and extrapolation by others of what Jon H “actually meant”. I’m happy to be corrected, though.
Donald Johnson,
-“Anyway, it’s outrageous that we don’t know.”
Indeed. But we can get in the ballpark. The sources I listed upthread I just tossed out from a quick look at a bit of my files and “favorites” and only scratch the surface of what’s available.
-“(I have no idea how we “know” any of the statistics for Saddam’s atrocities, btw.)”
Typically, the figures relating to casualties under Saddam show up in right wing sources, though some work has been done by NGO’s and the like.
(Sorry I’m a little rusty on this subject, but it’s been a year or so since I last really looked into it.)
Here’s a companion paper someone linked to over at Tim Lambert’s blog–
http://web.mit.edu/CIS/pdf/Human_Cost_of_War.pdf
(I misplaced my html guide, so you’ll have to paste).
It might possibly answer some of the questions people have raised, maybe including mine. I haven’t read it yet.
“Who are you? And what have you done with Rilkefan?”
Perhaps there’s an issue with “many if not most”? I take that to mean “a large fraction but probably not a majority” – if that’s not English, oops.
Otherwise, if you considered Iraq before the invasion (perhaps one of the great strategic blunders since Athens trying to take Syracuse), and someone told you that Saddam was going to die in five years leaving the country in the hands of his even more awful sons, what would be your expectation for the death toll there over the following five years? You might average Congo, Sudan, Bosnia and come up with a horribly high number. Ok, so 30% are on our ledger straight up, and maybe we’ve made things relatively worse than the horrible natural scenario – but I don’t see a realistic future after the Florida recount and 9/11 leading to a Saddam-status quo outcome for Iraq.
The bad part is that I doubt we’ve gotten near the really deep awfulness yet.
Donald Johnson, that’s good, thanks. There’s an appendix on how so many deaths could have occurred that is apropos.
mss: I would have liked a bit more detail, but it would have been of the “equations and more jargon” sort. This paper is unfortunately not the place for a long discussion of methods starting from the ground up.”
I wasn’t asking for such a discussion, just the numbers to evaluate the analysis. I think by the standards of my field that section is woeful – maybe they plugged their excel into stata and there wasn’t much more to report than the output, but it seems really skimpy to me.
As far as bootstrapping is concerned, as noted I know it’s a real technique but I got the impression from trying it that it’s a good way to make long tails.
I agree that their discussion of inference is very skimpy, and contains a rather odd emphasis on software rather than actual methods. An equation showing their model would be nice, and would be customary in my field (especially since there are a variety of log-linear models). However, like most articles I’ve seen from medical journals, this article omits such things. On the other hand, the discussion of sampling seems very detailed, and surely that is where the emphasis should be for this study, since that is where the “action” is in their estimates.
Back from work — Daniel at CT has a good summary of the cluster sampling thing. (Part of it recapitulates something that mss said above, but hey, it bears repeating.) Having said that there are two problems with the criticism of cluster sampling, he discusses them in turn:
Over half a million …Beyerstein at Majikthise has a public health blogger detail the methodology in layman’s terms
Donald Johnson: if there is an issue of lies, I always think that the position requiring the fewer sources of lies is likely true. Always seems to me much more plausible that a handful of government bodies are not telling the truth, rather than thousands of people who are not affiliated and have not coordinated with each other.
This excludes entirely the issue of whether either of these parties has an incentive to lie.
In my own mind, there is no grounds for skepticism about the statistics. The only real grounds for skepticism might be in the data gathering. Having said that, considering just how chaotic things are, I find it awfully hard to believe that IBC numbers aren’t low by a couple of multiples. It’s not hard to imagine that only one out of five or one out of 10 deaths gets reported either by a journalist or by a morgue.
600,000 out of a population of 24 million is hardly depopulation, in the sense that you’d expect to see empty streets.
I realize the point you were trying to make here, but 600,000 out of 24M is 2.5%. That percentage of the US population is about 7M people.
That’s a lot of people. It might not rise to “depopulation”, but I daresay we would sit up and take notice.
Thanks –
“The bad part is that I doubt we’ve gotten near the really deep awfulness yet.” …rilkefan
I am not sure about Iraq. About 20% of the Arab Sunni population is gone and we may be peaking. Can’t say if Shia on Shia violence will increase dramatically, or Kurds, or whatever.
America has not yet peaked on the ugliness of the war yet. I will just note that I find rilkefan’s comment of 7:36 deeply troubling, not meaning much particularly critical, but just mostly a sadness at what I think is coming in the discourse.
I care more about Riverbend than about the vast majority of Americans.
I realize the point you were trying to make here, but 600,000 out of 24M is 2.5%. That percentage of the US population is about 7M people.
That’s a lot of people. It might not rise to “depopulation”, but I daresay we would sit up and take notice.
Well, I think they do…it’s just that they may not consider it that unusual in times of such….civil unrest….
Wow, when I’ve saddened bob, it must be a red-letter day.
More saddening.
I wish that those who supported/support the war would state clearly what kind of evidence would lead them to change their minds. I’m happy to say that, if Iraq becomes a liberal democracy in the next five years, with a low level of violence and effective protections for the rights of women and minorities (of whatever sort have relevant conflicts with whatever majority ends up running the place), I will acknowledge that my opposition was misplaced. (I think that’s pretty generous, given the massive bloodshed that’s already happened.)
What kind of statistical or journalistic or other evidence would make people on the other side change their minds?
While the report cites an overall average of 2.5% of the population, clearly the percentage is quite a bit larger in some provinces (Anbar, Diyala, Ninewa, Samara). The clumping can be seen on the map, but I missed the discussion of the relative incidence by geographic region.
The data does seem to suggest that for the most impacted states the loss of life is moving to the catastrophic level.
Evidence of depopulating is there for people to see, in accounts like Riverbend’s most recent post and this heartbreaking blog from a 15-year-old Iraqi girl, whose family left Bagdad for the United Arab Emirates earlier this year. These deal with people moving rather than getting killed, of course, but the accounts (and many others like them) do indeed show a country suffering a very noticeable drop in population.
The US media aren’t, by and large, covering the kinds of places and people that would make the drop obvious to us, too, but that’s not the Iraqi people’s fault.
Summer of Goodbyes …Bruce’s link doesn’t work
BTW, a second volume of Riverbend’s blog published by The Feminist Press at CUNY came out about a month or so ago. Since women’s perspectives on this whole ball of wax can be a liitle scarce, some of you may like “I, Nadia, Wife of a Terrorist” by Baya Gacemi as well.
While the report cites an overall average of 2.5% of the population
Just for the purposes of comparison, I was curious what the figures for Vietnam were, and I found this
The Agence France Presse (French Press Agency) news release of 4 April 1995 concerning the Vietnamese Government’s release of official figures of dead and wounded during the Vietnam War.
HANOI (AP) – April 4. Cinq millions de morts: 20 ans apregraves la fin de la guerre du Vietnam, le gouvernement de Hanoi a reacute veacute leacute, lundi, le bilan d’un conflit dent le nombre de victimes avait eacute teacute minore a l’eacutepoque pour ne pas affecter le moral de la population.
Selon Hanoi, il y a eu pres de deux millions de morts dans la population civile du Nord et deux autres millions dans celle du Sud. Quant aux combats proprement dits, les chiffres sent d’un million cent mille militaires tueacutes et de 600.000 blesseacutes en 21 ans de guerre.
Ce dernier bilan comprend a la fois les victimes de la guerilla vietcong et les soldats nord-vietamiens qui les eacute paulaient. Les preacute ceacute dentes estimations de source occidentale faisaient eacute tat d’un bilan de 666.000 morts parmi Ies combattants Vietnamiens.
Translation
The Hanoi government revealed on April 4 that the true civilian casualties of the Vietnam War were 2,000,000 in the north, and 2,000,000 in the south. Military casualties were 1.1 million killed and 600,000 wounded in 21 years of war. These figures were deliberately falsified during the war by the North Vietnamese Communists to avoid demoralizing the population.
End Translation
Note: Given a Vietnamese population of approximately 38 million during the period 1954-1975, Vietnamese casualties represent a good 12-13% of the entire population. To put this in perspective, consider that the population of the US was 220 million during the Vietnam War. Had The US sustained casualties of 13% of its population, there would have been 28 million US dead.
LJ,
Bang! Thanks, LJ, I was just going to ask about a comparison with Vietnam!
Also, 2.5% only covers from spring 2003, not the last 16 years of war for Iraqis.
BTW, I just bought what I think is one of the best documentaries on the Vietnam War: “In the Year of the Pig.” Highly recommended. In it there is about a five-minute sequence with Gen. Mark Clark and Gen. Curtis LeMay where LeMay is lecturing an audience on the advisability of bombing everything in North Vietnam flat including agricultural works (“every work of man” . . . “so long as there are still two bricks stuck together”). It reminded me of the Coalition bombing campaign in Iraq since 1991, which similarly conflated military and civilian targets to the point of indistinction, despite the PR spin to the contrary. You don’t get casualty figures like Vietnam and Iraq by only hitting ammo dumps and motor pools. And of course that’s the point.
realize the point you were trying to make here, but 600,000 out of 24M is 2.5%. That percentage of the US population is about 7M people.
That’s a lot of people. It might not rise to “depopulation”, but I daresay we would sit up and take notice.
More than 800,000 Iraqis have already fled to neighboring countries. Did you sit up and take notice of that?
Cinq millions de morts: 20 ans apregraves la fin de la guerre du Vietnam, le gouvernement de Hanoi a reacute veacute leacute
Those ‘grave’ and ‘acute’ are funny.
Heh, I didn’t even look at the French, but saw that it was a lot longer, so I bunged the whole passage in. That’ll teach me.
The frightening thing is the vietnam figure seems to be for 21 years. We’ve gotten to 3% in Iraq in only 5 years.
More than 800,000 Iraqis have already fled to neighboring countries. Did you sit up and take notice of that?
Yes. I think I obscured my point by responding to Liz’s comments. The statement I’d really like to respond to was John H’s:
The number does seem wildly high. Those are Rwanda-class numbers.
I haven’t seen anything that suggests the country has been depopulated to that extent.
Depopulation is, hopefully, not required in order for the situation to be considered a disaster.
A death rate of 2.5% still leaves 97.5% alive, but if it happened to us I think we’d correctly call it a catastrophe. It’s a lot of dead people, and a lot of destroyed lives.
Here’s the thing. This is what war is. War is death, destruction, mayhem, havoc, and obliteration.
War is not “shaking things up”, it is not “creative chaos”. It is not wiping the slate clean in order to make a fresh start. It is not the “birth pains” of anything.
War is death. It is the death of people, people who are known and loved. It is the death of homes, livelihoods, communities. It is death.
The controversy about the goodness of the statistical methods in this report is an interesting one, and probably a useful one. It’s important, very important, to make decisions based on accurate information.
But the obvious subtext to the debate has little to do with statistical methodology. It has to do with whether our adventure in Iraq was, is, or will somehow turn out to be wise, useful, or beneficial to the Iraqis or anyone else.
As far as I can make out, the idea of going to war with a country so you can turn it into a different kind of country, one better suited to your interests, is nuts. Nuts, as in willfully ignorant of history and of human nature. Close your eyes, stick your fingers in your ears, chant “la la la” as loudly as you can, and assume the world will look like you want it to look when you stop. That kind of nuts.
And that appears to be our foreign policy.
We went to war with Iraq. Some Iraqis were happy about it. Some weren’t, and aren’t, and they have decided to fight back. Other enemies of ours have decided that, since we’re in their neighborhood, they’ll join in the fun as well. Other folks in the neighborhood, with agendas of their own, stir the pot a bit to see if they can steer things to their own advantage.
As a result, Iraq is turning into a killing field. No surprise there, that’s what war is. If we thought something different would happen, we were wrong.
It saddens, and sickens, me to know how carelessly all of this was done.
Thanks –
War is cheap for America, the vast majorities of the people being killed aren’t Americans, the cities being destroyed aren’t American and the environment being damaged isn’t American.
When the time comes for us to leave, we will leave and go on about our lives as if nothing had ever happened, except for a few nuts who’ll go on to write books about how we were betrayed by a bunch of leftist and prepare the ground for the next major war.
If there was a real chance that major cities on either coast were likely to suffer the consequence of war, we ‘d think about it twice before starting one.
But war is cheap, and we rarely get to pay the consequences of starting them, hell even the supporters of the wars we start don’t even have the basic decency of joining the Armed Forces and fighting in them.
Les Roberts was on Democracy Now this morning. He said reporters could easily verify or refute the Lancet study. Just go to the gravediggers in 4 or 5 villages or towns and ask them to compare burial rates in 2002 with the more recent past. The Lancet paper says death rates have quadrupled, so one doesn’t need a subtle statistical analysis to see an effect like that.
If professional gravediggers are a bottleneck. I don’t know that they are or aren’t, but it isn’t obvious to me that, for example, people aren’t burying their own dead.
Right, Lizardbreath, but maybe you’d see a doubling rather than a quadrupling. Or maybe you wouldn’t. It would not be a conclusive test, though I’ll point out that Les Roberts is one of the authors and the one most visibly defending this study and the previous one in public, so he’s not trying to discredit himself. He’s offering a good faith way for people to do an independent check.
Which is what all of us American and British citizens should be demanding of our government. I get the impression (mostly from the Lancet critics) that people don’t want an independent investigation that would settle this issue, yet to me this is a moral and ethical no-brainer–as invaders of Iraq we have a moral duty to know how many deaths our actions have caused, directly and indirectly.
Not that I want our government to actually conduct the study–just support independent investigators in every way possible.
“Just go to the gravediggers in 4 or 5 villages or towns and ask them to compare burial rates in 2002 with the more recent past.”
But, look: after you did that, you’d have to extrapolate to the whole country to get the answer, right?
And to justify the extrapolation, you’d need to know that the 4 or 5 villages were actually representative.
So in other words, you have reproduced the study-design of the Lancet study (i.e. selecting random samples and scaling up) except this time doing it *really badly*, with far too small of a sample size.
Anyone pretending there are problems with the Lancet study would just raise the same charges against the grave-digger study: “those villages were unrepresentative!” “The grave-diggers lied!” etc.
What *would* be interesting–though you’ll never hear it–would be to have one of these holocaust-denialists tell you exactly how *they* would design a study to produce the most reliable estimate of excess deaths. Instead of just criticizing all of the time, why don’t they come up with some positive proposals of their own?
Given the constraints of time, barriers to communication, danger in the field, breakdown of centralized reporting, and so on, I think the Lancet people have come up with about as good of a number as we can hope to get for a while. Claims to the contrary are just more negativity from the party of no ideas.
Sigh. Kid bitzer, did you post that not knowing that Les Roberts is one of the Lancet authors?
I thought it was to his credit that he’s trying to get journalists to do their own investigation, half-assed though it would be. Of course it wouldn’t be conclusive, but if several journalists did this and found that, yeah, gravediggers had noticed that burial rates had skyrocketed it would be strong supporting evidence. If they found no effect then it wouldn’t disprove the study, but it would lower the chances of it being correct. In other fields people don’t necessarily accept one study as conclusive, nor do they dismiss it because they “know” better.
The issue is extremely important and there’s no reason (yes, I know Iraq is extremely dangerous) why we shouldn’t be demanding our government sponsor (not control) a similar study on a much larger scale. And if they refuse that’ll tell us something. I think I know what they’ll do.
my response was not intended as a criticism of Les Roberts or of you. My point was simply that this would be extremely weak confirmatory evidence, open to all of the complaints that are being raised against the study itself–plus some more, based on the trivially small sample size.
Sure, it’s to Roberts’ credit that he says “good look for yourselves”. It’s as good an answer as any. And I am happy to join you in asking our government to keep track of its effects.
Except that there would kind of be no point in asking, because, you know, Rumsfeld doesn’t do body-counts and all.
I wasn’t trying to harsh on you, just pointing out the extreme limitations of this method of follow-up. E.g. this:
“If they found no effect then it wouldn’t disprove the study, but it would lower the chances of it being correct.”
strikes me as wrong, exactly because it would be so easy for a study-skeptic to cherry-pick a few villages that have escaped relatively unscathed and claim that they have debunked the study. Whereas in fact it would only show that those villages were unrepresentative.
As far as I can make out, the idea of going to war with a country so you can turn it into a different kind of country, one better suited to your interests, is nuts. Nuts, as in willfully ignorant of history and of human nature. Close your eyes, stick your fingers in your ears, chant “la la la” as loudly as you can, and assume the world will look like you want it to look when you stop. That kind of nuts.
And that appears to be our foreign policy.
Exactly, which was one of the prime reasons for opposing the war from the start (and my biggest reason, based on my own knowledge of the histoy of warfare). This central premise in favor of war was nuts.
And whatever belief people had in this possibility should be irrevocably dashed by now, with so much post-war incompetence and unending carnage.
Now, the current debate concerning Iraq policy has more to do with dealing with the irrational beliefs of the war supporters (who cannot face the consequences of the worst American strategic blunder since when?) than actual policy on what to do next.
I suspect the Lancet study is correct in estimating that total war related Iraqi deaths is between 400,000 to 800,000. Let your own personal bias select the number in that range.
So two points. If you just refuse to believe the Lancet study, what is the actual number, and based on what methodology? Anyone who rejects the study but will not engage in meaningful discussion of this question is simply being willfully blind. And that is the current policy of the Bush administration — refusing to even address the question.
Two, if rational thought requires accepting these big numbers, how can any future prediction of a semi-decent outcome for the war be based in reality? Carnage on this scale unleashes its own terrible monster. There are no mulligans in fighting wars.
“If you just refuse to believe the Lancet study, what is the actual number, and based on what methodology? Anyone who rejects the study but will not engage in meaningful discussion of this question is simply being willfully blind.”
Well as far as “excess deaths” goes, a strong methodological objection is that you shouldn’t trust the pre-war Iraqi death rates. As it stands, the reported death rate was dramatically below the US, UK, Germany, France, Brazil, and Mexico. (8.1,10.3,10.3,9.0,9.3,10.1 per 1000) {I used the easiest to find rates over the past four years for each country, which isn’t strictly sound but considering the lack of huge disasters in any of the reference countries I suspect isn’t the cause of an error}. The reported Iraq number appears to be around 6.4.
When I was looking at the numbers originally I suspected it might be a function of age, which is why I added two relatively younger countries–Mexico and Brazil.
So under the sanctions regime (which was allegedly killing thousands) Saddam had apparently a drastically better death rate than the US, UK, Germany, France, Brazil or Mexico. That is drastically as in 20% better than the US and 35% better than the UK and Germany. (Am I doing that calculation right? Normally I’m math-confident but that seems ridiculous.)
Well as far as “excess deaths” goes, a strong methodological objection is that you shouldn’t trust the pre-war Iraqi death rates.
But the study compares post-war Iraqi deaths as reported by surveyed households with pre-war Iraqi deaths as reported by the same surveyed households. It doesn’t depend at all on officially reported pre-war Iraqi death rates.
Sebastian: So under the sanctions regime (which was allegedly killing thousands) Saddam had apparently a drastically better death rate than the US, UK, Germany, France, Brazil or Mexico.
The sanctions were most lethal to infants and small children (as NGOs were pointing out from 1991 onwards) – dysentery, for example, which an adult may well survive, can easily be lethal to a child under five. As the authors of the Lancet study note (in, I think, the first report) infants and very young children were the age group least likely to have death certificates and most likely for their deaths to be underreported, especially if several years had passed. I have no idea if the deaths of children under the sanctions could by itself account for the difference you point out: but a filmed report by John Pilger released in 2000 said that 4000 infants and children were dying each month as a result of the sanctions.
Seb: a very good point, you make, I think. My only comment: weren’t most of the so-called extra deaths attributable to violence? Whatever you might think of health policy in pre-war Iraq, it seems unlikely that what was underreported was an awful lot of tribal and political violence.
The terrifying implication of the study, which people ae not really fessing up to, is that, if true, even those people who are following this as intently as possible have no idea what’s really going on over there.
My point was simply that this would be extremely weak confirmatory evidence, open to all of the complaints that are being raised against the study itself–plus some more, based on the trivially small sample size.
Complaints by whom? The methodology is textbook in epidemiological circles.
One thing I think both left and right should take from this is that we have no idea whatsoever what the consequences of our policies often are, whether they be sanctions or the invasion. That uncertainty ought to be figured into the deliberations much more than it is as of the moment. Instead, we con ourselves with false assurances of certainty and lots of armchair reasoning. Elsewhere, people may or may not be dying because of it.
Just that uncertainty itself is morally objectionable.
You are right, but when I read the report again the numbers they use are even more silly:
Their estimate allegedly independently finds that the pre-invasion Iraqi death rate was 1/2 of the German, French, Mexican and Brazilian rates and 2/3 of the US death rate.
And that should be “Lizardbreath, you are right, but…”
sven–you’ll find no argument from me on this point. The complaints are spurious and meretricious, and if we had not just been reminded to watch our manners, I would say they are intellectually dishonest.
Nevertheless, any of the people who complain about the original Lancet study (thereby showing their ignorance of standard epidemiological methods, as you point out) will also complain about the proposed grave-digger check–and will have *slightly* more reason for complaint, since the proposed number of checks really would be too low to show much.
But given that the point of the study is to measure the change in death rates, does the fact that the method used systematically underestimates the actual death rate undercut its usefulness for that purpose? Unless you’ve got some reason for assuming that the survey method underestimated deaths in the pre-war period and then stopped underestimating deaths in the post-war period, I don’t see that it does.
“A sample size of 12 000 was calculated to be adequate to identify a doubling of an estimated pre-invasion crude mortality rate of 5·0 per 1000 people per year with 95% confidence and a power of 80%,”
context shows that they were using the 5/1000 number for a power calculation, i.e. a preliminary estimate of how large a sample will be needed in order to come up with meaningful results.
The numbers you use in power calculations are not the same as the numbers you use in final calculations, and may be intentionally low-balled or high-balled in order to create a greater margin of error.
(I.e., the thought is “let’s ask enough people so that *even if* the pre-war mortality were as low as 5/1000, our findings would still be statistically robust”).
Power calculations are not actual measurements; they’re more like preliminary estimates to make sure that your measuring stick is calibrated correctly, and that you are using the right scale.
Don’t forget the doublespeak that spews from the Prez:
Reading through the thread I find the discussions about the study’s methodology interesting and potentially useful. But what I really find fascinating is the prevelant (both in this thread and elsewhere) gut-level reaction that the numbers are just too high. (Note that my initial reaction was similar, that 600k excess deaths by violence seemed very high). But, as I asked way upthread, is the headline number really so high on a gut level?
To revisit my crude arithmentic, the study’s lower bound of about 400k excess deaths by violence implies an average of 350 Iraqis being killed per day. The mid-level number of 600k brings that up to 500 per day.
Recent news reports had the Iraq government reporting a daily average of 90 civilians being killed by violence in Baghdad alone in September 2006. No definition of civilian but an obvious inference is that, if the current Iraqi army casualties (plus police perhaps? plus insurgents perhaps?) are included that number will be higher. Add in the rest of the country, some parts of which are reportedly less violent, some parts more and that lower bound number of 350/day suddenly appears to be not that high at all and the 500/day figure at least within reason on a gut level as well.
Obviously there are people with a very strong incentive to dismiss this study out of hand (e.g. George Bush and his strongest supporters feel it makes their decisions and actions look terrible and runs counter to the “things aren’t so bad and are improving” theme). But I think there are also many more people (and especially Americans) who just don’t want to accept that so many people are getting killed. It’s ugly in Iraq and that attention grabbing number brings the ugliness home.
SH,
So are you suggesting that the methodology under-estimates the pre-invasion mortality rate but over-estimates the post-invasion mortality rate?
“But given that the point of the study is to measure the change in death rates, does the fact that the method used systematically underestimates the actual death rate undercut its usefulness for that purpose?”
Sure it does. First, finding such an obviously wrong result has to call the methodology into question. Second, there is no reason whatsoever to assume that the error is evenly distributed over time. A well known problem with retrospective surveys which ask for information from many years before is that they underreport things from the beginning of the period and overreport the end of the period.
This well known methodological problem is magnified when you try to measure ‘excess’ against a baseline which is being determined by retrospective oral interview.
While I can see it for retrospective surveys generally, that seems like an unlikely critique of a retrospective survey of something as memorable as deaths within a household. Do you really think that the odds of recalling that a member of your household died are substantially different for the period 1996-2001 than for 2002-the present? I’d think that accuracy of recall for events of such magnitude would approach perfection over both periods. Does anyone forget the deaths of a household member at all, much less in the past decade?
One thing to remember here: the methodology in this study is NOT ad hoc (or at least not as ad hoc as some people would have you to believe). It’s based on methodology used in other epidemiological studies. The estimates tend to run high, but not overly so, and tend to be more accurate than other methods.
Some of the criticism seems to be in a vacuum, and not connected to its real-world usage, where there have been confirmations.
“Does anyone forget the deaths of a household member at all, much less in the past decade?”
The exact year is rather important if you are trying to estimate an annual death rate.
What does that have to do with systematic underreporting (again, something which seems very unlikely to me in the context of deaths within a household)?
“What does that have to do with systematic underreporting”
You have a cut-off and have to locate the deaths in a particular year if you want to calculate a yearly death rate.
*sigh*. Can we rename this thread? “Arguing about complex statistics between people who know nothing of complex statistics”?
I don’t know much about statistics — just the bare minimum I need for my job. However, I’m familiar enough with statistics methodology to realize that some objections here are rather silly. I mean, they sound good to those who don’t know statistics (IE: Most people), but in terms of actual “people using statistics” they’re ridiculous. It’s like first-year algebra students — fresh from learning about these crazy ‘function’ things — expressing grave doubt that one can easily find the tangent line at any point along that function, or reliably express the area under the curve with a simple procedure.
It sounds magical and crazy to them, but it’s such a solid and unremarkable part of calculus that no math paper is going to spend much time talking about how basic integration works.
Look, laymen have problems grasping the actual true probabilities of the Monty Hall problem. I’ve had people argue it voraciously, confidant that their “gut instinct” was correct. They were wrong — and could be proven wrong with elementary probability math.
A lot of the “objections” to the study are like Joe Average proclaiming the Monty Hall odds are 50/50. He’s utterly, totally wrong — but he doesn’t have the math background to understand how badly wrong he is.
Okay, I think this is going to be repeating something already said upstream, but it doesn’t seem to have sunk in.
Suppose that the prewar rate is being grossly underestimated (because–choose your paranoia: the Lancet people want to exaggerate the effect; Saddam was lying about how many were dying; the villagers asked by the Lancet people just forgot about a lot of deaths before 2003; whatever).
How would that affect the study? Well, the 650,000 is arrived at by subtracting pre-war totals from current totals. So if the pre-war totals were higher, it would make the difference smaller; it would mean that the 650,000 people who are being claimed as *excess* mortality are not really excess at all. Many of them would have died anyway, even if the old pre-war rate had continued unchanged.
But the problem with that proposal is simple: the vast majority of these people (like roughly 12/13) died by violence, “the most common cause being gunfire”.
These are not people who would have just died anyway had the war never happened. These are people who were shot. Some of them by our own troops, some of them by death-squads and militias.
So now if you want to claim that the estimated excess is being inflated by including deaths that would have occurred even in pre-war conditions, you have to claim not only that a few hundred thousand of those people would have died anyway, but that a few hundred thousand of those people would have been shot anyway (or died some comparably violent death).
This just seems to descend deeper and deeper into special pleading. Go there if you like.
“A lot of the “objections” to the study are like Joe Average proclaiming the Monty Hall odds are 50/50. He’s utterly, totally wrong — but he doesn’t have the math background to understand how badly wrong he is.”
Maybe so, but I do have the math background to understand the Monty Hall problem–enough that I never really understood why so many people thought it was a problem.
LizardBreath: Does anyone forget the deaths of a household member at all, much less in the past decade?
Well, LB, put yourself in the position of a member of an extended household. You and your husband and his brothers and their wives and his parents and a couple of cousins and their spouses. During the past decade, every single woman in the household has borne children and the children have often died within a year or two of their birth. Can you honestly say you think you would remember, just on being asked, every infant who died in the household over the past ten years?
I wanted to add: That doesn’t mean the study is necessarily right. It just means no one arguing it’s wrong (at least here) seems to have the relevant background to make an actual case. They’re relying on “There’s two doors! 50/50 odds!” arguments rather than — say — whipping out Bayes and plugging in some numbers.
I freely admit — I don’t have sufficient skills to do anything more than the crudest methodolgy check, and even then I’d probably screw it up.
But I’m going to lean towards the experts and let peer review handle it. Since this confirms the Lancet study — while taking into account the valid critiques (IE: From statisticians and such, not people on a blog) on Lancet — I tend to think they’re on the money.
I suspect people’s “Guts” don’t agree because Americans have no instinctive understanding of chaos or war — civil or otherwise. We blow up other countries. It doesn’t happen to us.
Sure. But your initial assertion was that events in earlier years are systematically underreported compared to later years because people forget the earlier events, or at least that was how I read:
A well known problem with retrospective surveys which ask for information from many years before is that they underreport things from the beginning of the period and overreport the end of the period.
And that’s generically plausible — people do forget things — but not really in the context of deaths within a household. But in a less memorable context, it’s the sort of claim I’d buy without support because it does make sense, people forget stuff.
Now, if I understand you correctly, you’re saying that there’s going to be a systematic tendency for people to get the year of death of members of their households wrong in a fashion that makes it look as though there were fewer pre-war deaths than there actually were (I suppose if they reported post-war deaths accurately, but had a systematic tendency to report pre-war deaths as having occurred before the study period, when they actually occurred within the study period.) And that doesn’t make sense to me — I’ll (in general) buy a tendency to forget things without support, but I’m not familiar with any general human tendency to think things happened longer ago than they did. Do you have any support for this, or am I misunderstanding what you’re saying?
Maybe so, but I do have the math background to understand the Monty Hall problem–enough that I never really understood why so many people thought it was a problem.
Because they saw two doors left, and the possibility to choose between them. Most people viewed that as a coin flip.
The most reliable way to explain it to laymen — without math, that is — is to simply expand the problem. One million doors, one car. You choose a door, Monty throws open 999,998 doors (leaving one closed) and asks you if you want to switch.
Anyone who can’t grasp the benefits of switching at that point — and who has been informed that the car doesn’t change doors — is someone who is NEVER going to understand.
Kid Bitzer, the actual reported death to violence under the study is 55%. An interesting note is that of all the pre-war deaths, only two were reported as violent, one in an explosion. That is a rather shockingly low number in itself.
“I suspect people’s “Guts” don’t agree because Americans have no instinctive understanding of chaos or war — civil or otherwise.”
That is why Jane Galt compared it to other war-torn countries.
What Ara said at 1:10.
But as long as the subject is on the table, thank you Sebastian for making the observation that I was about to make — that your fundamental objection is that “finding such an obviously wrong result has to call the methodology into question.”
Yes, your second objection is perfectly reasonable, but it’s just a Wild-Assed Guess, ultimately based on your first objection. You seem to be suggesting that this particular study is skewed in a way which takes it past the threshold at which the conclusions are no longer useful. But what is that threshold? How did you determine it? Not based on any empirical data, AFAICT.
Let’s grant that pre-war mortality was underreported and post-war mortality overreported (I personally think it must have been, though I’m open to counterarguments).
In order to do that we are postulating a formula f(n) which describes the likelihood that a death which took place n units of time ago would not be reported. Your implicit solution to this (known) methodological problem would be to re-weight the raw data according to f(n), and then recalculate the excess deaths estimate. But you haven’t offered any evidence that f(n) is statistically significant in this instance, other than the “obvious” wrongness of the end result. Science in all its glory, eh?
I don’t have the time to chase down reasonable values for f(n), or the knowledge of statistics necessary to apply them to the raw data, but I don’t see any reason why it couldn’t be done. In the meantime I’m going to accept an argument from authority and defer to epidemiologists who have lots of real world experience with what f(n) ordinarily looks like and whether it has an appreciable impact on a study like this.
“And that’s generically plausible — people do forget things — but not really in the context of deaths within a household. But in a less memorable context, it’s the sort of claim I’d buy without support because it does make sense, people forget stuff.”
See Jesurgislac above, you’re treating it like a US household.
“I’ll (in general) buy a tendency to forget things without support, but I’m not familiar with any general human tendency to think things happened longer ago than they did.”
I’m pretty sure you are wrong about this too, but I sold back my statistics books in college so I won’t immediately be able to help you out. But in my experience, bad events in the past either feel very immediate or well in the past.
“We estimate that as of July, 2006, there have been 654 965 (392 979–942 636) excess Iraqi deaths….Of post-invasion deaths, 601 027 (426 369–793 663) were due to violence.”
601/654 = 12/13.
“the actual reported death to violence under the study is 55%”
Can you explain the discrepancy here?
“In the meantime I’m going to accept an argument from authority and defer to epidemiologists who have lots of real world experience with what f(n) ordinarily looks like and whether it has an appreciable impact on a study like this.”
It seems to me that the epiemiological experts are saying things like:
“Given the conditions (in Iraq), it’s actually quite a remarkable effort,” said Steve Heeringa, director of the statistical design group at the Institute for Social Research at the University of Michigan.”
“This is the most practical and appropriate methodology for sampling that we have in humanitarian conflict zones,” said Brennan, whose group has conducted similar projects in Kosovo, Uganda and Congo.”
They seem to be saying “this is the best we can do” not “this is really good data”.
See Jesurgislac above, you’re treating it like a US household.
That wasn’t sarcasm? Seriously, are you sitting here saying that “Eh, what are the odds Iraqis are going to be able to accurately remember how many family members died in the last ten years, including all the infants? Who keeps track of dead babies?” Jeez, two hundred years ago here dead babies got names, and tombstones, and written into the family bible.
I’m pretty sure you are wrong about this too, but I sold back my statistics books in college so I won’t immediately be able to help you out. But in my experience, bad events in the past either feel very immediate or well in the past.
Mmmm. If you come up with any sourcing for this as a source of systematic error, do point me to it. I’d be interested.
Oooh! Math wankery!
enough that I never really understood why so many people thought it was a problem.
In my case, it’s because the Monty Hall problem turns on a very subtle phrasing which the person who originally told it to me botched. Specifically, it’s required that Monty know where everything is in order for you to want to switch; if he simply opens another door “at random” and it happens to show a goat, the odds remain even and there’s no point in switching.
“Can you explain the discrepancy here?”
Nope, but that is what their table 2 says, they must be doing something weird with the “excess” calculation.
“Eh, what are the odds Iraqis are going to be able to accurately remember how many family members died in the last ten years, including all the infants?”
See you did it again. You turned “household” into “family”.
Oh, come on. Household members are people you know pretty damn well. Most of them are going to be family.
But up here, LizardBreath quotes the study as defining a “household” as a group of people who live and eat together. Given that definition, I’d figure that people would be *more* likely to remember dates of death.
And the “Who keeps track of dead babies” critique doesn’t affect the study, given that the excess postwar deaths aren’t infants, they’re adults who died violently. Unless you’re asserting that an Iraqi household is unlike enough enough to an American household that people would be likely to forget deaths of adults as long as they happened more than four years ago, I don’t think you’re making sense here.
It looks to me like the disagreement here boils down to those who are suspicious of the study’s results based on the splashy number preferring to us IBC’s results as the calibrating feedback for the control loop rather than the Lancet number. The people supporting the Lancet number are arguing that since the cluster method tends to underestimate rather than overestimate that we should trust the Lancet numbers, even if at the lower end of the spectrum, as a preferable value for making adjustments.
So, accuracy and methodology aside, would a number like the one reported in the Lancet study be sufficient evidence for changing US policy in Iraq, or should we, like the administration, just ignore all feedback and continue with policy-as-given?
And what adjustments should be made based on feedback? Is this a system that can be adjusted?
How much of a reduction in the death toll will all this special pleading result in? My own initial reaction to the 600K figure was incredulity and I had and have some questions about the death certificate issue. Basically, my question comes down to whether the statistics are somehow fraudulent.
Assuming they are not, the Lancet team found 300 deaths in a group of about 13,000 people.
Roughly 2.5 percent. So you don’t have to be a statistics whiz to realize this implies hundreds of thousands of deaths. I don’t think we have to pledge allegiance to the Lancet’s confidence interval of 400,000 to 900,000 to realize that we’re talking about a violent death rate in the hundreds of thousands, because it would be rather strange that the survey team would have found 300 violent deaths in a group of 13,000 people if the true violent death rate was 50,000 out of 25,000,000. They should have found around tens of deaths, not hundreds if that were the case, unless they stumbled into several outliers like the Fallujah cluster in the previous study. And if they found several such clusters, that would be significant in itself.
But I’m happy to keep repeating the real point–what the freaking hell is wrong with a country that can’t be bothered to determine how many people we might be killing in Iraq? I feel a very strong desire to violate posting rules at this point. But nevermind. We’re all happy to casually spout off about the (similarly measured) death tolls in other places like Darfur. But let someone claim we’ve caused an even greater disaster in Iraq and rather than have the press and politicians united in demanding a serious impartial investigation, the press puts the story on page A12 (in the US) and people go out of their way to find some reason to discredit the numbers. One doesn’t need to know a Bayesian from a frequentist to understand what’s really driving this reaction and it’s not an ethically driven motivation to know how much misery we’ve inflicted.
Not “doing something weird”, I suspect. Something pretty straightforward.
Table 2 reports all deaths. Some of those really *do* go into the category of “would have died anyhow”, i.e. even if there were no war.
So the totals in Table 2 do not correspond to the 650k excess figure. Rather, the totals in Table 2 correspond to the 650k excess, *plus* the pre-war background rate.
Subtract the pre-war background, and you’ll be subtracting some of the post-war deaths. But not (as I was arguing above) the gunshot and carbomb deaths. You’ll be subtracting the infant deaths, the cancer deaths, and so on.
So: violent deaths are 11/20 (=55%) of *all* deaths in the post-war period. But after you subtract from those deaths, the deaths that would have occurred anyhow, then violent deaths become 12/13 of the remaining deaths, i.e. the excess deaths.
So the lower, 55% number is just the result of keeping in the deaths that are not part of the excess. When we look at the excess, the number is still around 12/13,or around 600k.
Now we’re back where we were at 2:50p. You want to say that the pre-war rate is being underestimated, and that we should re-assign some significant part of the “excess” group into the pre-war background. And now we’re back to where you have a problem, because you have to claim not only that the pre-war had a lot more deaths than this study estimates it to have, but a lot more deaths due to gunshot, deaths due to car-bomb, deaths due to IED, and so on.
So thanks for confusing issues with the irrelevant 55% number. But once again: This just seems to descend deeper and deeper into special pleading. Go there if you like.
Donald:
Donald, the problem is we love Iraqis TOO MUCH. This is particularly true for war enthusiasts who cited humanitarian reasons for their support. They simply care too strongly about Iraqis to be able to find out how many have died, even if the number is just one. They couldn’t bear it.
It’s DEFINITELY not the case that the humanitarian aspect was always transparent bullshit and vain posturing, and that they never could have cared less whether Iraqis lived or died. Don’t even think that.
Okay. I’m confused.
The numbers I’m seeing thrown around are five hundred Iraqi’s per day, for the entire war. My understanding was that violence has been steadily increasing for some time. As in, since ’02.
Doesn’t that mean that if you graphed deaths of Iraqi citizens, you’d see a steadily rising line? And doesn’t that mean it’s not “five hundred civilians per day”, it’s more like “fifty – no wait, 100 – er, 200 – um, 300..” and so on?
If these figures are true, it would mean that the current ‘wave’ of violence in Iraq is being vastly underreported, wouldn’t it?
What I’m trying to say is – doesn’t this mean there must be a thousand or more Iraqis dying every day?
zaeron
yeah. click on the link to get a copy of the study, and look at graph 2. It has just the kind of upward-trending line you’re expecting.
Zaeron — I think so. And it’s not as odd as one might think, given the huge limitations on where reporters can go. When you consider that the horrific number of deaths that make, say, the Washington Post are generally all deaths in Baghdad or one of a few other cities, and that the civil war (understood to include fighting between rival militias that are e.g. both Shi’a) is going on in a lot of smaller places that no reporter can get to, the idea that the death toll is an awful lot higher than the number of deaths we read about — possibly higher by orders of magnitude — is not, to me, at all implausible.
Er, that’s exactly the point. At a minimum, data has to be “the best we can do” in order to be worth working with at all. It doesn’t have to be “really good data” in order to be useful. What you want is some way of gauging how good your data actually is, so you can decide how useful it is. You have now (helpfully, thank you very much) quoted Brennan and Heeringa, neither of whom suggest that f(n) is a deal-breaker.
What some of us want to know is exactly why “the best we can do” (where “we” is a bunch of peer-reviewed epidemiologists) isn’t good enough for you (where “you” is Sebastian Holsclaw). The answer so far appears to be that the data is “obviously wrong.”
Seb: All these arguments about misremembering I don’t understand. In 80% of cases they saw death certificates. Shortly that resolves the issue of double counting. As far as I know people don’t issue multiple death certificates to separate households, extended family members, and cousins.
Correction: ‘surely’, not ‘shortly’. Darned dictation software.
“The answer so far appears to be that the data is “obviously wrong.”
Sure, just like we knew (or should have known) that high tension wires don’t actually cause cancer.
The problem is that Lancet is paintint a picture of WWII levels of killing. That is surprising. Maybe it is true, but it is surprising.
If kid bitzer is correct and they are nearly all violent deaths, that brings us back to the important question of why they aren’t bothering to compare combatants and non-combatants (which IMHO is really the important question). 600,000 combatants? That doesn’t necessarily sound as awful as 5 combatants and 599,995 civilians.
And I’m way to exhausted for a “Why it is important to distinguish combatants” fight today. 🙂
BTW, I mentioned it a couple of hours ago on crooked timber but on rereading the thread I realize I didn’t also post it here. On revisiting the numbers for Mexico, the death rate is reported very differently in different places, but the consensus appears to be much more in the 5-7 range. So my hypothesis on younger nations having lower gross death rates may have been correct–making the numbers I quoted look a lot less shocking.
why they aren’t bothering to compare combatants and non-combatants (which IMHO is really the important question).
Because it’s too difficult — they can’t figure out how to do it at an acceptable level of reliability. An answer would be interesting, but the lack of one isn’t surprising, and doesn’t render this study useless.
Donald beat me to it. The very fact that there’s any room for controversy about this is itself a very strong condemnation of my country’s behavior. There ought to be an official tally – and yes, I do expect that parts of it would be kept secret for a while to protect current operations, and parts longer for various good reasons under any administration. But we should have at least a broad overview of how the Iraqi people are doing, who’s living, who’s dying and of what, and so on.
The problem is that Lancet is paintint a picture of WWII levels of killing. That is surprising. Maybe it is true, but it is surprising.
Hm. Why?
With diffusion of advanced technology, communication is more rapid and sophisticated than in WWII. Diffusion and advancement of weapons technology would logically also raise kill rates.
“What some of us want to know is exactly why “the best we can do” (where “we” is a bunch of peer-reviewed epidemiologists) isn’t good enough for you (where “you” is Sebastian Holsclaw).”
Because the best we can do isn’t always “good”. The best brain surgery I can do would suck. The best a specialist can do for certain cancers sucks. The best top class economists do on lots of economics topics is barely science. The best psychologists can do is pretty goodish for certain ugly cases. “The best we can do in a war with self-reported retrospective studies” isn’t super great. The best we can do in calculating values of pi is pretty precise. We can compare to things we know about–like WWII. By reporting “excess” death it implies that we should measure it against civilian deaths in WWII and it suggests that the present war is worse than that. I find that surprising. I suspect the surprise is from mixing combatants and non-combatants. That is problematic (to me) maybe not so much to you.
What’s so unusual about a 600K death toll in a war?
As for the civilian/insurgent distinction, suppose half of those alleged 600K are insurgents. Then there were 300,000 Iraqis who died fighting the foreign occupier. And presumably they have a lot of active and passive support. So what’s our moral justification for being there? To save them from themselves if we have to kill every last military age male?
“Because it’s too difficult”
Heh, don’t want to “do the best we can do”?
But I’m happy to keep repeating the real point–what the freaking hell is wrong with a country that can’t be bothered to determine how many people we might be killing in Iraq?
whatever is wrong with ‘us’, it’s the same problem that leads us to kill ‘them’ in the first place.
but the violent/non-violent split is completely different from the combatant/non-combatant split.
What’s the dodge here–‘oh, if they’re violent deaths then clearly they were terrorists, so it’s a good thing we killed them over there’?
It may be more *obvious* when children suffer a violent death, but it is still the case that an adult’s suffering a violent death does not mean that the adult was a combatant.
And it is an even further step to the claim that a combatant death is one that we need feel no compunctions about. We created this hell. We even gave a lot of these people their IEDs, by leaving the arsenals unlocked and unguarded. The scale of our culpability is gradually getting clearer.
So some people want to grasp at any straw to obscure it.
Yukoner,
I think the gut reaction to the numbers being too high may be the result of being personally offended, not how well they stand up to daily death reports (“This can’t be true. Could we really be doing this? Not my country.”), especially in the face of the phenomenally potent myth of American exceptionalism, that this adventure indeed had something to do with “spreading democracy,” that the US only goes abroad to right wrongs, and that the military is something of a Peace Corps with guns (and Morat20’s 2:54 comment). I hear such sentiments all the time. Perhaps that also explains the reluctance to discuss the implications of these casualties (even if we seriously low-ball the numbers) and instead concentrate on methodology, margins of error, etc. (A second to nous 3:22 and DJ 3:23.)
-“what the freaking hell is wrong with a country that can’t be bothered to determine how many people we might be killing in Iraq?”
Remember the note from Rumsfeld asking just how many insurgents US forces were killing in order that the Pentagon might track progress?
SH,
Re death rates: You may like to check out coverage of this at
http://abstractnonsense.wordpress.com/
and Unfogged.
“but the violent/non-violent split is completely different from the combatant/non-combatant split.
What’s the dodge here–‘oh, if they’re violent deaths then clearly they were terrorists, so it’s a good thing we killed them over there’?”
There is no dodge, but if I’m going to compare it to other wars I can either treat them as civilians (in which case this looks much more horrific than WWII but for silly reasons) or I can look at it.
These numbers are reported for a moral analysis purpose. I find it difficult to do the moral analysis desired when everyone gets counted as a civilian.
Correction–since the US killed 30 percent of the 600K (accepting the numbers at face value), then we have at most killed “only” 180,000 insurgents. The rest of my point stands.
I’ve been looking for rightwing critics of the Lancet paper to come out in favor of a serious attempt to count the dead (or estimate them using statistical methods, since counts aren’t necessarily accurate.) Anyone seen any who have done this? I expect Iraq Body Count will be somewhat critical of this Lancet paper, but I suspect they will agree with the Lancet authors that there is a need for an independent body to do this.
Incidentally, Iraq Body Count criticized the previous paper, preferring the UN survey that was conducted in 2004. But when one compared the violent death toll of the 2004 paper with the UN survey, the UN number was about 30 percent less by IBC’s own estimate. In other words, they were in rough agreement and IBC’s only legitimate point was that this suggested the true death toll was in the lower part of the Lancet CI, not the upper part as the Lancet authors thought. But it wasn’t that much lower.
“I find it difficult to do the moral analysis desired when everyone gets counted as a civilian.”
Okay, maybe this would help you do the moral analysis:
In a country that is not involved in a war, pretty much everyone is a civilian.
Non-civilians aren’t just born; they are people who are involved in fighting a war. They may have chosen to fight it, or it may have landed in their laps.
By your reasoning, if someone invades your house and shoots you when you resist, then they really shouldn’t feel too bad about killing you–after all, you were no longer a civilian as soon as you resisted.
This is crap. If we are morally culpable for having visited a war upon an entire people, then we are also morally culpable for turning at least some civilians into non-civilians. We don’t get to then turn around and say “hey, so what if we killed them–it’s not like we were killing civilians.”
“By your reasoning, if someone invades your house and shoots you when you resist, then they really shouldn’t feel too bad about killing you–after all, you were no longer a civilian as soon as you resisted.”
Actually with sectarian violence being such a huge part of it, the analogy is more like: if someone invades your house and shoots you take out a gun and shoot your wife….
Hah! That is funny. Suddenly contemplating the 600K deaths caused by our invasion of Iraq hardly bothers me at all.
Thanks, SH!
“Actually with sectarian violence being such a huge part of it, the analogy is more like: if someone invades your house and shoots you take out a gun and shoot your wife….”
does that comment have any bearing on the earlier lines of argument? Does that analogy help in any way to understand the Lancet study, or what it shows about the number of people killed by the invasion, or what we as a country should think about those facts?
Did you have a point? Or were you just making a joke?
I was responding to your analogy. It is connected to the discussion inasmuch as you think your analogy was apt.
Does it have any bearing on the Lancet study? Yes. Presuming your statistics are correct, when you talk about “excess deaths caused by ‘X'” there are different levels of moral culpability between invading your house, and you intentionally murdering your wife ‘because’ I invade your house.
Moral culpability in the eyes of who? Are Iraqis to be swayed by this arguement?
Or rather, argument.
All of your argument amounts to putting scare-quotes around the word ‘because’.
If the sectarian strife would not have happened had we not started this war, then, yes, it is happening because of us.
You can’t just throw rocks at an unstable situation and then pretend you have no responsibility for the consequences of your destabilizing it.
Sebastian,
I see from your statement here that you apparently still don’t understand a very simple point I made way back yesterday here.
Could you go back and read what I wrote until you do understand this? Because if you don’t I’m going to have to rip off my head in frustration. Thanks.
Interesting stuff in here. I do some stats, but the fundamental difference is that I’m dealing with words/phrases, not dead people. When looked at in this way, statistics becomes another way to disconnect us from the reality rather than get us closer to what is actually going on.
The mention of the previous figures involved with the sanctions regime is also very interesting. DeLong had, about a year ago, a post with some back of the envelope calculations on child mortality and such during the period around the US Civil war and wondering how this must have affected their outlook and mentality. It occurs to me that if sanctions struck the weak and young disproportionately, it has the effect of supporting a fatalistic view of the world, while the current mortality rates in Iraq must be concentrated among the breadwinners and adults, which may have the opposite effect. I think, if this is true, it has some bearing on the use of sanctions in the future as an instrument of policy. It may also suggest why Iraqis may be much less reserved in their opposition to a US occupation as opposed to Sadaam’s rule. Unfortunately, some will argue that this proves that those who did not resist Sadaam but are resisting the US occupation are unregenerate Baathists, which I do not think is the case.
You know, there really are marriages that limp along for decades in a state of quasi-stability. They aren’t marital bliss, there are underlying tensions and grievances, but they raise kids, they provide for each partner’s financial security, and they don’t kill each other.
Do you think if you invade the house of such a marriage, tear apart their common life, and goad one of the two into shooting the other–even if the enraged shooter is acting in part on long-standing grievances–do you really think you escape all culpability?
Multi-sectarian Iraq was always a mess, and it was always going to be hard for it to evolve peacefully out of the Saddam era into a post-Saddam era. But for decades, Shia and Sunni lived in Baghdad without sectarian violence. They intermarried. They raised kids. Had things gone differently, it might have evolved peacefully.
But now that is not going to happen. Because we invaded, in what we now know was a completely needless, pointless war.
And it is simply inadequate to say “yeah, but they might have wound up squabbling even if I didn’t destroy their common life and set them against each other.” That is of course true. They might have.
But you can’t just declare an equivalence between bad outcomes that might have occurred without your interference, and bad outcomes that, in fact, are the direct result of your interference. They are not the same at all. For the second kind, you bear responsibility.
At least I’m glad that this exchange has evolved in such a way as to show the real issues:
The real line of objections to the study are not about statistical reliability and power calculations. They are about trying to evade responsibility.
Jon(S),I do understand that point. That is called wanting to have your cake and eat it too.
Deaths of combatants aren’t “excess deaths”. They are intended combatant deaths.
Lancet acts on the pretense that the combatant/noncombatant distinction is not worth investigating. It treats everyone as civilians. It isn’t 600,000 Iraqi soldiers killed, right? When trumpeting the number it isn’t making an ad for US military efficiency, is it? No, they are strongly implying that these aren’t soldiers.
That makes their numbers not particularly useful as far as I’m concerned, but insofar as you feel like using it, you can’t compare it to combatants plus civilians in other wars. The distinction was drawn in the past for a reason.
To try to do otherwise is to attempt to erase the combatant/non-combatant distinction in a way that I think is morally unuseful.
“They are about trying to evade responsibility.”
Quite. Though I think in the opposite way from how you intend it.
the opposite? what does that mean?
Who are you saying is attempting to avoid responsibility? And for what?
OT: UK Chief of the General Staff calls for British withdrawal “soon” – “our presence in Iraq exacerbates” the “difficulties we are facing around the world.”
There are other interesting things the General says too.
Sebastian: Lancet acts on the pretense that the combatant/noncombatant distinction is not worth investigating.
No: if you read the study, you will discover that the Lancet study consciously decided not to ask whether the deaths in the household were from people fighting in the insurgency, because they feared that asking questions like that in the middle of the war might well put the research teams at risk. This is saying that the distinction is “not worth investigating”, but not as a “pretence”: not a distinction worth putting the lives of the people on the research teams at further risk.
The snide tone of people who don’t like the answers people risked their lives to get is distinctly unpleasant.
When trumpeting the number it isn’t making an ad for US military efficiency, is it? No, they are strongly implying that these aren’t soldiers.
No, they are saying that these people wouldn’t have been killed if Bush hadn’t decided to invade Iraq. To argue that these high numbers could ever be said to be an “ad for US military efficiency” is fairly damn disgusting: are you really saying you see causing the violent death of around 600 000 people in a wholly unnecessary war as “efficient”?
Hey, Sebastian. A thought experiment. The war in Iraq ends, and a couple of years later, out of nowhere, China attacks us. For some reason, it doesn’t go nuclear — they land an invasion force in Mexico, and invade over the Californian border. Through superhuman skill and bravery, our armed forces defend us with such surpassing excellence that not a single American civilian is killed.
A whole hell of a lot of American soldiers die, though.
Have we got anything to be aggrieved about?
Sebastian:
Sebastian, I don’t think you actually believe this, because it’s so strikingly wrong.
But if you do, I’ll bet you money on this proposition:
As I say, I doubt you actually believe what you’re saying, so I doubt you’ll bet on it. But as always, I’m happy to take your money.
Seb: they are not “pretending” that all deaths are civilian deaths. They are not making any claim about civilians at all. They are just asking: how many more deaths are there than one would have expected had the death rate under Saddam continued?
They didn’t attempt to figure out which deaths under Saddam were ‘really’ caused by Saddam’s torturers, the sanctions, etc. Not doing that might, had the figures come out differently, have led (different) people to charge that the numbers were unfairly favoring the war, since of course when you remove the cause of a bunch of deaths, it looks like the death rate improves. That charge would have gotten the same reply from me, namely: all they purport to count is how many extra people — not civilians, people — died.
“Have we got anything to be aggrieved about?”
Sure, but that doesn’t change the fact that we would have an even further complaint if they had destroyed Fresno and Davis.
“They didn’t attempt to figure out which deaths under Saddam were ‘really’ caused by Saddam’s torturers, the sanctions, etc.”
Actually considering that they found only two people killed by violence in the prewar period (one of them allegedly by coalition bombing, they didn’t need to try to figure it out. They apparently didn’t exist–which is one of my points.
I’m still anxious to make this bet, Sebastian.
Sure, but that doesn’t change the fact that we would have an even further complaint if they had destroyed Fresno and Davis.
So for the purposes of your own, internal, moral calculus, Seb, how about you assume every one of those 600K Iraqis who died was holding a gun. That way you can be sure not to feel any worse about the war than you’re obliged to.
So, Seb. As a war supporter, what benefits do you see coming out of the war that justify the deaths of 600K Iraqi combatants?
Well then I will have to say that I find the Lancet study not worth the attention it has gotten. I will also say that in the media it is, like it or not (and I suspect “like it”) being played as if it meant civilians.
If it was 600,000 Iraqi soldiers killed, it would be regretable in the way war is regretable, but morally understandable in that context.
If it were 600,000 civilians (killed by the US), it would be atrocious.
600,000 of ‘we can’t be bothered to tell whether they were civilians or soldiers, whether they were killed by the US or by sectarian violence’ isn’t particularly useful. As I’ve said before in other threads, it isn’t particularly useful especially with the weird anomolies (like the low number of infectious disease deaths, like the one violent death before the war, see also perhaps the death certificate issue which d-squared doesn’t believe there was a genocide in the Sudan thinks is such a non-issue.
I will say that I should have made that objection of mine evident earlier in the discussion so as not to annoy people about ancillary issues while not directly confronting the big one. (I hate it when people do that to me. I talked with someone about school vouchers for at least two days before I found out that they thought the quality of US schools was pretty much high enough already. Well of course you won’t support vouchers if you think the schools are fine).
So sorry for that.
Deaths of combatants aren’t “excess deaths”. They are intended combatant deaths.
Well, except to the extent that they’re only “combatant deaths” because they’re fighting to get us the hell out of their country. You know, “but for” and all that. Alas, smarter people than me have tried to get you to understand this, but you seem stubbornly unwilling to do so.
If it was 600,000 Iraqi soldiers killed, it would be regretable in the way war is regretable, but morally understandable in that context.
Um, okay. What’s the benefit that outweighs that cost for you, and makes it morally acceptable?
My thought is that the whole process of modern war has been to try and make an argument that civilian deaths and military deaths are two sides of the same coin. One of the rationales for firebombing Japan and Dresden was that civilians were contributing to the ability to wage war and as such, constituted a legitimate military target. Because, the argument goes, japan dispersed its military industries into residential areas, so that parts were fabricated not in large factories, but in household workshops, it was necessary to destroy that infrastructure. That was certainly the pattern in Korea and Vietnam. I’ll agree there is a problematic tension between the Geneva convention and this notion, but the fault lies not with Iraqis taking advantage of GC, but the constant moving up and over the line by powers utilizing industrial strength to wage war. Of course, I understand that no country is going to say ‘ok, let’s not use industry to wage war better’, but it’s important to realize how we got to this point.
And that last line “So sorry for that” may sound sarcastic but I’m really serious. I hate it when people pick on the edges while ignoring such a key area of disagreement.
But I also want to make something slightly different VERY clear.
I think Bush and his administration did an awful job in Iraq. Presuming the 600,000 number is correct and even if it were all soldiers and even if none of it were sectarian violence, what we are getting out of it isn’t worth it because we aren’t getting to a positive end. Bush never treated this like a serious war, and it clearly shows. He declared victory too early, didn’t apply enough force early, let 3 separate insurgencies start up full force before bothering to sort of pay attention to them, and the list goes on and on.
Jon,(S) [and I see you’ve changed your handle recently]”But if you do, I’ll bet you money on this proposition:
A study of German deaths during World War II conducted according to the same criteria as the recent Lancet study in Iraq would have classified German combat deaths as “excess deaths.”
I’m sure under your wording you would win.
The problem is that in 1950 very few people would think it was worth treating the numbers like that.
Sebastian: Presuming the 600,000 number is correct and even if it were all soldiers and even if none of it were sectarian violence, what we are getting out of it isn’t worth it
Is this a response to LizardBreath’s question at 07:18 PM?
Sebastian:
No. That’s not the problem. The problem is something else, something so simple that the fact I can’t get you to address it is bringing me dangerously close to ripping off my own head.
You stated that the study is “obviously wrong.” Your reason for this? “The rate of civilian deaths in Germany during WWII was 20 in 1,000 measured over the whole war. The rate implied by the Lancet report is 27 in 1,000 thusfar.”
But these numbers are simply not comparable. When I (and others) pointed this out, you didn’t address this, but simply moved onto other ground.
So, let me ask once again: do you understand this? The reason it matters is because if your Germany comparison is invalid (which it is) so is your main basis for calling the study “obviously wrong.”
Now, it may be your statements about it being obviously wrong were just you picking around the edges, as you put it — and your main concern with the study all along was not the numbers it came up with but the fact they didn’t classify people as civilians or not.
Fine. But if that’s the case, both my head and I would appreciate a clarification on your part.
<>s wr spprtr, wht bnfts d y s cmng t f th wr tht jstf th dths f K rq cmbtnts? < hrf="http://www.bbc.c.k/hstr/wrldwrs/wwtw/htlr_lbnsrm_.shtml">lbnsrm t’ll tch Gd t pt r l ndr thr snd. Sb, knw y wr ChknHwk, bt ddn’t knw y wr sch cld clls bstrd.
<>s wr spprtr, wht bnfts d y s cmng t f th wr tht jstf th dths f K rq cmbtnts? < hrf="http://www.bbc.c.k/hstr/wrldwrs/wwtw/htlr_lbnsrm_.shtml">lbnsrm t’ll tch Gd t pt r l ndr thr snd. Sb, knw y wr ChknHwk, bt ddn’t knw y wr sch cld clls bstrd.
Oh Don Quijote….why, why, why?
spartikus: any moment now, it will be: wh, wh, wh?
Isn’t DOn Q banned?
Time to find a disemconsonanting script.
to those who argue (and I’m thinking specifically of Rilkefan and Sebastian) that combatant deaths are different from noncombatant deaths in this case, I would like to point out the example of:
South Africa.
Here is a country which many people believed would inevitably face bloody civil war as a minority group which held power through violence transitioned into minority status.
oops. managed it just fine without much outside interference, thanks very much.
the 600K violent deaths were not necessary nor inevitable.
A large college football stadium holds about 100K people. So the next time you see a Big 10 football game, or UCLA play at the Rose Bowl, imagine six such stadiums, full of dead people, and recognize that each one of those deaths is America’s fault.
Lizardbreath: repeatedly. For some unfathomable reason, he keeps coming back with new IP addresses.
“oops. managed it just fine without much outside interference, thanks very much.”
You’re aware of Iran I presume?
SH,
What is your definition of “soldier” in this context? Do you remember back before the first big attack on Fallujah when US forces were not permitting any male from the age of 15 (? or perhaps it was 16) and up from leaving the city?
LJ writes (7:20):
-“My thought is that the whole process of modern war has been to try and make an argument that civilian deaths and military deaths are two sides of the same coin.” etc.
Exactly.
Bye bye Don Q, who won’t be missed.
Sebastian: You’re aware of Iran I presume?
You’re aware that Iran did not invade Iraq, I presume? (The Iran-Iraq war of 1980-88, to which I suppose Sebastian is referring, was kicked off when Iraq invaded Iran.)
“South Africa”
Kind of a miracle, unless you think Mandelas are easy to find. Also I suspect SA didn’t have many of the problems faced by Iraq.
Anyway, I wasn’t really arguing so much as musing, but if you have a convincing story starting from 9/11 minus the Iraq war plus Saddam dying in the near-term not leading to 0.7*600k deaths (+ f_leave(0.7)*rather larger number to come), I’d be happy to hear it. I would feel vindicated in my opposition to the invasion simply on the basis of that 0.3*600k if I didn’t already have much more vindication than I ever could want.
“You’re aware that Iran did not invade Iraq, I presume?”
I was speaking of the “much outside interference” bit.
you know, there are scenarios under which I would share SH’s serene unconcern about the deaths of 600k Iraqis.
For instance, if they had invaded the US. Then I would have been happy to hear of their death. (I won’t say “happy to kill them with my bare teeth,” or indulge in the other keyboarder fantasies–I had the opportunity to serve in the First Gulf War, and declined.)
But not just that: if Iraq really had been the danger we were told it was, and if we had had to kill 600k members of the Iraqi army in order to neutralize that imminent danger to America, then in that case too I would not feel too bad about it.
But the fact is, we got off on completely the wrong foot in being there to begin with. There never was a credible threat, and the people in charge either knew as much, or willfully plugged their ears to that knowledge, while lying to the rest of us.
It was when the WMD rationale fell through that we were sold a new rationale: we were there on a mission of humanitarian intervention, helping the Iraqis for their own sakes.
Once that becomes the operative rationale for what we’re doing there, then it seems to me that we can no longer be blase about killing a lot of Iraqis–even ones who are fighting to expel us.
Because if the claim is that we are there to do good, then we had better be doing more good than harm.
And killing combatants who would not have been combatants if you hadn’t invaded their country does not get to go into the positive column any longer.
When the Hussein regime and the Iraqi army were our enemies? Then, fine–kill some combatants, s’okay with me.
When Iraq is our charity orphan and we’re on a humanitarian mission? No longer so fine to just kill any Iraqi who doesn’t welcome us with open arms.
What these new numbers do is to show two things:
1) That the humanitarian rationale, to the extent that it was ever sincere, is not being borne out by the consequences. We are not doing more good than harm.
2) That people who are willing to say on Monday that the Iraqis are our fledgling allies in democracy, and on Tuesday that we have a free hand to kill them cause they’re just combatants anyhow, were never really sincere about the humanitarian rationale.
That people who are willing to say on Monday that the Iraqis are our fledgling allies in democracy, and on Tuesday that we have a free hand to kill them cause they’re just combatants anyhow, were never really sincere about the humanitarian rationale.
You’re a hell of a lot more generous in your assessment than I am.
<>wh, wh, wh? Cs nc n whl smn hs t tll th gl trth. <>By by Dn Q, wh wn’t b mssd. Lv y t, J srgs d lc…
I note with dismay that despite my best efforts here and here and here, I’m apparently unable to persuade Sebastian to address a very simple point.
And I again wonder what difference it would make if we could all agree on a general figure for Iraqi deaths. Playing goldilocks with the figures doesn’t make them connect to anything else.
Is this only a fight over some undefined tipping point for public opinion where X-1 means stay and X means go? How will we know when we hit X? What will happen to effect the change once the tipping point is reached? If X is reached and we go, does X increase or decrease as a result of our leaving? Can we simultaneously pursue a successful counterinsurgency policy and minimize X (not that I am arguing that our current policy is anything like successful)? Does a successful counterinsurgency policy depend on minimizing X as much as possible? Does the actual size of X change any of the things that we need to decide which are connected to it?
And for any who disapprove of the mathematical abstraction here, assume that X = an apalling number of dead people whatever the actual value.
NY Sun – Baker’s Panel Rules Out Iraq Victory
Looks like we caused the death of a few hundred thousands Iraqis for nothing.
nous: While I agree with the general thrust of your argument, I would pray that this topic, if no other, would remain immune to sorites-style arguing.
kid bitzer: “SH’s serene unconcern about the deaths of 600k Iraqis.”
You’re talking about a fantasy version of SH. This was a favorite tactic of hard-right war supporters – if one opposed the invasion, one was serenely unconcerned about the babies dying there.
One way to reach a broad agreement would be to note that the confidence intervals for this study are quite large, precisely because all of the issues with cluster samples are known and factored into them. Nonetheless, the lower bound is 392,979 excess deaths. That’s an awful lot of excess deaths, even if it isn’t the even more horrific 654,965 (let alone the unbelievably awful upper bound of 942,636.)
ditto to rilkefan’s last comment.
I am not a statistician, but I play one when I practice historical demography, which I have done, though not recently. I have particular – though by no means conclusive – knowledge of two relevant cases of attempting to assess political/military crisis mortality: the US invasion of the Philippines in 1898ff and Cambodia under Pol Pot, 1975-78. From this experience I have only a few comments to add to what has gone before here (see also the threads at Crooked Timber, BTW).
– Everyone’s complaining about how bad the data are. From my perspective, I keep marvelling at how good they are, compared with what historians of Southeast Asia usually have to deal with! To comment only on their shortcomings is simply to condemn us to absolute ignorance of the topic. (Which may or may not be the desire of those making the comments: sometimes “Our best simply isn’t good enough” seems to be code for “We don’t wish to know that.”) You go to historical demography with the data you have, not the data you wish you had.
– “Excess” mortality has a fairly precise demographic meaning, which has nothing at all to do with the civilian/military or moral distinctions that Sebastian keeps trying to draw. “Excess” mortality is simply the number of deaths in excess of the number of expected deaths for the same period and the same population. (Because obviously thousands of people will be dying in any sizeable population over any significant period of time.)
Of course this very “expectation” is a matter of potential controversy, and can be complicated by the fact that catastrophes often do not happen in a vacuum. The US invasion of the Philippines occurred in the middle of a revolution and a “natural” mortality crisis involving epidemic malaria and cholera. Pol Pot took over Cambodia only after 5 years of American bombing and shelling had devastated the countryside, killing hundreds of thousands. So the “base level” is by no means obvious – but we do our best, sometimes reverting to earlier period to define “normality” (e.g., in the Philippines, pre-1896 {?}, in Cambodia, pre-1970).
(Demography does not usually take into account any consideration of counterfactuals, such as “What would have happened if X?” “Excess” mortality is defined in relation to expectations based on extrapolation from the base level. The argument that something worse would, or even might, have happened otherwise is simply not relevant to the demographic question, though it may inform one’s political perspective on the answer.)
Sebastian is thus on point in querying the 5.5% crude death rate for Iraq assumed by the study as the base level from which excess mortality is measured (though it seems reasonable enough to me, no expert on the Middle East). He is totally off-base, however, in suggesting that the failure to distinguish civilian from military deaths has anything at all to do with the demographic data he seems so anxious to dispute; at most it affects his own attitude toward them. Lately in the discussion he seems to have been sliding from “I don’t believe there were 600,000,” which we can reasonably debate, to “I don’t care even if there were 600,000, if they were combatants,” at which point I abandon him to the ethicists, if any remain.
– Based on what I’ve seen, the Lancet report seems plausible. (I’m not prepared to say more at this time.) If it’s correct, or anywhere near correct, between 400,000 and 950,000 Iraqis have died since we invaded their country above the number that would have died if mortality had remained at pre-invasion levels. We need to ponder the significance of that fact, and our responsibility for it.
With you more or less up to “he seems so anxious to dispute; at most it affects his own attitude toward them” which is I think unnecessary mind-reading.
Also “We need to ponder the significance of that fact, and our responsibility for it” begs the question – perhaps you dropped a “the extent of” before “our”, and I think even “fact” is unfair or else meaningless in context.
Quibbles:
Also “We need to ponder the significance of that fact, and our responsibility for it” begs the question – perhaps you dropped a “the extent of” before “our”…
Speaking for myself, I don’t regard responsibility as either binary or zero-sum so I wouldn’t regard the qualifier “the extent of” as necessary there. YMMV.
…and I think even “fact” is unfair or else meaningless in context.
On my read it’s still part of the conditional (“If it’s correct…”) and hence neither unfair nor meaningless, merely contingent on establishing the antecedent.
Sebastian: I was speaking of the “much outside interference” bit.
What “outside interference” from Iran are you referring to prior to the 2003 invasion of Iraq?
Deaths of combatants aren’t “excess deaths”. They are intended combatant deaths.
We seem to have got a little distracted away from this point. Sebastian seems to be saying that we (the US/UK, since the UK was also part of the invasion – 119 British military fatalities, plus at least four British journalists among the 80 killed so far) are not responsible for the deaths 600 000 Iraqis whom the Lancet estimates have been killed since the March 2003 invasion. Either because they might have been killed anyway if some other country had invaded, or because they were only killed because they took up arms to repel a foreign invasion/occupation, whereas if they had peaceably surrendered and accepted the foreign military occupation they wouldn’t have been killed.
But, as Sebastian does acknowledge, the reason there was an insurgency is not only the straightforward “because the US/UK invaded” but also because Bush & Co neither made or implemented plans for the occupation – aside from the plan: Let American corporations who are large donors to the Republican party make lots of money out of it. And it appears likely (see Baghdad Year Zero) that this focus on how US corporations could best profit from Iraq was a major instigator of the widespread resistence with popular support: though mass arrests, military attacks on peaceful demostrations, kidnappings, hostage-taking, torture, and murder would also play a part.
The Bush administration wanted to sell off every Iraqi nationalized business to the highest bidder, without any restrictions on who could bid: which sale, by a foreign occupation, would have been illegal. It appears likely that the delay in elections, putting into power a puppet “interim government” was intended to try and make the sale legal. (And of course once Iraqi elections had been delayed after a certain point, Bush/Cheney couldn’t permit elections to take place until after the November 2004 elections in the US: it was never about Iraqi democracy.)
The point is: even if someone wants to argue that the 600 000 deaths were probably mostly combatants (ignoring for the moment that the initial study found that a majority of the 100 000 deaths were killed in US air raids, mostly women and children), killed in the insurgency because they were fighting in it, that still doesn’t exempt the US/UK from blame, and in particular, does not exempt the US. It wasn’t only the invasion and military occupation that started the insurgency: it wasn’t only removing Saddam Hussein’s regime that started the insurgency. It was invading Iraq with the plan of setting up a military occupation to pillage it, and without any other plans for it. To argue that an insurgency might have started anyway, or that most of the people killed might not have died if they hadn’t been fighting in the insurgency (neither argument supported by any facts, but never mind that for the moment) does not take away any responsibility for what the US did, with UK support.
anarch–
I heartily second your point about this topic being immune from sorites-style argument.
And since this topic is immune, I think we can all agree that the topic immediately adjacent to it is immune as well. And if *that* topic is immune, then certainly the topic immediately adjacent to it is *also* immune. But….
SH is free to characterize his own attitude towards the deaths of combatants under various scenarios. Up above he suggests a larger number might reflect well on the efficiency of the US military–that is the attitude I would have under certain scenarios, and I would describe it as serene unconcern. But no one is barred from saying more about what they mean.
kid – I’d just point out to you (as others have pointed out to me on other occasions) that when you make comments about Sebastian rather than about Sebastian’s arguments about the Lancet study you distract the thread from arguing about the Lancet study and turn it to arguing either about Sebastian or about you or about civility in discussion threads. This is a fault I’m guilty of myself, and so I know how easy it is to slip into: indeed, if you hadn’t said it, I might have. But it is a distraction from the topic of the thread, and I am resolving all over again to try and be better at avoiding such distractions.
John Cole – Top Ten GOP Excuses Regarding the Casualty Estimates
You’re talking about a fantasy version of SH.
That’s not a fantasy version, most of us have had the dubious pleasure of reading SH unending support of a war in which he is unwilling to fight in.
Stewart: most of us have had the dubious pleasure of reading SH unending support of a war in which he is unwilling to fight in.
…and snarking about a poster supporting a war but not fighting in it is well over the line.
Thanks for the John Cole link.
I think that some people who go on about supporting the war but come up with shoddy evasions for their own involvement deserve harshing on. But for starters, Sebastian has as strong a respect for competent management as (say) I do, and if he has any tendency toward blood-and-iron war-glorifying nationalism, I can’t recall seeing it. Heck, if anything, just the opposite. 🙂 I sometimes think he’s too calm and mellow about things that warrant some passion. But when he’s asked right and doesn’t feel he’s in the midst of a dog pile, it’s quite clear that he is no friend to this particular war.
Objecting to some objections is not the same as defending.
…and snarking about a poster supporting a war but not fighting in it is well over the line.
It’s not a snark, it’s a statement of fact.
It does, indisputably, have one too many “in”s.
It’s not a snark, it’s a statement of fact.
It’s a snark in dire need of disemvowelment. There is rarely any good reason for demanding of someone why they haven’t joined the military to fight a war they support, or snarking at them because they haven’t. (As it happens, Sebastian has a perfectly valid public reason for not being in the US military: but this would apply to anyone.) Someone who is not in the military but supports a war may have reasons for not joining up so nakedly personal they would have a right not to share them with their best friend, let alone on a public blog. It is in general an improper question, and in this specific you have no excuse for asking it.
Steward: Jes is right. She really is. For starters, that Seb is not fighting in the war is a statement of fact. That he is unwilling to fight in it is an unwarranted inference, since it relies on the assumption that if he were willing, he’d be fighting. That is simply not true. There are lots of people who are not accepted into the military, or accepted only if they are willing to lie about themselves. When someone supports the war but is not serving, you need to assume that they are not gay, do not have some disqualifying medical condition or criminal history, etc., etc., in order to get to unwilling. And putting someone in a position in which s/he would have to explain this to an audience of random strangers reading a blog, or else accept insult without comment, is just wrong.
As it happens, Seb is open about being gay. I have no idea whether or not he has any other reason for not serving but it is, frankly, none of our business.
Moreover, as Jes also said, focusing on personalities rather than arguments is uncivil, and distracts from the matter at hand.
And more-moreover, as Bruce said, Seb is not an uncritical supporter of the war. That is a fantasy version of him. I can’t find the exact comment — it was a while ago, well before this study — but I seem to recall him being asked whether he would support the invasion given everything he now knows, and he said no.
Ask him what he thinks. Question statements he makes that you think are false. But don’t just create a straw man of your own devising, name it “Sebastian”, and go after it in comments.
hilzoy:
I hate to harp on this, but I’ve been doing that for quite a while now, with no response from Sebastian.
So the “base level” is by no means obvious – but we do our best, sometimes reverting to earlier period to define “normality” (e.g., in the Philippines, pre-1896 {?}, in Cambodia, pre-1970)
So for Iraq, that would be….the 1970’s. And if I take what I’ve learned over the past few days and hypothesize, Iraq’s pre-Iran/Iraq war mortality rate may have been a bit higher (than the 90’s) as it’s population would have been “older”. Or some such.
Anarch: “Speaking for myself, I don’t regard responsibility as either binary or zero-sum so I wouldn’t regard the qualifier “the extent of” as necessary there. YMMV.”
My attempted point was that the explicit “the extent of” makes it clear that the speaker acknowledges that there is some dispute about how the responsibility is assigned. I might for example claim you are one part in 10^-9 responsible for 9/11 and express that as “We need to discuss 9/11 and your responsibility for it” and have to call myself for question-begging or carelessness.
“On my read it’s still part of the conditional (“If it’s correct…”) and hence neither unfair nor meaningless, merely contingent on establishing the antecedent.”
Sure – just meant that there’s some dispute about the antecedent, so it’s not useful to ask SH to assume it. [Metaphor involving you and Keira Knightley suppressed since one strained metaphor/comment is enough.]
Majikthise – Innumerate cowards recoil from the facts: 655,000 dead Iraqis
She missed at least one of them.
I forgot where I saw it, but somewhere in the past two days I saw a statistician who said he thought the error bars should be bigger on the Lancet figure–in particular, he thought the lower one could go all the way to 100,000. Wish I remembered where I saw that. Of course, assuming that to be the case, a figure as low as 100,000 would still be unlikely, but if people realized that the study might be consistent with numbers as low as 100,000 with a different analysis of the raw data, then maybe there would be less fuss.
Actually, I don’t agree with what I just wrote. If the study had come out with a range of 100,000-300,000 violent deaths (which would have been my wild guess for insurgents and civilians together), we’d probably still have people making exactly the same complaints. I’d have been less skeptical. Everyone else, I suspect, would have lined up the same way.
OT: James Baker to join the loser-defeatist crowd after the election. It seems that admitting defeat over there may cause extra defeats over here.
Lower limit could go all the way down to 100,000, is what I meant to say. I don’t know what the argument was–probably something to do with how to quantify the uncertainty that comes from the way deaths come in clusters in wartime.
I sometimes get the feeling that a lot of people have a US-centric view on this; that all these deaths are from direct Coalition/insurgent fighting, and not considering that the general instability of the country contributes to deaths due to poorer health care, deaths from sect vs. sect fighting and so forth.
Right. What makes these numbers believable to me is the combination of actual fighting and the almost complete lack of policing power sufficient to control violent crime. Not everyone who gets shot was in a battle; what I read about Iraq suggests that there’s also an incredible amount of unrestrained criminal activity.
Yes, that rise in criminal activity may be indirect, but still attributable as excess–police would certainly depress those deaths in a more stable Iraq, but not when they are trying to handle insurgency actions or are themselves being killed off.
Sure, and I didn’t mean to imply otherwise. My point is that criminal activity in an anarchic society is going to push up the death rate on top of the deaths from organized (or disorganized) combat.
This thread may be dead, but I just want to announce that at the moment, I’ve reached a tipping point–I don’t believe this 600,000 figure. I might change my mind again tomorrow, but anyway, I ‘ve got what seems like a good reason.
The UN survey that was conducted in 2004 found that there were 24000 (the CI was 19,000-28,000) violent deaths in the first 13 months or so. That excludes criminal murders and taking that into account, it was about 70 percent of what the 2004 Lancet paper would have said for the same period. But this 2006 Lancet paper found 45 violent deaths for that same period, which corresponds to 90,000 violent deaths and while that’s not directly comparable to the 24,000 figure because it includes criminal murders, I don’t think it’s rational to think that would make up the difference. The first Lancet paper found that (excluding Fallujah) criminal murders were about a third of the total violent deaths.
Anyway, where you can actually compare the current Lancet paper with some other survey using similar methods, it doesn’t agree. It doesn’t agree with the earlier paper on violent deaths for the same time period and it disagrees even more with the UNDP survey.
I don’t think the reality-based community should embrace this one paper as though it represents science in its struggle with the dark forces of reaction.
BTW, I still think the death toll in Iraq is probably in the hundreds of thousands, just in the low hundreds of thousands, and this disavowal of the 600,000 figure has no great moral significance whatsoever. And it is still our duty to investigate how many people we are killing over there. Besides, someone might tear my argument apart in the next 24 hours and I’ll go back to thinking it possible.
Rilkefan: With you more or less up to “he seems so anxious to dispute; at most it affects his own attitude toward them” which is I think unnecessary mind-reading.
Also “We need to ponder the significance of that fact, and our responsibility for it” begs the question – perhaps you dropped a “the extent of” before “our”, and I think even “fact” is unfair or else meaningless in context.
On the first point, I think it is not very sophisticated “mind-reading” to conclude that Sebastian was initially anxious to dispute the 600,000 figure, as witness his numerous posts not just here but on Crooked Timber. If that’s unfair, I apologize. The final clause of my sentence could have been better phrased, and might have reduced the likelihood of misinterpretation – what I was stumbling toward (but clearly didn’t reach here) was the idea that the civilian/military distinction may affect one’s (rather than his [= Sebastian’s]) attitude toward the demographic reality, but not the reality itself.
On your second point Anarch has charitably (and understandably, I believe) explicated what I intended to say. You, reading without the (dis?)advantage of being related to me by blood and living in the same household for many years, are forced to rely on what my words actually said, and (thus, uncharitably) how they might mislead.
In any event, if I had meant “our sole responsibility,” I hope I would have said so. OTOH, I don’t see any way on god’s brown earth that the US can avoid a major share of the responsibility for what happened in/to a country we invaded and still occupy. Are you seriously arguing that this is not the case?
As for the word “fact,” yes, I meant this as contingent (on the Lancet being substantively correct). At the time I wrote, I sensed that this was not le mot juste but it was too late (2AM+) to come up with a more precise alternative. “Figure,” perhaps?
Spartikus So for Iraq, that would be….the 1970’s. And if I take what I’ve learned over the past few days and hypothesize, Iraq’s pre-Iran/Iraq war mortality rate may have been a bit higher (than the 90’s) as it’s population would have been “older”. Or some such.
I don’t know enough contemporary Middle Eastern history to be sure about this, but I’d guess that the period between the I-I war and the US invasion would be preferable to the 1970s as a base period for “normality,” as being more recent (and thus more in conformity with the current age pyramid and “expected” mortality) and sufficiently stable demographically for purposes of calculation. That would depend in part on a determination of the extent to which Saddam Hussein’s regime was not just extremely nasty, but actually demographically disruptive in a major way in the years before our invasion. (Many despots who delight in torturing to death their enemies have less demographic significance than, say, an outbreak of malaria or cholera, or even measles.) And on that point, I plead simple ignorance.
I haven’t read all the comments in this thread, and at this time, I don’t intend to, since I’ve already read numerous iterations of the basic arguments in many places, so if I’m misunderstanding, my apologies, but in the context of Steward Beta’s previous comments (I’ve read many of the comments in this thread), did he just call Sebastian a coward there?
Or am I missing another interpretation of that comment?
You, reading without the (dis?)advantage of being related to me by blood and living in the same household for many years, are forced to rely on what my words actually said, and (thus, uncharitably) how they might mislead.
FYI, I’m still blaming the parents.
Gary, who cares? If the guy wants to violate posting rules explicitly, then Sebastian or hilzoy or someone can deal with it. If he implicitly violates posting rules, one can just ignore it as though it hadn’t been written.
I suppose I’m a little annoyed because I think this is one of the most important issues out there–the death toll in Iraq, and the extent to which we have contributed directly or indirectly to it–and it gets very little attention. And when it does, I’d rather not let stupid snide comments distract from it.
Though this thread is near death anyway.
dr ngo, the word “anxious” is I believe unfair. I wouldn’t say that people arguing with SH were “anxious” for the number to be as high as possible, though I could refer to the large number of people carping against arguments why the number should be lower.
As for the rest, my (elided) point was that while I pretty well knew what you meant, I think it’s important for us liberals to be extremely careful not to kid on the square or allow penumbras of dismissal when arguing with more conservative commenters here. And I saw quite a density of penumbra in your comment.
“Gary, who cares?”
I care. I care about the posting rules being enforced with some consistency.
And I don’t give a damn about excuses.
I especially don’t care about excuses along the lines of “my feelings on this are so very strong that clearly the rules don’t apply to me, or anyone who agrees with me!”
Whatever the damn issue is.
Because the issue doesn’t matter; there a million issues on which “but this is so important, and therefore I am entitled to be mad and violate the rules!” illogic could be applied.
Pfui.
And I believe in community standards being upheld and created by the community, by every individual taking their share of responsibility; I don’t believe in “leave it to someone else; ees not my job” as regards creating community.
So: I care.
Formal policing of the posting rules is difficult on a real-time basis. Informal policing can of course go on at any time–and it doesn’t even really need to mention the posting rules. It can be dealt with by “that doesn’t help the discussion” and the like.
As for “She missed at least one of them,” I’m choosing to interpret that as a self-disclosure.
Whatever, Gary. Bringing it up long after the fact seems a little pointless to me–ymmv, as they say.
“Bringing it up long after the fact seems a little pointless to me–ymmv, as they say.”
I brought it up as soon as I saw it. And I find labeling 12 whole hours later as “long after the fact” pretty bizarre. A year or two, sure, it would be pointless. Even three or four months.
12 hours? I don’t think so.
Really, are you now going to not comment on anything 12 hours old, since that would be “pointless”? I assume that since that’s long after, that therefore half that time, 6 hours later, is still far too late. 3 hours might merely be significantly late. Probably you should hold yourself down to only commenting within an hour and a half after something is posted, so as not to be pointless.
Or maybe not.
I hope you realize I’d object to you being (possibly) called a coward just the same, Donald. I wouldn’t find that pointless, either.
Understood Gary. I just think ignoring it (which I think is what people did) was the best thing to do. Too darn many serious conversations around here get derailed by snide remarks and sometimes just treating the remarks with quiet contempt/disregard is the best way to go, IMO. Let people go right on talking about the serious issue. If the implicit violations start happening on a regular basis, then Sebastian or hilzoy or someone will crack down.
I choose to interpret Steward Beta’s comments as Sebastian did. (If it was meant otherwise, then it was an involuntary self-disclosure.)
For the record, however: calling present posters or commenters cowards is against the rules.
Can I call future commenters cowards? ‘Cause I predict I’m going to have a hated rival in about six months or so, and I’d like to take the opportunity right now to call him a pansy.
My future sell is quite possibly always apt to be a wuss, unless I call him on it.
What makes these numbers believable to me is the combination of actual fighting and the almost complete lack of policing power sufficient to control violent crime. Not everyone who gets shot was in a battle; what I read about Iraq suggests that there’s also an incredible amount of unrestrained criminal activity.
Pursuant to that, there’s a brief article in the latest Discover concerning the problems of the scientific community in Iraq:
It goes on to discuss possible motivation for these killings and the fact that “[n]early 85 percent of Iraq’s universities have been several damaged or destroyed. Looting has emptied laboratories.”
Beautiful.
There’s also this from Smithsonian about the Marsh Arabs. Remember how they were paraded about as one of the great success stories from the early part of the war, along with the statue-toppling. Not so much, it appears.
Emphasis mine. Read the whole thing.
Iraq Body Count has just released their analysis of the latest Lancet paper. Short summary–they think it’s a pile of dog feces.
They give pretty good reasons for their view and I’m inclined to think they’re right. I don’t doubt the true death toll is considerably higher than IBC’s, but I won’t be citing this Lancet paper at all– I think lefties are going to regret waving it as a banner against the hordes of ignorant rightwing science-hating skeptics.
Of course, I could change my mind tomorrow (he says, reconsidering that dog feces line).
But I am a little annoyed at how many people are lining up on this thing according to ideological lines–I expect the right to behave like this, but the fact remains that bad as Iraq is, one should at least wonder whether a death toll far higher than anything previously estimated might be wrong, and it’s not enough to say that the statistical methodology is impeccable. Scientists, including very good ones, have been wrong before, and one paper doesn’t settle anything.
Donald, I haven’t read the IBC response yet.
But BruceR, a Canadian military blogger who doesn’t have a history of approaching these kinds of issues ideologically, provides some context that makes the Lancet numbers plausible.
And he minces no words about the science-hating character of much of the right-wing response.
The Iraq Body Count response to the new Lancet study seems to me to misstate what the study shows. For example, they say that the Lancet estimate implies that:
This criticism would be easier to evaluate if the IBC cited what the relevant numbers are, so I may be misunderstanding it. But it sounds as IBC is conflating excess deaths in the absence of the Coalition/U.S. invasion and deaths directly caused by the Coalition/U.S.
The newest Lancet report says that the number of coalition-inflicted civilian casualties has gone up every year, though as a percentage of total deaths it has been going down as the Iraqi-on-Iraqi violence has increased even more. IBC is arguing that this is implausible. Their media statistics “show” that the initial two months of the “shock and awe” invasion were by far the bloodiest as far as US-inflicted casualties are concerned, with a little under 7000 civilians dying. Once you get past April 2003, the casualties which can be clearly blamed on coalition forces (in IBC’s account) become a very tiny fraction of the total inflicted, with the exception of the two months when Fallujah was assaulted (March or April 2004–I forget which, and November 2004). The total number of civilian deaths in Fallujah (mostly US-inflicted) comes to close to 2000, in their view. Outside of the “Shock and Awe” campaign and Fallujah, IBC’s media figures for US-inflicted casualties drop to literally dozens per month. In the third year of the occupation they have 370 civilian deaths attributable to the coalition, though they add that there are thousands they can’t pin blame on, or that’s what they said in a press release that was on their website earlier this year (and presumably still is somewhere, but I haven’t looked). But they can blame thousands of civilian deaths on Iraqi or jihadist forces in that same time frame.
I’m not a fan of IBC–I downloaded their two year analysis in that came out in late summer of 2005 and much of what I cite above came directly from that. I don’t blame them for reporting what they had–what was shameful about it was the lack of qualification in the paper. They presented it as though they thought it was a valid statistical cross-section of the deaths in Iraq, when a little thought would tell you that it might be very difficult for reporters to know how many civilians are actually being killed by us. Michael O’Hanlon of the Brookings Institute, who has been very critical of the Lancet numbers, uses IBC figures, but he also claimed (in a newspaper report in October 2005 which I can’t cite because I don’t remember the date) that the US was killing 750-1000 insurgents per month. So about 10,000 insurgents per year, with only a few hundred clearly-identifiable civilians as “collateral damage”. Compare that to any other guerilla war you’ve ever heard of.
All that said, IBC came under heavy criticism because of a campaign by the leftwing media criticism group called medialens early this year. I participated. So they struck back with their analysis “Speculation is no substitute” which is on their webpage alongside their newest assault on the second Lancet paper. I don’t agree with everything in the two IBC responses, but I’ve read through them and it has altered my opinion of the Lancet papers. And Les Roberts has been pretty sloppy in some of his public statements. His “derivation” of a death rate from an otherwise very interesting statistic in a New England Journal of Medicine paper was simply embarrassing. (The statistic was about the percentage of Iraq war vets who say they killed civilians. You simply can’t use that as a way of calculating the number of civilians killed, because 10 guys could have, for instance, shot up one civilian car.) He wrote an online paper comparing various Iraq death toll estimates and made an arithmetical mistake which made IBC’s daily rate seem much lower than the others, when that turned out to be wrong. IBC also says that some of Les Roberts’s other public statements are just flat wrong–he claims that the Iraqi Ministry of Health is missing most of the deaths from natural causes and IBC member Josh (as I posts at Tim Lambert’s blog) says this is simply incorrect. I haven’t checked that one out yet.
I’ve been a little overly emotional about this, because I’ve been a participant on the Lancet paper side of the argument and started realizing that it’s not the clearcut morality play of good guy Lancet supporters vs. silly arrogant mainstream media-loving IBC supporters and disillusioned people get angry. So that’s part of what’s in my response. But I’d urge people who think this is simply science vs. rightwing ignorance to read the IBC response . (Not that one should accept everything they say either.)
As for the true death toll, there’s plenty of space between 50,000 and the (unknown) civilian fraction of the 400-900,000 in the second Lancet paper and I think our knowledge of the true figure is pretty much where it was a week ago. I’ll read BruceR’s response. I’ve read Juan Cole and several sciencebloggers all defending the plausibility of the Lancet numbers. I’m convinced Iraq is a horror show, but don’t know about the actual scale.
I pretty much mirror Emerson’s take at CT, which is that if the number is 100,000, it’s too damn many, so this debate doesn’t really do much for me. But I’m wondering what y’all think about the discussion at CT about the number of airstrikes. Also, there’s this over at TPM.
If what we are doing in Iraq militarily still involves heavy use of air strikes, then we are a major source and cause of that violence to an even greater extent than I had imagined, and in a random and indiscriminate way which undermines anything we try to accomplish in Iraq politically.
I think that we’ve too often underestimated death tolls for me to simply blow off the Lancet survey. Not saying that anyone is here, but if we simply accept half the lowest bound, we still have caused an immense amount of misery.
My rejoinder to John Emerson on that point is that the difference between the mainstream/Iraq Body Count depiction of the war and the version in the Lancet papers lies in what they say about the behavior of US forces.
The Lancet version–we are fighting in Iraq the way we fought in Vietnam. Massive civilian casualties caused directly by US forces.
The version depicted in the press and reflected in IBC statistics–Once you get past the shock and awe campaign in March and April 2003 (7000 dead), the number of civilians killed by US forces is typically one or two a day in nearly all months. There are a couple of exceptions, but even the two months when we attacked Fallujah are small (2000 total civilian dead) compared to the sorts of death tolls that mainstream outlets are telling about now, inflicted by Iraqis on each other.
The difference, then, is not in whether the Iraq War was a good or bad idea. The difference for Americans is whether our forces have been killing one or two civilians a day or 100 or 200 a day. It’s a difference in magnitude that amounts to a difference in kind.
I fail to see how this is unimportant.
It is, however, a completely unimportant point to the American press or American political culture. I’m not dumping on you, LJ, but this has been driving me nuts for years. Watch the mainstream press–there are stories about our failure to “provide security'” (i.e., stop Iraqis from killing each other, not stopping us from killing Iraqis). There are stories about dead and wounded American troops. There are even stories about the occasional outright atrocity like Haditha and about the (handful of) people who have died under torture from US forces. And there are stories about the policy failures, the corruption, the incompetence and/or lies that led us to war. All very very important and all deserving a lot of coverage.
What you need a microscope to find are stories about the level of death that US forces might be inflicting on civilians in the normal course of military operations (as opposed to cases where soldiers go out of control, as at Haditha.) Maybe it’s small, maybe it’s large, but how the hell can anyone tell from reading the press? Though there are occasional hints of a problem, when debate surfaces over a hearts and minds approach vs. a more violent “kinetic” approach. George Packer was dancing around the issue in a New Yorker article last spring and so was John Burns in a June 4, 2006 Week in Review piece in the NYT. But the unstated consensus in the mainstream seems to be that the issue doesn’t matter, except to the extent that some unnamed cause might be leading Iraqis to favor attacks on US forces.
BTW, one number that supports the Lancet claim is the number of attacks on US forces claimed by Bob Woodward. Hundreds per week, I think, and if I recall, the point of that story was the level of violence had been covered up. So I waffle a lot on this issue in the past few days.
What you have in Lancet v. IBC is two opposing views of what is happening in Iraq. The Lancet view, based on their figures, is it’s a full-blown civil war, in which outside forces are playing a part. The IBC view is that it’s a serious disturbance but mostly life goes on in a fairly normal way.
So IBC’s latest “Reality Check” basically just says: if the Lancet is right (a) Iraq’s administrative infrastructure has fallen apart so that information supplied by the government is worthless; and (b) one in 15 adult males have been killed and that’s just the average.
To which Burnham, Roberts and Co. might reasonably reply: exactly so; that’s just what happens in a civil war, guys; lots of men get killed and the central authority either collapses or it becomes one of the belligerents – either way you trust the official figures at your peril.
Thanks Donald, no offense taken. There is just a feeling on my part like ‘what number do needs to die to take any of this seriously?’ which accounts for my indifference about the statistical argumentation. I’m not saying that this is a license to toss any old number into the discourse, but in confronting the stone cold silence by the MSM, I have a hard time getting outraged by a number that is bigger than it should be, which is a very bad sign, at least for me personally and morally. I think about the times when discussion was at attempt to try and figure out which factions were involved, and understand the politics, while now, it is just a sort of smear of irrational violence. Death in Iraq, regardless who is responsible, just seems to have reached the kind of position in the public eye that black on black urban violence, or disappearing non-white women, or prison rape seem to occupy. Which is to say, out of sight, out of mind.
Donald: “The Lancet version–we are fighting in Iraq the way we fought in Vietnam. Massive civilian casualties caused directly by US forces.”
The Lancet study says nothing about the way we are fighting. It only talks about the number of excess deaths, which it extrapolates from surveys, and about the number of those that are due to violence, Iraqi/Iraqi and US/Iraqi.
About the IBC response: a lot turns on how likely you think it is that a large number of deaths would go unreported by the media, and unrecognized by the Iraqi ministry of health. As to the former: my understanding is that there are large chunks of the country where the media simply don’t go. I find the idea that people are dying in these areas without its being reported completely plausible.
About the ministry of health: it would be very important, for these purposes, to know exactly how the ministry aggregated death figures. Does it, for instance, have the capacity to get death statistics from each municipality (or whoever gives out death certificates)? Is there any reliable way in which the number of death certificates is determined and aggregated?
I have read a number of accounts that suggest that the answer is: no, there isn’t. Some places — e.g., the Baghdad morgue — seem to keep statistics; other do not; and in any case the ministry does not do much of a job of aggregating them.
I do not myself know the answer to this question. But I don’t find the idea that this is one of the many things that has broken down in the midst of a civil war following the wholesale replacement of a government all that implausible.
For that reason, I find myself left with the thought: there are some researchers who have done a study that found a whole lot of deaths. I have read the study and haven’t found anything that calls its results into question. I’m not really competent to say, though. But I do note that the people who wrote the study are very competent, and moreover that other competent people think it’s a good study. Still other people have questioned their results, but on grounds that don’t seem to me all that solid. The upshot, for me, is that while I wouldn’t stake anything of value on any particular figure, I nonetheless have yet to see a reason to think that this study is off base.
Hilzoy, the Lancet says roughly 200,000 deaths caused by coalition forces and that’s just the identified part. I don’t have the paper handy, but there were also hundreds of thousands of deaths from unknown causes. It’s unlikely all those people killed by Americans are insurgents.
Among the 600,000 violent deaths there are about 13 percent (if my memory is correct) attributed to air strikes. That’s the kind of thing I’m talking about when I bring in Vietnam. The Lancet paper doesn’t imply that there are massive numbers of My Lais or Hadithas, but then, only a small proportion of the civilian casualties in Vietnam are usually attributed to those sorts of behavior on the US side. According to Neil Sheehan (A Bright Shining Lie) and others (including people like Chomsky, for that matter), it was the indiscriminate use of artillery and air power that did the bulk of the killing on the American side. The implication of the Lancet numbers is that something similar is happening in Iraq.
Something that would be interesting, in light of the IBC’s skepticism about the Lancet study, would be a cross-reference of the Lancet’s raw data against the IBC’s data. That is, the Lancet has a couple of hundred reports of named individuals who have died violently since the beginning of the war — it would be good to know how many of them showed up in the media reports collected by IBC. If all or most of the violent deaths reported in the Lancet study show up in the IBC’s database, that would indicate that there may be something to the IBC critique, whereas if only a small percentage show up, that would cut in the other direction.
LJ, what I’m saying (or part of it) is that it’s the US-inflicted violence, whatever the actual level, which is out of sight and out of mind. It’s the subject that is almost never even mentioned, with the exception of cases like Haditha. There is no shortage of stories on Iraqi-on-Iraqi violence, though if the Lancet paper is correct even that has been greatly under-reported.
The military deliberately chose not to engage in regular announcements of bodycounts. Bodycounts made them look bad in Vietnam. From a PR standpoint, their new policy has been very effective. In the US, anyway. Quite a few Iraqis seem to favor attacks on US forces for whatever combination of reasons.
Les Roberts claimed last summer that of the 21 violent deaths in the first Lancet report, 20 didn’t show up in IBC’s files. JoshD (a member of IBC who shows up online and defends IBC and attacks the Lancet team and who co-authored this latest response) said this was ridiculous, on the grounds that much of IBC’s data is in the form of morgue data and not specific accounts of actual violence–if the 21 violent deaths were among those represented by death certificates, it’s entirely possible (according to Josh) that all of them showed up in IBC’s dataset, but there would be no way to know.
So it may not be easy to check.
I liked Les Roberts’s suggestion–have (suicidally brave) reporters go out and do checks with some random gravediggers.
it was the indiscriminate use of artillery and air power that did the bulk of the killing on the American side. The implication of the Lancet numbers is that something similar is happening in Iraq.
The Sunday Times reported in January 2006 that US airstrikes had stepped up in the last months of 2005. I recall reading this in other UK papers, too, but this one was the first to come up when I googled for a news report just now.
Here is a Knight-Ridder article from March/06.
A review of military data shows that daily bombing runs and jet-missile launches have increased by more than 50 percent in the past five months, compared with the same period last year. Knight Ridder’s statistical findings were reviewed and confirmed by American Air Force officials in the region.
IBC claims (in their response last spring “Speculation is no substitute”) that the number of air strikes in the “shock and awe” phase was much much greater than it has ever been since. I don’t know if that’s true–I also don’t know how much it tells you if it is. An airstrike on a house occupied by insurgents with civilians present will kill more civilians than a strike on an empty office building or on military positions out in the desert.
The military deliberately chose not to engage in regular announcements of bodycounts. Bodycounts made them look bad in Vietnam.
This is true, but not for the reasons you’re suggesting. Body counts in Vietnam placed the focus in the wrong area, as a counterinsurgency fight isn’t won by killing people, but body counts encouraged just that. Body counts also created confusion about the war’s progress, since body counts in no way correlated with success in the war. So the DoD made the decision to avoid body counts in Iraq to avoid those problems.
Which is not to say that it may be leading to other problems.
I’ve read that Andrew, but didn’t see it as conflicting with what I said (which just repeats what I’ve seen somewhere else.) Bodycounts had a ghoulish flavor.
I probably misunderstood what you were saying, then, Donald. That’s what I get for coming in late.
Andrew: Body counts in Vietnam placed the focus in the wrong area, as a counterinsurgency fight isn’t won by killing people, but body counts encouraged just that.
*blinks* Well, from all reports and from all evidence, the US military is not discouraged from killing lots of people just because they’re not running a tally. I was going to say “rather the reverse”, but in fact, there is a rather horrid similarity between Vietnam and Iraq in that way, as well: the bulk of direct victims killed by the US military are probably dying by US airstrikes, and there is a general contemporary denial that US airstrikes are killing civilians – still less that the US military knows that when an airstrike is ordered, this means US military killing civilians.
There is an interesting discussion going on at scienceblogs comparing the Lancet 2004 report with the Lancet 2006 report.
The short version is that Lancet 2004 found a huge increase in excess deaths, of which a huge portion was “non-violent” deaths. At the time we talked a lot about how it made sense (from an infrastructure point of view) that there were lots of non-violent deaths which were nevertheless caused by the war.
If you take Lancet 2006 and examine it for the exact same period covered in the Lancet 2004 reports, you see an even larger number of “excess deaths”. You also see that 100% of them are “violent” deaths and that there was a small reduction in the “non-violent” death category. Taken together, the 2 Lancet reports seem to paint a hugely different picture of people dying in Iraq. Lancet 2004 painted a picture of hundreds of thousands of excess deaths–40+% of which were non-violent excess deaths. Lancet 2006 paints a picture even more hundreds of thousands of excess deaths–none of which were non-violent. That seems very odd.
The standard response appears to be something along the lines of this (by ‘Robert’ at Tim Lambert’s:
Statistically speaking that is correct to a point. The problem is that it is a bad explanation for the magnitude of the difference between the studies. Posit a highway accident rate of actually 1,000 per day consisting of (on average) 600 multi-car accidents and 400 single-car accidents. On any given day it might be 550/450, or 300/400 on a light-accident day. It might be 490/510 on some day. If you sample the wrong days, when you extrapolate, you might very well get a pretty wrong picture and think that 50% of the accidents are single-car. But if your methodolgy shows 0% single-car accidents, and 10,000 per day (an order of magnitude above many other reports) I would strongly suspect that your sampling procedure is flawed.
Compare also to the UNDP ILCS report which was a survey of 21,000 households (10 times the number of the Lancet Study).
Oh, and I’m in the middle of writing a proxy statement, so don’t take a further lack of response as a lack of interest.
Compare also to the UNDP ILCS report which was a survey of 21,000 households (10 times the number of the Lancet Study).
Minor, minor point: comments like these are not particularly useful when talking about methodology and sampling.
the US military is not discouraged from killing lots of people just because they’re not running a tally.
Not having been there, I can’t speak to that. But the fact remains that if you set a marker up like ‘body count’ and use that as a proxy for success, you will encourage people to add to that number however they can. Removing that incentive doesn’t mean that people still won’t get killed, nor necessarily even that people won’t get killed at the same rate, but removing it as an incentive still seems a wise policy.
As I said, I haven’t been to Iraq, so I don’t know what the situation is on the ground. But I do know that when we train units to go to Iraq, rules of engagement (ROE) is an important part of the training, and an important part of ROE is determining when it is permissible to engage a target when that attack will also kill civilians.
I think this is right. The “Shock & Awe” part was mostly military: bombing tanks, weapon emplacements, logistics, etc. It seems reasonable that the largest number of airstrikes should have occured in the (roughly) four weeks of armed (conventional) conflict.
The fact that (post conventional conflict) the Air Force has ramped up airstrikes does speak to increased civilian casualties. A significant percentage of recent airstrikes should be aimed at civilian/insurgent targets in urban areas. These (no matter how precise) run the risk of killing nearby civilians (not to mention the risk of dropping the bomb on faulty intelligence, such that the Air Force may have bombed the right house, but it wasn’t an insurgent house).
I’m not sure the the US/Iraqi figures (from Lancet) are particularly reliable. As noted, the methodology was self-reporting (teams asked households about deaths pre/post March 2003); while deaths were substantiated with certificates (about 80% were), the cause of deaths was less so. After all, blaming a death on American airpower may be both smart (laying blame with local insurgents may get you killed) and politic (not many like the US these days). Thus, it seems the overall number (655k) is more reliable than the number of deaths from US forces (200k).
(NOTE: This is deliberately not an argument about the overall reliability/accuracy of the Lancet study, but about the relative reliability of one sub-category within the study, compared to the numbers the overall study generates.)
Andrew: But the fact remains that if you set a marker up like ‘body count’ and use that as a proxy for success, you will encourage people to add to that number however they can.
Yes. Of course, one solution would be to set a marker up like ‘body count’ and use it as a measure of failure. Which is, in fact, what is happening -= but on an extra-military basis. It would be deeply interesting to see what would happen to a military that believed that civilian deaths were always a mark of failure, and routinely punished/demoted/shamed soldiers who were responsible for killing civilians. What would in general happen, I imagine, is that the lower-ranking soldiers who were directly killing civilians would get landed with all the blame, while the generals who gave orders for airstrikes on cities using cluster bombs would get rewarded, but it is a momentarily interesting idea: a military that behaved as if it really believed that You’re not allowed to kill civilians and imposed penalties on its own personnel – no matter of what rank – accordingly.
Baltar: I’m not sure the the US/Iraqi figures (from Lancet) are particularly reliable. As noted, the methodology was self-reporting (teams asked households about deaths pre/post March 2003); while deaths were substantiated with certificates (about 80% were), the cause of deaths was less so.
That may have been a factor: it would be interesting to know if the death certificates were clear enough about the cause of death to know that it must have been airstrike rather than a ground assailant (US or Iraqi).
Except that war isn’t as neat and tidy as we’d like it to be. Pushing hard to minimize civilian casualties is a good thing, and perhaps a body count measure that counted strictly civilian deaths would be a good thing (although difficult-to-impossible to pull off in a counterinsurgency fight) to use as a measure of success (fewer civilian deaths=better success in COIN). But you can’t just say that no civilian deaths are acceptable, because that doesn’t work. What do you tell a soldier who’s about to be engaged or has been engaged by an attacker in a crowd of civilians? Can the enemy engage us with impunity as long as he hides among civilians? How long do you think such a policy would last?
What are the rules of engagement, Andrew? If it’s a long detailed statement about what to do in various different situations, maybe you could provide a link, if one exists.
But you can’t just say that no civilian deaths are acceptable, because that doesn’t work.
*points at self* Pacifist talking, remember?
What do you tell a soldier who’s about to be engaged or has been engaged by an attacker in a crowd of civilians?
That they’re not allowed to kill civilians. Soldiers who know they can kill civilians with impunity will do so.
I’ve been reading the inquest results on Terry Lloyd this past weekend: he and his crew were caught in crossfire between Iraqi and US forces. The US forces started firing: the Iraqi forces fired back: two of Lloyd’s crew were blown apart: Lloyd himself was hit by an Iraqi bullet, and might have survived, but the US tanks then fired on a civilian minibus that had stopped to pick up the wounded and had made a U-turn to get away – and, among others, killed Terry Lloyd. The coroner has said this was unlawful killing: as it surely was under the Geneva Convention. It remains to be seen if the US soldiers who killed him will be prosecuted for their criminal behavior.
I don’t doubt, though, that though we know specifically about Terry Lloyd because he was a well-known journalist, one of his crew survived to testify, and he was a British subject, therefore getting a full-scale investigation and public inquest. How many Iraqi civilians were killed by US soldiers who will never be prosecuted for their criminal behavior?
I got into this argument on the Slacktivist thread with several people who argued that it couldn’t possibly be true that the law requires a soldier to die rather than kill a civilian. It reminded me, rather unfortunately, of young male drivers in a class I once took who couldn’t believe that UK law held them responsible for any accident involving a pedestrian: the same utter disbelief that the person who is controlling lethal machinery shall be held responsible for killing or injuring someone who isn’t so armed or protected.
Donald,
The ROE are not available to the public. Because they establish rules for when a soldier may or may not engage the enemy, they’re kept under wraps to prevent the enemy from determining how to protect themselves.
The ROE are not available to the public.
The US army may not like to make their ROE public, but the British Army has long since learned the value of making clear to civilians how to avoid getting shot by British soldiers: see Yellow Card and Green Card.
Huh. Well, that makes sense, Andrew, but it has the unfortunate side effect of making it difficult to determine how plausible it is that US soldiers could be killing a lot of civilians and still be operating within the rules.
As I said, Donald, I can’t speak to that. I know that we take the matter very seriously in training, but things are always different when you get on the ground.
Donald- Comment 150 in the Crooked Timber airwar thread has a neat analogy that answers one of your objections.
“Anyway, you’ve given us pies, custard, sandwiches and bank accounts, but here’s the analogy I have in mind. You have a jar with a number of roughly nut-sized things in it. That’s your population. You reach in without looking and pull out a handful. That’s your sample. On counting, most of the sample turns out to be pistachios. The rest are peanut M&Ms. So now you can make a statement about the likely proportion of peanut M&Ms v. pistachios in the jar.
But looking closer at the M&Ms, you find that three of them are blue. You can now make a statement about the likely proportion of blue M&Ms v. pistachios in the jar.
My suggestion here is that your estimate of blue M&Ms is going to be a fair bit less confident than your estimate of M&Ms of all colours.
And consider your first estimate. Was it ‘made up’ by adding together a series of separate estimates of M&Ms of various colours? No.
Posted by Charlie Whitaker · October 16th, 2006 at 3:58 pm ”
Thanks Frank. I saw it. That’s more of an answer to Mike H’s objection–the violent vs. nonviolent question. I’ve left that one alone, and thought the M & M analogy was a good way to handle it.
It’d be nice to have all the objections, good and bad, in one spot for the epidemiologists and Iraq experts and human rights experts and military experts (and whoever else might be relevant) to examine. I’ve had a few–
1. The death certificate thing–Not sure what to make of this. The LA Times did call various hospitals and morgues and only came up with 50,000 deaths. Did people lie or did the individual hospitals not keep good records or did the LA Times not call enough places or the right places or what? Don’t know. I don’t have a problem saying the Iraqi government itself would lie or cover up this sort of thing, because there have been stories about that already.
2. My other objection is that the ICLS survey gave 24, 000 deaths with a confidence interval of about 5000 on each side for the violent deaths (excluding criminal murders) in March 2003-April 2004. The violent mortality rate for the same period in the new Lancet paper is 1.8 to 4.9 per thousand per year, or (using 25 million as the rough population) 45,000 per year at the low end and about 125,000 at the high. Correcting for an extra month gives about 50,000 at the low end and the midrange figure (3.2 mortality) is 80,000 per year or nearly 90,000 for 13 months. So you’d have to add 20-30,000 criminal murders to bring the ICLS Confidence interval up to the lowest end of the Lancet CI. And this is the Lancet’s CI for that time period–it’s not me taking out a few of their colored M&M’s and arguing about it. To me it suggests that either the ICLS study or the Lancet study had some kind of bias that skewed the results, though it’s barely possible that the low end of the Lancet CI is consistent with the ICLS CI.
3, Iraq Body Count also says there are a huge number of missing wounded, which also occurred to me. But I’m not sure about this either. With 40 percent of the deaths coming from car bombs, air strikes or other explosive ordnance, that’s 240,000 deaths, so you’d expect several times that many wounded. Taking the low end of the CI, you’ve got 160,000 deaths and still a lot of wounded. I don’t know if that’s a big problem for the study or not–Tim Lambert doesn’t think so. There’s also all the gunshot killings, but if most are death squad style executions there might not be many wounded. There would be a lot if it’s US forces inflicting the deaths with gunshots (because only a tiny fraction of our troops perform Haditha-like killings.)
Too bad the threads on most blogs on this subject are such zoos. You have to wade through so much posturing and insults (some of them funny, admittedly) to have any kind of discussion. Not that I’m completely innocent in all this–my dog poop comment above was the kind of thing I’m complaining about.
“My suggestion here is that your estimate of blue M&Ms is going to be a fair bit less confident than your estimate of M&Ms of all colours.”
I don’t think this is a good analogy. The first survey showed almost half non-violent deaths, the second almost zero. That is too big of a difference considering that violent/non-violent is a binary distinction that you aren’t likely to get wrong very often.
Methodologically I think the selection of households was problematic and the explanation of the survey was almost certainly off–considering that they did 40 per day. The first household was selected at random, but the subsequent 39 were selected by proximity and word-of-mouth eliciting volunteers. This could easily cause some double counting of deaths if any of the households were related to any of the other households and the survey wasn’t very well explained to the Iraqis in terms of the definition of “household”. Since 15 minutes per household gets you to a 10 hour day (not counting travel time to the neighborhood or travel time between households), there likely wasn’t much explanation of the survey.
Mark Kleiman was also bothered by the idea that one team of four surveyors could survey a cluster of forty households in a day.
I don’t see the problem. In the 1849 households visited there were 629 deaths. So for at least two-thirds of households there were no deaths to report; it was simply a matter of determining the number of people in the household in 2002, noting any births or other arrivals during 2002-2006 and any departures. How long does that take? I reckon about ten minutes.
Since the IBC critique is basically an argument from incredulity, I suppose it was only to be expected that d-squared has come up with the best response to date, responding with his own argument from incredulity. If the IBC count is correct, how bad does that make the situation in Iraq? About as bad as Northern Ireland in 1972 or Jamaica in 2005.
I should add that IMHO that’s why the debate is going nowhere. One side has the idea that Iraq really is like Northern Ireland in a bad year and the other side is thinking it’s more like the Bosnian civil war. To see how the other guy might be right you really have to reject your own reading of the situation completely and think of Iraq as a different place from the one you have been following. If you normally rely on Juan Cole, try telling yourself that Red State or Winds of Change really have the facts right – and vice versa.
Where’s dsquare’s analogy with Northern Ireland and Jamaica? I’ll look for it later, but I’m lazy and if you’re still around, Kevin, a pointer would help.
Double-counting of deaths should be prevented if they had death certificates.
The question of Northern Ireland came up here, but I’m not sure if dsquared brought it up.
There is also Daniels Guardian comment is free column, which is a pretty full throated defense of the study. If you’ve seen those, apologies.
Sorry Donald, I should have given a link to d-squared. It’s here at his own blog. Of course IBC don’t pretend to pick up all or even a majority of deaths, but many of the people (like GWB) who cite their figures affect to believe they do. It’s people like that he is really getting at, not IBC; he doesn’t go in for attacking them.
As to double-counting, I really can’t see how that would be possible. If two households in the same cluster reported the deaths of two people in the same age-group, around the same time, with the same cause, the researchers would have to be asleep not to spot it. Most households report no deaths at all.
Thanks for the links, LJ and Kevin. I’d seen one of them, but not the others.
In the spirit of on-topic linkage: Lenin’s Tomb on the IBC response.
For the interested, one of IBC’s authors has turned up at Tim Lambert’s blog
Mr Dougherty posted precisely the same spiel at the Persistence of Vision Message Board. I chatted him up about it. He strikes me as somewhat excitable in defense of his confederates.
Baghdad Burning – The Lancet Study…
Courtesy of the Whiskey Bar – Down the River
Nice Job, all ye great patriotic war bloggers.
Rosinante’s Rider, please read the posting rules (05/19/2004 update). Overly-long cut’n’paste comments are not welcome here, and this comment (IMO) is verging on that.
This is Don Q, after all, and he’s decided the posting rules need not apply to him.
Don Q?
*double-takes*
Oh, yeah.
He usually posts with spelling mistakes.
Kid Bitzer wrote:
‘They seem to be saying “this is the best we can do” not “this is really good data”.’
Sure. But if it’s the “best we can do,” than on what basis are people rejecting the findings? Sure, we want to recognize they might be wrong — but that means they might even be too low!
“I just don’t believe it!” is no counter-argument at all.
Gene C.–
thanks for the shout-out and all, but I think you’re confusing me with Sebastian Holsclaw. As far as I can tell upthread, the prose you quote was his, not mine.
We actually hold different positions on certain issues. The appropriate response to the Lancet study being one of them.
Hard-science objections discussed in Science.
Thinking that this thread was an ex-thread, dr ngo contributed to this discussion in an open thread. Given that It lives!, I’m linking (and recommending) his comment.
The Science article is subscriber-only. rilkefan, if you have access to the full article, it would be a favor to this discussion for you to quote crucial criticisms Bohannon makes of the Lancet methodology, and to let us know what is his area of expertise. (Given that the article appears under ‘Epidemiology’, I’m assuming Bohannon’s an epidemiologist, but the excerpt available to non-subscribers doesn’t say.)
Is it known who were the peers who reviewed the newer Lancet study? [Is there a convention about naming peer reviewers?]
Lenin: [Josh Dougherty] strikes me as somewhat excitable in defense of his confederates.
Yes, and also to have time to spend hashing out the issue in online discussions. I wouldn’t think anything of that if not for this note at the end of the Iraq Body Count response to the Lancet study on their site, which implies strongly that the staff don’t have time to discuss or defend their criticisms with reporters:
The insinuation that all the Lancet study did was to spend a lot more time to document 300 deaths than the IBC routinely does is a nasty piece of work, undercutting the far more respectful acknowledgement in the introductory section of their reponse:
Everyone could benefit from taking a deep breath and reflecting on our shared agreement with that last sentence. Including its authors.
Nell: Bohannon’s website doesn’t include a bio, but you can view the articles he’s written (for Science and other media outlets) here.
According to one archived piece, “John Bohannon earned a DPhil in Molecular Biology from Balliol in 2002. He moved to Berlin as a Fulbright scholar last year.” Elsewhere, he is identified as a Berlin-based writer.
From what I’ve read, Bohannon’s articles are straight journalism with little opining on his part, so I assume the article Rilkefan linked is objectively conveying the objections of other experts in the field.
At least, one would hope it’s not simply more territorial pissing from IBC.
Tim Lambert has a go at this latest critique here. It’s a bit soon to say whether there is anything in it.
Sorry about the link. The bit that caught my physicist eye:
Hm. THe methodological problem pointed out would tend to UNDERCOUNT, wouldn’t it? Because where they died isn’t the same as where they lived, hm?
The (disputed) claim is that they sampled near arteries, hence should expect more casualties. Dunno if that’s relevant for aerial bombardment, would think not.
Weird about deleting some of the data.
I could understand deleting the data–the reason for doing so is given. If Iraq is as dangerous as the study suggests, then you’d have a moral obligation to protect the interviewees that would outweigh normal scientific practice. Of course, it does make it difficult to tell if there is a neighborhood effect in this study.
There were only 47 clusters, though. I find it hard to believe the surveyors couldn’t remember all 47 locations if they gave it some thought.
As for IBC, there’s been a pretty unseemly amount of backbiting and dislike between them and various Lancet supporters (including me for a short time). I think that for some of the participants it’s become more important to “win” the argument over how many Iraqis are dying than it is to either stop the deaths or hold the real villains to account. It would be humerous if the subject matter didn’t make all this egoizing more than a little appalling. But it’s rather like debates you read about in other areas of science–I remember something in SCIENCE once, where an ornithologist who did not believe birds came from carnivorous dinosaurs referred sneeringly to the “paleobabble” of the paleontologists who believed differently. So much emotion about Archaeopteryx. Very funny from the outside, but in this case the issue is whether the Iraqi death toll is 60,000 or 600,000. People should grow up.
I should tone down that accusation–it’s not more important for some of the participants to win the argument about the deaths than to stop the war or get the correct figure. But it is obvious that winning the argument has taken on too much importance for some. (And so as not to sound too self-righteous, I actually just have a few people in mind and for a short period, one of them was me.)
You just enter the neighborhood data into a blinded database and you never use the key where you can see the results. That’s what people do in physics now to avoid bias – you never know the result until the analysis is complete. So in this case one’s code could do some sort of distance-to-artery comparison but no one would know where the neighborhoods were.
I hate to play the paranoid liberal, but given what the admin has been willing to do to demonize opponents, I could see a demand to the Lancet/researchers to turn over their data because clusters could reveal something about the insurgency, just to have them refuse so as to create even more FUD (the current situation makes the original application of the acronym to Microsoft seem quaint)
The original surveyors, the people who actually asked the questions, would know where they went. I would think the Iraqi member of the team could go back and ask them, if he doesn’t know himself.
Rilke, are you’re talking about how to keep the data secure from outsiders? I didn’t follow your point.
I continue to feel guilty about my post about the ego-driven pro-and anti-Lancet debaters. I’ll narrow the target of my accusation still further–if you were hanging around the medialens comment boards last spring, it was kinda obvious on both sides. I’ll leave it at that.
I’m talking about keeping the data invisible to everybody, only open in the guts of some program. You encrypt the data when it’s entered and you keep the decryption key elsewhere, and you don’t design your software to expose dekeyed info.
The admin can’t care about such a small sample. They don’t have the troops to deal with the insurgents they already know about.
rilkefan, thanks for the excerpt.
… the scientific community should call for an in-depth investigation into the researchers’ procedures. “It is almost a crime to let it go unchallenged,” adds Johnson.
The tone does not strike me as particularly scientific. “Almost a crime?”
This kind of language is an explicit slam at the quality of the peer review of the Lancet study.
It would be reassuring to know that the information on which streets were surveyed was recorded and encrypted at the time, and was still available.
Is there anyone who regularly reads scientific or medical journals who can answer my question about the customs wrt identity of peer reviewers?
Have returned from reading the Tim Lambert link provided by Kevin, and have formulated a new internet law:
Anyone who tells Donald Johnson to ‘put a sock in it’ automatically loses the argument.
The Science piece is science journalism, not a critique by statisticians or epidemiologists. Here’s the part dealing with criticisms by researchers (as opposed to the part dealing with criticisms by e.g. Bush):
The (disputed) claim is that they sampled near arteries, hence should expect more casualties
Um, no. They sampled HOUSEHOLDS near arteries. The casualties are in marketplaces, roads, which are, in truth, near arteries, but the people in those marketplaces are not necessarily from households near the marketplaces. The critique makes the unwarranted assumption that the people killed are disproportionately living in the heavily travelled areas (folks from less travelled areas could well use the arterials/marketplaces and could be the majority of those in the area of attack).
In other words, this may be a methodological flaw, but it’s one that quite possibly underestimates the true count.
rilkefan,
I appreciate the skepticism, but the hindsight works both ways. One can either accept the reasons for destroying the data, and suggest that it was possible to do it better, or one could take the fact of the destruction and present that as evidence that the study was biased and determined to get a number that was higher than it actually was. Again, I know you are just being cautiously skeptical, but some of the argumentation takes the data destruction as reason to reject the study out of hand. There is a possibility that the researchers feel a higher calling and are skewing the results to match their own biases and they know it and are destroying the data to cover up that fact. When looked at in that way, this is not simply a charge that they let their biases sway their conclusion unconsciously, it is an accusation that they purposely lied. I think simple fairness would have us step back from that accusation.
“Is it known who were the peers who reviewed the newer Lancet study? [Is there a convention about naming peer reviewers?]”
Nell,
I believe they call them referees in the UK. They normally remain anonymous. When the 2004 study was published Professor Sheila M Bird disclosed that she was one of them, when she published a related comment in the Lancet. She suggested that if the UK government wanted better information there were various steps they could take, including declassification of relevant military data and commissioning a larger survey (she suggested using four time the number of clusters which would halve the uncertainty). If you want to see that article you’ll have to register (which is free) and then can read it here if you’re interested.
I haven’t seen any report of the 2006 referees going public. Most likely they would only do so if, like Bird, they were publishing a comment on the paper.
lj, I see no reason to destroy the data, that’s all. If the researchers can’t defend their study from bias estimates generated from reasonable models consistent with an agreed-upon understanding of the possible range of sampling methods, they should widen their uncertainty estimate.
There seems to be a claim in the Lambert thread that this sort of objection has been raised before with the group in question; if so, and the bias claims hold up, I’ll be rather unhappy about the study.
gwangung, I’m not a sociologist, but I can well imagine causal relationships between living near arteries and having characteristics that are risk factors. It’s the job of social scientists to understand the correlations and either design against them or control for them. The Lancet people ought to be able to say “Our sample is flatly distributed across variables x and y to level L +/- blah and we put that into our uncertainty estimate after correcting for the estimated bias, which is B”, and I would look to see if delta-B is small relative to the claimed effect and the quoted uncertainty. Maybe they can say that and this objection fails. Maybe there’s a missing large positive bias and the corrected number looks too large. I don’t know.
Also note that it’s perfectly normal for studies to be argued over.
gwangung, I’m not a sociologist, but I can well imagine causal relationships between living near arteries and having characteristics that are risk factors. It’s the job of social scientists to understand the correlations and either design against them or control for them. The Lancet people ought to be able to say “Our sample is flatly distributed across variables x and y to level L +/- blah and we put that into our uncertainty estimate after correcting for the estimated bias, which is B”, and I would look to see if delta-B is small relative to the claimed effect and the quoted uncertainty. Maybe they can say that and this objection fails. Maybe there’s a missing large positive bias and the corrected number looks too large. I don’t know.
This is quite true, but it cuts both ways—if the flaw systematically biases the data, then the critics have a responsibility to indicate how. All methodologies have flaws; focussing on them without indicating how it affects the data isn’t very useful in a scientific debate and does nothing to improve the science. This seems to me more a political objection since it doesn’t improve or extend the science.
“if the flaw systematically biases the data, then the critics have a responsibility to indicate how”
That’s wrong – simply showing the sample isn’t homogeneous is sufficient to raise doubts about the result if the study doesn’t consider that inhomogeneity. And in any case there are clear ideas about bias mechanisms in the various discussions going on. And on top of that the very data needed to clearly challenge the study has reportedly been destroyed. Your position is the political one.
But since the study’s authors claim that they did include alleys etc., why argue about whether it would have biassed the results if they hadn’t?
rilkefan: There seems to be a claim in the Lambert thread that this sort of objection has been raised before with the group in question; if so, and the bias claims hold up, I’ll be rather unhappy about the study.
I’m not sure which remarks in the Lambert thread this refers to. However, nearly all objections to both reports (2004 and 2006) boil down to the claim that something went wrong in the sample selection. The thing is that a random sample is a beautiful thing in theory but how to get one? Strictly speaking, everyone in the country should have precisely the same chance of inclusion as everyone else. But that’s impossible. Once you decide to sample households you lose everyone who sleeps rough, as well as every household which has been completely wiped out. Given that participation is voluntary you lose everyone who doesn’t want to talk to you. It’s very probable that people who sleep rough and people who don’t like talking to strangers are, on average, more likely to have lost relatives.
One of the most contentious questions is, having narrowed your choice to a particular town, how do you give everyone in town an equal chance of being in the cluster? Ideally you would take into account the fact that some areas have higher population density than others and weight those areas accordingly. In practice nobody has adequate data to do that. So you end up assuming, in effect, that the population is evenly distributed around the town. This is unsatisfactory because it is very likely that those who live in cramped conditions have a higher mortality rate.
The irony is that most of the biases which get mentioned in this context point to an underestimate of the mortality rate.
I continue to fail to see why the Lancet team couldn’t ask the Iraqi surveyors to tell them where the 47 clusters were located. That ought to go far towards determining whether there is a possibility of a main street effect.
“But since the study’s authors claim that they did include alleys etc., why argue about whether it would have biassed the results if they hadn’t?”
Because that claim is disputed, because there’s an claim that they admit they don’t understand the sampling at that level, and because the data is gone.
“Ideally you would take into account the fact that some areas have higher population density than others and weight those areas accordingly. In practice nobody has adequate data to do that.”
How can that be? Iraq has been sampled before – _somebody_ knows how the population is distributed there – and if not then there must be sensible models available. And if not, well, you oversample one town and make an estimate of the inhomogeneity in your method if you care.
Anyway, I’ll wait for more data.
Rilkefan: Because that claim is disputed, because there’s an claim that they admit they don’t understand the sampling at that level, and because the data is gone.
What this amounts to, though, is that the Lancet study must be wrong because the people who did the sampling (or the people who gave them the instructions) were (a) dishonest (b) incompetent.
Making charges like that generally requires some positive evidence, not just “I don’t like the results” and “They can’t prove they weren’t”.
And no one has actually come up with any positive evidence for either dishonesty or incompetence, nor any reason to suggest it, other than the obvious one: not liking the results.
Given the push down automaton nature of blogs, a post at TiO has been made on this.
How can that be? Iraq has been sampled before – _somebody_ knows how the population is distributed there – and if not then there must be sensible models available. And if not, well, you oversample one town and make an estimate of the inhomogeneity in your method if you care.
rilkefan,
When I read that, it conjures up an image of a man waving his arms around in an agitated manner – something like Col. Mannering in Dad’s Army, if you’ve seen that series. Unfortunately the world is not a tidy place. I doubt whether anyone has even a remotely accurate notion of the distribution of the population in Bray, County Wicklow. It has grown quite a bit since the last complete census and rising rents and property prices have surely prompted an increase in population density. I don’t know whether immigrants are living ten to a room but it’s entirely possible. By contrast, the wealthy have large gardens surrounding detached houses from which their grown-up children have fled. So population densities are in flux.
But Ireland is a haven of stability compared with Iraq. If you take a look at Appendix 2 of the UNDP-ILCS survey you will see such statements as these: “in the northern governorates, no census exists” and “some villages that are listed in the [sample] frame turned out to be uninhabited”.
Incidentally it is unclear whether “the data is gone” – names and addresses of respondents were not recorded (for good reasons) but Burnham denies that data was destroyed.
Jes: “And no one has actually come up with any positive evidence for either dishonesty or incompetence, nor any reason to suggest it, other than the obvious one: not liking the results.”
No, you’ve got it completely backwards. You’re biased in favor of the study, so science be damned, and have to resort to a pile of straw over a pile of something else. Certainly “the data is unavailable” is better than “the data was destroyed”, but that’s not relevant to my position.
Kevin: “When I read that, it conjures up an image of a man waving his arms around in an agitated manner – something like Col. Mannering in Dad’s Army, if you’ve seen that series.”
When I read that, it conjures up an image of someone who knows he’s losing an argument and deciding it’s time to pound the table. Look, either people can do social science, in which case they should do a good job and minimize/measure/encorporate bias and present the data and defend it, or they can’t, in which case this conversation is really spectacularly useless.
“names and addresses of respondents were not recorded (for good reasons)”
As argued above I think this is a pile of something else.
As argued above I think this is a pile of something else.
That seems a bit much. We’ve already seen the argument here that because they were killed, it was likely that they should be defined as combatants rather than civilians. Is it that much of a stretch that if the logic suggests someone’s death rules out civilian status, someone might further suggest that they be visited by the authorities? Ironically, we are a lot more defensive about harvesting information on the internet than in a situation when having the wrong name might be fatal.
rilkefan,
how do you reconcile your claim that “somebody” knows how the population is distributed with the statements I quoted from the UNDP-ILCS study?
Look, either people can do social science, in which case they should do a good job….
But apparently a critic is free to make wild statements about what “somebody” must know.
There is a news story I’ve seen cited in the discussion elsewhere about a man being shot at a hospital for saying that he knew who killed his brother. So I can certainly understand why the Lancet team feels an ethical obligation to suppress or destroy information that could reveal who participated in their survey.
Hopefully they can release enough to defend their methodology, however.
Should this discussion go over to the Taking It Outside blog?
lj, I argued that the data could be unbreakably encrypted in the bizarre scenario that the data fell into the hands of the evildoers but not the surveyors and support. I skipped over as too obvious to mention data logging which would avoid identifying the exact location but answer the bias question. I also omitted as to obvious to mention much more likely scenarios leading to the evildoers finding out about who was surveyed, which combined with the collected data would be as problematic.
Kevin, you’ve managed to conflate an area with the whole as well as miss two “and if nots” in my argument and (as far as I can tell) effectively argue against your own position as noted in my previous comment.
Anyway, it seems clear enough to me there’s no progress to be made here until there’s more clarification from the various claimants.
rilkefan, I am just suggesting that they may have had reasons for not collecting/ensuring that they did not get/destroying the data that, in hindsight, might have been dealt with in a different ways that only appear obvious in hindsight. Injecting words like ‘evildoers’ seems a bit banging on the table-ish itself.
The Lancet podcast is here, it talks to one of the editors and one of the authors, FWIW.
rilkefan,
I saw the two “if nots”. The first one introduced another dubious assertion. Yes of course there are models, but you need data to estimate models. Lack of adequate data is precisely the problem. The second “if not” proposed oversampling another town, when I had been referring to the problem of sampling a given town. This could be deduced from the fact that I wrote: “having narrowed your choice to a particular town, how do you give everyone in town an equal chance of being in the cluster?” (The text wasn’t bolded in my original comment of course.) Anyway, what reason is there to suppose that the situation would be any better in a second town? These problems are likely to be widespread in Iraq.
As to conflating an area with the whole, are you suggesting that in a country where there is no census for entire governorates, there is likely to be adequate information regarding neighbourhoods within a town? I really don’t know what you are trying to say here, nor what you mean about arguing against my own position. It seems to me you are being unreasonably hard on the Lancet authors and the Iraqis who did the fieldwork for them. I’m pretty sure I haven’t argued against that position.
“I am just suggesting that they may have had reasons for not collecting/ensuring that they did not get/destroying the data that”
There may be ok reasons for not recording names and addresses (though I am frankly very skeptical). There is absolutely no acceptable reason to avoid collecting routine demographic data so that you can find out how closely your sample reflects a random cross-section of the population as a whole. That is Stats101. If you take a poll, there is a reason why you have all those demographic questions at the beginning–the sample verification is absolutely impossible without it. I don’t even understand what “peer-reviewed” statistical study means without the ability to verify against the demographic data. You essentially have removed the ability to compare your sample to what passes for a control group in statistical studies.
Is it my imagination, but aren’t the last two comments talking past each other?
Seb: it was my understnading that they did, in fact, collect demographic data, and that the claim that they didn’t was in error. I think I saw this on deltoid, but could be wrong.
Donald Johnson: I continue to fail to see why the Lancet team couldn’t ask the Iraqi surveyors to tell them where the 47 clusters were located.
This or a similar claim has been made several times, implying that the authors of the study do not actually know where the clusters are located. However, from the article it is my understanding that one of the authors, Riyadh Lafta, was acting as a “field manager” of the survey and was part of the survey team. From p. 2 of the Lancet article:
The two survey teams each consisted of two female and two male interviewers, with the field manager (RL) serving as supervisor.
While not all of the authors might know the details of the sampling, at least one of them does know exactly which neighborhoods have been surveyed.
Incidentally, the presence of one of the authors in the survey team also has some bearing on the question of possible misrepresentation by the Iraqi surveyors, on which some people have speculated. For this to take place, you would have to assume the cooperation of at least one of the authors, so this would in effect also amount to accusing at least one of them of fraud.
hilzoy: http://scienceblogs.com/deltoid/2006/10/les_roberts_responds_to_steven.phpYou did see it on Deltoid:
So, there appears to have been some confusion on that front.
Darn it, that was supposed to be a link.
LizardBreath: now it is a link!
Thanks — a few days ago, prompted by I don’t know what, I sat down and read al the Lancet stuff in one go. (I hadn’t done this earlier, since I basically subscribe to the Riverbend theory: as long as the numbers are anywhere close to what the Lancet study says — as in, give or take a few hundred thousand — they’re horrific; nothing I have seen even begins to indicate that they’re anything like that far off; that said, only an interest in the minutiae of study design would make me want to read all these arguments.) Having done that, all the posts tended to blur together, and I didn’t have the heart to track down the reference.
So: thanks.
Rilkefan: You’re biased in favor of the study, so science be damned, and have to resort to a pile of straw over a pile of something else.
Wow, Rilke, that’s a classic tu quoque. Sure, I’m beginning from the presumption that an article published by The Lancet, with four authors all of whom have respectable prior publishing histories in peer reviewed journals (check it out on Google Scholar, if you don’t believe me) is probably neither dishonest nor incompetent unless someone can show evidence for either one. And, whatever you may like to think, not liking the results of a scientific study is not evidence of dishonesty or incompetence in the scholars who carried out the study.
It’s not impossible for bad science to sneak past even a Lancet peer review: but no one has shown that this has happened, only that they don’t like the results. It’s not impossible that the four authors might all have decided that their scholarly reputations be damned, they were going to present two fraudulent papers to make a political point: but no one has shown that this has happened, only that they really don’t like the results.
It’s funny, I’ve seen a fair amount of criticism of the study that appears to be based on a remarkable level of confusion. An article in Science on the study reported one of the authors (Burnham) as saying that they had destroyed the data on the location of the clusters, which whould be very very weird if true. Of course, Burnham says that he told the Science writer no such thing.
The criticism seems to be circling around trying to figure out how the study was done, rather than taking issue with any actually identified problems.
The previous post at Deltoid also has this quote from Burnham
I did not ever tell the writer from Science that the raw data have been destroyed. Absolutely NOT! It is sitting right here! What I did say is that our Iraqi colleagues are very concerned about security, not just theirs but the neighborhoods they surveyed. They have asked us for the moment not to release the data to others as there might be some identifiers there. I am sure that we can remove any unique identifiers, but I am bound to honor their requests, as they have staked so much in collecting the data. We will be discussing this over time with our Iraqi colleagues, and I would imagine that in due course we can make it available to those interested. …
Under human subjects regulations we could not keep unique identifiers, so we limited the information collected — such as street and house numbers. The team did not write down information on the forms on the specific decision making process for each location.
It’s useful to realize how shortened this process has become. Aren’t we talking about less than 10 days since the study came out?
LB: The word, or rather phrase, that leaps to mind here is “FUD” — as LJ noted above.
Which is perhaps the primary reason I’m less inclined towards treating skepticism kindly at this juncture since, willing or no, it plays into the entire strategy of FUD in ways that aren’t easily counterable.
Hmm. I was a fairly ferocious defender of Lancet I. I’ve been more cautious about Lancet II, in part because I was half-convinced by the Iraq Body Count counterattack in which they argued, seemingly based on an intimate knowledge of every bit of reporting that has come out of Iraq, that it is sheer paranoia to imagine that the violent death toll could be an order of magnitude greater than the media reports. Their argument as I understand it, is (in a nutshell) that if the violent death toll really was ten times higher, if American air strikes really were killing tens of thousands of people, if car bombs really were killing vastly larger numbers of people than we realize, that some brave Iraqis somewhere would get the information out somehow to Al Jazeera. There are shadowy insurgent groups in Iraq that want the US out–why haven’t they produced evidence of these massive casualties from American air strikes? And so on.
IBC might be wrong. On a personal level I don’t much care for IBC’s reactions to criticism. Sloboda said things on British TV earlier this year about his critics that disgusted me. My personal and ideological biases are entirely in the direction of believing Lancet II. And I despise the way the mainstream press treats this whole issue as deserving very little coverage and furthermore, my own private term for the American military action in Iraq is “the ghost war”, because you hear almost nothing about it. It’s almost impossible to tell how much violence we’re inflicting.
But I still don’t think one should dismiss the IBC critique–it’s possible these guys know what they’re talking about. As for other critics, they can talk about their own motivations. It’s got nothing to do with mine.
Oh, just to be clear, I don’t disbelieve the Lancet II numbers either–for all I know, the excess death toll could be anywhere from 100,000 (probably too low) to 1,000,000 (probably too high). But the details matter, not just because of the sheer numbers, but also because in the Lancet II version of events a very large fraction of these deaths are coming directly at the hands of Americans and we are hearing nothing-literally not a damn thing–about it. To my mind, if it is true, that is a vastly greater scandal than Abu Ghraib and Guantanimo.
DJ: I am treating the ’caused by Americans’ numbers more skeptically, for the rather obvious reasons that everyone has pointed out (the more you subdivide your data and try to characterize subgroups, the smaller your relevant sample, the larger the confidence intervals, etc.) However, I’ve been trying to follow this, and I have yet to see a good reason for skepticism about the study.
The IBC critique seems to me to rely heavily on assumptions about the level of civic disruption in Iraq, which would affect things like: how likely anyone his to have a broad picture of anything in Iraq, which obviously affect the likelihood of anyone being in a position to say: ye Gods, the following large number of people have died in Iraq recently! We must tell someone!; and also how likely it is that anything that happens outside the rather small areas that the media frequents ever gets reported.
I mean: IBC is using a method of counting that is guaranteed to count only a subset of actual deaths. Their objections seem to me to turn on assumptions about the relationship of the number of deaths they count to the total number of deaths (about the visible part of the iceberg to what lies beneath, you might say.) I don’t see any obvious reason to think that they know what this relationship is — the obvious way to find out about it would be to, um, do a study…
I should add that the press coverage of this has imho been more than usually inane (and I have a pretty low view of most press coverage, so this is really saying something.)
Sebastian There is absolutely no acceptable reason to avoid collecting routine demographic data so that you can find out how closely your sample reflects a random cross-section of the population as a whole. That is Stats101. If you take a poll, there is a reason why you have all those demographic questions at the beginning–the sample verification is absolutely impossible without it. I don’t even understand what “peer-reviewed” statistical study means without the ability to verify against the demographic data. You essentially have removed the ability to compare your sample to what passes for a control group in statistical studies.
I’m genuinely curious on this point – assuming, arguendo, that this is a flaw in the study, rather than the necessary precaution the authors imply, just how significant is it likely to be? How different (in basic age/sex profile) would the sample data have to be from the national data to substantially alter the results?
I don’t know the answer to this, and I’m not even sure how one would figure it out. But since you feel so strongly about this, Sebastian, perhaps you can supply a “fr’instance.” (Anyone else is free to jump in here, by the way.)
E.g., if the national age/sex pyramid is X, what would the sample profile (Y) have to be to, say, double the perceived excess mortality rate, and thus reduce the estimated total from a mere 650,000 to a (more acceptable?) 325,000?
I suspect that the answer is that it would require an enormous difference in average composition, one that any good-faith researcher could simply recognize by eye (e.g., one “household” was a children’s hospital, or a militia barracks, or the like). But I don’t really know, and would appreciate it if you could supply some example(s) of how an otherwise unnoticeable variance between sample and national profiles could significantly alter the outcome.
Lacking this, I will continue to surmise that this factor is trivial, rather than substantive, at most a character flaw of the researchers (like sexual pecadillos in government officials?) rather than a genuine reason for doubting the main conclusion of the study: that there have been 400,000-950,000 excess deaths since we invaded Iraq.
It’s easy to speculate – say that the risk to people under median SES is twice that of people over the median, then say they sampled 100% submedian, because wealthier people live away from major streets – or say it’s dangerous to be wealthy and they didn’t sample the poor people in the twisty narrow streets of the casbahs. That gives a factor of 1/3 too high. Probably if one discovered one’s sample was unrepresentative to that degree one would have messed up the uncertainty estimate too. Then there’s the question of whether living near a main street is inherently dangerous. Still, I’d guess it’s hard to get to a central value of 300k unless they missed some significant bias and messed up the pre-war rates and had bad luck with the fluctuations. It’s rather easier to imagine some of the above putting a true value of 300k in the confidence interval. Of course it’s easy to think of mechanisms that bias their number down. I’m waiting to see if what the other group publishes stands up to scrutiny and how that shakes out.
But as noted above I see Iraq as having been a disaster waiting to happen, so I could get to 300k over expectation trivially (integrating over the future) if I thought it mattered to the overall argument.
The fact that they actually did collect demographic data has been mentioned before, but Burnham’s response to Tim Lambert on the “main street bias” question is new and worth a look. Real addicts will want to follow the argument Ragout is having with Lambert and dsquared about the definition of households and the possibility of double-counting, but I can’t see how it can be a big deal.
rilkefan: I appreciate your efforts here, but I had understood Sebastian’s remarks to refer not to SES or location, but specifically to age/sex breakdown, which is, AFAIK, the only kind of “demographic data” that is routinely collected. (Thus the great T-shirt for sale at demographers’ conventions: Broken Down By Age And Sex.) What I was asking was whether the failure to record these data constituted a real threat to the soundness of the conclusion.
dr ngo – I can do the same pararectal thing with age/sex – young people are more likely to be killed in war or have babies who die due to whatever, and they oversampled young families – or males are more likely to be killed in sectarian violence and they oversampled apts where young men live – or they had a good sample and the respondents made up dead sons. I don’t know if there’s a purdah issue. Anyway, as noted above the issue’s moot.
From Kevin‘s link:
“As far as selection of the start houses, in areas where there were residential streets that did not cross the main avenues in the area selected, these were included in the random street selection process, in an effort to reduce the selection bias that more busy streets would have.”
Sounds like a) there is an issue but b) they designed against it. That’s reassuring.
Hilzoy–I’d like to know the confidence interval for the coalition-caused deaths, but though they give CI’s for violent mortality year by year, they don’t have them for the different causes of death, or if it’s there I’m missing it.
My wild guess is that since coalition caused deaths are a bit over a quarter of the total excess deaths (28 percent), the CI for those deaths would be a bit over half the CI for the total excess. That’s an amateur’s guess. Anyway, that’d come to about 186,000 deaths plus or minus about 150,000.
It’d be nice to have the actual CI. I may email them for it, but suspect Burnham has emailers with more expertise than me that he’d prefer to respond to.
DJ: without the software, it’s all wild guesses. Mine would be that the CI might be considerably higher.
True about the wild guesses, but the true figure presumably has a 50 percent chance of being above 186,000 and a considerably better than even chance of being above 100,000. But I’m not positive about that either.
Donald,
When you get down to specific causes of death, I’m pretty sure the CIs would explode. You are trying to estimate a really small parameter (probability of being killed by coalition forces for example) from a small number of observations.
If they don’t quote a CI, that’s likely to mean it is so wide that no useful inference is possible.
Well, they did compute a CI for violent deaths year by year–45 violent deaths in the first year (CI for mortality rate 1.8-4.9), 90 deaths in the second period (CI for mortality rate 4.0-9.8) and so on. So it was possible to compute CI’s that aren’t too ridiculously wide when the data is broken down into three time periods. The number of coalition-caused deaths is 95 in total, so it seems like a comparable size subset.
But as I am probably inadvertently demonstrating, I’m not an expert.
Lord, I’m blind. I was looking for the CI’s in tables. It’s right there in the text. Page 5
“Deaths attributable to the coalition accounted for 31% (95% CI 26-37) of post-invasion violent deaths.”
Okay, so if the Lancet II data is to be believed, it’s fairly certain that Iraqis tend to blame coalition forces for between 26 percent and 37 percent of the violent deaths.
The next question would be whether to believe them, I suppose. Anyway, that would explain why so many Iraqis favor attacks on coalition forces, if they think this is true. (And explains it even better if it really is true.)
Okay, my earlier comment was silly. I should have looked at the paper first. They had 95 deaths, not a small number by any means. And such a narrow CI suggests that there wasn’t nearly as much variation between clusters as in the 2004 study.
I’ve just been watching a BBC documentary filmed in a Baghdad hospital. It should be required viewing for the guys who responded to the Lancet study by asking “where are all the wounded?” Sit down and watch them being treated, guys – often without anaesthetics. A doctor said that 90% of all the injuries he sees are war wounds. It’s amazing how people cope. They had a blood shortage, so now, when they give a transfusion, they take the patient’s ID card. When his relatives come in and donate as much blood as they have given him, he gets the ID card back. I guess if you’ve lived through wars and economic sanctions you figure out how to keep the system working.
Relevant to Donald’s current concern: a remarkable number of people said “No Iraqi would do this!” even though it was pretty obvious that some Iraqi did, unless the Marines are using car-bombs these days. Denial, denial; I think that’s at least part of the story with the Lancet figures. I doubt that the respondents invented deaths, but I suspect they blamed the foreigners any time they could.
Kevin notes that the newer Lancet study shows the sampled households holding coalition forces [responsible] for between 26 percent and 37 percent of the violent deaths. … I doubt that the respondents invented deaths, but I suspect they blamed the foreigners any time they could.
Maybe. But the proportion of violent deaths laid at the coalition forces’ door is lower than in the 2004 Lancet study (and in Iraq Body Count figures from the same period):
I tend to agree (or guess) that US-caused deaths are a large part of the total, but in IBC’s case about 2/3 of the US-caused civilian deaths came in the March and April 2003 months. After that they drop to a very tiny fraction, except in the two months when we assaulted Fallujah. This is one of my main gripes against IBC–I see no reason to believe that the press is able to keep track of the collateral damage since the opening months. IBC is being somewhat disingenuous in that statement (which I remember reading last winter during the earlier Lancet/IBC wars).
I suspect the press did a better job regarding March and April 2003 counting US-caused dead because Iraq was relatively peaceful (relatively) in the early days and Westerners could wander around and go to hospitals and also, in those days, it was also clear that if someone had been killed, he’d been killed by US firepower. But Iraq got steadily more chaotic after that.
I don’t really disagree with anything you’re saying, Donald, but I was responding to Kevin’s raising the possibility that respondents to the survey were blaming US/coalition forces for deaths that were actually caused by other Iraqis.
My point is that Kevin could be correct about that, but the percentage (26-37%) in Lancet 2 is not an argument for that, since the percentage is lower than in the 2003-4 period.
Point taken, Nell. Anyway one shouldn’t read too much into a documentary about one Baghdad hospital. It is highly probable that US forces really are taking quite a toll of guerrillas and civilians in Anbar, for example. What struck me was the readiness of patients to blame Americans in a situation where there really were no grounds (except in the general sense that the invasion created the whole bloody mess in the first place). But that could be just their first reaction, it might not be what they would tell a Doctor doing a survey months or years after the event.
If and when we ever get a really detailed breakdown of the data gathered for Burnham et al., it will be interesting to see the dates and approximate locations of deaths attributed to the coalition. It should then be possible to judge whether the attributions of blame are plausible.
Mainly though I just wanted to say it was a remarkable documentary.
From the ORB Group:
I wonder how many studies showing that the US invasion and occupation of Iraq has caused the deaths of over a million people can be ignored in favor of a study that shows that the media have been able to report something like 8% of the casualties?