Friday, January 05, 2007
# Posted 9:51 AM by Taylor Owen
I do have a bit of a thing for counting bad things, and have been involved with several projects using modelling techniques at least tangentially similar to those in this project, however, only as a statistically-less-inclined co-author. In lieu of my own assessment then, let me provide the following two links that may be useful for those wanting to move beyond knee-jerk dismissals of these quantitative techniques.
First, the Human Security Centre, responsible for the Human Security Report released last year, has a very useful list of links to responses to the Lancet study. The Human Security Report was instrumental in bringing together considerable new data on conflict mortality rates and has substantial research connections to those on the frontier of this emerging field.
The Royal Holloway economics department provides another good resource on the methods used in the study.
What both these sites make clear is that this is an ever evolving field with its own internal debates. This is disputed by no one. That being said, there is a big difference between this recognition and the types of quasi critiques that have been abound on this issue.
I wrote a friend who has intimate knowledge of the methods used in the Lancet Iraq study, asking the following question:
What is your professional opinion of the new Lancet study of Iraqi casualties? Is the real story not the 650,000 but the 392,000 at 95% certainty? Would be very interested in your assessment.I received the following reply:
No, message is 650,000 excess deaths is most probable estimate (probabilityThis person has worked in just about every major conflict zone of the past two decades, and developed statistical surveys that are at the forefront of the field. He is intimately knowledgeable on the indirect human costs of war. His point is a clear one. The principal question is why are we so surprised that this level of conflict would result in such levels of excess mortality?
I would argue it is a direct result of our sanitised view of war. We consider the costs of war to be limited to direct conflict casualties. Bombs killing our soldiers, bullets killing insurgents, end of story. This of course is only the beginning, excess death levels tell the other side. The failure to provide humanitarian protection has real human costs, far beyond those directly killed by munitions.
And here in lies the rub. Indirect costs of war are both incredibly difficult to measure and generally not considered part of the overall war calculus. Iraqi Body Count, for example, only measures specific killings, not excess mortality – a substantial difference. Particularly since it was their figures which most news accounts (as well as the President) compared to the Lancet totals. Yet they are measuring different things.
But are excess deaths the responsibility of the intervening country? I would emphatically argue yes. Particularly if there is even an iota of humanitarian rational for the occupation. More importantly, it is these very excess deaths that are both symbolic of the failures of the humanitarian component on the occupation, as well as a causal factor of the high levels of anti-occupation sentiment and outright insurgency support. The fact that the numbers would be dismissed wholesale, is in my view symbolic of the disconnect between many war proponents and the humanitarian realities of the mission.
UPDATE: I must add one more very useful set of links provided by the Human Security Center. In their words:
The recent Lancet article attempting to quantify the death toll since the 2003 invasion of Iraq has been the subject of much controversy. What is clear is that there is a dearth of accurate data and huge practical and methodological challenges inherent in calculating conflict death tolls. This special issue of Human Security Research features a series of recently-published reports and articles on conflict-related mortality and is designed to shed some light on what we know, what we don't know and the challenges facing those who try to measure the human costs of conflict.For those interested in the state of the art (or maybe I should say science, but that is a whole other debate), check it out. (35) opinions -- Add your opinion
Recalling the disputes over the study, I believe the process was to take the pre-war death rate and compare it to the current death rate.
The study used a pre-war death rate as a baseline, and found it was 5. I believe the source was the CIA. Others disputed the death rate, finding other sources (WHO and Unicef) providing pre-war rates of 9 & 10, approx.
The lowest death rates are supposedly in populations with lots of kids compared to old folks. Hungary, which doesn't have lots of kids, has a death rate of 13.5, for example.
Some commenters looked at other countries with very young average ages--lots of kids, apparently--and discovered no death rates as low as 5.
So the pre-war death rate is in doubt, and no matter how accurate everything else is, if the pre-war rate is wrong, the study is wrong. However, the study is wrong in the desired direction.
Besides, since the evil sanctions killed half a million Iraqi kids, they aren't currently around to be not dying and thus holding the death rate down. Something's wrong, here.
Regarding your comment on the sanitised view of war - notice how when Bush/Cheney & Co were disputing the 650k number they didn't offer or discuss an alternative number. It was just that the true value was likely "lower".
650k, 400k, 200k they are all very sad numbers. (Add to that all the people who have simply left Iraq.) The general public and would rather not associate our actions (the war) with such horrible consequences.
But what number, then, is acceptable to us and the White House? Recall Madeleine Albright saying that 1 million dead children as a consequence of the sanctions was "worth it".
The question as to how many are too much is answered pretty simply:
Fewer than would be killed if we didn't. With a multiple for us vs. them, whoever them is.
nr, I think they went with the U.N. numbers.
Who took the survey? Was it possible to become a doctor in Iraq without being Sunni and a Bathist? How do we know the people responding are being honest?
There was a mention in the survey that they had found a neighborhood in which there were twenty-five or thirty deaths.
Keep in mind, the study didn't talk about causes of death, just whether the rate was higher, and, if so, how many more deaths that meant.
The survey in that neighborhood found that about 80% of the deaths had generated a death cert, or at least that many could be found.
Somebody suggested that meant there were about twenty-five percent more deaths than issued death certs. So find out how many death certs had been issued nationwide, add 25%, and there you are.
Unfortunately for the study's supporters, that resulted in a very low number, which is why it didn't get much play.
Taylor, you linked to the 2004 Lancet study, the one that estimates 100,000 excess deaths in the first 18 months of the war, not the new one estimating 650,000 in the first 36 months; do you know whether the authors plan to to continue the series, sextupling the estimate every year and a half, until they pronounce Iraq devoid of human life around 2011? At what point in such a series would your friend decide the larger question was, "why are these guys' numbers so much higher than anybody else's?"
nr, if you would follow some of the links Taylor provides, you might notice that Bush gave an estimate in late 2005 that about 30,000 Iraqis had died violently since 2003.
Richard Aubrey: If you read the study, it used the same methods to assess the pre-war death rate and the post (during?)-war death rate. The rate of 5.0/1000 may match some CIA report somewhere, but the report clearly explains that the 5.0 number is from the 3.7 to 6.3 confidence interval from the study.
Since the prewar rate was determined using the same method as the post-war rate, a comparison between the two is valid -- the 12.3/1000 post-war rate is 2 to 5 times worse than the 5.0/1000 pre-war rate. The CIA, WHO, or Unicef might use different methods the get their numbers, but Iraq is too unstable for those agencies to re-apply their methods. Apples-to-apples, The Lancet study shows that things are very likely significantly worse post invasion.
If you desire to show that the invasion made the death rate better, come up with something better than an argument from incredulity.
The problem has always been with the implicit moral accounting, not the math, whatever disputes might exist about that. The war came in two stages: A relatively bloodless war by us to overturn a dictatorship, and then a bloody war by the partisans of that dictator to reimpose tyranny. We are being blamed for the casualties caused by both sides, in both phases of the war.
Sorry, they're responsible for the latter deaths.
Reader. I didn't say the war made things better. I don't get it. I would fully expect you to tell somebody who had no way of knowing what I said some crazy lie about what I said.
But how is it supposed to work out when you lie to me about what I said? Am I supposed to be confused, or something? I'm the one who said what I said. I just don't get the math.
Anyway, back to the substance: You can lie about me to somebody else, and probably will, Reader, but spare me another attempt to lie to me about what I said.
The point I was making is that the study had numerous flaws, among them implying accuracy to the sixth digit. The pre-war death rate is questionable, especially if we want to continue to flog the half million dead Iraq kids theme.
The Lancet pushed their previous estimate without peer review because the review would have made it late for the election.
I didn't mention it, but it appears that men of military age are disproportionately represented in the study, which ought to be a matter of interest.
The Lancet pushed their previous estimate without peer review because the review would have made it late for the election.
That's incorrect. It was peer-reviewed.
Richard Aubrey wrote: "The study used a pre-war death rate as a baseline, and found it was 5. I believe the source was the CIA."
Your belief is incorrect. Both studies used a cohort technique to estimate pre- and post-invasion mortality rates. The estimates may or may not be flawed, but comparison with an inappropriate pre-invasion estimate isn't one of them.
"The Lancet pushed their previous estimate without peer review"
This, too, is incorrect. In fact, two of the reviewers' comments were published at the same time and in the same issue as the article. Once again, the estimates may or may not be flawed, but lack of peer review is not one of them.
Mr. Aubrey, both of these misconceptions could have been avoided had you read the actual article.
Furthermore, Richard Aubrey, who precisely is 'flogging the half million dead Iraq kids theme'? (what a distasteful choice of words).
I dont see this being made into a big issue in the Lancet report. Are you merely conflating the Lancet authors with the political opponents you continue to do battle with in your own mind?
You can't have the lowest death rate in the world--pre-war--without having a huge number of kids. Kids are not supposed to be dying. They hold the death rate down.
If you take half a million kids out of the population you either count their deaths, if they occurred during the study period in which case you have a pretty high death rate, or your study period is after they died and they are no longer available to hold down the death rate. In which case, you don't have the world's lowest death rate.
So, either the folks who were crying about the effects of the sanctions were pulling our leg, or the numbers are screwed up.
The Lancet article I referred to with the timing vs. peer review was the earlier one with the 8000-198000 count.
The 2nd lancet survey doesn't say anything that surprising about post-war Iraq.
The area where it is surprising, contradicts other accounts, is the low pre-war death rates.
There are two reasonable explanations for that:
1. sanctions and saddam had killed off the poorest and most vulnerable, and oil-for-food meant that few new people were starving in 2002.
2. the baghdad-based doctors who ran the survey found it difficult to perform interviews in the Kurdish and Shi'a areas that were suffering (i.e. being bombed by the Baghdad regime) pre-war.
Further research required, I guess.
Using pre-invasion death rates assumes that nothing else would have changed in Iraq had the US not invaded.
That al Qaeda would not have entered the country after being chased out of Iraq.
That Saddam would be unmoved by reports of the Iranian nuclear program, and seek to live in peace.
That Ahmadinejad would likewise have less interest in destabilizing Iraq if it were under the control of Saddam, friend of the Shia.
That Saddam would either achieve immortality and keep Iraq in stasis, or use his remaining years to set up a successor government that would not fall victim to sectarian strife.
Owen, could you outline how Iraq could have lived happily ever after without us?
Richard Aubrey wrote: "The Lancet article I referred to with the timing vs. peer review was the earlier one with the 8000-198000 count."
Right. That's the one you claimed was "without peer review", but if you had looked at the actual journal article itself, you'd have seen that: 1) the editor's comment included this: "Roberts and his colleagues submitted their work to us at the beginning of October. Their paper has been extensively peer-reviewed [...]"; and 2) that the comments of two reviewers (Bird and Al-Rubeyi) were printed at the very same time in the very same issue.
Mr. Aubrey, you have demonstrated that you held two misconceptions (about lack of peer-review and that an improper comparison had been made with an external estimate of pre-invasion mortality) that would easily have been avoided had you read the actual article. This is not a good sign.
You can't have the lowest death rate in the world--pre-war--without having a huge number of kids. Kids are not supposed to be dying. They hold the death rate down.
That's not generally true, and it's exactly opposite the truth in Iraq. Kids, especially under 5, have higher mortality rates than teens & young adults in most of the developing world; and did everywhere for most of human history.
(Which is why child mortality is a problem unto itself.) In pre-war Iraq, both UNICEF and WHO report far higher under-5 mortality rates than overall rates; the CIA Factbook and Roberts et al. report very high infant mortality rates, though not as high as during the sanctions. The Roberts et al. figure, for instance, is 29/1000 for infant mortality--and the other figures, which use data for earlier years, are higher.
Therefore, the deaths of young children would tend to lower the overall death rate in the next few years, not increase it.
When people say that a younger population means a lower death rate, they're not talking about a child-dominated population, but about a teen-to-twentysomething-dominated population with few seniors.
Good point, although putting Iraq into the "developing" camp might not be entirely accurate.
Among other things, they had to have a working medical system for it to be ruined by the sanctions. You can't ruin nothing.
The primary killers of kids in developing countries are malaria and intestinal problems, both of which Iraq seemed to have mostly licked. So I would expect them to be doing better than some of their neighbors in the 0-5 y/o category.
A widely hated dictator is deposed by an outside country. Many members of his ethnic group enjoyed privileges under his rule. They do not accept the change and go on a killing spree.
Who is responsible for the killing? The people who deposed him or the people doing the killing?
Why not hold the people doing the killing responsible for the killing?
Who is responsible for the killing? The people who deposed him or the people doing the killing??
The occupying power is responsible for security.
And just to be clear on the peer-review issue, both studies were fully vetted before publication.
You said "So the pre-war death rate is in doubt, and no matter how accurate everything else is, if the pre-war rate is wrong, the study is wrong." The study's finding was that the post war death rate was 2 to 5 times worse than the pre-war rate. Perhaps I misunderstood that you believe a non-wrong study should find the post-war death rates to be less or equal to the pre-war rates.
What would invalidate the study's finding that the death rate has significantly increased is to more accurately measure the pre and post-war death rates and see a different result, but you'd have to measure them using comparable procedures. The Lancet study used comparable procedures for before and after estimates, and came up with an estimate of a significant increase in the death rate as 2-5 times higher.
The 2-5 times increase in death rates found by the study isn't a 6 digit accuracy, in those terms it is more like a 1/2 digit accuracy. All the confidence intervals stated in the report don't even pretend to claim a 6 digit accuracy.
The conclusion of the study is that the death rate post war is significantly higher than it was before, and I've seen nothing that proves the opposite, including mischaracterizations like your "The study used a pre-war death rate as a baseline, and found it was 5. I believe the source was the CIA." While your statement of belief may not be a lie exactly, it is directly contradicted by the paper itself and is unsupported by any source you've shown.
"The occupying power is responsible for security."
By your reasoning civilian murderers are never responsible for murder. Some government is always responsible for security.
Are you claiming that whenever a murder occurs it is the government's fault because it is responsible for security?
There is a legally elected government in Iraq. Why are they not responsible for security? If that government is not fulfilling its obligations, who is responsible?
It's really not that difficult to understand.
The person who commits murder is responsible (leaving aside the deaths directly attributable to U.S. forces). The occupying power is legally responsible for preventing civilian deaths.
The U.S. - officially the Multinational Force in Iraq - remains the occupying power despite the formation of a government (just as Israel remains the occupying power in the West Bank). This is explicitly stated in UN Resolutions 1546 and 1637.
I never stated that things were better or the same. Believing I said that might be the defensive reaction of someone who desperately wants things postwar to be catastrophic. Because any doubt....
I said six-digit accuracy because the study had six digits in the number of excess deaths. The last one was, I think, 7. So they're saying it wasn't umpty-ump and six or umpty-ump and eight. They are confident it was umpty-ump and seven.
Whatever the source of the pre-war death rate, two things apply: One is that I haven't heard anybody applying the same methodology to some other nation, like, say, us to see if there is some kind of reality check. The other is to assert the same methodology would be as accurate post invasion as pre-invasion, even if they tried to use the same.
There was one report that the surveyors had to hurry out of a neighborhood out of fear of terrorists. I can imagine. What other shortfalls resulted from local conditions are not clear, but it would be foolish to say they did not exist.
The most puzzling thing about the entire exercise is the decision not to use death certificates. The only possible answer is that using them would have resulted in an unacceptably low number.
I acknowledge that I was wrong about the first study's peer review. The report I read said the authors wanted to avoid it in order to save time and get out before the US election. But it appears they got both.
If you aren't arguing that the death rate is not significantly worse post invasion, what the heck are you arguing?
Results of statistical studies make determinations like "X is significantly more than Y" versus "X is not significantly more than Y". If your meaning with "You stated "So the pre-war death rate is in doubt, and no matter how accurate everything else is, if the pre-war rate is wrong, the study is wrong," is that it is not the case that exactly 654965 Iraqis died between the invasion and June 2006, I'll agree with that vapid statement. But if you are meaning that the deaths in Iraq post invasion are significantly outside of the 392979 to 942636 range (not even the first digit is the same) with only a load of 'recalling', 'I believe', 'others disputed', 'I imagine', 'most puzzling' suppositions, your argument seems mainly that you find the number incredible.
If you want to convince me that the Lancet study's 2-5 times increased post-war death rate is in error based on the your readings of the CIA, WHO or Unicef pre-war numbers, get the CIA, WHO or Unicef to re-estimate the death rate and then show me the numbers. Else, you can't even imagine what you can do with your argument from incredulity.
Richard Aubrey wrote:
"I haven't heard anybody applying the same methodology to some other nation"
That's because you haven't been reading the proper things. The same methodology has been applied to many different countries: it is the standard technique used by all of the DHS surveys; furthermore, the same researchers used the same methodology earlier in Rwanda and the Congo.
Mr. Aubrey, it is evident that you have not read the actual article. It is evident that you are unfamiliar with the techniques and methods involved. Who, then, has been filling your head with these ideas and criticisms? I ask because it is also evident that they have not been serving you well. You should be worrying what other misconceptions and errors might they have been telling you.
Mr. Chung and Reader:
I am not arguing that things are better, or worse, or anything else about the actual results of the study. It would be reasonable to presume things are worse after the invasion, primarily due to the ongoing terrorist disruptions of infrastructure, not to mention the slaughter of several dozen civilians per day, at least.
My problem is that the pre-war death rate is questionable. I went to the DHS links--thank you--and was not comforted.
Iraq did not appear there, perhaps someplace further into the website. However, I read the methodology as regards data collection.
The data are self-reported. The pre-war data comes from Iraq where Saddaam got better than 99% of the vote in his last election and Eason Jordan felt it politic to fudge CNN's coverage in order to protect their access and the lives of their Iraqi associates and sources.
Initially, I was questioning the pre-war rate because it looked so low. It didn't agree with other studies. I had completely forgotten the likely effect of the whole thing taking place in Saddaam's Iraq.
The problem with the reactions of some to my questioning of the accuracy of the study reminded me of a trip to Central America with a faith-based peace group in 1987. We met with a Catholic Human Rights organization, Tutela Legal, where we found the death squad killings were down by about 98%. As the only non-peace-activist there, I was the only one pleased.
When big numbers of dead civilians are necessary to reproach US foreign policy, big numbers are supposed to be provided. One way or another.
We were told of the cops machine-gunning a demonstration. Gasps of horror. How many were killed or hurt, I asked. One injured was the reluctant answer. I remarked that my son, then nine, could have done better with his BB gun and no BBs. It was not well received.
I later wrote about my experiences for the sponsoring group, Center for Global Education, then and probably now at Augsburg College in Minneapolis. I got a call from the editor wanting to "discuss" my relation of the machine-gunned demonstration. Maybe M16s don't do the same thing real machine guns do....? Or....?
In fairness,I should say they published my manuscript straight, only cleaning up a sentence fragment of which I had been fond.
So. Experience as the above and in other circumstances makes me particularly cautious about such studies. And I can see why a suggestion that something may be wrong with it would generate such reactions.
Umm, no Richard.
Think about it.The data may be self-reported, but ALL data was collected by Lancet Iraqi researchers AFTER the invasion! The survey linked to states that it took place between May and July 2006 (maybe you should take the advice of Robert Chung - now offered to you twice - and read the Lancet report)
They simply asked people in all the households they visited to name all the deaths that had occurred in that household back until 2002. The pre-war data was not actually collected before the war. Therefore, fear of 'reprisals' from Saddam Hussein would not apply (and, in fact, it probably would have been in Saddam's interest to inflate, rather than minimise, the numbers of deaths in pre-war Iraq due to sanctions anyway, so your point does not even stand on these grounds).
Moreover, there is no logical reason why it would be in the interests of the Iraqi interviewees to make up extra dead after the war. Who would be forcing them to do so, and why? If you are going to allege a massive conspiracy to lie to the authors of this study, you have to hang it on solider evidence than this.
Your South American anecdote is amusing, but, frankly, your desperate and risibly unscientific attempts to rubbish a study you have not even read suggests that the anecdote tells us more about your state of mind than that of anyone else on this thread.
wars have indirect deaths, in part because wars disrupt economies, leading to deaths?
How many deaths around the world were associated with the economic downturns associated with the oil price shocks in the 70s? Is the govt of Iran responsible for the deaths associated with the second price shock?
There was considerable economic disruption in the US after 9/11. Has anyone done a study of death rates in the US after 9/11? Or did we 'only' suffer the 3000 direct deaths.
How many deaths occur due to disruptions like the fall of the Soviet Union? Was it therefore not justified?
OTOH, that raises the question of sustainability. If no fall of the Berlin wall, the USSR would still have spiralled downhill in all probability. So the question of what the right base case is comes up.
Thats the real problem with the Lancet study, and all "you broke it" arguments. It assumes not only that Saddam could have been contained from further aggressions, but that his regime was indefinitey stable.
Thats highly suspect, IMHO. The regime could have fallen "peacefully", esp at the point when the sons of Saddam took power. At that point the problem of a ruling Sunni Arab minority displaced by a vengeful Shia majority and its Kurdish allies would still have come up.
Y'know Aubrey, I was playing poker with my PNAC buds back in early 2003, and they said there was no doubt that Saddam had Weapons of Mass Destruction, and that snuffing Saddam might take 6 days or six weeks, but couldn't foresee 6 months. My experiences with their later backpedaling, revising, underestimations, and unforeseen circumstances made me feel that I should share my valuable insights about the veracity of politicians with people I call liars.
And other sources which used self-reporting had higher death rates. WHO and UNICEF had to deal with Saddaam in power.
Was 2002 a very good year?
As one commenter supposed, as a practical matter, it's possible. First, you kill the vulnerable by sanctions and then you feed the strong with whatever food is let out to the public under Oil For Food.
2002 is not satisfactory by itself as a pre-war baseline, whatever happened or didn't happen in that year.
It's hard to say what reader is referring to, catching SH, or the continuing trouble.
If it's a matter of catching SH, people can estimate, speculate, hope. They may turn out to be wrong. Many were wrong about how the battle-hardened Iraqi army would chew up our soft boys and girls in 1991.
I'm glad they were wrong.
The point is that in war the other side has a vote, as the saying goes, and predictions are a bad idea.
WRT the current troubles, P. J. O'Rourke had an interesting observation. He, and he supposes others, expected a Kuwait-style ending. Once the bad guys were gone, the locals would get to work and everything would be hunky-dory.
It would have been racist to suggest the Iraqis weren't up to it, that they prefer their internecine murders to just getting through the day.
Consider that we now see Iraqi society generates an appallingly large number of people who actively slaughter their fellow citizens.
I suppose you could say we should have foreseen that, but who would have wanted to say so in public?
I was appalled and shocked by the violence, mostly by the way it's almost entirely directed by Iraqis at Iraqis with little or no interest in whether they have anything to do with the government.
I did not expect it.
I would not have liked myself if I had expected it.
I would have strongly reproached anybody who tried to make the case.
I had some friends whose parents came from various ethnic groups in the Balkans and I was not surprised at the violence there, since I'd heard about the history of the place. Perhaps I should have generalized more widely.
So. Who's up for insisting that our next move be made with the presumption that the folks we're dealing with are really, really vile? Not just the leaders. Tens and tens of thousands of what would ordinarily be ordinary citizens?
Didn't think so.
Richard Aubrey wrote: "I went to the DHS links--thank you--and was not comforted.
Iraq did not appear there, perhaps someplace further into the website."
Mr. Aubrey, your exact claim was: "I haven't heard anybody applying the same methodology to some other nation." When I pointed out a site that demonstrates that the same methodology has indeed been applied in other nations, you change the target. This is not encouraging behavior on your part.
"And other sources which used self-reporting had higher death rates. WHO and UNICEF had to deal with Saddaam in power."
Right. Those higher death rates to which you refer were: 1) from an earlier time period (the mid-1990s) during which import sanctions were being applied to Iraq; and 2) almost entirely due to higher infant, not adult, mortality. These two factors mean that those rates you cite aren't as directly comparable as one would wish.
Mr. Aubrey, it is clear that you are unfamiliar with the actual studies in question and have been getting your information from elsewhere. I tell my students that it is fine to read others' criticisms of an article but, at a very minimum, they should themselves read the article in question before criticizing it (when I tell them this I have been known to write on the blackboard the letters "RTFA").
"The problem with the reactions of some to my questioning of the accuracy of the study reminded me of [snip]" Hmmm. Mr. Aubrey, I cannot help what this may remind you of, nor can I stop you from speculating about my motivations, but they do not immunize you from being corrected about misconceptions that could have been avoided had you read the articles in question rather than relying on whatever sources you have been using. Before posting more criticisms, please, please, please RTFA.
Richard Aubrey: Cripes! Could you not read that as a possibly fabricated anecdote used to justify non-responsive blather? Reader's masterful mirrorlike riposte to your pythonesque argument is scathingly brilliant! I think we all owe Mr. Reader a beer.Post a Comment