POAC III

The Project for an Old American Century's parade of ineptitude continues ...

The talking point
Lancet report on Iraqi civilian casualties isn't credible

The facts
Report of 650,000 dead Iraqis relied on large sample base, checked death certificates, includes more reliable methodology than previous reports.
(POAC)

This claim concerns the second survey published in the Lancet and designed by researchers at Johns Hopkins. An earlier Lancet survey on the same topic was criticized because of methodological problems and its unusually wide margin of error.
The new study utilized a broader sample, but many questions remained about collected information. Iraq Body Count issued a fairly prompt critique of the Johns Hopkins/Lancet study.
The "facts" offered by POAC come from a partisan source (Richard Horton, publisher of the Lancet--like he's going to say he published rubbish).

So, POAC offers us a single, partisan source as "the facts" and ignores the legitimate criticisms of the study.
Is that great or what?


Summary of Problems

1) The number of clusters (47) does seem small for such a diverse region as Iraq. One defender of the small number of clusters claimed that a small number of samples from a swimming pool would give about the same results as using four times the number of clusters. The whole point of using clusters, however, is to allow data collection where data collection is difficult, and larger numbers of clusters become desirable where the samples are likely to be diverse (as with Iraq, since some parts are much more violent than others).
2) Were the interviews reliably done? The astoundingly high response rate alone gives rise to suspicions that the data were fabricated to some extent (I don't see how the degree can be known).
3) The report claims that cluster sites were chosen "entirely at random" but surveyors were permitted to change the survey site where the randomly selected was deemed unsafe. How could they do that randomly?

This is just the tip of the iceberg regarding the data collection (I recommend the Iraq Body Count critique, and also this one). While the study may not be entirely worthless (given the broad definition of deaths associated with the war), there's no reason to place great faith in the results.

Three in a row, POAC. What kind of streak will you end up with?