Cooking the Iraq body count books

Listen to any ‘Stop the War’ spokesman, and you’ll likely hear quoted a number representing the excess mortality count in Iraq since the multinational military invasion and removal of the Ba’athist dictatorship. By excess mortalities we mean deaths resulting from the invasion and occupation, and subsequent terrorist activity carried out by those who refuse to recognise the authority of the elected Iraqi government.

The number of excess deaths quoted by political activists opposed to the invasion of Iraq is based largely on a study published in 2006 in the medical journal The Lancet. In this highly controversial paper, Gilbert Burnham, Riyadh Lafta, Shannon Doocy and Les Roberts claimed, based on a door-to-door survey, that over 650,000 people had died as a consequence of the war, of which some 600,000 suffered violent deaths. Add to this the statistical uncertainty, and the grand total is around a million. As you might expect, if you are of a cynical disposition, one million is the figure quoted by innumerates with a political agenda as the actual number of excess deaths due to ‘western imperialism’.

Burnham et al.‘s findings have been challenged on a number of fronts, yet owing to repeated quoting in popular news media, the one million excess deaths figure is widely accepted outside the scientific community. Among experts there is a general acknowledgement that the number of excess deaths exceeds 100,000, but is nowhere near the total presented by Burnham and his colleagues and quoted with monotonous regularity by political commentators.

The latest contribution to the debate surrounding the number of war-related deaths in Iraq comes from the economist Michael Spagat, who in a paper in the journal Defence and Peace Economics questions the ethical and data integrity of the Lancet study, presents evidence of data fabrication and falsification in that work, and calls for an investigation. Spagat’s conclusions are damning, and his criticisms of Burnham and his colleagues go way beyond previous accusations of ‘main street bias’, which results in an over-estimation of a measured quantity through sampling only the more accessible urban locations within a polled area.

In addition to the Burnham et al. study we have a survey carried out by British opinion pollsters ORB, which in 2007 published an estimate of 1.2 million (±2.5%) excess deaths in Iraq. That figure was later revised downward to one million, but it remains an order of magnitude greater than the number of excess deaths accepted by the majority of experts. Michael Spagat, along with Josh Dougherty of Iraq Body Count, an organisation which has regularly published estimates of Iraqi deaths due to the war, looked in detail at the ORB survey, and found serious irregularities in its mortality data, and systematic errors that render those data worthless.

Spagat and Dougherty’s dissection of the body count over-estimates was discussed last week by Independent columnist John Rentoul, for whom the scientifically credible number of excess deaths should be enough for the ‘stoppers’ to make their case without having to exaggerate the count by a whole order of magnitude. Indeed, but Rentoul would do well to acknowledge that his own trade of newspaper punditry is no stranger to the use of hyperbole in the service of political rhetoric and vulgar point-scoring.

Further reading

Burnham et al., “Mortality after the 2003 invasion of Iraq: a cross-sectional cluster sample survey”, The Lancet 368, 1421 (2006)

Michael Spagat, “Ethical and data-integrity problems in the second Lancet survey of mortality in Iraq”, Defence and Peace Economics 21, 1 (2010)

ORB opinion poll data on excess deaths in Iraq, (2007)

Michael Spagat & Josh Dougherty, “Conflict Deaths in Iraq: A Methodological Critique of the ORB Survey Estimate”, Survey Research Methods 4, 3 (2010)