After an animated debate at the Royal Society of Arts, The Spirit Level’s authors—Richard Wilkinson and Kate Pickett—have issued a written defence of their work via the website of their political campaign group The Equality Trust. The authors of the Taxpayers Alliance critique responded here last week and I have issued my own response on my website. Between us, we posed 30 questions, the answers to which only demonstrate the exhausting and seemingly endless task of fact-checking Wilkinson and Pickett’s claims.
Most of the replies manage to duck the question with the skill of a practised politician, but when more concrete answers are forthcoming, Wilkinson and Pickett play so hard and fast with the facts that one can only assume they expect no one to check their sources. How, for example, do they expect to get away with claiming that only 3 per cent of US charity donations go overseas? (The actual figure is 38 per cent.) Who, other than Wilkinson and Pickett, really believes that life expectancy in Cuba and Costa Rica is “as high or higher” than in the United States? You almost have to admire the audacity of a duo who insist that “there are several pieces of research which show that homicide rates are inversely related to suicide” and then point you to a study which concluded: “The overall correlation between homicide and suicide rates was weak and statistically insignificant.”
Admiration turns to awe when two British epidemiologists try to convince three Swedish economists that, despite their own experience, “in Sweden, people don't bother to check your tickets on the train or bus.” At the heart of Wilkinson and Pickett’s defence is the repeated assertion that their work is based on a mountain of peer-reviewed studies. In a recent letter to The Guardian, they referred to "hundreds of other academic research papers which show similar patterns." This is a gross exaggeration at best. Although it now suits Wilkinson and Pickett to present themselves as the face of a scientific consensus, the academic evidence they cite in support of their theories ranges from ‘hotly debated’ to ‘non-existent’.
The only one of their claims to have a significant body of scientific literature behind it is the association between inequality and health, and many of these studies disagree with Wilkinson and Pickett’s hypothesis, not least because it is so difficult to disentangle the material effects of poverty from the less tangible, ‘psychosocial’ effects of inequality. The idea that the health of whole nations depends on the level of inequality is particularly contentious. Summarising three decades of research, the Oxford Handbook of Economic Inequality (2009) concluded: The preponderance of evidence suggests that the relationship between income inequality and health is either non-existent or too fragile to show up in a robustly estimated panel specification. The best cross-national studies now uniformly fail to find a statistically reliable relationship between economic inequality and longevity.*
At the RSA debate, and in all of their published rebuttals, Wilkinson and Pickett have been eager to mention the study by Kondo et al., published in the British Medical Journal in 2009. This, they say, "shows unequivocally that inequality is related to significantly higher mortality rates." In fact, the study concludes that: “The results suggest a modest adverse effect of income inequality on health, although the population impact might be larger if the association is truly causal... The findings need to be interpreted with caution given the heterogeneity between studies.” If this is unequivocal proof, you can imagine how cautious the conclusions of other studies have been. Wilkinson and Pickett now accept that that their inequality/life expectancy claim is “one of the weaker associations in The Spirit Level” and that it falls down entirely if one uses inequality figures from the OECD. It is no coincidence that the claim which has been put under the most scientific scrutiny now turns out to be the weakest. But as contradictory as it might be, at least the body of evidence exists.
Elsewhere in The Spirit Level, Wilkinson and Pickett largely have to rely on studies they have written themselves. In some instances they cannot even do that. When they argue that inequality leads to higher rates of infant mortality, they fail to provide a single reference to a study that even vaguely supports their hypothesis (for what its worth, my own brief chapter on infant mortality refers to 17 peer-reviewed studies). In a recent interview with International Socialism, Richard Wilkinson freely admitted that the bulk of The Spirit Level’s claims have “very few” peer-reviewed studies to support them: "There are about 200 papers on health and inequality in lots of different settings, probably 40 or 50 looking at violence in relation to inequality, and very few looking at any of the other things in relation to inequality. In a way, the new work in the book is all these other variables—teenage births, mental illness, prison populations and so on—and the major contribution is bringing all of that into a picture that had previously been just health and violence."
Wilkinson and Pickett have every right to take the credit for their “discoveries” (as they describe them in The Spirit Level). They are very much their own theories. Even the life expectancy hypothesis is almost uniquely associated with Richard Wilkinson, who popularised it in the early 1990s. Then, as now, he was accused of being rather selective about which data he included. Wilkinson’s favoured method of research has long been the ecological study, ie. using aggregate data from whole nations. This type of study is notoriously unreliable because it allows almost unlimited scope for misinterpretation (so much so that it has its own term—the ‘ecological fallacy’).
Using simple scatter graphs showing international comparisons is both a strength and a weakness for Wilkinson and Pickett. On the one hand, the evidence is accessible and easy to digest, helping The Spirit Level’s political message reach a wide audience. On the other hand, it is easy to check the figures, since they all come from publicly available sources. It does not take a PhD in social epidemiology to plot this data on a graph, and figures from respected sources such as the UN and OECD do not require further peer review.
By fact-checking The Spirit Level, we can see that both the use of the empirical data and the authors’ interpretation of that data are seriously flawed. In response to numerous criticisms of their work, Wilkinson and Pickett have offered only evasive, disingenuous and factually inaccurate replies. When not making baseless accusations of racism, they appeal to authority on the basis that they have had articles published in public health journals.
Now, after having enjoyed tremendous success in spreading their message through the mass media, they have now decreed that “all future debate should take place in peer-reviewed journals”. This is the sound of a door slamming. As Brendan O’Neil wrote recently in Spiked: “It is ironic that Pick ett and Wilkinson, so very keen on the idea of equality, don’t like the idea of an equal right to speak and critique.” Regardless of whether Wilkinson and Pickett choose to engage in debate in the future, the serious questions hanging over their research will not be going away.
* Andrew Leigh, Professor of Economics, Australian National University, Christopher Jencks, Malcolm Wiener Professor of Social Policy, Harvard Kennedy School, Timothy Smeeding, Distinguished Professor of Public Affairs, University of Wisconsin, ‘Health and economic inequality’ In W. Salverda et al (eds), The Oxford Handbook of Economic Inequality 2009, (p.4)