Yesterday the Guardian posted an article with the headline ‘‘Biased’ fit for work tests penalise poorer people’, based on a soon-to-be-released piece of research. This seems to have caught a few people’s eye – but sadly this is one of those times that I think the media has got it wrong (much as it’s written by the usually excellent Frances Ryan). In this post, I’ll explain why the headline goes way beyond the evidence.
First, a quick recap. The paper uses publicly available data on WCAs for people moving from the old Incapacity Benefit to the new benefit ESA, and looks at which areas were most likely to find people fit-for-work. Its key finding is that current Incapacity Benefit claimants were least likely to be awarded ESA in areas with more poverty and lower educational achievement. Hence the claim (in the words of the Guardian piece) that “the controversial work capability assessment [WCA] is disproportionately removing benefits from people in more deprived regions”.
Before disagreeing with this completely, it’s worth flagging three good points here. Firstly, the newspaper article does accurately reflect the paper. Secondly, the article still isn’t out yet, but the author Jonathan Hume helpfully shared a draft to his Twitter followers yesterday. (Jonathan unusually isn’t an employed academic, but has an article in the journal Radical Statistics – this is a journal for leftwing statisticians whether in academia or outside of it, and it sometimes has really thought-provoking articles; I should mention that I know some of the people on the editorial board). And thirdly, the paper itself is well-written enough that I’ve managed to replicate the analysis in it, and it holds up.
Yet I still think the article and Guardian piece are wrong. The author’s key assumption here is that if the test is fair, then the rate of finding people fit-for-work during their benefits assessment should be constant across different areas (as he said explicitly on Twitter). But I see no reason whatsoever for this to be true.
Put simply, some disabled people find it easier than other disabled people to find and keep work. If Stephen Hawking had previously worked for Primark rather than as a theoretical physicist, then he would have stopped working much sooner. While everything obviously depends on the individual, generally people with genuine but less severe disabilities are more likely to be working if they have higher qualifications and live in areas with more jobs. If people have more severe disabilities, then their chances of working get much smaller, even if wider advantages probably still help (I’m currently doing research on this as part of the Rethinking Incapacity project, so watch this space – but for the time being, see this and the studies cited in the paper behind it).
So my guess is that the average incapacity benefit claimant in a poorer area is less severely disabled than in the average claimant in a richer area. (It would be possible to check this out using different data such as Understanding Society – I’ll try and do this later in the year). And if you make it harder to claim incapacity benefits (such as the transition from Incapacity Benefit to ESA), then what you would expect to see is a greater proportion of people failing to get the new benefit in poorer areas. Hence this would explain the pattern that Jonathan Hume sees, without any unfairness at all in the WCA.
Does controlling for life expectancy in that area help? Well, probably not. The issue is about the health selectivity of the benefits system in different areas – i.e. not about whether people in the area are healthier or not on average, but how likely unhealthy people are to be claiming benefits. So to my mind, what the paper shows is a pattern that you would expect to see, and one which doesn’t (in itself) show there’s anything wrong with the WCA (though see my final words below on this).
I just don’t think the study can justify the very strong claim that “it has been established that there are significant biases in the WCA”. Showing bias is a tricky task where you need to have information on people’s actual eligibility for benefits, which would need a completely different type of study, backed up by supporting evidence. The study does show an interesting pattern for researchers to go away and look at, but it wasn’t predicted, there’s no other supporting evidence, etc – the study just isn’t strong enough to justify this kind of headline.
[Note: I edited this shortly after putting up the original post, when my wife pointed out that I hadn’t really explained it very well first time round…]
Fairness and the WCA
It’s worth clarifying a couple of things here:
- Overall, incapacity benefits are disproportionately claimed by lower-qualified people in more deprived areas (see e.g. the maps by Tina Beatty here). I find it very strange that this isn’t mentioned in either the research paper or the newspaper report. And the Government’s decision to means-test ESA for less-disabled people after a year further pushes the benefit away from
lessmore deprived people.
- The DWP is right to say that the paper “makes a huge leap in attributing local variation in work capability assessment outcomes to ‘bias’”. If you want to criticise the DWP’s response to research, then it’s better to use e.g. the DWP’s dismissal of the excellent paper by Ben Barr et al on the WCA last year.
Despite all this, at heart I agree with Jonathan, Frances and others that the WCA is a disaster and needs to be changed. Indeed, I’ve detailed a lot of the problems with it (and how it could be changed) here, and will keep writing about this over the next 12mths. But if anything, I think this makes it even more important to criticise the WCA for the right reasons, so that the many deficiencies of the WCA can’t reasonably be denied by anyone.