Improving our understanding of UK poverty will require better data

Policy makers across the political spectrum want to improve the living standards of the UK’s poorer households. The Leader of the House of Commons recently stated that “We are committed to our manifesto pledge to reduce child poverty”, while the last Labour manifesto pledged to “eradicate in-work poverty”. So statistics about poverty matter. We need to know not just whether aggregate measures like GDP grew last year, for example, but whether the living standards of the poorest improved or not, and by how much. Ideally, we would learn quickly when policy or economic changes have an impact on poverty.

It is therefore welcome that the Office for Statistics Regulation (OSR) has begun a ‘systemic review’ of poverty statistics, “to enhance understanding and decision making in this area”. There are a lot of choices that can be made about poverty statistics – after all, there is no strict scientific guide as to how to define ‘poverty’, let alone operationalise any definition in a measure – but this blog argues that the most fundamental change that’s needed is to improve the accuracy of the underlying data. Put simply, the UK’s existing poverty statistics are wrong, and fixing them must be a priority.

The problem: benefit under-reporting

As we set out in detail in 2018, existing UK poverty statistics published by DWP should be treated with some caution because they rely on survey-based data (the Family Resources Survey) that appears to be missing a large fraction of income from social security benefits and tax credits. We know, for example, that the government spent £214 billion on benefits in 2016-17 (in Great Britain), but the key survey used to estimate poverty statistics records only £170 billion. This means that income of around £40 billion a year is missing from our key income and poverty statistics (and this is having accounted for some definitional differences, such as benefits going to people who live overseas).

Although we do not know for sure why there is so such under-reporting or under-recording of benefit income, it seems likely that at least part of the explanation is a desire to speed up the questionnaire, and part could be genuine confusion or forgetfulness. When answering a large array of questions about their household’s circumstances, many people may temporarily forget (or try to save time by ignoring) some of their sources of income, or perhaps get confused about the ever-changing forms and names of benefit support. This effect varies for different benefits. It seems that Tax Credits, for example, have been particularly likely to be forgotten about or under-estimated by respondents, with around 30 per cent – or £8 billion – of all this spending missing

Figure 1: £40 billion of annual income is missing from key income and poverty statistics

And this inaccuracy has got worse over time. As best we can tell, around the turn of the millennium under 10 per cent of benefit spending was not being recorded in the Family Resources Survey data, but this then rose to peak at 19 per cent in 2015-16.

Figure 2: Household surveys have become less good at capturing income from government benefits

This is a not a minor problem. Recording the missing £40 billion a year would go a long way in reducing measured poverty. Figure 3 shows the results of our earlier modelling, which imputed the missing benefit spending and then recalculated poverty statistics. Although these numbers should not be considered certain, they are probably closer to the truth than the existing National Statistics on poverty, and they show the rough scale of the error. Relative child poverty could be overestimated by 6 percentage points (25 per cent) and pensioner poverty overestimated by 5 percentage points (over 40 per cent).

Figure 3: Adding in the “missing” income would make a big difference to recorded poverty statistics

And the change in the scale of this problem over time means that poverty trends may also have been misleading. Thankfully our modelling suggests the direction of travel has not tended to be wrong, but it is likely that the pace of changes has been underestimated. In fact, it is possible that the high-profile child poverty target set by earlier governments for 2004-05 (and perhaps even that for 2010-11) was actually met, in contrast to what the official statistics said at the time. That would be a significant failure of National Statistics.

Figure 4: With corrected poverty data, child poverty in the 2000s may have hit some of the Blair/Brown government’s targets

The solution: linking to administrative data

The solution follows directly from the obvious question, of “Why does the Government need to ask people what is their benefit income when it is the one paying them that income?” Parts of the government do indeed have accurate administrative data recording who is being paid what; the challenge is to link that administrative data with the responses to the household survey.

We will not dwell here on the technicalities of linking survey responses to benefit records. But the DWP has already managed to link 89 per cent of cases (with consent) for the purpose of estimating benefit take-up rates; and it is working on the problem for income and poverty statistics. At the same time, the ONS is building up its ability to use administrative data, and recognises that fixing incomes at the bottom of the income distribution is a priority. The problems caused by Covid-19 for data collectors in 2020 and 2021 only increase the need to make use of administrative records as quickly as possible. However, we are yet to see any real public progress on fixing benefit under-reporting from either DWP or ONS.

And there is a still more radical possibility. The Government holds not only data about people’s benefits, but also about their earnings, other forms of income, taxes and household composition. The ONS has therefore been able to build up some experimental statistics that calculate incomes for every household in England and Wales. The focus of this work seems to have been on creating a particular snapshot for census-related research. But this powerful resource seems underutilised so far: it should also be used as a supplementary measure of statistics like typical income and poverty rates. In doing so, it would form a valuable check of the purely survey-based estimates, as well as allowing for extremely detailed geographical breakdowns, and would open up the possibility of much more timely household income statistics.

How to define poverty in the UK

So, having accurate income data needs to be the foundation of improving income-based poverty statistics, and should be a focus for the OSR’s review. In the shorter-term, thought also needs to be given to how the ONS’s Household Finances Survey could be used alongside the DWP’s Family Resources Survey to base poverty statistics on a larger sample of households and make statistics more timely. (Note that the Resolution Foundation currently produces nowcasts of poverty rates, in part because the survey-based data has a significant lag of 1-2 years.)

But once we have those stronger foundations, there remain many conceptual questions left to discuss. Three are particularly tricky:

1. How should we define income? A key part of this question is how housing costs should be accounted for. Poverty statistics have typically been presented for disposable incomes both before and after housing costs. In the near future (based on work by ONS), there may also be the possibility of including ‘imputed rent’ as income – under which owning a house outright would be counted as a boost to a household’s ‘income’. Relatedly, the Social Metrics Commission argues that all mortgage repayments should be deducted from disposable income – not just interest costs. How best to treat housing costs is not a question with a simple answer, but we would argue that there is little justification for poverty statistics that do not account for housing at all.There are many other complexities in defining incomes. We have noted previously that tuition fee loans are included as income, which has the bizarre consequence that increases in tuition fees, which automatically increase the value of tuition fee loans, result in fewer students being recorded as living in poverty. There is a case for deducting childcare, commuting, and disability-based costs from disposable income; and the Social Metrics Commission tries to accounting for liquid assets. Then there is the even more important issues of whether we are ‘equivalising’ income (accounting for family size) correctly, so it is welcome that the Commission and ONS are discussing this.

2. Should we use fixed or moving poverty lines? The OSR review would be right to look at whether the common use of two separate definitions of poverty is confusing and allows commentators on all sides to cherry-pick the statistic that makes their point better. There may be scope for new norms or terminology here. But we believe that both measures are very valuable. Fixed poverty lines provide a measure of long-term progress for the poorest, with a current threshold (after housing costs) of around £7,900 a year for a single person: 60 per cent of the 2010-11 median but adjusting for inflation. Relative poverty – usually 60 per cent of the current median – is a good measure of how far we willing to let some people fall behind relative to typical living standards. It would be unfair to focus exclusively on relative poverty if real incomes were growing rapidly – though in practice the last 15 years have been unusually bad in this respect. But it would also not be useful to stick with the same absolute yardstick indefinitely. As Figure 5 shows, if we adopted a 1970 or 1980-based poverty line, poverty would now be almost abolished (and the stubborn rump shown may in large part be due to the data issues discussed above). That is indeed a genuine improvement in living standards, but society should aim for – and therefore seek to measure – further progress for the poorest. There are, of course, other alternatives, with one in particular being proposed by the Social Metrics Commission. But we believe that their suggestion to measure poverty by benchmarking incomes to the average of the median over three years is – on balance – more confusing than useful. So the continued use of both absolute and relative poverty seems warranted.

Figure 5: Over long periods of time, absolutely poverty measures lose their relevance

3. How should relative poverty be defined? The Government’s approach to defining relative poverty is ultimately arbitrary. In the UK, the focus has been on those with equivalised household incomes below 60 per cent of the median. However, other thresholds are just as meaningful, and looking at more than one poverty line can give a sense of changes in the depth of poverty: Figure 6, for example, shows the fraction of the population in households with less than 40, 50 and 60 per cent of the median.50 per cent of the median is a common standard internationally (and it is easier to envision relative poverty being abolished by that metric). The 40 per cent threshold might in theory be a good measure of severe poverty – and this level is now at record highs – but missing benefit income and other data issues are a particularly big problem for this measure. On balance, it is probably best to limit confusion by focusing on only one relative poverty line in the UK, but it is also important to keep track of how the depth of poverty is changing.

Figure 6: Tracking multiple poverty lines allows you to assess the depth of poverty

We look forward to seeing what the OSR concludes, and how the communication of poverty statistics can be collectively improved. But definitional choices should not distract DWP and ONS from what should now be a priority: fixing the large inaccuracies in existing income statistics.