Fallout from breakdown of UK labour force survey spreads

Unlock the Editor’s Digest for free

Roula Khalaf, Editor of the FT, selects her favourite stories in this weekly newsletter.

The fallout from the breakdown of Britain’s labour force survey (LFS) is spreading, with the statistical regulator warning on Thursday that the quality of more official data had been affected.

The statement from the Office for Statistics Regulation — the regulatory arm of the UK Statistics Authority — coincided with the release of figures on the health of the population in England, Northern Ireland and Wales.

These are usually calculated directly from survey responses, but the Office for National Statistics, the UK’s statistics agency, said it had been forced to switch to a new methodology because the sample size of its Annual Population Survey — based on the troubled LFS — was now so small that it had produced “implausibly” extreme results for some local areas.

These results exaggerated the extent to which inequalities in expectations for a healthy life had widened since the pandemic.   

The OSR said the figures for healthy life expectancy were among 14 sets of statistics that it could no longer badge as “official statistics” because of the problems with the quality of the LFS, instead terming them “official statistics in development”.

The ONS was forced to scrap publication of the LFS in October 2023 after a long-term decline in response rates left them so low that they were no longer reliable.

The lack of reliable local data matters because the government has set a target to halve the gap in healthy life expectancy between England’s poorest and richest regions. The Health Foundation think-tank pointed to the experimental figures as evidence of “stark inequalities” that had worsened over time.

As well as the main labour market data, the OSR has stripped official status from data sets on smoking habits, personal wellbeing, the composition of the UK population by country of birth and nationality, and from regional breakdowns of data on labour markets and households’ disposable income.

Household disposable income is one of the metrics that Sir Keir Starmer’s government is now targeting as a “milestone” for rating progress in achieving “higher living standards in every region of the country”.

But the warning from the OSR suggests that the data fog that is obscuring the true state of the jobs market, making it harder to set fiscal policy and interest rates, could be a wider problem.

“We recognise there is still more to do,” Siobhan Tuohy Smith, OSR head of assessment, said in a blog accompanying the regulator’s statement.

“We are working to understand to what extent response issues are impacting other household surveys used across the statistics landscape. We have asked the ONS to consider whether the issues, concerns and lessons it has learnt from the LFS apply to its other surveys,” Smith added.

The ONS, which last week admitted it may not be able to replace the LFS until 2027, said it was “addressing the challenge of falling household survey response rates . . . in different ways with different statistics”.

But it added that small samples were particularly problematic for statistics at local authority level, especially when the results were split further by age group and sex.

“We have developed a new approach for estimating healthy life expectancy, which creates more reliable data and deeper insights at local level,” the agency said, adding that where other surveys fed into key statistics, “we triangulate the results with other sources to ensure the statistics remain of high quality”.