Can we trust official statistics? The data gaps shaping our view of the economy

Unlock the Editor’s Digest for free

Roula Khalaf, Editor of the FT, selects her favourite stories in this weekly newsletter.

I have absolutely no idea who will win the US election, but at least that uncertainty should be resolved soon. If only the same could be said for the sort plaguing our official statistics. In theory, we live in an age of data abundance. But in some high-profile cases, we simply know less than we once did.

The sorriest case is Britain’s labour force survey (LFS). Whereas in 2014 roughly half of households bothered to respond, these days the share is closer to one in five. This collapse has exposed figures to volatility and potential bias, enough for the Office for National Statistics to have downgraded measures of inactivity and unemployment to “official statistics in development”. (I prefer the “numerical naughty step” or “nonsense number territory”.)

This is tricky for monetary policymakers, who need a clear view of the labour market’s strength when setting interest rates. I’m also sure more of that clarity would have been helpful for the Treasury before it loaded extra costs on to employers in the Budget. And if some of Britain’s measured inactivity problem is really a product of dodgy data, surely everyone would like to know.

Other labour market indicators, including from tax records, can be helpful. They suggest that employment growth was healthier than shown by the LFS between the summer of 2023 and spring of 2024, though weaker since then. But those don’t cover the self-employed, and reveal nothing about unemployment and inactivity.

The ONS is trying to fix the problem, by boosting the LFS’s sample size, strengthening incentives to respond and working on a new “transformed” version. But that involves grappling with a different threat to our knowledge. Extracting meaning from data often relies on having consistent series over time. And when you start fiddling with survey methods, that can be hard to achieve.

That issue probably gave statisticians the most grey hairs over the pandemic, when in-person interviews became impossible and it was unclear how useful funky new real-time data sources were.

More recently, Ryan Cummings of Stanford University and Ernie Tedeschi of Yale University argued that the past few months of consumer sentiment data coming from the University of Michigan had been distorted by a switch towards online respondents. Whereas the unadjusted data looks as if sentiment fell in spring of this year, according to the adjusted data the vibes aren’t quite that bad.

A third threat to our knowledge comes from the world changing in ways that statisticians find hard to capture quickly. Like how the shift to lending from non-bank financial institutions has made trends in credit murkier. Or how the rising importance of intangible capital makes investment harder to pin down.

In the US, it seems that the real change messing up official statistics has been a surge in immigration. That affects how responses in the Current Population Survey are weighted, and probably means a key measure of the labour market has understated its capacity to grow. Meanwhile, a different survey of employers suggests that employment growth is stronger. It will probably take until 2030 and the next census to resolve the puzzle.

Whereas I have seen plenty of analysts in Britain and the US agonising over which data sources can be trusted, such statistical soul-searching in the EU is less obvious. Why?

Holger Schmieding of Berenberg Bank suggests that identity cards and the European welfare state mean that many EU members probably have a better handle on their population size. Carsten Brzeski of ING Research says that survey data is less important for policymakers than data from unemployment registers. Then there is the fact that the EU’s labour force surveys boast response rates to make British statisticians weep with envy.

Before the Europeans get too smug, there are some caveats. Eurostat, the EU’s statistical agency, publishes response rates to labour force surveys with a hefty three-year lag, and some national agencies do not deign to publish anything more up to date (hello France and Italy). Overall trends are not entirely encouraging, with recent improvements in Germany, Spain and Portugal not reversing the decline since 2017.

And then if dropping response rates were causing difficulties, Europe’s relative lack of rival data sources would make it harder to tell. Problems in America and Britain have been laid bare by different statistics telling conflicting stories. Knowing less than we did before is stressful. Perhaps the secret of a calmer life is just to have known less in the first place.

[email protected]

Follow Soumaya Keynes with myFT and on X

The Economics Show with Soumaya Keynes is a new podcast from the FT bringing listeners a deeper understanding of the most complex global economic issues in easy-to-digest weekly episodes. Listen to new episodes every Monday on Apple, Spotify, Pocket Casts or wherever you get your podcasts