How stats chiefs left Rachel Reeves flying blind

How many Britons are unemployed? How fast is the economy growing? How many people migrate in and out of Britain each year?

These are crucial facts that politicians, central bankers and civil servants must know when drawing up laws, making economic decisions and measuring the impact of policy. Such calculations affect us all.

Yet decision makers ranging from Rachel Reeves, the Chancellor, to Andrew Bailey, the Governor of the Bank of England, have been flying blind after a breakdown at Britain’s official statistics body that has forced it to admit that it is now producing highly unreliable data.

The scale of the problem was clear to see on Thursday when the Office for National Statistics (ONS) admitted it had somehow missed 166,000 migrants who came to Britain in the 12 months ending in June 2023. It means net migration over the period was actually an unprecedented 906,000, not 740,000.

It is not just migration figures that are unreliable: the ONS faces scrutiny over its labour market data, with the statistics body admitting it cannot say with confidence how many people are unemployed or in work.

The fundamental breakdown in the reliability of Britain’s data has been blamed on the death of the landline and the overly long surveys used to collect data. Yet the stats watchdog also faces questions about the ability of its workforce. The ONS appears to struggle to attract top talent and its unionised civil servants who work from home are now in open rebellion against management over attempts to drag them back to the office. Critics say this row over working from home is distracting them from the day job.

MPs are now demanding answers. Dame Meg Hillier, chairman of the Treasury select committee, warned national statistician Sir Ian Diamond last week that MPs “have major concerns” about the UK’s ability to set interest rates and make tax and spend decisions because of the data issues. Bank of England officials have also warned that the job of controlling inflation has been made more difficult.

Jonathan Portes, a professor focused on labour markets and immigration at King’s College London, says the errors with migration figures “suggests that the communication between the ONS and the Home Office is not as good as it could be”.

“There is a significant number of Ukrainians who the Home Office recorded as giving visas to and entering, and the ONS had for whatever reason not counted them in previous statistics.”

An independent review undertaken by Professor Denise Lievesley at the start of 2024 found that officials were reluctant to share data with other government departments because of fears over privacy issues or that problems with quality of the information would be exposed.

It’s not the first such issue with migration figures. The statistics body said in 2019 that it had underestimated the number of EU migrants coming to the UK from the mid-2000s until 2016. Experts at the University of Oxford said the ONS had “systematically” underestimated the true numbers.

Yet problems go beyond communication between government departments. The statistics body cannot say how many people are in and out of work. This makes it hard for politicians to know the true scale of an apparent drop in the employment rate and rising economic inactivity since Covid.

One reason for this is that the share of people agreeing to respond to its jobs survey has plunged since Covid.

“I worked on the labour market figures from 2003 until I retired in 2019. I remember in the 2000s, the ONS at the time were concerned the response rate had fallen below 80pc. It is now below 20pc,” says statistician Richard Clegg, who spent 27 years working for the ONS and led on the data.

“If dropping below 80pc sounded alarm bells then you can imagine dropping below 20pc really affects quality quite substantially.”

Death of the landline

Clegg says part of the problem is the length of the survey. It has “grown and grown” and the questionnaire is more than 200 pages long.

“The more questions in the survey, the more time it takes and the more likely people are to say, ‘No thanks, I haven’t got the time to do this,’” Clegg says.

Questions on smoking habits or sexual orientation, while important, may be better collected in other surveys.

“Whenever the ONS have tried to cut down the survey, whenever they’ve consulted, people have always said no. Can we take these smoking figures off? The Department of Health will say no. The ONS haven’t been prepared to say: we have to because the response rates are going down,” he says.

Some issues are outside the statisticians’ controls. A majority of households no longer have landlines, Ofcom figures show. Of those who still do, many are reluctant to pick up a call from an unknown number. The plethora of scammers, meanwhile, makes those who do pick up are wary of sharing detailed personal information over the phone.

Meanwhile, people who are at home during the day and willing to go through all the questions are more likely to be out of work, potentially skewing the results.

Even still, Andrew Bailey has told MPs that other countries’ data appears to be more robust in the face of similar challenges, suggesting the UK has struggled more than its peers.

Swati Dhingra, a member of the Bank of England panel that decides interest rates, has little sympathy for the statisticians’ struggles.

“I grew up in India. There are a billion people there. We managed to get the labour force survey answered. I don’t find it particularly plausible that that’s hard to do,” she told a conference in London earlier this week.

Loss of London-based talent

The ONS has faced hurdles in the past in retaining staff, with an independent review finding that the decision to move it to Wales in the late 2000s led to a 90pc loss of London-based talent.

While nearly two decades have passed since then, Clegg says attracting the best mathematical minds is a challenge.

ONS roles advertised on LinkedIn for data analysts and senior statisticians are listed with salaries of £35,000 to £45,000, and candidates with 2:2s are accepted.

Meanwhile, a lead economist role is advertised with a salary of £56,020, lagging well behind what banks, insurers and consultancies in the private sector offer.

“It may mean that you’re not really getting the best,” Clegg says. “If the private sector pays more, then logically the cream of the crop is going to go into the private sector.”

Then there is the question of working from home. More than a thousand ONS staff who belong to the Public and Commercial Services Union (PCS) have refused to go into the office since May in protest at being ordered to come in two days a week.

The ONS’s own data published earlier this month suggest people who work from home sleep longer and work less on average. Sir Jacob Rees-Mogg, the former minister for Brexit opportunities and government efficiency, told The Telegraph at the time that civil servants were “literally asleep on the job”.

Unreliable data matters as our assumptions about everything from productivity to living standards and employment may be wrong, says Adam Corlett from the Resolution Foundation.

His analysis of tax data and other sources suggests that the statistics body has undercounted nearly 1m workers in the jobs market and overstated economic inactivity.

Underestimating the rise in net migration may have played a role in this error, he believes.

‘Behind schedule’

The ONS is aware of its own shortcomings and is trying to address them. It is in the process of overhauling how it does the labour force survey and modernising it. But anyone holding their breath for a fast upgrade will likely be disappointed, says Clegg.

“It’s quite substantially behind schedule, as pretty much everything that the ONS seem to do. They don’t have a good record of managing major projects.”

An ONS spokesman said issues with migration and labour force data were unrelated. On migration, the spokesman said the revised figures reflected changes in the way the figures were drawn up and said the new system was an improvement.

They said: “The improvements we are making have been against the backdrop of significant changes in migration in the UK and across the world, including due to the war in Ukraine, the post-Brexit immigration system and the pandemic.”

On the labour force survey, the spokesman said: “To tackle this challenge we have increased the Labour Force Survey’s sample size, reintroduced face-to-face interviewing and increased incentives for those taking part. We will also be re-weighting the survey responses to our latest information about the UK population from next week.”

The statistics body is also testing a shorter version of its questionnaire “with early responses looking positive. We will publish findings of this test in the new year”.

Broaden your horizons with award-winning British journalism. Try The Telegraph free for 3 months with unlimited access to our award-winning website, exclusive app, money-saving offers and more.