Surging inflation led interest payments on government debt to hit the highest amount for May on record.
Interest payments paid by the government for last month hit £7.6bn, up £3.1bn from a year earlier.
Government borrowing fell in May from a year earlier but still remained higher than pre-Covid levels.
Borrowing - the difference between spending and tax income - was £14bn, down £4bn from a year earlier, the Office for National Statistics said.
But the figure was still the third-highest May borrowing since monthly records began in 1993 and was also £3.7bn more than the Office for Budget Responsibility (OBR) had forecast.
The recent high levels of debt interest payments are largely a result of higher inflation, the ONS said.
This is due to the interest paid on government bonds rising in line with the Retail Prices Index measure of inflation, which hit almost 12% in May.
So far this financial year, interest payments have totalled £14.1bn, up £4.7bn year on year.
Where does the government borrow billions from?
The ONS said May's figure was the third highest debt interest payment made by a government in any single month.
The OBR estimates that debt interest payments will cost the government £87.2bn over the financial year ending in March 2023.
Chancellor Rishi Sunak said rising inflation and increasing debt interest costs "pose a challenge for the public finances, as they do for family budgets".
Danni Hewson, a financial analyst at AJ Bell, said the rise in interest payments on government debt was a "prime example" that there is "nowhere inflation is not making its presence felt".
Michal Stelmach, an economist at KPMG UK, said reducing debt this year remained a "long shot", due to due extra financial support being provided to households hit by rising fuel, energy and food prices.
"We expect borrowing to overshoot the OBR's March forecast by around £20bn this year, largely on account of higher spending and weaker economic growth," he said.
Central government receipts were £66.6bn in May 2022, £5.7bn more than May 2021, with an annual increase of £3.4bn in taxes.
Tax revenues in total rose to £48.3bn in May, with National Insurance Contributions (NICs) raising £2bn more than last year.
Employees, employers and the self-employed started paying 1.25p more in the pound for National Insurance from 6 April.
Ms Hewson said tax income had risen thanks to a "mix" of VAT returning to normal levels on hospitality, extra cash from the uplift in National Insurance and "simmering" housing market and a workforce back at work.
"But any household pouring over their own books will know that a budget that's still got more going out than coming in is one that's not healthy or sustainable," she added.