At the Conservative Party’s constituency office in Wellingborough, a town two hours north of London, a sign tells us that local Tory MP Peter Bone, who was first elected in 2005, is “listening to Wellingborough and Rushden”. But for how much longer?

A report from the parliamentary standards committee has recommended that Bone be suspended from the House of Commons for six weeks for serial bullying and “one act of sexual misconduct”. (Bone denies the allegations.) If the punishment is confirmed by parliament, a recall petition will be triggered, and a by-election will probably follow.

The disintegration of the Conservative Party is happening faster than the Conservatives feared and Labour dared hope. The remarkable thing about the party’s recent double defeat in the Tamworth and Mid-Bedfordshire by-elections was not just the scale of the drubbing, with swings of 23.1 percentage points and 20.5 points, respectively. It was the party’s reaction. Nadine Dorries used her Friday night show on Talk TV to denounce prime minister Rishi Sunak’s leadership — as if her failure to turn up to her seat for months had nothing to do with the result. Personal responsibility, it seems, is no longer a Conservative virtue. A chorus of MPs called on the party to turn hard right by clamping down on immigration and reducing the top rate of tax — as if the most persuasive appeal to moderates is madder music and stronger wine.

The Observer, a Left-leaning newspaper, claimed, on the authority of several “senior Tories”, that chancellor Jeremy Hunt will not contest his seat in the next General Election. Hunt denied the rumours, but the striking thing is how plausible they are. In May 1997, Michael Portillo, the party’s presumptive leader, lost his seat in a 17.4-point swing to Labour, sending a surge of joy through the Labour Party and producing the popular meme “Were you up for Portillo?” (The result was announced at 3.10am.)

Politicos are circulating a “Portillo list” of leading Tories who are in danger of losing their seats and the swings required to unseat them: Grant Shapps (10.5 points to Labour), Michael Gove (17.5 points to the Liberal Democrats) and Hunt (7.4 points to the Liberal Democrats).

Calls for the party to move to the Right make some superficial sense. The Reform Party — the heir to the Brexit Party, which in turn was the heir to UKIP, which in turn was heir to a million pub rants about the decline of the country — certainly helped to deny the Tory Party victories in both Tamworth and Mid-Bedfordshire, collecting more votes than the Labour winning margins. Richard Tice, the Reform Party leader, pledges to punish the Tories for betraying Brexit by targeting “red wall seats”; Reform’s fortunes could surge if Tice steps aside for the party’s ringmaster, Nigel Farage. The hero’s welcome that Farage received at the Conservative Party conference three weeks ago — including from some former Cabinet ministers — shows how strong his appeal is to true-blue Tories.

But Sunak’s party will pay a heavy price for moving rightward. Since 2016, the party has been able to protect its right flank by absorbing Brexit voters because people were so terrified of Jeremy Corbyn: better a slightly noxious Tory Party than a poisonous Labour Party. But Sir Keir Starmer’s detoxification of Labour has enormously increased the cost of that approach.

Middle-of-the-road voters are not only willing to vote Labour because they think that the ghost of Corbyn has been exorcised; they are more willing to vote Liberal Democrat, as well, because they don’t feel the same need to keep Labour out of power. Disillusioned Tories are also more likely to stay at home, as so many did in Tamworth and Mid-Bedfordshire, if the fear of Corbyn is not there to motivate them.

There is also a sense that the party’s time has come. Thirteen years is a long time in office in any political era, let alone one of 24-hour news cycles and nanosecond concentration. The Tory Party has prolonged its life span by constantly reinventing itself: David Cameron’s Notting Hill Toryism gave way to Theresa May’s dutiful middle-of-the-roadism, which gave way to Boris Johnson’s cad regime, which gave way to Sunak’s desperate last act. Sunak is trying yet another relaunch by claiming that he is running against the past 30 years of failed policies. But the chances of success are low. If voters may not be particularly excited about Starmer’s Labour Party, they are thoroughly sick of the Tories.

Bad luck seems to be haunting the Tory Party in a way that good luck blesses rising parties. Following hard on the heels of the two by-election defeats, Jamie Wallis announced that he will be standing down at the next election, having failed to report a serious car crash before being arrested. Even the party’s ability to choose the timing of the next election up to the hard deadline of January 28, 2025, is turning from a blessing into a curse. That the party is trailing in the polls by more than 20 points means that it has an incentive to keep waiting for things to look up. But the longer it delays and keeps its zombie government in suspended animation, the angrier the public become. The chance of a repeat of Tony Blair’s 1997 landslide grows by the month, if not the day.

• Adrian Wooldridge is the global business columnist for Bloomberg Opinion. A former writer at The Economist, he is author of The Aristocracy of Talent: How Meritocracy Made the Modern World