Hungary’s controversial Prime Minister Viktor Orban has suffered a shock political defeat. His Fidesz party’s two-term mayor of the capital Budapest, Istvan Tarlos, was ousted in elections at the weekend. It represents an important first-ever check for the increasingly authoritarian Orban, who clearly imagined his racist blockade of migrants and his barely-disguised Islamophobia continued to be widely popular.

Fidesz mayors were ousted from ten of the 23 cities in which votes were held. That is the good news. But less encouraging is the reality that it took a series of deals among opposition parties to bring about these results. Essentially it was agreed among them that they would agree on and back just one candidate to challenge the incumbents of the long-dominant governing party, which still holds 133 seats in the 199-member national parliament.

This tactical coming together is inherently weak. It has seen liberals, social democrats and Greens team up with Jobbik, a party which is arguably even further to the right than Fidesz. Thus under the opposition deal, Jobbik won two provincial mayoralties, with votes from liberals and Greens who must nevertheless have held their noses as they cast their ballots.

An attempt to strike a similar deal ahead of last year’s general election fell apart. It must be wondered if the opposition parties can now maintain their cohesion for the next national poll three years hence. The chances are it will be unlikely. It is an unpleasant reality that Jobbik is actually the second largest single party in parliament with 23 seats, having won just over 20 percent of the popular vote. Orban’s government has recently been caught up in a sex scandal and allegations of corruption. Jobbik will be hoping that by 2022 it will be in a position to vacuum up the votes of disillusioned Fidesz supporters.

Moreover, if the only common ground on which the opposition parties can come together is the shared distaste for Orban, then it is hardly promising. Much of Jobbik’s political platform is even more extreme than that of Fidesz.

Orban did his best to dismiss the Budapest and other mayoral defeats. “Fidesz is the biggest party in Hungary, and we will act according to that knowledge,” he told supporters on Sunday night. Local analysts believe it unlikely the Hungarian premier will seek to copy Turkey’s President Recep Tayyip Erdogan and find some way to annul the Budapest vote and have the election rerun, in the hope that this time Fidesz supporters can be galvanized into voting, which it appears many of them did not do at the weekend. However, to try this tactic could risk the same enormous humiliation that Erdogan suffered when the voters of Istanbul once again returned the opposition mayoral candidate, but this time with an even bigger majority.

Budapest’s new liberal mayor Gergely Karacsony’s vow to bring the city “back into Europe” is a direct reference to Orban’s confrontational attitude toward Brussels. The prime minister is likely to do everything he can to push back against such a challenge, made the greater because the opposition now has most of the capital’s district mayors and controls the city council. But open warfare with his capital’s leaders is a high-risk proposition that could bring damaging legal challenges from the European Court of Justice.