Were the Venetians to blame for the diversion of the 4th Crusade to Constantinople?

[General Pilates]

Myriocephalon, or rather, Manuel's reaction to that battle, did quite a bit of damage as well.

 

[Masada]

In before someone mentions 1350's and the Kantakouzenoi-Palaiologoi civil war.

Even that didn't manage to kill the Empire off. John VI Cantacuzenus was a solid emperor and managed to keep things going: Dushan couldn't land the killing blow, Orhan's Turks got kicked out of Thrace and he managed to finally defeat the Zealots in Thessaloniki. Granted, the Empire of John VI was weaker than it had been for any time since the restoration in 1261 under Michael VIII. But it was still capable of holding its own against its neighbours.

On that note, the Empire of 1261 under Michael VIII was anything but a rump including as it did the old core of the Empire: western Anatolia, Thrace and northern Greece. In terms of foes, the principal Latin Lords of Greece were in his hands, the Seljuk Sultan had fled to his court, Michael II Ducas Despot of Epirus and Thessaly was on the ropes and Trebizond and the Seljuks were under the Tartar's yoke.

Hell in 1262, Michael's armies were fighting on three fronts: in Achaea, the Aegean, in Epirus and still managed to deal Constantine Tich of Bulgaria a stinging blow when he invaded. Even then, the Emperor expanded his holdings: grabbing Achaea, some of the islands in the Aegean and managed to turn Michael II into an Imperial vassal. He left behind a strong army and a growing treasury. Things weren't so rosy under Andronicus II but Andronicus III was a good ruler and made still more ground.

Andronicus III's death in 1341 bought about the aforementioned Kantakouzenoi-Palaiologoi civil war, which lasted six years and the reign of John VI. What ultimately bought about the end was the plague, which crippled the Empire's armies and tax revenues, the ad-hoc power sharing arrangement between John VI and John V and the fall of Callipolis to John VI's own Turkish troops. Those events made it impossible for John VI an otherwise good ruler to right the ship of state. Had the plague not happened it's possible John VI would have made good on his military skills and stabilised the situation.

The thing only became hopeless (not impossible mind) when John V took over full control of the Empire at age 23 in 1354 and that was because the Empire had lost a good general in John VI and had little to nothing left to fight with. Annoyingly it still survived for two generations more.