What defines empire? Imperium means "territory." The Imperium Romanum was literally "territory of the Romans." Since the Romans ceased to exist as a defined cultural and ethnic group since it's fall--displaced by either Greek or "barbarian"--there was no longer such a thing as an Imperium Romanum because there was no longer such a thing as "Romans."
Byzantium was a Greek Empire built upon the political foundations of Rome; it was composed of territories belonging to or subservient to the culturally and ethnically Greek.
The Holy Roman Emperors were given "Roman" in their title to invoke the former Roman Empire--sort of like invoking "Sid Meier" in every Civ game despite his decreasing importance in the actual programming.
The Popes have less ties to the Romans and more ties to the Slavs, Greeks, and Ethiopians. They were never around during the Roman times, and they were only important to the Holy Roman Empire because the idiot Franks made it important for the Pope to name an Emperor. Imagine, if they didn't do that, claims to the title would be settled not by a solemn Popely word but with blood and tears.
The Russian Tsars were just "inheriting" the Byzantine Empire by virtue of marriage and its defeat. It inherited a title from the Byzantines, who inherited a title from the Romans, but they're an empire of Russians with a Roman name (The Third Rome).
The Turks are Turks. They are farther away from Romans than everyone else combined.
If you have anything to contest, contest my definition of Empire.

It really depends on what definition you use, and I used the basest, most literal one.