You forget, the way the governments are run in Civ 4 is determined by civics. The fact that TGE/TIR are run differently is a consequence of different circumstances necessitating different -that's right!- civics.
But yea, you could have Pellaeon as the IR leader for TGE. I like Pellaeon. He's cool. Plus, he has that mustache that just reeks of BAMF.
Anyhoo, regarding different unit graphics, it can be explained through pure tech advancement via Tech Tree, as they both used the same tech lines, both used Stormtroopers, both used Star Destroyers, both used TIE Fighters and their successors. As I said, the reasons you have brought up to distinguish between them are easily explained by the Civ 4 Civic System and advances in the Tech Tree. Even the Civ 3 (<3) Government system could explain it.
Edit: And with your suggestion of separating TOR and TGR, I again say that it's a matter of Civic changes and Technological Advancement. One could say that they went from Free religion, to Pacifism, then back to Free Religion over the course of its history.