The point of the tower of mastery is that it is a quick way to win if your empire is developed to the point where you're basically sure to win, but driving it home on a large or huge map would take a long time with any other victory condition. Between the tech requirements, land requirements (you need 3 or 4 nodes, even with dispel) and gold rush options, it's a way to channel your empire's output directly into a win, if said output is suitable impressive. Every 4x game needs a victory condition like that. Everyone hates the whole "you've practically won but not really" syndrome of many strategy games, ToM's a perfect cure.
If you made the ToM itself only buildable with all mana types you'd be left with the old situation where ToM and domination were basically the same victory - or, if you made rites of oghma repeatable like is being suggested, a case where commerce doesn't help you, since you need to build that ritual repeatedly and rituals can't be gold rushed or GE rushed.
The current victory is not overpowered because you still need to pay the 7,200 hammer cost for the 5 towers. If you can generate enough gold to buy them, that means your empire is on top and should win. If you try and win this victory by spamming dispel on 3 nodes in a crappy empire that couldn't win any other way, you won't be able to cash rush the final tower. You'll have to try defending your puny little civilization from everyone else's attacks while building the tower the hard way. And you will fail.
