Here is my take on the Tower of Mastery option.
It takes way, way too long to build. I know its been said already, but its worth repeating. I've taken the trouble to build the Tower of Mastery in four or five games. Typical build times are over a hundred turns--I play at epic speed. In all of these ToM wins, I finished the Tower with fewer than 15 turns to go before the game's end.
What I've found is that I end up finishing this build out of sheer stubborness. Because the truth is that in each of my ToM games, I could have accomplished a military victory well before the ToM was complete. Is this observation unusual? There was no need for the ToM victory. But I go for it anyway, and my fun factor drops as I manage my empire and watch the AI fumble about for a hundred turns or so.
Incidentally, this is observation is similar to the production of the more powerful rituals. They take so long to build that by the time they are near completion, I've already won a military victory.
For just this reason, not since my first two games (FFH v1) have I even bothered with the rituals. They just take too long! They look like a heck of a lot of fun, and I want to build them, but I can't justify the ~100 turns.
My opening suggestion is simple enough: cut their build time by 50-75%. Give me a reason to build them!