The big thing that's been only briefly touched on is civics.
Theocracy is great for warmongers. As said, it allows you to run Bureaucracy while still having an XP boost. Alternatively, if you're really set on war, you could run it alongside Feudalism. Combine that with Barracks and possibly a Stable, as well as either AGG or CHA (or both, Boudica) and you're looking at some pretty ferocious troops coming right out of the gate.
Pacifism, on the other hand, is great for development. +100% GPP is potentially massive (though, as with PHI, it doesn't actually equal out to double the amount of GP). It's gold cost per military unit is frustrating for military leaders, but if your army is sparse, it's the cheapest civic in the game.
Both Organized Religion and Free Religion are versatile civics that are effective for both warmongers and pacifists. A +25% bonus to building construction (including wonders) on the one hand, and an all-around research boost on the other. Not too shabby.
But all of these bonuses (except FR) only applies to cities where your state religion is present. No religion, no benefits. FR gets happiness bonus for each religion, so it benefits from spread as well.
The other bonus is religious buildings. Keep in mind that you can have one of each building per religion in every city, and their bonuses are cumulative.
Monasteries are great. +10% science for minimal hammer commitment. That means potentially up to +70% science in a city. You stack that with Academy, Oxford, etc... Well, I'm sure I don't need to spell it out for you. The only disadvantage is the fact that it becomes obsolete.
Temples aren't bad. The happiness boost can be valuable. Whether it's worth the hammers or not depends on the situation, but think of it like this: Every religion you have in a city is an extra population that city can grow that it ordinarily wouldn't be able to maintain. Double that with FR. Not too shabby. It helps smooth the transition from HR to Rep, or lets you grow cities bigger if you build Mids and skip HR altogether.
Cathedrals are invaluable for culture games. Utterly indispensable. That 50% boost is incredible. Whether they're worth it in other games depends. They are very situational - or rather, their hammer cost is very, very high for what you get. Even with the double production bonus, you're still spending 150 hammers for, at most, 3 happiness - technically more efficient than many other happiness buildings, but keep in mind that temples are a prerequisite for it, and that if a cathedral's not your state religion, and if you don't have incense, then you receive NO happiness from it. Also, 2 priest slots... But how often do you use priests?
That 50% culture boost is what makes them great, and man, it can be amazing. But if that doesn't interest you, skip it.
Anyway, these are just some of the things unlocked by religions that, in my opinion, make them very worthwhile