Ancient Era
Barracks
For certain, appropriate for cities dedicated to generating units. In practice, many cities create quite a few units because units tend to hit the sweetspot when you can pop two population and get lots of overflow, and even if you aren't at war Hereditary Rule makes military police a useful thing.
Also, there's a happiness bonus available when you are parked in Nationalism.
My games don't often feature Nationalism (I tend to jump from Bureaucracy to Free Speech) so the happy bonus isn't relevant. The troop promotions are important for troops that are going to be used, but not so much those who stand around looking pretty.
In builder mode, a lot of the troops I generate aren't going to be used, so I'd rather invest the barracks hammers into commercial infrastructure (40 hammers is another happy in HR, after all).
Aggressive leaders who leave their troops standing around looking pretty are wasting a trait, so they should be rolling barracks into every city.
Granary
As noted previously, it's really hard to come up with a scenario where granaries are a bad thing, especially so if the slavery civic is being run. These go everywhere - the only question is whether there is some more pressing immediate need.
More pressing immediate need is fairly common in captured cities, where you need to deal with cultural pressure or maintenance costs in a hurry. The granary is on the to do list, but since it only begins to offer benefit when the city is growing, I don't fret about it when I'm whipping the angry citizens away, only when I'm growing them back.
Library
Lilnev hits the main points. The capital normally has food surplus that I'll want to leverage into a Science Academy fairly quickly.
The cities that are currently generating a lot of research are candidates for libraries. That normally doesn't include cities 2-5 in my list, because the they aren't generating a lot of commerce yet, and the gold slider eats into that for a while. As a result, I've been trying to get out of the habit of rushing libraries.
Basically, this means that cities which are going to be running science specialists, and cities that are going to be specialized for commerce, typically get a library eventually, but the urgency generally isn't there. In particular, I'm trying to lose the habit of rushing libraries for the border pop.
Lighthouse
The value of a lighthouse is determined by the number of water tiles you expect to work.
I used to make a lighthouse a top priority in a coast city, but have not been doing so recently. An early coastal city is normally placed to (a) give me a good shipyard (so I'm going to be working a lot of production tiles) or (b) collect a bunch of yummy resources, which I'll be working. Coastal tiles just don't generate enough commerce to be really interesting.
Exceptions are those cities that don't have enough interesting land tiles. Archipelago calls for a lots of lighthouses, as are those cities that you drop into the tundra to bring you silver, furs, etc.
But in land based maps, lighthouses are now an "eventually" build.
Monastery
The main benefit of Monasteries is building missionaries, which can be accomplished for free with Organized Religion. So that's not such a big deal.
The research bonus is nice, but it's only a 1 beaker boost for every 10 base beakers you feed into it. Again, the capital can make nice use of that early, but for other cities it takes a while for that bonus to matter.
As a result, Monasteries are pushed far enough down the priority list that I may not ever get them built, because Sci Method is going to make them obsolete anyway. So I'll get one done for each religion that I'll need to throw around in the late game, so that I can continue generating missionaries after the swap to Free Religion.
Monument
Cultural boost. You don't need it if you are creative, or if you have a religion to spread around, or if you build Stonehenge instead, or if you can afford to wait before worrying about expanding into the second ring. Or if you can run an artist specialist briefly, or if you can run a Sistine enhanced specialist of any kind, or if you are about to found a religion and you are fairly confident where it will land, or if you are about to drop a Science Academy into the city, or you have a more efficient way to invest hammers in culture....
I'm actually building more of these than I used to. The boost for Charismatic is part of that, but I'm coming to the conclusion that 30 hammers wasted on the monument, and 60 hammers invested in something useful, are a better deal than a 90 hammers prematurely invested in a library.
So when I do build them, they tend to show up higher in the priority queue than the granary, because I want whatever is in ring two NOW.
Besides forcing the pop to the fat cross, the extra culture can be useful in a culture battle, or offers a trickle of value in one of the legendary cities when shooting for a cultural win. In practice, I don't see these cases too often.
Temples
Culture, happy, a priest slot, and fraction of a coupon redeemable for a cathedral.
The happy is usually the big part of the puzzle here. 80 hammers for a temple may not be a good deal compared to two military police at 40 hammers each. But temples come sooner, and I'm a big fan of early growth.
Also, temples are the key to getting Great Prophets in a hurry, a key element to doing well in a religiously themed game.
Like other happy inducing buildings, whipping temples is "free" as far as cruel oppression is concerned.
Walls
Almost useless. The main issue being that if you need walls, you don't have the troops to adequately defend your tile improvements, so the city is about to be in a world of hurt economically anyway. The only time I find myself building these are in games that I expect to lose: immortal, raging barbarians, always war sorts of things.
Otherwise, I would consider them only within the context of being a pre-requisite for castle, adding them into that calculation, and backdating the start of them from when I wanted castle construction to begin.
. The forge's production bonus applies to whipping too.
You poor misguided fools. There is no such thing as buildoholism. That which you describe is not a symptom of a building disease. It is a sign of one attaining a higher sense of self. In knowing that we create that which will survive for many thousands of years we feel an inner warmth from the thought of the betterment to our peoples lives. Isn't it better to build with one's own hand rather than to fill that hand with steel and take from our neighbor? Unless that neighbor is a psychotic circus clown named Monty. 
