I would very much like this feature as well - i just ran into the problem yesterday. I understand that mousing over the production stats can help you figure out what you need more of, but there are other reasons I want to know quickly whether a city has something. The most obvious one is the coliseum because that actually lowers unhappiness. The other one is the stronghold/citadel as someone mentioned, b/c that leads to unique units. It'd be nice to see at a quick glance what cities have what buildings other than scrolling around and eyeballing the map (granted, they will undoubtedly make the buildings more illustrative of what they are), or mousing-over things using process of elimination.