Yes, I specialize cities, but not the way most people are thinking.
In any empire, you'll have a core of "original" cities where you'll build every structure. Every one of these cities will have a Barracks, Library, etc.; note that National Wonders require every city you possess at the time the thing is built to have a certain building. Coincidentally, many national wonders come pretty early in the game. (Many of the later ones aren't much of a downside. For instance, look at the Ironworks: needs a Workshop in every city. No matter what you want to specialize a city in, a Workshop is a good investment.)
So your first five or six cities have Barracks, you build the Heroic Epic, and then from that point on you can expand without having to build more Barracks. This isn't Civ4 where you needed a large standing army; a few good combat units will suffice, so you don't need to support unit production in every city. Besides, most of the prerequisite buildings are the dinky 1-cost ones; it's not a big deal to have a few more of these than you really need, so even if that Library wasn't really necessary in a city, build it anyway.
That being said, in the long term I'll usually have a half-dozen high-production cities, two of which will be specialized for land military units (Forge, Barracks, Armory, etc.) and one for naval units. Again, these will almost always be my starting cities; cities you acquire later in combat, or that you settled later on, just won't have the time to build all the necessary structures, so those will specialize.
(Ironically, the +50% railroad bonus will apply to all of these except for your capital, so invariably the capital ends up being a money/research town surrounded by heavy industry.)
