I think the problem here isn't that AI's don't use specialists correctly; it's that wide is just a much better strategy for the AI. I'm a 4-city Tradition player myself, but when happiness is no object (as with high-difficulty AI's), wide will beat tall almost every time. Better specialist use would definitely help the AI in some cases, but since AI's can support specialists even when going wide, tall empires would still get left in the dust (just slightly less),.
Exactly. If you have no happiness issue and a substantial culture discount, settling anywhere any everywhere makes perfect sense.