The problem is we are so used to the govener settings not really effecting the workers builds, that we now forget that they do in a big way.
I "Rush grew" a city after a poison water attack, by putting farms everywhere I could, and moving a manual stack, with govener on FOOD. So Im watching this city every turn, and workers keep turning up to change all the mines to windmills. As soon as the city was a size I wanted, I turned FOOD off and guess what, no more autoworkers turned up.
Dont get me wrong, I would never put wokers on auto in the early game because we dont have enough contral over auto (No GO Build the Pastrue, then build the road on the way back, rather than the other way arround). However I think a lot of the problems are that we arent used to the new system yet.
As to forts, AHHHHHHHHHHHHHHHHHHHH, if Ive built a mine on something outside my city limets, the worker shouldnt build a Fort on it AT ALL, its just giving invaders really big defence bonuses when they run in and take those square first attack