Towards the end of the game, when I've built all the important improvements in all of my important cities, I sometimes automate a few workers. I set to them leave old improvements and not chop forests, but that isn't enough. I sometimes feel like they go out of their way to stuff things up.

Nar, they aren't that bad, but they do annoy me sometimes. The main thing is that they don't seem to understand irrigation chains. I have to build all the farms manually because otherwise they start building cottages and factories on vital irrigation tiles that will supply food to cities that don't have fresh water. Another annoyance is when an improvement is destroyed somewhere, they often like to replace it with something else - usually I'd prefer to just rebuild what was destroyed.
Setting them to build trade routes is usually safe enough -- unless you're trying to grow your national park forest, then automated trade routes is a bad idea. I don't want roads (or anything else) on any non-forest tile in the BFC of my national park city, but the automated workers don't understand that. Also, if you're going to use automated workers, you really have to keep an eye on any jungle tiles for your national park - there's no "leave jungles" option.
Sometimes I imagine a kind of semi-automation system that I might like. My idea is that you don't touch the workers, but you choose which improvements you want on which tiles. So you select the tile itself and say "I want a farm here" or whatever, or "I want _nothing_ here", and when a nearby worker has nothing better to do it will build those improvements for you (or leave the tile alone if you want nothing). If you don't specify what you want, the workers would treat it much as they do now - ie. they basically choose what to do with the tile themselves.
So, if workers were automated in that kind of way, I'd use them. But as it is I only automate workers when it becomes tiresome finding little tasks for them to do after all the major work is done.