There are special commands that make your automated worker leave existing improvements alone. Andy you can also set them to only clean pollution.
In both cases, the human player can still manage the workers more efficiently, but if manually handling your workers is reducing the enjoyment of your game, then that may be a good trade off to you? who knows.
The most difficult thing perhaps is getting rid of the automations. At the start of each turn, the AI handles all automated workers, and if there is no task for a worker to do, the AI parks the worker in a city and hits the "skip turn" button for you. (Instead of fortify/sleep, what a human would do)
So if you wanted them to do something, you need to wait an other turn. That is, after you managed to track them down. (the ones that where automated)
At the later stages of the game I create stacks of workers and keep them in stacks of the right size to do a task in one turn. You can move stacks of the same units at once.
For example, I have a stack (or more) for railing flat land, a stack for railing hills and a stack for railing mountains. Same for pollution cleaning tasks.
Especially with rails up, it is no problem to move the mountain stack from one lonely mountain to an other lonely mountain.
And if they have nothing to do, I fortify/sleep them at some easy to find spot.