Thumb-twiddling is no reason to avoid grouping workers. Let's say you've got a group of three workers that you assign to do something which takes 5 worker-turns. On the second turn, when the job is finished, that thumb-twiddling worker still has his movement points available. Just click on the group, then select that worker to have him leave the group and start something else. I do this all the time, especially once railroads come around and when I'm improving captured cities with worker-gangs. 4-turn mines, 5-turn farms, 3-turn railroads, etc., mean I want some flexibility in my worker-gangs.
On decay, there is none, or if there is, it's insignificant. I just ran a WB test where, on turn 0, I put 4 turns into a 7-turn floodplains watermill (all techs known from the start). As expected, my fifth worker, who did nothing, showed 3 turns left when I moused over the watermill button. I then did nothing but cycle through the turns, checking to see how many turns were left on the watermill every five turns. At turn 100, it was still 3 turns. That satisfies me that it's nor a problem with Airey's technique.
To clarify what Airey's talking about, imagine the following situation. you have a worker two flat tiles away from an unimproved resource tile on a river. The river will bring the resource into your cities. Instead of moving over both tiles on turn X, and starting to improve the resource on turn X+1, or roading the in-between tile fully, then starting the improvement on turn X+2, you move to the tile between the worker and resource on turn X and lay down one turn of road. On turn X+1, you then move into the resource tile and begin improving it. On turn X+2, if you want, you go back and finish the road, or you just wait until you either really need the road or you have a similar situation that allows you to finish it, essentially at no cost. Compared to finishing the road first, you've gotten the resource a turn earlier, making you a little bit better off for not having finished the road once you started it. Compared to moving straight to the resource, you're a little bit better off for not having wasted a worker turn.
Back to the general OP question: I tend to prefer gangs of workers, but for somewhat different reasons in the early and late game. Early on, I want my workers to be just ahead of my cities, so that there are one or two improved tiles that a city can grow into. This is easiest with grouped workers, since they can finish improvement A, then work on improvement B while the city is using, or growing into, A. Using lone workers, I run a greater risk of the city working an unimproved tile while two workers finish side-by-side improvements. Later in the game, I have a much larger empire, and many more workers, and I want them concentrated on high-priority tasks, like getting the railnet extended to my military frontiers, or getting that new city's hinterlands developed so it can just grow without me keeping workers tied to it. It also helps to only have to direct, for example, 1/3 of my workers (if they're in groups of 3), from a turnplay standpoint. I'll use the opposite approach for less important projects for the last reason, too, and direct a lone worker to build a route to that near-useless tundra city, knowing he'll be out of my hair for several turns.