Wasnt there an option in Civ 3 to add dormant workers to a Cities Pop? That would be useful in Civ 4.
Anyways, when i have idle workers, i tend to group them into two's and place them between cities ready for Railroad, so that i can connect all cities as quickly as possible once i have discovered the tech. Make mobilizing and form of attack or defence a lot easier. With workers grouped into two's (and the right civis) they will build a railroad in 1 turn, allowing super speedy railroad networking.
If i really have nothing for them to do, i'll "sleep" the remaining workers somewhere safe until i need them. They quickly become important once you conquer a city or two, once peace is made get them there and improve the tiles.