I think Civ3 handled it the best - they worked slower but had no upkeep. My problem is I had an entirely slave-based economy until the 21st Century. There needs to be some mechanism for abolition. Ironically, the removal of stacking of workers means the concept would actually work better in Civ5 because the slower workers are actually a penalty (in Civ3, I would have so many free workers that I would usually have stacks of ten). Still, if they were combined with a happiness penalty, perhaps, that increased over time, they might be worthwhile (maybe even make them work the same speed).