It is probably something to do with worker automation in the late game (usually with a large game).
Work around: I find that if I have one unit (maybe at the captial) that I don't end it's turn, always hit the wait button on that unit (I always have one unique looking unit, maybe a balloon), that never ends it's turn. I find that that eliminates any automated worker loop problems and when it I'm done my turn and it gets back to the balloon, I can just hit the next turn button.