I tend to restart only for the following:
1) Occasionally, barbarian Axemen will come knocking at my door *before* it would be theoretically possible for me to discover Bronze Working, lay a road to Copper, and build an Axeman. To me, that's just silly, and I have none of it.
2) If there are absolutely NO strategic resources. Unfortunately, due to the game's AI demands, if you don't have a strategic resource, they demand pretty much 187 Gold per turn for it, which is tantamount to not getting it at all. So basically it's control or nothing, but the game's balance precludes you to be able to take over a strategic resource when you don't have any of them to begin with.
3) I'll let the "AI builds the Pyramid when I had one turn left to go" thing slide once, usually twice. But if it happens more than that, then I know the game is done for me.