If you are using your GG for a supermedic, its usually best not to upgrade it at all, so that it will never get called upon to defend your stack. And if you aren't using it to make a supermedic, I find it almost always best to settle it in a city that's making lots of units. But there are rare situations where making a GG offensive unit is worthwhile, too, and then he should be upgraded... but not to a machine gun, wait for infantry.

Note, this "free upgrade" property of GG means that its usually better to attach him to an obsolete unit than a state-of-art one.