I have a habit and a preference for defensible borders, but sometimes an ai unit slips through before I can place my cities up to that limit. Usually Im building from a core outward, with the vast majority of my early empire placed as a perfectionist with no gaps. The problem with this is allowing the other civs to get too large during their otherwise weak years generating serious war in the late mid game as much of the world is filled.
Depending upon the stage of the game (and the civ in question) I may let the city go on for a while. If it is early in the game, save a few $$ and buy the intruder (my preferred alternative). Usually the city is small (size 1 to 3) and usually were talking governments Republic or earlier.
For a mid to late game situation, and/or where the civ in question is an ally or in Democracy, you can treat the city like an infection -- build a fort two spaces away (on the diagonal outside of its radius for a few military troops), build nearby cities (soon, not necessarily ASAP) and make sure that your citizens work as many of the overlapping spaces as possible, thus keeping the size of this infection small. Also, if the foreign civ ever opts out of Democracy, buy this thorn.
But in your situation (2356? my Civ II games seem to end at 2020 or sooner), you may have to make a choice -- if youre leaving for Alpha Centauri, leave the cities (for now) & focus on space -- once you launch, war may soon occur & your focus can be adjusted. On the other hand, if youre thinking of conquest, spend a few turns, build up some more units, place them where theyll do the most good & have at it.
My preference? Either way, Id buy the sites at the first opportunity.