What are the best buildings/units to build first after creating a new city near the outermost borders, especially if there are other civs nearby? How do you balance between military units for defense, and buildings that would allow the city border to expand to get nearby resources.

If they're exposed to attacks and you don't have an overwhelming defending force, walls and barracks. If they're stuck in foreign cultural borders, theaters and libraries. In any case, courthouses soon.

Granary first, then start whipping whatever else you need. If you need a source of culture, whip a library or theater (or maybe an obelisk). If you need to start bulding a military, whip a barracks. I don't think I'd ever recommend walls, for the same reason I don't recommend longbows. They encourage you to sit on your ass instead of going out and conquering the world.

First i whip a granary, then i whip a forge, then i whip whatever else i need, using the granary and forge bonuses for the whip. If under organized religion, i might wait a bit for the forge and send a missionary there to get the same bonus.

Presuming I've got the techs for it, I'll whip one of a granary (faster whipping) and theater (border expansion) first, and after that a forge and courthouse. If I expect it to be fought over a lot I might whip walls, but generally not. Basically, I try to get the city's production off the ground (granary+forge, side benefit to health/happy), borders expanding (theater), and costs lowered (courthouse).

Under normal circumstances (and outside of the very early game), I won't be creating any units in the newly captured city, any units will be built by well-developed cities pouring out 6xp or 10xp units. An archer/longbow plus spear/pikeman can easily march to my new conquest, buildings can't.

Granary is virtually always first, unless you need a culture building to get your borders to include an important resource. Doesn't make any difference whether it's near or far: if you're going to defend it, it's going to be with units produced elsewhere.

I normally build a granary followed by a courthouse

I normally build a granary followed by a courthouse

Same here, except if it's on a hostile front, then get walls and culture up first, then granary and a courthouse, preferrably whipping the granary.