I usually build a defensive unit; it can defend if needed, but more likely will be used to explore.
However... (in ToT) If I have the option of building a skald (diplomat), that's the first unit I will build -- it has a movement of 2, can bribe/conduct diplomacy, and if I should stumble upon the Gate of Hel early on, the skald is the unit I want exploring first of all (since once you get over 287 gp you can bribe the Dwarf). Also, a skald is essential as early as possible for bribing a Buteo unit, to explore the Cloud World.
I just started a new ToT game, playing the Goblins. I began with 2 settlers on the Surface World and 1 Underground. So I have 1 Surface city, with no option for building a skald. I can build a Goblin defense unit (warrior), a Miner (settler), a Sorceror, Barracks, Temple, or Sacred Menhir (a Wonder). Since the food situation is not good yet, I'm opting for a Sorceror, as production can always be changed later. Depending on the map, my second settler will explore a bit and hope to find a friendly crag wolf unit. The wolf can explore/defend, and the settler can found another city. The Underground settler will build a city ASAP that it finds a source of water.
For improvements, I always start with a library and push my tech rate.
The first Wonder I will build as soon as I get the requisite tech will be The Crown's Emissary, so I can communicate with the other civs and trade tech and maps, and demand bribes.
Note that this is for ToT, Chieftain level, miniscule Barbarian activity in the early stages.