In single player, there is really no downside to accepting delegates and inviting civs over when you meet them. You can also send some, which gives you more diplomatic visibility in the early game, but this costs you money. I usually accept all delegations and send some back, except to the first civ I meet, because it usually declares war soon anyway.
In multiplayer things can be a bit different. If you meet a civ a bit away from your capital, I don't invite them over, because I don't want to reveal my capital to them. And I don't send delegates in multiplayer at all - friendships and alliances, sure, but no delegates.