According to several posts in the multiplayer forum, I am not alone with my problem of "connecting to peer..." forever.
It seems to be a general civ4 problem, apparently worsened by the last patch, and it has nothing to do with firewall configuration. Or, to be more exact, if the firewall is configured correctly, the problem still exists for quite a lot of users.
At least it doesn't seem like it's a RFC-related problem.
edit: Also, fyi, it seems that the problem rarely is on the side of the user encountering those problems, but rather caused by incorret firewall/router configuration of other users (port 2056
MUST be forwarded/opened
for every player in order to cause no problems
for other users). Unfortunately, I can't give you any guarantee on that due to lack of testing