Few things (and what is all you need for "Direct IP connection" setup):
1) Civ 4 game needs only port 2056 (UDP). All the other ports are not
necessary. If you do not need GameSpy stuff or built-in VoIP you should be fine with only UDP 2056 port.
2) You only need to open port if it is really blocked

. By this I mean that if you are not hosting the game (just connecting to the server out there), you probably do not have ports blocked. Most router firewalls automatically allow (by default) all outgoing connections from local network, they block only incoming connections - and I mean really connections here (not packets).
3) I did not test in-game VoIP, but the UDP 3784 port is (coincidentally?) used by Ventrilo (
http://www.ventrilo.com/). I do not know, whether this is because Civ4 uses internally Ventrilo technology, or they simply found this port convenient, but if you plan to use Ventrilo, you might encounter problems when mixing both.
Facts 1) and 2) were tested in situation where I hosted "Direct IP" game (have router configured) and two friends were connected to my system. They also use routers (with NAT and firewalls), but they did not need to configure anything at all, because theirs routers handle this kind of traffic by default.
Note: When configuring home router/firewall(s) you might need to configure firewall (adding the proper rule to let the incoming traffic in)
and router (i.e. configuring NAT or NAPT so the incoming traffic is routed to the right machine on local net).