Could be worse. The lag between turns doesn't bother me much, I'm a very slow player anyway. I just use this time to think about strategies. And I think the game has become noticeably faster compared to Warlords.
What bothered me in vanilla Civ4 was lag within your turn, when scrolling the map. But I've got 1 GB RAM now, and the graphics programming has become *much* more efficient. I still play with all settings maxed out and there's still no lag when scrolling.
However, I'm not sure whether I can play this game to the end, it's an experiment. Usually the game slows down noticeably when Paper gets discovered and the civs start to swap maps. Currently I'm at 800 A.D., so I wouldn't expect much lag at the moment anyway. Civ4 is only using 560 MB of memory despite the huge map and many civs at the moment.
However, whether any lag is bearable or not is highly subjective. As I said, I'm a slow player, so I wouldn't be bothered much. It also wouldn't bother me to lower the graphic detail if I had to. Other people's mileage may (and will) vary. Personally, I recommend to get hold of a modern-times savegame or scenario of a game similar to the one you'd like to set up, play a few turns there, and see whether it runs fast enough for you. It's hard to predict gamespeed because of the many factors involved, and it's nearly impossible to predict how it will *feel* for someone since that is highly subjective.