I personally think that Civ4 even with all of it's flaws and deficiencies is by far the best game in the series. It has the best graphics, the best game play, and the best AI. It is better as a whole than any of the civ games (including SMAC) that came before it even though some of those games have better parts.
Civ3:
This game seriously lacked any strategic depth. In vanilla civ3, the gap between the strongest units and the weakest units was fairly small. Which when you coupled that with the default 3hp system it resulted in unpredictible combat outcomes. Civ3 had a rather high rate of spearmen defeating tanks, amongst other nonintuitive combat outcomes. Combat in general wasn't that strategic. Contrast this to Civ4's combat system which is mostly an improvement. I miss armies, but the Civ3 implementation of armies wasn't very good. I also loath the jump points in the Civ4 combat model, and how after a unit takes some damage it becomes completely fragile. However, outcomes for the most part are close to what they should be, and the promotion system makes the game more strategic, more flexible, less predictible, and more interesting. Though I think firaxis could upgrade the promotion system quite a bit. Also bombard units are a weak point in Civ4. Instead of going back to the Civ3 system I'd like to see something better. As for armies, the civ3 system isn't very good, but the fleet/logistic system from Galciv2 would be a great addition.
As far as buildings goes, Civ3 didn't have that many, and it there wasn't much strategic choice in buildings anyway, because all of the cultural buildings also just happened to be the science or happiness buildings. Plus it was quite easy to run out of things to build in vanilla civ3.
Specialists in civ3 also presented few strategic choices because in vanilla civ3 at least, specalists were underpowdered and usually only good in very special situations. Compare that to civ4 where proper specialist use can open up some new strategies.
Tech trees also limited the strategic choices in the game because of the tech eras. This prevented beelining to some extent, and made most games play out the same way. Civ4's tech system is far superior (though it's surely possible to create a better [more fun/more strategic options] tech tree than what civ4 currently has using the same civ4 and/or tech rules).
That's just a few of the shortcomings I can think of in Civ3, but granted I haven't played it in years. Those were just the most egregious examples of civ3's lack of strategic depth. Though I do miss civ3's colony system, and wished that civ4 had kept it. I think that airunits in civ3 might be slightly better implemented than in civ4, but from the newest reviews it seems like BtS may fix civ4's air unit system. I also prefer the civ3 bombard units (except for the automatic capture feature which is fairly ********) to civ4's suicide cats, though I'd like a better system than either of those.
Civ2:
It has worse graphics (speaking absolutely here not relative to the time), worse AI, and fewer ways to play the game. Most games are exactly the same. ICS, howitzer blitzes, and other strategies are without a question the BEST way to play. However it does have civil wars (unless you play modded civ4), a better space race (till BtS comes out), partisans from conquest, caravans, and arguably a better or easier set of scenario creation tools. Though certainly for modding teams that have the proper skills Civ4's mod tools give them more chances to alter gameplay.