I really like the implementation of policies and the effects they have on the game (although I do play with mods which tweak the policy system even further.) Basically, it allows for varied and specialized games despite unique civilization flavor. Even civilizations as clearly streamlined towards war, such as Mongolia, can do variations such as war/culture or war/commerce due to the policy trees. I really hope they do a better variant of policies in VI.
As for what I really don't like? There are several contenders, and all of them have been mentioned here. Espionage really annoys me because it doesn't do some things which should be really obviously possible, such as more detailed information gathering, possible sabotage (as annoying as it can be to deal with when it's directed at you, the sort of disruption spies can cause in Civ IV makes sense. Albeit, it would have to be drastically toned down for CiV, where costs are higher.) The things that it does seem very underwhelming, especially given the limited amount of spies you can have (though intrigue is great.)
Diplomacy seems a bit weak in some areas. It's a bit annoying that prices don't scale with income. When civs are making 100+ GPT, having the same 7 GPT/luxury default seems really petty. It would be nice to have prices that scaled at least a little bit. Even 20-25 GPT seems better. Also, I don't like that other civilizations can threaten and have permanent repercussions for things like missionaries being used to convert their cities, but don't treat any such requests from you with any sort of relevance. I've NEVER seen a drop in either missionary or espionage activity, even if the civ acquiesced to your request. It would be nice to have several options, ranging from a polite request to 'I will make you wish I never sat down to play this game if I see one more goddamn missionary, Haile.'
The AI's behavior seems downright irrational in some places. I know that's less a game feature than a coding feature, but still, I've had some instances where, say, Alexander would be brought back to life by me, and promptly denounce his two closest neighbours, Monty and Genghis. Any human player would realize that that is a VERY BAD IDEA, but the AI never seems to. Surely a few simple value analyses could solve that? IE, AI becomes much less willing to antagonize someone with 10X+ their military prowess? Just in general, it would be nice to have a more reactive AI. An AI that has no iron and is locked in a war should be willing to pay more for it than an AI with plenty of their own, at peace. For that matter, the AI shouldn't want to buy resources far in excess of what they need. So if Catherine has 20+ iron she's not using, why would she buy any of mine?
To end with something I like? The variety the game provides. I know there are some flaws with that, and a lot of the early game is neglected, as well as much of the late game being somewhat irrelevant, but even from the parts of the game that matter most, there's so much customization and variety. Most of this is done through policies, but there's enough other things out there, such as city states and the tourism system, that allow you to make your games differ (except of course the overwhelming importantce of science. That's something that's annoying from a gameplay perspective, even if it is kind of true historically. It just seems a bit too much.) There are enough different civilizations and flavors that you need a lot of hours in the game to even scratch at the surface of everything you can do.