The AI in that regard definitely needs to be better.
Also, Civ II had the best perception as far as real life treaties go. I remember being able to fairly easily get a city through negotiation.
During one game of Civ IV, Spain, with whom I thought I was having very good relations joined a surprise war against me, and seized a fairly important city for me early on. Since I shared the border with Spain, I concentrated my efforts on them. Since the other Civ (I believe Greeks) had great distances to fight me, I made them do all the work.
As the war went on, I reduced Spain to their capital, and one connected city, and the city that had seized from me--a good distance and disconnected from their main territory, which I wasn't ever able to recapture. I ended the war, but no matter what I threw in the hopper (including tens of thousands in gold or even hundreds per turn), no dice on the city.
I could've easily destroyed their entire Civ with a few more turns, but I wanted to move away from defense production. I figured I would surround the city with culture.
While Spain struggled to maintain the city and civil disorder constantly plagued the city reducing it's size to less than 5 for hundreds of years the city never turned, nor would Spain ever negotiate for it. Even all the resources for the city were in my territory.
I have had the same frustrations on "new worlds" where I'll have basically the entire landmass save for one small city that constantly has the same problems as the city above.
Civ II this was much more negotiable. Now the only way to take is a city is to draw blood.