I think the key is who initiates the deal, or at least what position each person is bargaining from.
On my current game (Regent level) I'm in the process of beating down the Indian empire. I tried to initiate a peace treaty (with no extras thrown in), but my advisor said it would be insulting

, so I backed off. A few turns later, with nothing really significant changing, the Indians approached me and asked for a peace treaty, and I managed to get 3 techs (really significant ones, too) with the deal. Something similar to this has happened multiple times.
It seems the AI is insanely unfair whenever you come asking for something. But if they ask you for something, they expect the same kind of unfair treatment, so you can get away with murder.
Though the whole thing reeks of Civ 1, where opposing civs would demand 1000's of gold (even though you had <100) to refrain from attacking you...
One good thing about the AI's love affair with cities on the bargaining table is that you can trade one of your cities for a helluva lot of goodies. In a recent game I gained this pathetic size-1 city (through cultural expansion) the AI had built in the middle of a desert. First chance I got, I traded it to another civ for 3 techs, gold, and their map!

And of course, it's only a matter of time before I get the stupid thing back through cultural expansion again...
