...
I'm not claiming to come up with a grand solution to this, and I honestly don't know if one really exists. I'm just saying that CIV does a relatively good job at balancing some peoples wishes at having a game in which the AI competes to win (a GAME) while having a game in which the AI does a bit of roleplaying at the same time.
On that point I agree. I certainly don't recall having the same problem getting into CIV that I had getting into CiV. It's certainly better now but the crux of the problem is essentially the same as it was at CiV's release - nobody understands why the AI does as it does. Even if you take apart the XML and take paper notes the whole game (which to my shame I've done) you'd be hard-pressed to understand the AI's foreign policy. If there's a single more immersion-breaking and/or frustrating feature in CiV other than the erratic/unclear diplomatic system I'd be very surprised.
Just a minor clarification but I assume when you say 'to play its role as near as possible to its historical context' you mean conduct its diplomacy with respect to its ingame situation adjusted for flavour rather than anything more prescriptive, like England and France hate each other until the late game, then become friends.