Diplomacy AI Development

[Cokolwiek]

Predictability vs Erratic is when the AI is making suboptimal decisions (either because of performance constrains, or role-play constrains). If those are too predictable, they become exploitable. If those are too erratic, they break the player immersion / trust that the AI will behave rationally.

I am not familiar with terminology, I was applying it to whole AI foriegn policy.
Can you elaborate or give an example of predictable versus erratic diplomatic situation?

 

[Moi Magnus]

I am not familiar with terminology, I was applying it to whole AI foriegn policy.
Can you elaborate or give an example of predictable versus erratic diplomatic situation?

I will not take examples in the actual state of VP diplomacy, because I don't know enough of the current code, but here is an hypothetical example:

Assume two situations for an AI, identical (up to the AI knowledge) in everything but a declaration of friendship with the player. The AI is currently weighting if it should declare war to the player. The friendship declaration was not a fake one made in preparation of a backstab. How should he consider friendship? On one side, friendship is something valuable (and backstab give diplomatic penalties), on the other side, backstab give some tactical advantage from surprise.
+ If you put a weight way too high to friendship, the AI will be too predictable, as having a declaration of friendship might just immune you to wars with that civ.
+ However, if some AI personality make them MORE likely to declare you war to friends than to non-friend (cf Napoleon in BNW), then it is too erratic because the player will just remain puzzled at the situation. [You know, because having a declaration of friendship is supposed to be something good to have...]

 

[Recursive]

Current game: Emperor/Epic/Standard size map. Songhai going Domination.

Talking about DoW I think the first thing to be addressed is how AIs evaluate human military power.

In this game:

_ started on center of my continent. Coastal start with Pearls (ok, this one is BIG, cannot pass =)). Korea my closest nightbour South, India in the East and Denmark to the North. After 80 turns or so Denmark and Korea as friends DOW me. It was the right decision: i was expanding quite fast with a lead on cities. Bad execution though. Korea tried to attack a walled city behind a mountain with 4 units losing them all and basically ending their attack there, Denmark went towards my recent unwalled founded city but with the bulk of my army (Warriors, Cbows) and just lost. They had barely the numbers but had massively underestimated both my tech level and my defense placement. In the end they feed me science and culture and money through a favourable peace treaty.

_ later in medieval: vassalized Korea as catapults came into the fight, Denmark stalled me thank his Berserkers in the North that did very well vs my Mandekalu cavalry. I had to fight three wars vs Bluetooth and finally got him deploying cannons and caravels.

_ India DoF me since ancient, Smart decision but his capital was really exposed. Vassalized him without deploying the fleet just a massive artillery barrage from 5 field cannons. Delhi went down in 4 turns.

Other continent something strange:

_ Zulu was the runaway vassalized Portugal, Netherland but left Egypt alone ??? DoW India same time as I did. Again good strategy decision but poorly executed: India managed to defend from his fleet easily then capitulated to me and war over for them.

_ Zulu liberated Netherland with 1 city left but did no DoW to finish the job. Strange. He kept Portugal as vassal.

_ Egypt was left alone with 5 cities (3 in mainland and 2 on isles) with lilke 14 Wonders and still wonder spamming and this was before I accepted a DP with them (smart). Why ? Well in 2 turns my whole fleet (28 Corvettes, 2 Ironclads, 8 Frigates) and my recently embarked eastern army will storm Egypt and Zulu managed to stay just helplessly dead in the mid of his ocean rift.

To be honest so far AIs played fairly well but need help to finalize his plan decision wise. Zulu should vassalized Egypt taking all wonders and giving me a bad time but I think he was frozen by my military might and still in the scoreboard his military is better than mine. Why ? Maybe he produced 70 Impis waiting for my invasion.

I'm working on this.

@Moi Magnus proposed an excellent idea where all players are given a military rating that goes up or down when military units are killed, based on your kill/death ratio, the strength of the units, and the military rating of the target. The AI could compare their military rating to other players' and adjust their desire for war based on this. It could also play into other functions, like an AI's choices for friends or Defensive Pacts.

I accept that an AI that is offering my a luxury in its turn may not have it for trading next turn if it manages to trade it away to other civs. This won't be an issue.
What drives me nuts is having to decide which side I must join before I can carefully look at my neighbours stats. Right now, I am forced to involve myself in a war or risk to be rude to someone, blind eyed. I'm only asking for that AI to wait just to the very next turn for a response. Exploitable? In the sense that the unprepared player would commit fewer mistakes, yes, but I can't see other drawbacks.

I feel this too, very much. I think what Recursive is saying though is not that such a thing would be inherently exploitable but that it would be difficult to implement in a way that isn't exploitable.

Ideally for me I would be able to alt-tab out of the pop-up or something so I could check the situation and then give a response straight away. I don't know if that's even possible though.

Delayed trade offers are potentially exploitable (for instance, wait for a DP offer, denounce that AI, then accept to dissuade them from retaliating), not really the case for coop war requests - a few potential issues come to mind, but nothing unresolvable. I'll implement an option to wait one turn and assess the situation at some point.

You have a point. So these reminders should be not too invasive. I'm talking about diplomatic options that an experienced diplomatic player know and use, but inexperienced players miss. For example, we all know that a civ that wants to be friends with us comes offering a DoF. What they don't do is asking that we Denounce their enemies, and that's a very powerful diplomatic tool. When an ambassor knows of an intrigue, new diplomatic options open, but unaware players may miss the chance to share the intrigue.
Perhaps they could come and visit just to say that they'll love that the player share the religion with them, had they not founded theirs. Maybe (only maybe) a very friendly civ that opens a policy tree before the player and wants to stay friends could come and suggest the player that he picks the same policy, that the civ would be very pleased if the player chose the same.

I wish Civ had a more useful advisor system, because this is actually the kind of thing you could use advisors for if they actually gave good advice.

Well, that's another way to achieve the same, but I fear that I would forget more often than not to unlock the button. I think it would be better that the AI gets angry just when the player uses the diplomatic action with the diplomat or the great diplomat on the city state in case, after being warned. If I am not actively trying to make an ally, AI should not be angry. AI could be concerned and could request that we make no further advancement, but fulfilling quests should not be considered as such.

I'll poll the community on this.

I'll try to be more specific. You know that giving gifts to major civs increase their opinion, but we can't say how much gold we should give to them. If I am giving to them some gold and resources, I would like to see how much this would influence their opinion before making the offer. If I am going to share an intrigue, or insult them or denounce them, how much would this influence the opinion? I understand that a few modificators remain hidden so the AI can act mischievously, and I don't really need to see them all unless I play with Transparent Diplomacy, but at least I would like to know in advance how much my actions will piss them off.

Perhaps there could be a general statement somewhere on the deal window, like "The leader will be very grateful to you for this offer" or "This offer will not change the leader's view of you". Or the Opinion number change from making the deal could be displayed, although that's potentially immersion-breaking.

Sharing intrigue is a slight buff to Opinion, I could note that in the description.

I honestly don't think you need an exact number to know that insulting or denouncing the AI is going to make them very angry with you.

I am talking about the Coalitions system in Europa Universalis 4. There, you raise a flag against another nation, and some other nations may join your side, depending on their opinion and their interests. You can even trade with some of those nations so they enter the coalition for a reward. Once the coalition stands a chance, the leader of the coalition is able to declare war. In that moment, all nations that are inside the coalition must declare war too or suffer a big reputation hit. This is the only way a few minor center european countries can stand a chance against a huge empire like the Ottomans, France or Russia.

We could adapt this system for Civ5. See how we can mostly ignore the lesser civs that have been beaten by some big contender? How would it be if they could join a coalition and work together towards some goals? Right now they can only use a defense pact, but that's self defense. I don't fear a vanquished civ that is only strong at defense. But 3-4 civs ganging up against the biggest empire with the goal of reconquering some of their former cities that would be something.

Anyways, if you think coalitions are too hard to implement, then drop it. The current method is not that bad.

I'll keep that idea filed under "someday/maybe". It sounds potentially fun, but complicated to handle, and my focus is on improving the existing mechanics first.

I just want to say that diplomacy is in fact the best developed part of the vp, and I do not even try to comprehend what enormous amount fo work it taken to get vp to this point. Do not overwork yourself, because diplomacy is the most realistic and fun to do phase of the game right now. Especially do not give in to people requesting favorable treatment or requesting an AI that do not know where its interest lay. I feel that pushing anything right now could result in instability.

I'm glad you enjoy the diplomacy aspect of the game. I'm not programming the AI for easy mode diplomacy; disabling victory competition is an option for a reason.

I disagree on this. There could be no unpredictable AI eithout being erratic. If the AI will behave according to its own best interest, it will be predictable for every player who is not half-witted. You most of the time know their capital, their cities, tech, policy count, army potential, their neighbours, their luxes, they strategics, their possible victory type they aim for, who the fear, who they can cooperate with, who is their next target, who is next targeting them etc. you cannot not make clear assement of how they would act to secure thie interests. I disagree though that predictability entails exploitability or passivness. You should have enough competitiors, and if you choose not-friendly attitude for them, they should turn neutral and deny you opportunities you would gain by cooperating with them.
Contrary I think deceitful AI should ask you if you would be a luxury several turns when it is already moving its troops against you , agree to deal, even offer better prices. It would be no erratic, but consistent lulling a player into its plan.

Predictable VS erratic: the AI makes approach calculations every turn to decide how to behave towards other civilizations, and adjusts these values gradually every turn if they change. If the AI considered only a few factors in these decisions, then it would be far too predictable and players would exploit it mercilessly. For instance, many players know that in vanilla BNW, you can sell your excess luxuries for absurdly large amounts of Gold (7 GPT if you have neutral relations), and take advantage of their overvaluation.

If the AI changed its approach every single time conditions changed, rapidly fluctuating from NEUTRAL to FRIENDLY to HOSTILE to WAR, etcetera, then it would behave too erratically, be impossible to comprehend, and it would never get anything done either.

"Predictable" and "erratic" are not necessarily opposites, but my point is more that neither of them is desirable at an extreme level.

A well-designed diplomacy AI behaves in a way that players can observe, understand and respond to (predictability), but that is also able to change, adapt and surprise them (erraticness).

For this reason, the AI chooses its approach using a large set of factors and adjusts its approach gradually on a curve, so sudden changes in the situation don't upset it - although in some cases, sudden changes in approach are justified. Credit goes to Gazebo and co. for the approach curve and generally making diplomacy a lot better compared to BNW prior to my arrival on the scene.

Footnote: how would you feel if declaration of friendship be changed into something more appriopriate? Like declaration of strategic coopeeration or just alliance. Beside, there is no friendship between states, nations, and corporations. There are only interests.

I think it's fine as Declaration of Friendship; it expresses the meaning just fine, it's been that way for many years and changing it now would cause more confusion than anything else.

And there are certainly friendships between nations, both historical and current. The world may be a dark and scary place these days, but there is a human element in our nations' governments.

 

[burleigh]

Would it be possible when you get the instant requests e.g. for a declaration of friendship (or a war) to get a shortened breakdown of their relationships with that request? For example, I’ve just accepted a declaration of friendship with Greece and been slagged off by other friends who were- unknown to me- Greece’s enemy. Similarly, I’ve gone to war on request with someone who had a defensive pact with a powerful neighbour.
Generally, the diplomacy is really fantastic- thanks! There seems to be less anti-human diplomatic dog piling in this beta (though I haven't hit the later stages yet, and it could just be down to me not doing as well this game, so I’m less of a threat

 

[Recursive]

Would it be possible when you get the instant requests e.g. for a declaration of friendship (or a war) to get a shortened breakdown of their relationships with that request? For example, I’ve just accepted a declaration of friendship with Greece and been slagged off by other friends who were- unknown to me- Greece’s enemy. Similarly, I’ve gone to war on request with someone who had a defensive pact with a powerful neighbour.
Generally, the diplomacy is really fantastic- thanks! There seems to be less anti-human diplomatic dog piling in this beta (though I haven't hit the later stages yet, and it could just be down to me not doing as well this game, so I’m less of a threat

If you play with EUI, the relationship breakdown is visible if you mouse over a leader's portrait in the sidebar or their visible approach.

 

[JamesNinelives]

@Moi Magnus proposed an excellent idea where all players are given a military rating that goes up or down when military units are killed, based on your kill/death ratio, the strength of the units, and the military rating of the target. The AI could compare their military rating to other players' and adjust their desire for war based on this. It could also play into other functions, like an AI's choices for friends or Defensive Pacts.

That's a cool idea! Maybe included cities gained/lost? Becuase sometimes it's worth sacrificing units to capture/defend a city .

I wish Civ had a more useful advisor system, because this is actually the kind of thing you could use advisors for if they actually gave good advice.

Yes. Imo they exist mostly for comedic effect at the moment lol. I like the science advisor who, on turn 1 goes: can found found a city already? Carrying all this equipment if very tiring.

 

[Recursive]

Upcoming change for next version:

Adjusted diplomacy AI evaluation of military strength
- Slight bump to AI valuation of DPs/vassals
- AI now considers teammates when evaluating military strength
- New military rating system affects diplo AI's perception of a player's military strength

Military Rating
- Each major civilization gets a military rating which starts at 0 (cannot go below 0)
- Losing units/cities/trade routes and having your tiles pillaged reduces your military rating
- Killing units, plundering trade routes, capturing cities, and pillaging tiles increases your military rating
- Barbarians don't affect the rating
- Rating does not decay over time, but changes in later eras matter more than in earlier eras
- When considering a player's military strength, AI will calculate the percentage difference between that player's rating and the global average rating of other (living) players
   Above average: the % difference will be added to your military strength (max +100% bonus)
   Below average: half of the % difference will be subtracted from your military strength (max -50% penalty)
- This means that if you're successfully fending off waves after waves of AI units or conquering other civilizations, all AIs will now recognize your military skill and be more reluctant to attack you,
even if you have a smaller army
- Military rating modifier is also applied to teammates, DPs, masters/vassals (which are considered to be a portion of your own military strength by the diplo AI)
- Should increase the odds of human players obtaining DPs, as well

This change should help teach the AI that skilled humans and players with a lot of military advantages are not to be trifled with lightly, although it won't prevent all wars from occurring, and humans will not be able to exploit the system by, say, conquering someone and then disbanding all their military units.

I'm not skilled at UI work so the military rating will not currently be visible to the player, but I'll work on getting it displayed if the system works.

Change is experimental, let me know how the AI behaves in the next beta in the feedback thread: https://forums.civfanatics.com/threads/vox-populi-diplomacy-feedback.652134/

EDIT: Additional adjustment

Removed a few AI-only diplo mods, should hopefully result in humans receiving more DoFs/DPs

 

[Guynemer]

That sounds very interesting, and just reading the concept it seems to make sense. Excited to see it in practice.

 

[andersw]

A very ambitious list, I wish you all the good luck.
Diplo in this game is NOT my strongest part and I fear I will have to adjust.
Think the Ai gets a bit tetchy? then you havent seen me getting surprise ganked or beaten on a wonder by one turn.

 

[CrazyG]

They believe you are expanding your empire too aggressively

Spoiler Screenshot :

This modifier is really annoying me. I'm playing progress with 6 cities. I have good land to put 2 more down, but I wanted to try and have the AI on my side at least for a while, and they get so mad if you have more cities. Egypt seems upset about Aachen, but I settled that city on turn 90. I gave him plenty of time to put a settler there, he didn't.

This is actually a reload, my first playthrough I dropped two more settlers and got denounced and warred by everyone instantly (Egypt was so upset he backstabbed me). I'm not expanding too fast, its well within reason and happiness constraints. I think they are all upset that I have 2 more cities than they do. You know what they could do to address that? Build settlers. It's turn 100, most of them have 100% happiness, there's plenty of land. I don't see what's stopping them.

 

[kenneth1221]

They believe you are expanding your empire too aggressively

Spoiler Screenshot :

View attachment 546465

This modifier is really annoying me. I'm playing progress with 6 cities. I have good land to put 2 more down, but I wanted to try and have the AI on my side at least for a while, and they get so mad if you have more cities. Egypt seems upset about Aachen, but I settled that city on turn 90. I gave him plenty of time to put a settler there, he didn't.

This is actually a reload, my first playthrough I dropped two more settlers and got denounced and warred by everyone instantly (Egypt was so upset he backstabbed me). I'm not expanding too fast, its well within reason and happiness constraints. I think they are all upset that I have 2 more cities than they do. You know what they could do to address that? Build settlers. It's turn 100, most of them have 100% happiness, there's plenty of land. I don't see what's stopping them.

Dumb question, but where's your army? Your empire looks a tad fragile.

 

[Kim Dong Un]

I'm really intrigued to see this new system in action, but I don't necessarily know if it's a good idea to have trade routes be factored into the equation. A lot of times there's no way to protect multiple routes from being pillaged in a single turn, the first turn after a DoW, and I fear this could skew numbers; there are many instances in late game where you can lose basically all of your routes due to the AI's victory block kicking in when playing for cultural victory. At the same time, having the ability and prowess to sustain protection and ensure your routes travel safely throughout the course of a game should serve as an indication of strength toward the other AI, so I guess it makes sense.

Only one way to find out!

 

[Recursive]

They believe you are expanding your empire too aggressively

Spoiler Screenshot :

View attachment 546465

This modifier is really annoying me. I'm playing progress with 6 cities. I have good land to put 2 more down, but I wanted to try and have the AI on my side at least for a while, and they get so mad if you have more cities. Egypt seems upset about Aachen, but I settled that city on turn 90. I gave him plenty of time to put a settler there, he didn't.

This is actually a reload, my first playthrough I dropped two more settlers and got denounced and warred by everyone instantly (Egypt was so upset he backstabbed me). I'm not expanding too fast, its well within reason and happiness constraints. I think they are all upset that I have 2 more cities than they do. You know what they could do to address that? Build settlers. It's turn 100, most of them have 100% happiness, there's plenty of land. I don't see what's stopping them.

I looked at the code for this and facepalmed.

There's a < sign where there's supposed to be a > sign.

THAT explains a lot. Will fix.

 

[CrazyG]

Dumb question, but where's your army? Your empire looks a tad fragile.

All I have is 3 chariots and a pathfinder, but if someone wars me I'll immediately swap to building units everywhere. Where would I get attacked though? Egypt gave me friendship. Polynesia is far away. Aztec and I fought a war already, the city states bear his attacks. I'm confident I'll have time to react while the chariots stall.

 

[Recursive]

I looked at the code for this and facepalmed.

There's a < sign where there's supposed to be a > sign.

THAT explains a lot. Will fix.

Wait, never mind, the calculation is correct. It probably has to do with the AI's slower settling speed.

I'll adjust it so you need 80% more than the global average # of cities (up from 50%) to get the modifier, and see if that helps. There have been a number of reports about this modifier.

Also, the AI will likely be expanding more next version. If their expansion speed increases I'll tweak it back down, but this should be fine.

 

[JamesNinelives]

All I have is 3 chariots and a pathfinder, but if someone wars me I'll immediately swap to building units everywhere. Where would I get attacked though? Egypt gave me friendship. Polynesia is far away. Aztec and I fought a war already, the city states bear his attacks. I'm confident I'll have time to react while the chariots stall.

The point though is that if you have lots of cities and few military the AI is supposed to see that as an opportunity to attack you. At least, that's how I understand it.

Honestly, I think the modifier is doing what it's intended to do - punish reckless expansion. Sure, you can defend that land if you have to, but if the AI has the same attitude you would never have been able to found so many cities. And if the AI left their cities so sparsely defended surely the human player would see that as an opportunity to attack.

 

[Recursive]

The point though is that if you have lots of cities and few military the AI is supposed to see that as an opportunity to attack you. At least, that's how I understand it.

Honestly, I think the modifier is doing what it's intended to do - punish reckless expansion. Sure, you can defend that land if you have to, but if the AI has the same attitude you would never have been able to found so many cities. And if the AI left their cities so sparsely defended surely the human player would see that as an opportunity to attack.

I made reckless expansion a worse modifier and AI is expanding slower, so I think it makes some strategic sense to bump up the % threshold a bit. I'll tweak it again next version if needed.

 

[CrazyG]

The point though is that if you have lots of cities and few military the AI is supposed to see that as an opportunity to attack you. At least, that's how I understand it.

Your understanding is off. You get the modifier regardless of how strong your military is, it is based solely on the number of cities. I can edit in some giant death robots and I still get the penalty.

Honestly, I think the modifier is doing what it's intended to do - punish reckless expansion. Sure, you can defend that land if you have to, but if the AI has the same attitude you would never have been able to found so many cities. And if the AI left their cities so sparsely defended surely the human player would see that as an opportunity to attack.

I don't think my expansion was "reckless" at all. Do you think 6 cities by turn 105 is too many? At the rate the AI goes the continent won't even be filled by Renaissance.

 

[Recursive]

Your understanding is off. You get the modifier regardless of how strong your military is, it is based solely on the number of cities. I can edit in some giant death robots and I still get the penalty.

I don't think my expansion was "reckless" at all. Do you think 6 cities by turn 105 is too many? At the rate the AI goes the continent won't even be filled by Renaissance.

No, military strength is factored in.

    // If this guy's military is as big as ours, then it probably means he's just stronger than us
   if (ePlayer != GetPlayer()->GetID() && GetPlayerMilitaryStrengthComparedToUs(ePlayer) >= STRENGTH_AVERAGE)
       return false;

 

[CrazyG]

No, military strength is factored in.

    // If this guy's military is as big as ours, then it probably means he's just stronger than us
   if (ePlayer != GetPlayer()->GetID() && GetPlayerMilitaryStrengthComparedToUs(ePlayer) >= STRENGTH_AVERAGE)
       return false;

Correct me if I'm wrong.

It does affect AI's opinion of you.
But the "They believe you are expanding your empire too aggressively" is based solely on city number, no?