Diplomacy AI Development

[usadefcon1]

Just finished up a game as Rome and lost to Brazil via CV. Brazil would take turns bribing Japan, The Iroquois, and America to attack me. Very smart on Brazil's part, however kinda dumb on Oda, Hiawatha, and Washington. Brazil was the clear leader the entire 2nd half of the game. I was no threat to win, he was. At some point the AI has to prioritize taking down the leader, and right now that is happening too late.

Also I tried to get Attila to join me in attacking Brazil, but he just sat there the entire game with 4 cities. Never declared war. It's not like he was winning or anything, Brazil was. He should have been much more aggressive.

 

[Tonas1997]

Not sure this is intended (or otherwise impossible to program against) but I've had a recent war with the Zulu, who were at the time attacking a CS.

I entered the CS's territory and destroyed a bunch of the enemy's units, therefore saving their city... but I was getting trespassing penalties instead of being rewarded for attacking a common foe, just as it already happens with major civs.

This was particulary jarring because at a point the Zulu did manage to capture the CS. I wore down the city's defenses and was about to recapture it... and then the Persians rushed in with cavalry, captured the CS and returned it. So I ended up with a very angry CS that gave the Persians a ridiculous influence bonus (and allyship). I feel like that my previous war effort should have been rewarded instead of losing the diplomacy game to a civ that arrived much later just to collect the spoils.

TL;DR: "we fought against a common foe" should be implemented for city-states

 

[Stalker0]

So with Tech Trading on, the AI is way too considerate with techs as gifts. I will be 1 tech away from the leader, and the leader gifts me a tech. That's much too good, and techs are way too value to be given as gifts.

The only time I would consider that is for my vassals (to keep their military competent with mine), or with like the last place civ that I'm taking pity on and want good relationships with. I would never grant a tech to a civ that is near the top tech score. even if we are the best of fiends.

 

[Recursive]

So with Tech Trading on, the AI is way too considerate with techs as gifts. I will be 1 tech away from the leader, and the leader gifts me a tech. That's much too good, and techs are way too value to be given as gifts.

The only time I would consider that is for my vassals (to keep their military competent with mine), or with like the last place civ that I'm taking pity on and want good relationships with. I would never grant a tech to a civ that is near the top tech score. even if we are the best of fiends.

I haven't done many changes to any functions involving deal or dialogue logic because it's by far the most annoying part of the diplomacy AI. It's my next big project, though!

Not sure this is intended (or otherwise impossible to program against) but I've had a recent war with the Zulu, who were at the time attacking a CS.

I entered the CS's territory and destroyed a bunch of the enemy's units, therefore saving their city... but I was getting trespassing penalties instead of being rewarded for attacking a common foe, just as it already happens with major civs.

This was particulary jarring because at a point the Zulu did manage to capture the CS. I wore down the city's defenses and was about to recapture it... and then the Persians rushed in with cavalry, captured the CS and returned it. So I ended up with a very angry CS that gave the Persians a ridiculous influence bonus (and allyship). I feel like that my previous war effort should have been rewarded instead of losing the diplomacy game to a civ that arrived much later just to collect the spoils.

TL;DR: "we fought against a common foe" should be implemented for city-states

I agree, this shouldn't happen. I'll open an issue on Github about this.

 

[Recursive]

So with the rewrite of peace logic, I'm happy to say that the "core" diplomacy AI elements are finally all in good shape, aside from a few details and bugfixes.

I felt it was necessary to start with the mechanical aspects of the AI over its visual presentation, since no one will like a stupid AI no matter how visually polished it is, and it's been a big project to get to this point.

My next big task is more illusory in a sense, but only a little less complicated: the dialogue and leader interactions and general polish. There are some technical limitations here that I'll have to work around, but hopefully nothing insurmountable.

I am very open to ideas on how the dialogue/interaction system could be improved. Some thoughts have included:
- A "What can we do for you?" button where the AI gives you advice on how to improve relations with them;
- When an AI makes an offer for something on their turn, preventing the thing they want from being removed from the deal table;
- AI distinguishing between an AI plotting against you (Deceptive) and planning war against you when sharing intrigue;
- AI actually acting on shared intrigue (currently this is not handled well);
- Remembering the last line of dialogue sent, allowing for "followup" text (might not be possible due to memory limitations, I will investigate - although a lot of unnecessary memory has been deleted recently).
- The ability to warn the target AI if an AI confides in you that they're plotting or planning war against them (same penalty as warning the target of a coop war);
- The ability for one AI to send multiple messages in the same turn;

And I'd love to hear any other ideas people have.

 

[Stalker0]

So with the rewrite of peace logic, I'm happy to say that the "core" diplomacy AI elements are finally all in good shape, aside from a few details and bugfixes.

I felt it was necessary to start with the mechanical aspects of the AI over its visual presentation, since no one will like a stupid AI no matter how visually polished it is, and it's been a big project to get to this point.

My next big task is more illusory in a sense, but only a little less complicated: the dialogue and leader interactions and general polish. There are some technical limitations here that I'll have to work around, but hopefully nothing insurmountable.

I am very open to ideas on how the dialogue/interaction system could be improved. Some thoughts have included:
- A "What can we do for you?" button where the AI gives you advice on how to improve relations with them;
- When an AI makes an offer for something on their turn, preventing the thing they want from being removed from the deal table;
- AI distinguishing between an AI plotting against you (Deceptive) and planning war against you when sharing intrigue;
- AI actually acting on shared intrigue (currently this is not handled well);
- Remembering the last line of dialogue sent, allowing for "followup" text (might not be possible due to memory limitations, I will investigate - although a lot of unnecessary memory has been deleted recently).
- The ability to warn the target AI if an AI confides in you that they're plotting or planning war against them (same penalty as warning the target of a coop war);
- The ability for one AI to send multiple messages in the same turn;

And I'd love to hear any other ideas people have.

Probably not what you had in mind, but my number 1 request on the trade screen.

I want the "what will make this deal work" to actually....work. I just want it to equalize whatever deal using GPT....done and done. That would be an amzing QoL improvement.

 

[usadefcon1]

Right now when you Request Help from others, the button to execute says "Demand" which is misleading. The button should say "Request"

 

[EGA]

So with the rewrite of peace logic, I'm happy to say that the "core" diplomacy AI elements are finally all in good shape, aside from a few details and bugfixes.

I felt it was necessary to start with the mechanical aspects of the AI over its visual presentation, since no one will like a stupid AI no matter how visually polished it is, and it's been a big project to get to this point.

My next big task is more illusory in a sense, but only a little less complicated: the dialogue and leader interactions and general polish. There are some technical limitations here that I'll have to work around, but hopefully nothing insurmountable.

I am very open to ideas on how the dialogue/interaction system could be improved. Some thoughts have included:
- A "What can we do for you?" button where the AI gives you advice on how to improve relations with them;
- When an AI makes an offer for something on their turn, preventing the thing they want from being removed from the deal table;
- AI distinguishing between an AI plotting against you (Deceptive) and planning war against you when sharing intrigue;
- AI actually acting on shared intrigue (currently this is not handled well);
- Remembering the last line of dialogue sent, allowing for "followup" text (might not be possible due to memory limitations, I will investigate - although a lot of unnecessary memory has been deleted recently).
- The ability to warn the target AI if an AI confides in you that they're plotting or planning war against them (same penalty as warning the target of a coop war);
- The ability for one AI to send multiple messages in the same turn;

And I'd love to hear any other ideas people have.

what about a button to request units? for example you are in a defending against someone much stronger, your ally may not want to participate in a full war but may want help you a little or make more costly the war

a button to request to keep nuclear missiles out of reach of your cities like in https://en.wikipedia.org/wiki/Cuban_Missile_Crisis

 

[Recursive]

Probably not what you had in mind, but my number 1 request on the trade screen.

I want the "what will make this deal work" to actually....work. I just want it to equalize whatever deal using GPT....done and done. That would be an amzing QoL improvement.

I'll see what I can do. Deal AI is hard.

Right now when you Request Help from others, the button to execute says "Demand" which is misleading. The button should say "Request"

Yes, the code does not properly handle this, it visually treats it like a demand. Will fix.

what about a button to request units? for example you are in a defending against someone much stronger, your ally may not want to participate in a full war but may want help you a little or make more costly the war

a button to request to keep nuclear missiles out of reach of your cities like in https://en.wikipedia.org/wiki/Cuban_Missile_Crisis

Request for them to assist you with units or give you units? G disabled unit gifting because it was exploitable but if it could be fixed...

I like that last one.

 

[Stalker0]

Deal AI is hard.

Just to clarify, I am not looking for any AI changes in this request. All I'm asking for, is to put some things in the trade screen, press the "what will make this work" button, and for the proper amount of GPT to appear on either side of the trade screen to change the diplomatic number to "acceptable". Only if the deal cannot be balanced with GPT (because there is not enough GPT from that civ to do it, or the deal is just not acceptable period) should it say "Impossible". So more of a macro than any true change to the system.

And its completely fine if its just GPT. I know the button can in theory balance with all sorts of things, but that hasn't worked probably in YEARS, and there's no reason for it. 95% of my deals I just want to balance out with gold, if I want to make a complex deal not using gold, am I happy to propose and tweak the deal myself.

 

[azum4roll]

And please maximize the deal in the human's favour, so we don't have to manually add that last 1-2 GPT to check if it's still acceptable.

 

[BornDownUnder]

As a 'my two cents worth' on the topic of deal negotiations, I think the AI needs to be far more frugal in their offerings as at present it is far too accepting of deals which can quite easily send them towards effective bankruptcy which is easily exploitative in 'controlling' their ability to remain competitive throughout the game.

I feel also the AI is far too lenient in their decision making towards trade deals based upon their disposition towards the player and other AI.

To note, I play on Emperor difficulty, marathon game speed. In testing I have been able to go from a deficit of ~-5gpt to ~45+gpt with just a few trade deals on occasions before/first half of the Classical era. After the 3rd era it becomes much easier to have a commanding gpt rate without even trying to be exploitative (Going for maximum 'unfair/bad' deals) which effectively hamstrung the AI that I tested deal negotiations with.

 

[EGA]

Request for them to assist you with units or give you units?

I would say both

G disabled unit gifting because it was exploitable but if it could be fixed...

In which way it was exploitable?

 

[Milae]

As a 'my two cents worth' on the topic of deal negotiations, I think the AI needs to be far more frugal in their offerings as at present it is far too accepting of deals which can quite easily send them towards effective bankruptcy which is easily exploitative in 'controlling' their ability to remain competitive throughout the game.

I feel also the AI is far too lenient in their decision making towards trade deals based upon their disposition towards the player and other AI.

To note, I play on Emperor difficulty, marathon game speed. In testing I have been able to go from a deficit of ~-5gpt to ~45+gpt with just a few trade deals on occasions before/first half of the Classical era. After the 3rd era it becomes much easier to have a commanding gpt rate without even trying to be exploitative (Going for maximum 'unfair/bad' deals) which effectively hamstrung the AI that I tested deal negotiations with.

I think this has been much improved in the 5-24 beta.

Also @Recursive it feels like AIs may now be a bit too slow on making peace.

 

[wabab]

Playing a new game, Emperor difficulty, on the 5-28 beta (5-24 with the hotfixes), the AI seems way too lenient with each other. Have a bunch of warmongering AI's while I'm playing Austria, at least three of them have just devoured a few other nations and... everyone likes them but has it out for me since I'm gaining favor with with a handful of city-states. My one major ally (Korea) is obsessed with warring against the Inca, despite that they're on a different continent. Meanwhile, the Shoshone (same continent, much closer) have vassalized two civs and took over France. The major civs around the Shoshone apparently don't see them as a threat, since they have defensive pacts.

Also, I think it may have been mentioned elsewhere, but the turn your defensive pact expires, the AI will declare war on you immediately with no chance to renew it. In this case, Korea had a Defensive Pact with me and also with the Shoshone, so only once it expired could the Shoshone declare war. You can't renew it the turn it expires either, you must wait the turn after. I survived the war with the Shoshone, so we'll see how Korea treats them later on.

That said, the AI still feels really good, but it seems like there's more than a hint of "let's kill off the human player" feel to it. Weaker civs are much better at survival, Byzantium and Maya (the Shoshone's neighbors) voluntarily vassalized themselves rather than be conquered. Stronger civs seem way too focused on victory conditions though; it's clear the Shoshone are going to go after Korea (much larger civ than mine, bigger military too), but apparently Korea is more worried about winning than surviving long enough to win. I was pleasantly surprised that when Carthage also declared war on me at the same time as the Shoshone, the Huns offered me a deal where they declare war on Carthage.

 

[InkAxis]

I am very open to ideas on how the dialogue/interaction system could be improved.

And I'd love to hear any other ideas people have.

@Jednooki_John mentioned it in another thread but I thought it was a great idea, if you are at war with the same target then if the AI has a specific target (ex. attack city XYZ) then it should tell you. And you can also tell the AI similar things.

 

[Jednooki_John]

Thank you @InkAxis for mention! Here's what I wrote, I copy it here:

"You are definitely right! I was always thinking that this could be somehow improved, maybe I even wrote about it before on forums here. You know when a spy can tell you that another civ prepares a sneak attack on a city X? I assume then that this "warring" information reg. AI tactical plans is stored somehow. What if this information could be accessed somehow in a friendly allied scenario?

We can get some tactical information reg. our enemies (by spying on them), but we are unable to get any info reg. tactical information when it comes to our allies and our joint wars, isn't that weird? What if let say, when a defensive pacts kicks in, our ally tells us about his tactical plans for the upcoming conflict, to add some visibility? And just how a spy can tell us about "surprise" city-target attack, maybe AI could plainly tell us when defensive pact is activated - hey, I am targeting city XYZ. And maybe we could have another "talking" option available, just like "what do you think of...", which would then tell us about AI's war-mood. Is it still trying to capture city XYZ? Maybe it switched its priorities to ZYX? Or maybe it is actually trying to peace out, which would give a player an early warning that player might be on his own soon?

I feel like this information is there stored in a backend, we'd just need some way to present it, to get a better visibility into AI's ally thought proccess. I always thought it could turn out meaningful."

The full thread is here if you are interested in the context:
https://forums.civfanatics.com/threads/war-war-never-changes.670611/

 

[wabab]

Ended up winning my game with DV, ended up using an interesting (borderline cheese) strat—if you vassal yourself to a powerful AI, the other AI's won't bother attacking you. I ended up doing that since it was like a better version of a Defensive Pact. Your AI master will also never attack you either. Interestingly, you never have to adopt your master's ideology either; you can choose your own as a human vassal.

Now, to get a DV you can't be a vassal, but the way around this is to just to declare independence right before the WC vote, so you only have 1 turn of war (if the master civ doesn't grant you independence) to deal with. Not sure if you can win SV or CV while being a vassal, but it certainly seems possible.

My takeaway from this:

1. Either Defensive Pacts need to be made more "solid," or there needs to be an Alliance feature that acts as mutual Vassalage (but without the master/vassal relationship). The lack of auto-renewal makes it so that you can be attacked when it expires, and if the attacker has a DP with your ally, then even if they agree they won't join the war after the fact (or the they just will refuse the DP).

2. There doesn't seem to be many clear ways to sway the AI's opinion of either or other AI's through diplomacy/dealmaking. Perhaps there needs to be another mechanic for this (e.g., having a spy/diplomat perform an "improve relations" action).

3. Sanctions don't appear to damage the AI that much, nor do banning luxuries. I was able to get multiple AI's sanctioned and their main luxuries banned, but they didn't seem to suffer from any happiness issues. This would be generally result in a player losing if both their main luxury were banned and they were sanctioned. They treat it seriously in the WC, but its effects seem to negligible—on Immortal and Deity I'm assuming this barely tickles the AI.

 

[EGA]

Thank you @InkAxis for mention! Here's what I wrote, I copy it here:

"You are definitely right! I was always thinking that this could be somehow improved, maybe I even wrote about it before on forums here. You know when a spy can tell you that another civ prepares a sneak attack on a city X? I assume then that this "warring" information reg. AI tactical plans is stored somehow. What if this information could be accessed somehow in a friendly allied scenario?

We can get some tactical information reg. our enemies (by spying on them), but we are unable to get any info reg. tactical information when it comes to our allies and our joint wars, isn't that weird? What if let say, when a defensive pacts kicks in, our ally tells us about his tactical plans for the upcoming conflict, to add some visibility? And just how a spy can tell us about "surprise" city-target attack, maybe AI could plainly tell us when defensive pact is activated - hey, I am targeting city XYZ. And maybe we could have another "talking" option available, just like "what do you think of...", which would then tell us about AI's war-mood. Is it still trying to capture city XYZ? Maybe it switched its priorities to ZYX? Or maybe it is actually trying to peace out, which would give a player an early warning that player might be on his own soon?

I feel like this information is there stored in a backend, we'd just need some way to present it, to get a better visibility into AI's ally thought proccess. I always thought it could turn out meaningful."

The full thread is here if you are interested in the context:
https://forums.civfanatics.com/threads/war-war-never-changes.670611/

This also could be used fix an exploit.

As things are right now, on a past game my neighbor started a war against my other neighbor and fought for long time until one of them managed to break the enemy lines and begun to siege his cities, I toke this opportunity to move two spearmen(they were using tercios) and two cannons and by timing my attacks and blocking some of his units, I captured three cities including the capital. That would alright except the fact the AI was pretty pleased that stole three of his hard earned cities, so not only got three free cities but a permanent ally until ideologies. note I was fairly weak at that game as Venice and mi friend could wipe off at any time

If I was the AI, at the first stolen city I would be pretty pissed and considerate DW, especially how easy was to get rid of me at that time.

A system like the above could help fix this issue. For example by the AI saying something like "I want X city and his capital, please don't get in my way" or maybe a way to discuss how we are gonna split the territory like secret pact between Germany and Russia, but now that I think about it there could be simpler ways to solve it, like if the AI does +75% damage to a city, then it gets mad if not then pleased.

 

[InkAxis]

That would be very cool, it would feel like the AI is actually a person that has goals. So overall, you could ask what/how they are attacking (navy or army, what city). And then you can tell them what cities you want for yourself and what cities they want. Probably only 1 city at a time, to simplify the UI, and so it's easier for the AI and so things can change over the course of the war. But, you still can capture any city, if you do capture their city they will get a little mad.

But it would probably be difficult to code.