Diplomacy AI Development

As I've alluded to a few times, I plan on a major overhaul of diplomacy AI intelligence in Vox Populi throughout the upcoming year or so.

Diplomacy in Civilization is complicated because it has a lot of different goals that it needs to balance, and when the balance is done poorly you get an AI that makes nonsensical decisions.

These are my objectives for the rework (in no particular order):

1) The diplomacy AI should aim to win the game where possible. If it cannot win the game themselves, it should focus on survival, forming alliances to overcome opponents together, and/or propping up players who treat them well over players that don't. It should always be looking out for its own interests.

2) The AI should be neither too passive (BNW) nor too aggressive (last beta). It is better to err on the side of aggression because most players seek a challenge, but it shouldn't be stupid aggression (endless phony wars accomplish nothing, and ramming tons of weak units into a skilled human player's defenses just gives them XP and yields, making them even stronger). The AI should gravitate towards neutrality as a default, not friendliness or aggression.

3) The AI must strike a balance between being too predictable (exploitable) and being too erratic (impossible to comprehend).

4) For flavor and smarter decisions, individual civilization leaders should behave differently from each other, but not radically so unless the situation calls for it.

5) The AI should be strong at evaluating the situation and making good decisions. However, the AI's situational assessment should not be the only factor in its decisions - see 4 and 7.

6) AIs should not treat humans differently from other AIs unless there is a good reason for it, and should also avoid cheating (e.g. using information they shouldn't know) where possible.

7) Diplomacy should be rewarding for players that invest time and resources into maintaining good relations and treating the AI well. This does not mean always being able to evade challenges (e.g. bribing the AI to avoid war constantly), or being stupid (e.g. allowing enormous warmomgering with no consequences) but there should be a return on investment for assistance, long friendships, etc. even if it sacrifices the AI's win chances slightly. This is a game and meant to be fun, after all.

8) In contrast, however, diplomacy should be punishing for players that do not take the time to invest resources into it - and because resources are limited and also because of global politics, civilizations that are friendly with everyone should be a very infrequent occurrence.

9) If the AI can send any statement to a human that has an impact on gameplay (a promise request, for instance), the human should also be able to do the same.

10) Diplomacy should feel realistic and immersive, within reasonable limits. My goal is to make the AI feel less robotic and more like actual civilization leaders.

(Note: Memory and performance limitations unfortunately make the addition of large amounts of diplomacy text difficult, although I'm searching for some way around this problem.)

11) Diplomacy decisions should feel meaningful. If one choice is always best it is simply annoying - for this reason the AI should also avoid doing things like asking the human for Open Borders every 2 turns.

12) Diplomacy should generally be fun, at least for the majority of players. Options can be added if enough people want a specific playstyle (for instance, if they want non-competitive or hyper-competitive AI) so it can be fun for everyone.

13) Good documentation/a guide of some sort should be available to teach players how to conduct effective diplomacy.

14) In general, the AI should be adaptive and able to change as the circumstances do.

15) Diplomacy AI should be less opaque to developers, more open to modding and easier to debug.

In addition, I aim to improve trade logic, since the AI there could use improvement in a number of ways. For instance, they should be able to trade embassies and open borders in a single turn - they should not say "there is no way to make this work" when they are willing to accept something...

...and most importantly of all, when requesting help the button should NOT be labelled "DEMAND" but rather "REQUEST". It drives my OCD mad! Mad, I say! Wait, where was I...oh, right, this thread.

I started this thread so the community could suggest ways they think the diplomacy AI could be improved, to get feedback on the rework I'll be doing, and to update people on my progress.

Note that this is going to be a longrunning project, diplomacy is complicated (as I've outlined above) and I have other things to do in my life, so patience and reasonable expectations are good ideas.

Also note that I do not plan on making alterations to non-diplomacy AI like military, tactical, or city production AI.

With all of this said, I'd be happy to hear what others think of this and any ideas they would like to contribute.

 

I think my biggest pet peeves are the frequent “impossibles” that come up. To me that should only occur if the AI literally cannot do it. Otherwise make it crazy expensive, but make it possible

 

I am very glad that this is being treated as an explicit design goal as opposed to a desirable side effect.

 

im very happy that someone is taking time to adjust diplomacy and as i wrote before.. last version feels much better.

i have some desires that annoying me pretty much.

1. As Stalker0 writes.. it´s the frequent impossibles, it´s like you can remove some questions/Demands because you know the answer is impossible (it could be a good deal)
and lately civ´s doesent want their capital back (think it´s a bug but anyway)

2. it should be possible to have friends (much better last version)

i like these changes.

3. This thing is disturbing me a lot, it´s the world congress Sanction.

i like the mechanic Sanction and it can possibly be more devastating and stronger BUT....

your friends or neutrals should not use it against you and think one more time before propose it..

what can they win/lose on it.. is it worth it??

the Sanction proposal can destroy the game diplomacy that you have built up during years..

 

Happy that you've posted this, lots of good stuff (I did find it annoying when the AI would ask for my Furs for their embassy every turn)

What I'd love to see personally is the AI stop making defensive pacts with their enemies. It makes no sense

 

Many of these are designed to avoid the AI making stupid decisions or to block certain trade options (e.g. giving away all their Gold income, blocking diplo options from.players who've denounced them), but I'd rather address this by making the AI smarter where possible.

It takes a backseat to functionality and good gameplay, of course, but I think it'd certainly improve the singleplayer experience.

1 - See my reply to Stalker0 above. Not being able to trade the capital back is likely intended; unless at war with a civ, you can't trade a damaged city to them to prevent exploits, you must heal it to full HP first. However, I don't like that the UI just removes the city from the list without explanation; I plan to grey it out and have a red text tooltip explain why the city cannot be traded.

2 - That is among my goals (although if every AI has much more to gain by making you an enemy than a friend in a particular game, they're not going to be stupid - the AI is intended to look out for its own interests).

3 - Voting logic is not part of the diplomacy AI but as the world congress is really a global diplomacy mechanic I'll take a look at this. Your friends are probably trying to sanction you because you're too successful and they want to win for themselves (that, or they aren't really your friends).

Trade spam is annoying. The AI should at least make different offers if their original one fails.

DP logic does need adjustments; that said, they might make one with an enemy because they think it'll deter another enemy from declaring war on them, etc; there are reasons why this can happen that aren't bugs.

 

Good list overall!

For 6) I think the AI should probably take into account the player's tactical superiority when evaluating the player's military strength (perhaps this is what you had in mind with the 'unless there is a good reason' bit). Maybe have the AI give the player's strength a bonus based on difficulty level when doing the evaluation which uses the assumption that the higher the difficulty the stronger the player's ability to use their military?

For instance, a warmonger AI choosing a target to conquer might be better off if he goes after a nearby AI of medium military strength than going after the player with a weaker military just because the player tends to still be tactically tougher. This of course has to be balanced around the fact that a player who neglects military needs to be punished for it. It's probably a tight rope to balance on...

For the 'not using information they shouldn't know' bit- can you expand on what that entails? Would that be something like not giving them vision of the whole map? Or not making them aware of an agreement to DoW between two other players? Or perhaps there are other examples?

I think I'd probably be ok with some cheating by the AI. Maybe not total vision, but giving the AI a little extra sight so they have more information when doing military operations could be ok, for instance. The human is capable of inferring things without direct information at times, though I'm having trouble thinking of very many examples.

 

There are some circumstances where discrimination has a good reason behind it; for instance, the AI can know what another AI wants most in trade, but can't know what the human would want, and must guess. The AI can't be as certain of a human's approach towards them compared to an AI's, because humans don't use the approach system. There's also the AIDeclareWarProb difficulty setting, which makes the AI more aggressive towards humans close to winning the game as difficulty increases (the AI also gets a penalty, but it doesn't scale based on difficulty). What I'd like to avoid is something unfair and unfun; e.g. in Civ 4 if TVTropes is to be believed, the human using nukes would get a negative diplo modifier with every civ, whereas the AI doing the same would only trigger a "you nuked our friend" modifier. All AIs ganging up on the human player like a hive mind rather than looking out for themselves obviously isn't fun either.

I'm thinking the diplo AI's assessment of the human player's military at higher difficulties could perhaps use the AIUnitSupplyPercent bonuses when comparing. For example, an AI with a 25% unit supply bonus could assume the human player's military is 25% stronger than it actually is. I do think the AI is having trouble properly estimating military strength right now, is too aggressive as a result, and could use some help with that. On the other hand, it's indeed walking a tightrope, because if the AI never attacks the human player, they'll never have to invest in military and can do whatever they please. I'm also considering a mechanic whereby the AI could maintain some memory of the results of past wars, although that would be difficult to implement and Civ has memory limitations.

The diplo AI (or the AI in general, for that matter) doesn't really cheat all that much, I'm just noting it as a design goal as it's a restriction on potential solutions. The AI does always get an estimation of your military and economic strength at all times, but this isn't cheating since the human can more or less do the same (military advisor/war details popup). Having every AI calculate every other player's strength estimates every turn only based on the information they know would be super expensive performance-wise and make the AI stupider.

The AI definitely can't see the entire map; they get +1 or +2 extra sight on Immortal and Deity, respectively. However, difficulty handicaps, like military and tactical AI, are not part of the diplomacy AI.

In terms of what I mean; the AI can, if it's programmed that way, can pluck any information they want about other players directly from the game engine. There's a distinction between "cheating" that enhances the game experience or is otherwise acceptable, and "cheating" that detracts from the game experience.

To give some examples of each, in my view:
Acceptable: When deciding whether to add a "dogpiling" bonus to jump on a war declaration bandwagon, the AI can't talk to the other warring player and ask "how's the war going?", so it just grabs the other AI's war projection (the human has a shadow AI which calculates this). It's easier than doing a complicated estimate that would offer no real benefit over the alternative, and the human can generally tell how a war is going just by looking at the map and/or war score.

Acceptable: The AI reading the player's unhappiness value when deciding to insult them for being too unhappy, but not using this information in any other way. Insult messages were disabled in VP, I believe, but this is the case in base civ.

Unacceptable: The AI always knowing the position of every military unit, including the ones it can't see, and popping up to warn the player to move their invisible troops.

Unacceptable: The AI wants to decide whether to stop spying on a player, and to do this it checks to see if the other player has a spy in one of their cities, which it can't possibly know, and the human can't do the reverse.

(The unacceptable examples do not happen, by the way)

 

Pretty sure the AI does mock you about having a weak army despite you don't. And claim that they'll get the Global Hegemony/other victories when there's no way they could.

 

True, and it is annoying. I don't think this is based on knowing anything about you though. They're kind of just saying 'this is the victory condition I have decided to aim for'.

Granted there's no need to be rude about, especially if those people are supposed to be your friends/allies lol.

 

Yes, but they don't pop up on their turn just to tell you "your army is weak" / "your people are unhappy" / "your empire is uncultured" / etc. like they do in BNW, as far as I'm aware, they reserve their insults for when you go to talk to them. Claiming that they're going for a victory condition is a new type of statement that was added by VP to indicate the AI's pursued victory condition, not an insult statement. Military strength evaluation could use some tweaking, though, as I've said.

Perhaps the dialogue could be changed slightly if you have a positive relationship.

Oh, one other design goal I forgot to mention in my original post: I realize this will probably take a long time, but I have hopes of making the AI able to talk and trade with human players in multiplayer (with the option to turn it off). This one might take years before it could become a reality, though, with all the issues MP has already.

 

If the AI dislikes you a lot they will send these kind of messages. You can either go 'whatever', or you can say 'you will pay for that' which may provoke them to war. That's how I remember it anyway. I'll see if I can get a screenshot of one.

That would be lovely!

 

You know? I'm not very proactive in diplomacy. I never seek the other civs to talk with them, just react to what the want to say to me. Only exception is when my cities want some luxury, I try to buy it.
If a civ has been nice to me and they ask for some support and I'm ok with it, then I admit it. Otherwise I tell them to go to hell.
But what I find most annoying is a civ asking me to join in a war, when I didn't put a thought into it before, so I'm not sure if it suits me.

When I have a resource that other civ wants, I am approached by that civs asking for the resource, and I am not sure if I should sell the resource to this civ or perhaps another civ is willing to pay more, or perhaps I should give it to another civ that is less threatening to me.

So, what I'd like to see changed is:

1. Delayed responses. If a civ offers me to go to war, let me think about it the next turn. Let me see the other neighbours, let me see if I have any chances against your enemy, let me think whether your friendship is convenient to me. I promise to give you an answer during my turn.
2. Remind me what my diplomatic options are. AI already comes to ask for Open Borders when it wants it. AI asks for luxuries or resources when they want it. But many times they don't come asking for a trade deal that they could accept if I proposed to them. If my friend is at war, and didn't asked for a joint war, maybe he will be glad that I send him a few units. Well then, come say hi, and remind me that if I were to gift a few units to them, it will help them greatly in their current wars, and they'll be grateful in the future.
If the AI could not found a religion, maybe it could come to remind us that sharing our religion with them will make them happy and will strengthen our ties.
3. If I'm not expending diplomats to get city state allies, then they should not hate me for stealing their city states. I would understand a requirement for a promise to not use diplomat on a certain city state, though.
4. Have it cristal clear how my actions affect opinions. Does this slightly piss them off, or will they hate me forever for this? Let me know how much your opinion of me is going to change if I choose this or that dialogue option, and how high your opinion of me currently is. Keep hidden your strategic approach if you like. It may be possible that we are best friends and you still want to backstab me when it suits you, but at least I will know that you didn't backstab me because you hate me.
5. Coalitions. Let me hold a banner with the requirement to defeat runaway civ X. Then let's see how many civs show up the 10th turn. Maybe some civs could ask for a payment, maybe they could ransom civ X for not joining the coalition.

 

1 - Achievable for coop war requests with some work, already the case for DoF requests. Trade deals can be made the next turn unless the AI has traded whatever you want away already (delayed responses on trade offers would open a number of potential exploits, although it's not impossible). Don't think there's any other statements where this is important.

2 - The reason for the AI asking only when they want specific things in trade (and for cooldowns) is to avoid spam. If every AI had a trade offer every turn, it would become chaotic. The ability to gift units to major civs was removed in VP.

I could look into a system of giving more hints at how to improve diplomatic relations, but again, there can be a lot of civilizations in a single game and if all of them popped up to tell you what they'd like frequently it would be spammy and annoying. If there was a better way of conveying this information, then perhaps.

3 - I've proposed on Github that we add a button to City-States to cancel/stop all quests, to prevent unintentional Influence gain.

4 - The AI keeps some of this information hidden for strategic reasons. If you can see that they have an enormous negative modifier because of your warmongering, for example, then you can more easily deduce when they aren't really FRIENDLY. If hidden modifiers and invisible number scores are an issue for you, there's already Transparent Diplomacy.

There are a couple dialogue options that could use a mouseover popup to explain the diplomatic consequences. I'll add that to the to-do list.

I'm not sure how else it can be made clear - I suppose raw Opinion numbers could be added to these popups, but I think that would cause confusion for players who aren't playing with these values visible.

5 - This is too vague to be actionable. Are you proposing a mass joint war, or...?

 

One thing that might help is a tutorial mode, or something that explains where to find such information. Unfortunately civ has a lot of different things going on in it, and it's not always ituitive where to find the relevant information. The Civilopedia for example is relatively sparse on such things. That's part of why it's been so useful when someone explains what the code says so we can know how the game actually works.

 

Current game: Emperor/Epic/Standard size map. Songhai going Domination.

Talking about DoW I think the first thing to be addressed is how AIs evaluate human military power.

In this game:

_ started on center of my continent. Coastal start with Pearls (ok, this one is BIG, cannot pass =)). Korea my closest nightbour South, India in the East and Denmark to the North. After 80 turns or so Denmark and Korea as friends DOW me. It was the right decision: i was expanding quite fast with a lead on cities. Bad execution though. Korea tried to attack a walled city behind a mountain with 4 units losing them all and basically ending their attack there, Denmark went towards my recent unwalled founded city but with the bulk of my army (Warriors, Cbows) and just lost. They had barely the numbers but had massively underestimated both my tech level and my defense placement. In the end they feed me science and culture and money through a favourable peace treaty.

_ later in medieval: vassalized Korea as catapults came into the fight, Denmark stalled me thank his Berserkers in the North that did very well vs my Mandekalu cavalry. I had to fight three wars vs Bluetooth and finally got him deploying cannons and caravels.

_ India DoF me since ancient, Smart decision but his capital was really exposed. Vassalized him without deploying the fleet just a massive artillery barrage from 5 field cannons. Delhi went down in 4 turns.

Other continent something strange:

_ Zulu was the runaway vassalized Portugal, Netherland but left Egypt alone ??? DoW India same time as I did. Again good strategy decision but poorly executed: India managed to defend from his fleet easily then capitulated to me and war over for them.

_ Zulu liberated Netherland with 1 city left but did no DoW to finish the job. Strange. He kept Portugal as vassal.

_ Egypt was left alone with 5 cities (3 in mainland and 2 on isles) with lilke 14 Wonders and still wonder spamming and this was before I accepted a DP with them (smart). Why ? Well in 2 turns my whole fleet (28 Corvettes, 2 Ironclads, 8 Frigates) and my recently embarked eastern army will storm Egypt and Zulu managed to stay just helplessly dead in the mid of his ocean rift.

To be honest so far AIs played fairly well but need help to finalize his plan decision wise. Zulu should vassalized Egypt taking all wonders and giving me a bad time but I think he was frozen by my military might and still in the scoreboard his military is better than mine. Why ? Maybe he produced 70 Impis waiting for my invasion.

 

I accept that an AI that is offering my a luxury in its turn may not have it for trading next turn if it manages to trade it away to other civs. This won't be an issue.
What drives me nuts is having to decide which side I must join before I can carefully look at my neighbours stats. Right now, I am forced to involve myself in a war or risk to be rude to someone, blind eyed. I'm only asking for that AI to wait just to the very next turn for a response. Exploitable? In the sense that the unprepared player would commit fewer mistakes, yes, but I can't see other drawbacks.

You have a point. So these reminders should be not too invasive. I'm talking about diplomatic options that an experienced diplomatic player know and use, but inexperienced players miss. For example, we all know that a civ that wants to be friends with us comes offering a DoF. What they don't do is asking that we Denounce their enemies, and that's a very powerful diplomatic tool. When an ambassor knows of an intrigue, new diplomatic options open, but unaware players may miss the chance to share the intrigue.
Perhaps they could come and visit just to say that they'll love that the player share the religion with them, had they not founded theirs. Maybe (only maybe) a very friendly civ that opens a policy tree before the player and wants to stay friends could come and suggest the player that he picks the same policy, that the civ would be very pleased if the player chose the same.

Well, that's another way to achieve the same, but I fear that I would forget more often than not to unlock the button. I think it would be better that the AI gets angry just when the player uses the diplomatic action with the diplomat or the great diplomat on the city state in case, after being warned. If I am not actively trying to make an ally, AI should not be angry. AI could be concerned and could request that we make no further advancement, but fulfilling quests should not be considered as such.

I'll try to be more specific. You know that giving gifts to major civs increase their opinion, but we can't say how much gold we should give to them. If I am giving to them some gold and resources, I would like to see how much this would influence their opinion before making the offer. If I am going to share an intrigue, or insult them or denounce them, how much would this influence the opinion? I understand that a few modificators remain hidden so the AI can act mischievously, and I don't really need to see them all unless I play with Transparent Diplomacy, but at least I would like to know in advance how much my actions will piss them off.

I am talking about the Coalitions system in Europa Universalis 4. There, you raise a flag against another nation, and some other nations may join your side, depending on their opinion and their interests. You can even trade with some of those nations so they enter the coalition for a reward. Once the coalition stands a chance, the leader of the coalition is able to declare war. In that moment, all nations that are inside the coalition must declare war too or suffer a big reputation hit. This is the only way a few minor center european countries can stand a chance against a huge empire like the Ottomans, France or Russia.

We could adapt this system for Civ5. See how we can mostly ignore the lesser civs that have been beaten by some big contender? How would it be if they could join a coalition and work together towards some goals? Right now they can only use a defense pact, but that's self defense. I don't fear a vanquished civ that is only strong at defense. But 3-4 civs ganging up against the biggest empire with the goal of reconquering some of their former cities that would be something.

Anyways, if you think coalitions are too hard to implement, then drop it. The current method is not that bad.

 

I feel this too, very much. I think what Heathcliff is saying though is not that such a thing would be inherently exploitable but that it would be difficult to implement in a way that isn't exploitable.

Ideally for me I would be able to alt-tab out of the pop-up or something so I could check the situation and then give a response straight away. I don't know if that's even possible though.

 

I just want to say that diplomacy is in fact the best developed part of the vp, and I do not even try to comprehend what enormous amount fo work it taken to get vp to this point. Do not overwork yourself, because diplomacy is the most realistic and fun to do phase of the game right now. Especially do not give in to people requesting favorable treatment or requesting an AI that do not know where its interest lay. I feel that pushing anything right now could result in instability.

I disagree on this. There could be no unpredictable AI eithout being erratic. If the AI will behave according to its own best interest, it will be predictable for every player who is not half-witted. You most of the time know their capital, their cities, tech, policy count, army potential, their neighbours, their luxes, they strategics, their possible victory type they aim for, who the fear, who they can cooperate with, who is their next target, who is next targeting them etc. you cannot not make clear assement of how they would act to secure thie interests. I disagree though that predictability entails exploitability or passivness. You should have enough competitiors, and if you choose not-friendly attitude for them, they should turn neutral and deny you opportunities you would gain by cooperating with them.
Contrary I think deceitful AI should ask you if you would be a luxury several turns when it is already moving its troops against you , agree to deal, even offer better prices. It would be no erratic, but consistent lulling a player into its plan.

Footnote: how would you feel if declaration of friendship be changed into something more appriopriate? Like declaration of strategic coopeeration or just alliance. Beside, there is no friendship between states, nations, and corporations. There are only interests.

Moderator Action: Inappropriate content removed. Please do not use slurs in your posts in future. leif

 

Predictability vs Erratic is not about when the AI is making optimal decisions, that are "objectively better in every point".
Predictability vs Erratic is when the AI is making suboptimal decisions (either because of performance constrains, or role-play constrains). If those are too predictable, they become exploitable. If those are too erratic, they break the player immersion / trust that the AI will behave rationally.