Diplomacy AI Development

[tu_79]

I've been thinking on the reckless expansion thing, and I came out with a very early game problem.

See. If you use flat differences (civ has 3 more cities) it can be useless once empires get some real size. If you use relative differences, the second city is a difference of 100%, clearly a value too high for later turns.

So I say skip the first two cities. Wait until a civ settles its third (even fourth) city to start the reckless expansion test. Chances are that the median is already at two. Having three cities when the median is two surely is no reckless expansion. Having four cities when everyone else has two, is.

In pseudocode:
If player has three or more cities and player #cities is more than 175% of median then
Player is city aggressive.

I'm still uncertain about the best way to handle land expansion. Your approach could work, but I have doubts of the efficacy of the logic addition (an OR operator in this case). More likely, a player that has more cities will always have more land. A player that has more land is also a player with more cities.
But tall players (and Shoshone and Russia) have relatively more land per city, so that should be accounted.

An idea is to have the relative territory per city modify the value of what is considered to be aggressive.

For example, I said it could be considered aggressive to have more than 175% of the median of number of cities. Well, maybe, if the land per city is, say, 30% bigger than the median (meaning this player has cities that occupy 130% of the territory more common cities have), then reduce the value what is aggressive by 30%. This player with these large cities would be an aggressive expansionist when it comes to have 122% the median of the number of cities.

In pseudo code,
RelativeTerritory = (Territory/#cities) / (TerritoryMedian/#citiesMedian)
If RelativeTerritory < 1 then {
Relative Territory = - 1 / RelativeTerritory
}
AggExpan = 1.5 - RelativeTerritory

If player has more than two cities and #cities is bigger than AggExpand then player is aggressive expansionist

It is not polished, just an idea.

 

[wabab]

1) The diplomacy AI should aim to win the game where possible. If it cannot win the game themselves, it should focus on survival, forming alliances to overcome opponents together, and/or propping up players who treat them well over players that don't. It should always be looking out for its own interests.

Based on my playthrough's with VP (and Civ in general), it would be more interesting for the AI to prioritize survival first, then victory. The AI right now feels a bit erratic, seemingly thinking it can still win, then a few turns later trying to survive, etc. I think this would also allow for more "peaceful" playthroughs of the game to occur, with cultural/diplomatic/science victories possible without the need to conquer to half the world.

Tying into the above discussion of aggressively expanding empires, weighting strategic/luxury/bonus resources should probably play a larger role—granted, I'm not sure what role they play already, but it doesn't feel like AI cares too much about the quality of land you have, just the quantity. If a player has is monopolizing strategic resources or attempting to starve out another player, that should count for more aggressive expansion than if a player doesn't have as much of a strategic resource and tries to take more. Also, I haven't seen AI's really ever go to war trying to take over or raze another city for the sole purpose of claiming a resource, which I find strange.

I'm sure this is much easier said than done, but just wanted to give my feedback. There are some games where clearly I'm going wipe an AI player out, but rather than become a vassal state (maximize survival), they would prefer to lose most of their cities first. Defensive pacts don't seem to calculate distance either, with a lot of them being useless—their big ally isn't going to be coming to their rescue when they're half the world away.

 

[Bromar1]

On patch 3-15-2, wonder diplomacy modifier seems really high, I'm getting -100 with almost every civ. Minimum is -40 with a civ that has just one less wonder than me...

 

[Recursive]

On patch 3-15-2, wonder diplomacy modifier seems really high, I'm getting -100 with almost every civ. Minimum is -40 with a civ that has just one less wonder than me...

Hmmmm, that does seem excessive. Perhaps I'll have the opinion scaling use the median as well.

 

[Recursive]

Based on my playthrough's with VP (and Civ in general), it would be more interesting for the AI to prioritize survival first, then victory. The AI right now feels a bit erratic, seemingly thinking it can still win, then a few turns later trying to survive, etc. I think this would also allow for more "peaceful" playthroughs of the game to occur, with cultural/diplomatic/science victories possible without the need to conquer to half the world.

Tying into the above discussion of aggressively expanding empires, weighting strategic/luxury/bonus resources should probably play a larger role—granted, I'm not sure what role they play already, but it doesn't feel like AI cares too much about the quality of land you have, just the quantity. If a player has is monopolizing strategic resources or attempting to starve out another player, that should count for more aggressive expansion than if a player doesn't have as much of a strategic resource and tries to take more. Also, I haven't seen AI's really ever go to war trying to take over or raze another city for the sole purpose of claiming a resource, which I find strange.

I'm sure this is much easier said than done, but just wanted to give my feedback. There are some games where clearly I'm going wipe an AI player out, but rather than become a vassal state (maximize survival), they would prefer to lose most of their cities first. Defensive pacts don't seem to calculate distance either, with a lot of them being useless—their big ally isn't going to be coming to their rescue when they're half the world away.

Thank you for the feedback! You raise a number of concerns I'll be looking at in time.

 

[Aqinu]

I have 6 wonders (VP 3-15-2)

1st Civ 5 wonders (40 penalty)
2nd 1 wonder (100)
3rd 3 wonders (60)
4th 1 wonder (120)
5th 2 wonders (80)
6th 0 wonder (140)
7th 0 wonder (120)

I think too that is a lot of penalties coming only from number of wonders.

 

[Recursive]

I have 6 wonders (VP 3-15-2)

1st Civ 5 wonders (40 penalty)
2nd 1 wonder (100)
3rd 3 wonders (60)
4th 1 wonder (120)
5th 2 wonders (80)
6th 0 wonder (140)
7th 0 wonder (120)

I think too that is a lot of penalties coming only from number of wonders.

I'll reduce it to something more reasonable.

 

[Bromar1]

I'll reduce it to something more reasonable.

I think it could be a little bit more forgiving for weakened civs. It's hard to make alliances with a snowballing warmonger's many victims if they all hate you for having wonders.

 

[Recursive]

Well I've made some adjustments to the logic for next version:

- Only "conqueror" and "cultural" AIs will apply a penalty for wonder spamming when calculating DoF/DP willingness

- Only neighboring AIs (or close conquerors) will apply the wonder spamming penalty if they aren't cultural

- Threshold for both modifiers now uses the median instead of the average, as already discussed

Under consideration:

- Base Opinion penalty on the difference from the median or the difference between the civs, whichever is smaller

 

[Recursive]

Well I've made some adjustments to the logic for next version:

- Only "conqueror" and "cultural" AIs will apply a penalty for wonder spamming when calculating DoF/DP willingness

- Only neighboring AIs (or close conquerors) will apply the wonder spamming penalty if they aren't cultural

- Threshold for both modifiers now uses the median instead of the average, as already discussed

Under consideration:

- Base Opinion penalty on the difference from the median or the difference between the civs, whichever is smaller

All of these are now implemented for the next version. I'm hoping that'll be good, but I'm open to further adjustments if necessary.

 

[Omen of Peace]

Will the Wonder penalty be capped?

 

[Recursive]

Will the Wonder penalty be capped?

Not sure a cap is necessary, I've adjusted the mechanics of the penalty (instead of being based on the Wonder difference between the two civs, it'll take that difference and the difference from the median, and apply the lowest of the two), and the penalty won't apply at all under some circumstances, as described above.

The penalty was excessive, but with these adjustments it should be functioning more as designed; i.e. curbing excessive Wonder building, but allowing a reasonable amount of Wonder building.

The AI will also be a bit less aggressive about it.

So for now, no; but if it becomes necessary, one can be added.

 

[Recursive]

@Omen of Peace Here is the new code:

If you have any suggestions on how this could be improved further, feel free to comment!

/// Is ePlayer spamming World Wonders?
bool CvDiplomacyAI::IsPlayerWonderSpammer(PlayerTypes ePlayer)
{
   if (GC.getGame().countMajorCivsAlive() < 2)
       return false;

   // We don't care what our teammates do.
   if (IsTeammate(ePlayer))
       return false;

    // One or two Wonders is always okay.
   int iNumWonders = GET_PLAYER(ePlayer).GetWondersConstructed();
   if (iNumWonders <= 2)
       return false;

   if (ePlayer != GetPlayer()->GetID())
   {
       // If the player hasn't built more Wonders than we have, don't worry about it
       if (iNumWonders < GetPlayer()->GetWondersConstructed())
           return false;

       // If we're not a cultural civ, we only care about this if they're nearby
       PlayerProximityTypes eProximity = GetPlayer()->GetProximityToPlayer(ePlayer);

       if (!IsCultural())
       {
           if (IsConqueror())
           {
               if (eProximity < PLAYER_PROXIMITY_CLOSE)
                   return false;
           }
           else
           {
               if (eProximity < PLAYER_PROXIMITY_NEIGHBORS)
                   return false;
           }
       }
   }

   double fMedianNumWonders = CalculateMedianNumWondersConstructed();
   double fAverageNumWonders = 0;
   int iNumPlayers = 0;

   // Calculate the global average
   for (int iPlayerLoop = 0; iPlayerLoop < MAX_MAJOR_CIVS; iPlayerLoop++)
   {
       PlayerTypes eLoopPlayer = (PlayerTypes) iPlayerLoop;
       CvPlayer* pPlayer = &GET_PLAYER(eLoopPlayer);

       // Only major civs who have built Wonders are counted
       if (!pPlayer->isMajorCiv() || pPlayer->GetWondersConstructed() <= 0)
           continue;

       // For the mean, exclude the player we're looking at
       if (eLoopPlayer == ePlayer)
           continue;

       iNumPlayers++;
       fAverageNumWonders += pPlayer->GetWondersConstructed();
   }

   // Find the mean value
   fAverageNumWonders /= max(1, iNumPlayers);

   // Must have built several more Wonders than the median player in this game
   if (iNumWonders < (fMedianNumWonders + /*3*/ GC.getWONDER_SPAMMER_THRESHOLD()))
   {
       // Must also have at least 50% more than the global average, just to prevent anything stupid
       if (iNumWonders < fAverageNumWonders * 1.5)
           return true;
   }

   return false;
}

int CvDiplomacyAI::GetWonderSpammerScore(PlayerTypes ePlayer)
{
   int iOpinionWeight = 0;
   if (IsPlayerWonderSpammer(ePlayer))
   {
       iOpinionWeight += /*20*/ GC.getOPINION_WEIGHT_WONDER_SPAMMER();

       int iCivDifference = GET_PLAYER(ePlayer).GetWondersConstructed() - GetPlayer()->GetWondersConstructed();
       int iMedianDifference = GET_PLAYER(ePlayer).GetWondersConstructed() - (int) ceil(CalculateMedianNumWondersConstructed());
     
       // If we got this far, neither of the values above are negative
       // For scaling, go with whichever value is smaller
       int iWonderDifference = min(iCivDifference, iMedianDifference);

       if (iWonderDifference > 1)
       {
           iOpinionWeight += ((iWonderDifference-1) * /*10*/ GC.getOPINION_WEIGHT_WONDER_SPAMMER_PER_WONDER());
       }
       if (IsCultural() || IsConqueror())
       {
           iOpinionWeight += /*20*/ GC.getOPINION_WEIGHT_WONDER_SPAMMER_STRATEGIC_MOD();
       }
   }

   return iOpinionWeight;
}

 

[Zuizgond]

I have 6 wonders (VP 3-15-2)

1st Civ 5 wonders (40 penalty)
2nd 1 wonder (100)
3rd 3 wonders (60)
4th 1 wonder (120)
5th 2 wonders (80)
6th 0 wonder (140)
7th 0 wonder (120)

I think too that is a lot of penalties coming only from number of wonders.

Those levels of penalties are very similar to what I have in my current game. Not sure the next changes will drastically fix this as all close neighbors and cultural civs will stay at this level, if I understand correctly. Median vs average won't change a lot.

I'd like those modifiers to be cut by half, just to give an order of magnitude. Maybe the -20 base modifier should be toned down to -15 or -10.

Curious to see what others think about it.

 

[AnemoneMeer]

Personally, I would very much like to see a severe reduction in defensive pacts. In my games, I've noticed a lot of cases of people having defensive pacts that they cannot enforce due to distance. People agreeing to defend eachother while a world away because they're friends with good military. In my experience, it makes the game become much more passive later in the game for AI v AI combat and also makes it nearly impossible to convince AI's to declare war with you even if there's no reason why the other member(s) of the defensive pact could even reach you. Little hard to imagine them sailing around a continent and across the ocean to do it, but the AI considers it a major threat that prevents war.

The AI should only make a defensive pact if the person they are pacting with can be reasonably expected to reinforce them in a timely manner imo.

-----------------

Another change I think would be beneficial is that AI's who have basically been destroyed should seek to become vassals to a strong, nearby, and most importantly friendly power. If the AI has like 2 cities left and a third of the score of their ally, they should probably request it because the alternative is that they die horribly.

 

[Recursive]

Those levels of penalties are very similar to what I have in my current game. Not sure the next changes will drastically fix this as all close neighbors and cultural civs will stay at this level, if I understand correctly. Median vs average won't change a lot.

I'd like those modifiers to be cut by half, just to give an order of magnitude. Maybe the -20 base modifier should be toned down to -15 or -10.

Curious to see what others think about it.

Alright, I think I have a solution. The new penalty will be based on the difference between the median plus 3 (the "wonder spammer threshold") or the difference between the civs, whichever is smaller. It doesn't make sense that the penalties appear instantly in a large amount, but it does make sense that they should scale aggressively if the player continues to spam World Wonders, to prevent runaways.

int CvDiplomacyAI::GetWonderSpammerScore(PlayerTypes ePlayer)
{
   int iOpinionWeight = 0;
   if (IsPlayerWonderSpammer(ePlayer))
   {
       iOpinionWeight += /*20*/ GC.getOPINION_WEIGHT_WONDER_SPAMMER();

       int iCivDifference = GET_PLAYER(ePlayer).GetWondersConstructed() - GetPlayer()->GetWondersConstructed();
       int iMedianDifference = GET_PLAYER(ePlayer).GetWondersConstructed() - (int) ceil(CalculateMedianNumWondersConstructed());

       // For scaling, go with whichever value is smaller
       int iWonderDifference = min(iCivDifference, iMedianDifference);

       if (iWonderDifference > /*4*/ (GC.getWONDER_SPAMMER_THRESHOLD() + 1))
       {
           iOpinionWeight += ((iWonderDifference - /*4*/ (GC.getWONDER_SPAMMER_THRESHOLD() + 1)) * /*20*/ GC.getOPINION_WEIGHT_WONDER_SPAMMER_PER_WONDER());
       }
       if (IsCultural() || IsConqueror())
       {
           iOpinionWeight += /*20*/ GC.getOPINION_WEIGHT_WONDER_SPAMMER_STRATEGIC_MOD();
       }
   }

   return iOpinionWeight;
}

Or in plaintext:
Base penalty: 20
Cultural/conqueror civs: +20
Difference of 3 to 4 Wonders: no extra penalty
Difference of > 4 Wonders: +20 per Wonder above 4

So it'll be lower at first (20-40 when it first appears, not 60-100+ ), but scale more aggressively if you ignore it after it appears. The intent of the penalty is to prevent a single civ from building a huge Wonder lead without diplomatic repercussions, much like you can't settle everywhere or conquer the world without diplomatic repercussions.

These repercussions will be more localized however, with the new changes.

You can build Wonders, but you can't build Wonder after Wonder with impunity. So choose wisely!

 

[Recursive]

Personally, I would very much like to see a severe reduction in defensive pacts. In my games, I've noticed a lot of cases of people having defensive pacts that they cannot enforce due to distance. People agreeing to defend eachother while a world away because they're friends with good military. In my experience, it makes the game become much more passive later in the game for AI v AI combat and also makes it nearly impossible to convince AI's to declare war with you even if there's no reason why the other member(s) of the defensive pact could even reach you. Little hard to imagine them sailing around a continent and across the ocean to do it, but the AI considers it a major threat that prevents war.

The AI should only make a defensive pact if the person they are pacting with can be reasonably expected to reinforce them in a timely manner imo.

-----------------

Another change I think would be beneficial is that AI's who have basically been destroyed should seek to become vassals to a strong, nearby, and most importantly friendly power. If the AI has like 2 cities left and a third of the score of their ally, they should probably request it because the alternative is that they die horribly.

Re: Defensive Pacts - I'll look into this.

Re: vassalage - that's already under consideration, but will have to wait until I redo interaction/deal logic.

 

[Stalker0]

With defensive pacts, it may be similar to just reduce their duration. Its what 50 turns? That's a hell of a long time, global politics can completely change in that span. Even a 20 turn pact is a pretty big deal.

 

[tu_79]

You can build Wonders, but you can't build Wonder after Wonder with impunity. So choose wisely!

As long as the AI messages make it clear that they all hate you because of your world wonders, I'm ok with that.

 

[Recursive]

As long as the AI messages make it clear that they all hate you because of your world wonders, I'm ok with that.

The opinion modifier table is your friend.