AI War Approach and Trade Routes

I decided to start a new thread related to this hotly debated issue because the new perspective after the release of the DLL deserves its own thread to avoid getting diluted into an already very long and somewhat rant-like thread. I copy all the posts I made below, hopefully to make the new debate more clear and fresh.



That block of code comes from the BNW CvDiplomacyAI.cpp, and is one of the most relevant changes in that file as compared to G&K. It may explain a lot of what is being reported by some here. We may agree or disagree on this particular approach, but look into the code, and the comments ("sanity check" ) and draw your own conclusions.

Now first thing that seems calling for testing is to play without having any trade route with neighbours, and see what happens then... did anyone test that particular scenario?

EDIT: carefully consider iDeltaGPT. For the less inclined to C++ endeavours, basically what that formula says is: "the more negative the balance will become if I loose these trade routes with this particular potential target, the less likely I am to go to war".

Maxym said:

It doesnt matter how strongly they benefit by trading, only if they can have a reasonable chance of winning the war? If so trade should be sacrificed to gain lasting advantage, yes meybe losing friends will hurt them but they don't need friends to trade just not war. If they have large, technologically superior army and player has three archers not attacking him is plain silly.

Click to expand...

Or is it? Debatable. With the above piece of code much more rational debate and conclusions can be drawn, regardless of if we agree or not. The devs defined this as WAD (so, not a BUG, finally) ("sanity check" ), and there are strong arguments in favor of that. How long will it take to replace those routes? Is there another opportunistic AI lurking in the vicinity? If my GPT goes negative after I declare war on this pesky human, how can I sustain the army for enough time while other caravans are being built? If I cannot sustain such army, even after defeating this pesky human, will the other neighbours take advantage of such situation and jump on me? Etc etc etc.

This also explains why the more we advance into the game, the more likely it becomes for the AI to declare, and they do (and a lot): trade income becomes less and less relevant in that formula, so it decreases the GoToWar weight less and less with time (as long as the player has developed its economy well enough).

Interesting stuff.

Buccaneer said:

Kind of goes back to a fear I mentioned last week - are we able to easily pacify an aggressive military civ simply by trading through DoF? If they are delaying warfare until end game, how will they be effective without promoted units? Aren't there about 10-12 civs that would be really good at warfare (beyond just Zulus and Huns) that are being neutered?

Click to expand...

Well, trade routes are limited; for the iDelta modifier to become meaningful, the GPT proportion coming from trade has to be really high as compared to total GPT, thus implying that most/all trade routes are concentrated in one (maybe two?) relations... that's why we see the AI's warring between themselves a lot WHEN we actively engage in trade, especially with the dangerous neighbour (and remember, trade routes are also limited in range, and in the early eras, it's only the neighbours we have to trade with.

In other words, the block of code shows that trade is indeed a powerful tool, and more so in the early game. We can debate about the sweet spot on that approach now that we have the code, and some adjustment may or may not be justifiable, but I don't think the "o-no-you-builders-ruined-my-game" whine is valid anymore... if it was at all.

pilot00 said:

Another thing that buggers is me, is why would the AI would DoW Morocco (as an AI) for example when it more lucrative to maintain trade relations with it and consider only its trade relations with the human player?

Click to expand...

I do not see any differentiation in the code regarding human or AI... so, that block of code acts exactly the same for any player in the game. Granted, there are other modifiers (plenty) that affect the "Go To War" weight, but I couldn't see any big change in those from G&K... I might have overlooked something, because I quickly went over the file, and the file is huge (and there are others), so the whole package calls for a thorough investigation for the curious.

What I wanted to show if that there is a rationale behind what some are seeing, and that it is WAD. Not a BUG, in other words. We can continue the debate now in terms of how good/precise/appropriate the design decision was, and how to change it if necessary. In other words, I wanted to kill the whine and move this to a more rational ground.

Also, see the comments "To Do: constant/XML", clearly pointing to an eventual extraction of some of the components of the modifier from the hard code --> modding. I wonder why the did not do that right from the beginning... I am sure it will come with the first patch, especially now that they are seeing some reactions.

viApproachWeights[] is a vector of weights defining how each AI will "approach" any other player... whatever is inside the brackets is the index of the vector, and there is an index for each of the possible approaches (WAR, HOSTILE, FRIENDLY, DECEPTIVE, etc). The diplomacy code is truly complex, and takes into account many different factors (and even more now with all the new systems). In the case of the piece of code being analyzed, MAJOR_CIV_APPROACH_WAR is the index that points to the "cell" of the vector that holds the weight of the "WAR" decision (approach); each of these weights is modified by many actions/situations/history/treaties/etcetcetc, and the modification is stored in the corresponding cell.

So, in this case, whenever the GPT of the AI is positive, and the GPT without the trade routes to that target is negative, then the weight for the approach WAR is reduced by that amount (summed but is a negative value according to the IF condition). In simple numbers, say the GPT of the AI is 100, and the total gold of the trade routes to/from the target is 120, then the weight for the WAR decision will be reduced by 20.

Later on, the vector gets sorted by highest weight, so that the highest weight at the end of the calculations is chosen. If the highest weight corresponds to the index MAJOR_CIV_APPROACH_WAR, then WAR is the decision. More or less, that is how it works.


EDIT: all that has been extracted and affects this calculations can be seen/edited inside the file GlobalDiplomacyAIDefines.xml. To analyze the specific war decision, every variable that starts with APPROACH_WAR is related in some way to the algorithms that modify the weight of index MAJOR_CIV_WAR_APPROACH. The file is located in


...\Steam\steamapps\common\Sid Meier's Civilization V\Assets\DLC\Expansion2\Gameplay\XML\AI

eric_ said:

To break down the crucial part of the code, here's how I read it:

Code:

// sanity check - if we will go negative from war with this player, don't go to war

This threw me at first, as it sounds like it's staying, if revenue goes negative, don't war. But, I'm pretty sure it's saying something else. See *** below

Code:

int iGPT = GetPlayer()->calculateGoldRate();
int iDeltaGPT = iGPT - iCurrentTradeValue;

iGPT = player's overall gold per turn
iDeltaGPT = the change in gold per turn if the AI goes to war.

Code:

if (iGPT >= 0 && (iDeltaGPT < 0))
{
viApproachWeights[MAJOR_CIV_APPROACH_WAR] += iDeltaGPT;
}

This is the crux of it all. If GPT is not negative AND if iDeltaGPT IS negative (i.e., going to war will reduce gold), then add iDeltGPT to viApproachWeights[MAJOR_CIV_APPROACH_WAR] (which I take to be a constant for a given civ's likelihood to war with a major civ). If iDeltaGPT is negative, then adding it to viApproachWeights[MAJOR_CIV_APPROACH_WAR] will cause the latter to be reduced.

My guess is the sanity check means *** If adding iDeltaGPT (reduction in GPT due to DoW) to viApproachWeights[MAJOR_CIV_APPROACH_WAR] (Civ's likelihood to war) causes the latter to go negative, then don't go to war.

The viApproachWeights[MAJOR_CIV_APPROACH_WAR] is very likely different from civ to civ, and, iirc how flavors are handled in Civ V, even different for the same civ from one roll to the next...so that comes to bear on how much of a drop in GPT it takes to deter a DoW in any given situation.

Click to expand...

OK, here is something for all you warmongers that want to burn the world in 3000 BC...

I may have found a way for you to try to get some more wars without touching the dll code. If my understanding is correct, and I am not missing something else, this may work for you but will need some testing. Look at the following piece of code:


This block is the last before the approach weights vector gets sorted; the code in the file runs for every turn, and apparently in strict order. In other words, after all modifications to the weights have been done, the Difficulty level of the human player is fetched, and one specific value is used to adjust the final weight for MAJOR_CIV_WAR_APPROACH and MAJOR_CIV_DECEPTIVE. That value is read from the Civ5HandicapInfos.xml file inside

...\Steam\steamapps\common\Sid Meier's Civilization V\Assets\DLC\Expansion2\Gameplay\XML\GameInfo

The value read is the variable <AIDeclareWarProb> </AIDeclareWarProb> for the corresponding difficulty level; if you read that file, you will notice that the value increases from zero (Settler) to 100 (Prince and above). The value is then applied as a product to the weight for WAR, and divided by 100 (percentage). In other words, in Prince and above the weight that resulted from all the modifications stays the same, and below Prince it gets reduced to a certain percentage (and is zero for Settler, thus Settler has no AI war declarations ). That is the iWarMod argument in the code above.

Now, if this is correct and it works as I think it does, then you just need to adjust the values of <AIDeclareWarProb> </AIDeclareWarProb> for King and above in your HandicapInfos file to something bigger than 100. I cannot guarantee any result from that, but if you miss war so much it is worth trying. I would recommend the following:

1) COPY your original file first as backup.
2) Increment the value in 10 points per dif level and see what happens
3) If nothing happens, add some more (15 per level?)
4) Report back.

I will not test it as I am still fine with how it is working for me, and if I decide to change something I would work on the dll. But this approach, if it works, is really fast and simple enough for anyone to modify their own game without waiting for magic.

Test it, and let us know if it works and how. I would expect imbalances as a result (for example, you may have more early wars, but the mid-end game might become a troll festival), but it should make for a fun experiment (with very little risk).

On another note, and related to the specific trade code, the first thing that comes to my mind is that using the absolute gold value of the projected deficit in the GPT might be disproportionate. The key word is proportion; maybe the reduction in the weight for MAJOR_CIV_WAR_APPROACH should be a proportion of something instead of the absolute value; not sure what something, but I am just brainstorming.

For example, if I had cash reserves of 2500, and the projected deficit of a declaration of war against the AI is -50, would I go to war? If the prize is nice, heck yes! because I have savings for 50 turns of deficit. The AI right now is not doing that, it is just applying the -50 to reduce the weight. Right or wrong? I don't know, perhaps the word is incomplete. It should test for deficit ("sanity check"), but perhaps for more...? Cash reserves? Proportion of science points (as the deficit after ZERO cash gets deducted from there)?

Food for thought.

I think one of the biggest problems is people are trying to use the AI going to war for their own gains. "I want to war, but don't want to declare!"

I see a lot of people complaining the AI doesn't declare in the first 50 turns but you know what? I wouldn't either. And this code just makes that this much more clear. The AI should do what is advantageous to it. If it will destroy its economy by warring early, it should probably avoid it, just like I do. This is completely rational thought.

Instead people seem to want the more warlike civs to destroy themselves by declaring very early on, before a lot of key infrastructure is up. Does crippling itself for the rest of the game really make the AI better to people?

Now there could be some alternatives and obviously there are, since I have been declared on early in the game by Alexander. I have also seen Genghis in the past 2 games never bother building settlers and just conquering the 2 or 3 nearest CS. Perhaps if the AI is contemplating war, they should switch their trade routes to a slightly less cost effective route, probably a CS, but then they can do their war. This may be a good alternative.

People just have to keep in mind that AI is programmed by a human who thinks "How would I react?" and "How can I make the AI react as close to a human as possible while still having the game be fun to play?"

I personally don't mind if the AI doesn't declare early that often, especially if they have gone through some process and determined it is not cost beneficial to them at the moment. The point is they do occasionally, so its not an inherent problem.

If you want early war, you can always declare yourself but I have a feeling quite a few people wouldn't think it advantageous to them that early.

I agree with you, Yzman, and that was my position from the beginning in the other thread, but I took this new perspective out to avoid any continuation of the ranting/flaming and to focus more on the objective code that is now available, and on the possible rationale the devs used to design it that way (and maybe exercise some positive criticism that may or may not make the system better). So let's just discuss the code and implications, yes?

Hah, I was just thinking the same thing - perhaps the AI is so reluctant to go to war now because doing so would plunge its economy into a recession due to its trade routes being cancelled.

I honestly see no problem with it. Maybe early aggression needs to be raised somewhere else, but this particular part of the code makes perfect sense to me. And really when I think about it, this is likely only going to affect the AI very early game when you may be their only trade partner and they have no infrastructure to give them gold.

Once again I think to myself. Would I go to war if it puts me into negative? I can't remember the last time I was negative in a game past the early parts of the game so I don't feel this affects late game at all.

So lets look at early game. I have 2 ways that could perhaps "improve it" I don't know whether the AI is already doing this, so I can't say it would for sure.

1. Have the AI do a check at the end of its turn that it wants to war to see if it could change those trade routes to another civ to make up for the deficit. If it can, switch them at nearest opportunity even if there is a slight cost and then proceed with war.

2. Have the AI consider gold on hand. For example, in the early game you may have 100 gold. Lets say going to war will take you down to -5 gpt. Is it reasonable to go to war? This is an opinion and I am not sure what the correct value would be. The problem is, you don't know how long a war will be so I can't say " I have a surplus for 20 turns to last me" when the AI logically will not know how long that war will last. So this would be tough but maybe if its a reasonably small % of gold on hand and the AI weights the war as very favorable, it would proceed anyway.

Just my thoughts.

I can't follow all of this, not being a programmer, but it makes sense that there'd be a fail-safe to keep the AI from a disastrous drop in income. Namely, it makes sense to program a clause to the effect of

I won't DoW this player if the result is a loss of X gold within Y turns

where X and Y may vary depending on any variety of factors (AI personality, ease of rebuilding trade-units, etc.)

This is somewhat what Aristos says in the post right before Yzman's first post. The difficulty is in setting the table of variables. There's no clear way to predict what will happen after the first turn of the war: will the player attacked try to draw in another player? Will the war result in sufficient losses for another player to jump in and try and overwhelm the distracted aggressor? These result in altering the possible trade-routes and recovery of GPT only after at least several turns of war. To me, this is the crux of the issue: getting the AI to try and make some prediction about the imminent yet not immediate consequences of going to war, and how this will change the GPT situation after 5 to 15 turns of war or whatever.

***

A human is indeed behind the decisions, as Yzman said, since the programmers/designers ultimately set the variables to what they consider 'human-like', and innately we deem something 'human-like' in how we would react ourselves in many cases.

Because the consequences of going bankrupt are reach such high negative magnitude (auto-disbanding, beaker-loss, etc.), it's safest to be conservative and not let the AI DoW if the GPT loss is even simply modest-to-meager. This is amplified when the decision is to DoW a human player, as even us somewhat-decent players can manage to beat an AI that, on paper, has a much better military.

***

War is a tough variable in itself. The AI is not going to be able to calculate force-multipliers (Great Wall, rivers, hills, certain promotions) as well as a human. Even when calculating these while considering attacking another AI, the ability to algorithmically check every possible tile/unit combination for the era across all tiles and all unit-types in the likely battlefield is not incredibly accurate. Because of all these things, the AI has a very hard time predicting how quickly, or even if, it will likely recoup it's GPT losses by conquering sufficient land, forcing a lucrative peace deal, or taking a vital wonder. And this, too, is a significant hurdle.

***

Lastly, at the end of the day, the better the coding for major decisions like this, likely the longer the turn waits. To constantly check all these variables for each AI in the game would slow down the game perhaps too greatly for either everyone, or the vast majority of players, due to hardware constraints. There are trade-offs for improvements in the AI on any issue, and too many improvements leads to too much slowing.

Now that there's trade-routes to calculate, World Congress decisions to make, etc., the AI is already moving slower and many people have noted slower turns in BNW. So, again, even if some ideal set of code exists to really even out war-peace decisions and other major AI choices, there's also the slow-down this produces, and at some threshold players will begin to get bored or lose interest if too much inactivity of the player results.


All just some thoughts. Again, not a programmer, but interested in the logic here probably about as much as you guys

Incidentally, I suppose this also makes getting your friends into a war with you much more important now. In G&K I rarely bothered asking my allies to declare war on a common enemy, as their price was rarely worth the benefit they'd offer. Now, even if they just sit the war through, the benefit of the enemy being denied yet another trade route destination may be worth it alone.

It would be nice if this thread stays on topic of testing the various parameters in code and discussing relations between trade, income, diplomacy and DoW, while the thread They need to hotfix AI agression now be used for general discussion on non-DoWing and the usual rant.

Yzman said:

1. Have the AI do a check at the end of its turn that it wants to war to see if it could change those trade routes to another civ to make up for the deficit. If it can, switch them at nearest opportunity even if there is a slight cost and then proceed with war.

Click to expand...

After re-reading and considering, this is the biggest issue, and greatly what I was getting at before.

The AI should try to do this as Yzman said, but at the same time it needs to calculate the likelihood of the new trade-route destinations being blocked (should the defender counter-attack, or ally currently neutral CSes, or draw in the destination's owner into the war). And this is a complicated calculation in itself.

Further, the AI is always not great at risk-reward analysis, either:

Yzman said:

a slight cost

Click to expand...

is relative to the possible or likely gains:

- A human, knowing they have a reasonable chance of taking a city with a wonder currently crucial to them, will risk losing more than an AI would almost every time. Because even if the cost is to lose all existing gold, a human knows there's only such-and-such a window to take something, as well as how much chance they stand to succeed without it.

- An AI can be pre-programmed to value certain things, but is often poor at reacting to the relative value of things as conditions change. Thus, for example, setting Siam or France or another culture-oriented AI to always prioritize attacking a civ that owns the Sistine Chapel is not necessarily a good idea, as these civs may not always be in a position to really use it as well as they could use another wonder, depending what other wonders they already built or what terrain they have (or have lost). We already had some of this in G&K, where 'coveting wonders' was added. The affect was not noticeably superior to anything already done before.


The TL;DR version is: the AI is not currently reliable in its ability to determine its chances in recouping gold, GPT, or important land or wonders. Because of this, it's hard to have the code for the GPT-loss-threshold be all that dynamic, as the AI simply doesn't have time/processor-power/etc to determine how DoWing someone may or may not affect alternate trade-routes, possible peace deals, etc. (getting back to a lot of which I brought up earlier). The AI can be fairly well programmed for static, non-relative valuing of things, but struggles to do what most humans do fairly well, which is, to be programmed for dynamic, relative valuing of gold vs food vs faith vs land vs wonders vs the risk of trying to get any of those things, as well as which are worth more risk than others given various fluctuating factors.

Yzman said:

1. Have the AI do a check at the end of its turn that it wants to war to see if it could change those trade routes to another civ to make up for the deficit. If it can, switch them at nearest opportunity even if there is a slight cost and then proceed with war.

Click to expand...

This is not hard to code, but given the model, almost impossible to implement. Remember that once set, trade routes have a fixed duration (and I would not change that as it adds a strategic level to it: you have to carefully consider with whom you set trade routes because they are not breakable at whim). Add to that the spread out starting time of each trade route, and you can see that the check would work and basically stop all war; the AI waiting for the routes to cancel, then reassign them, then war. By then, the entire scenario will have certainly change radically.

ahawk said:

The TL;DR version is: the AI is not currently reliable in its ability to determine its chances in recouping gold, GPT, or important land or wonders. Because of this, it's hard to have the code for the GPT-loss-threshold be all that dynamic, as the AI simply doesn't have time/processor-power/etc to determine how DoWing someone may or may not affect alternate trade-routes, possible peace deals, etc. (getting back to a lot of which I brought up earlier). The AI can be fairly well programmed for static, non-relative valuing of things, but struggles to do what most humans do fairly well, which is, to be programmed for dynamic, relative valuing of gold vs food vs faith vs land vs wonders vs the risk of trying to get any of those things, as well as which are worth more risk than others given various fluctuating factors.

Click to expand...

The code for the AI is not that bad now; they certainly put a lot of effort to clean up part of the initial vanilla mess. It will certainly need balance, but the code is there. For example, some of the modifiers for MAJOR_CIV_WAR_APPROACH check the probability of success AND the degree of success, and assigns modifiers to the weight of the WAR decision accordingly. That is pretty good; I still need to dig into the calculations of the probability and degree of success, but the modifiers are there and that is pretty amazing. Same with many other modifiers, they are there, some of them I didn't even imagine before checking.