To the rescue! Healing the AI

Based on the discussion started in this thread (in the RaR subforum) I have created this thread here to discuss topics related to improving the AI which are not mod-specific.

I will first quote two postings from there to get you familiarized with the current state of the discussion.

Nightinggale said:

Originally Posted by Commander Bello View Post
Actually, I think combined efforts of modders would make more sense in the area of the AI - as it is in a terrible state, anyway.
That would be something from which all of us might gain a profit.

Click to expand...

I too would like an improved AI. However improving the AI is quite unspecific. You might as well have said "I want to make the game better". The main question is how and who have the coding skills to do it? Coding the AI is quite hard.

I would like the AI not to cut down all forests, but I haven't figured out even where to start.

Click to expand...

Well, I don't know if it really is the coding part which would be so difficult. Actually, I think it is the concept which holds the most problems.

Let me give you some examples:
1) First example
There's a function AI_updateYieldValues (something like this, would have to look up the correct naming and placing) which loops over all yields to either value them with European buy or sell prices. Later it loops over all cities (of the player) and over all professions and tries to assign half of the estimated value of final products to the ingoing raw materials.

Here I see many parts which should be improved.
First of all, one may very well discuss if yields shouldn't be valued with the price for rushing them (emergency replacement costs, or whatever you would like to call it).
Second, looping over all cities is questionable. Even it one would regard it necessary, there's a CvCity::doTurn(), switching to various other functions in which most probably the looping over professions is already done. So, why not placing the valuation at a proper place there?
Third, and most important, as far as I have seen from short browsing over this function it doesn't take into consideration build orders, planning (Revolution_Preparation and so on, or however they may be called) nor current peacetime/wartime conditions.
Fourth, it may be nice to add value from clothes to cotton (just as an example), but that doesn't help us (better: the AI) at all, especially not in the vanilla environment of randomly changing prices for randomly selected yields.
If I have the capacity of producing say 6 items of clothes per turn, this doesn't make my capacity of producing 24 cotton more valuable. Not a bit. The only thing I should care for is to have at least the 6 needed cotton.

So, here I see big flaws in the valuation "concept" already (and the coding of course may be improved, too, as indicated above).

2) Second example:
At least when fighting the Natives, all you need is a little army of say 5 cannons and some infantry. March from native settlement to native settlement, bomb down the defences, kill the defenders, collect some treasure. Rinse and repeat.
The AI won't do anything against this approach. It will happily stay sitting in the settlements and get killed.
In fact it should check if there are units with city attacker capabilities in the attacking stack and counterattack this stack before it can start bombing the city.
On a sidenote: there aren't any incentives to make peace with natives, to make things worse. Latest after having destroyed the first settlement, the natives will be that furious that even after having made peace a new war at least will be quite likely. But since you've made peace, they are swarming across the countryside again and may be next to your own settlements then.
So, the best option just is to completely kill them. Genocide, anyone?
And as added bonus you now can happily spread your culture over the territory, as there isn't anyone left to complain.

3) Regarding you question about the chopping:
I currently don't have the slightest idea, either.
But my approach would be to count all lumber providing tiles in all city radii and their potential output (once again, CvCity::doTurn() comes to mind). Check this against the assumed demand for lumber (add some percents on it to stay on the safer of both sides) and now redistribute a "need for lumber tiles kept" to each city.
Then, when the AI assigns professions, it has to check this number of lumber tiles kept not to destroy a forest which is needed.

So, we already have three topics which might be worth looking at, no?
And I am not saying that my assessments are perfect... Others may have other ideas and may have identified better options for improving the AI.
But as I indicated above already: the most important part is the concept behind the AI - and that means that the modders need to have a proper idea, first.

Update:
Since Nightingale mentioned it, I add another example:

Commander Bello said:

The AI is very weak in assigning military professions. A typical behaviour is sending a promoted military unit to a city, dropping all weapons and working as field worker or whatever (I have even spotted general-lead troops as pioneers or worse, as missionaries in the past). As soon as the need for a military unit arises, it takes the unit which it thinks it can spare - which in many cases is not the military unit.
In the end you may have the promoted unit work somewhere and a weaker unit do the defense (in the worst case).

Click to expand...

I looked into the forest removal issue. I figured that since it is done by a unit, it must be in CvUnitAI.cpp and started searching that file for function names, that it would have to use (such as canBuild) and found the code, which handles this.

It goes as follows (greatly simplified)

1. AI pioneer calls CvUnitAI::AI_improveCity(CvCity* pCity)
2. Pioneer moves to a plot next to the city
3. The pioneer builds whatever comes from CvCityAI::AI_getBestBuild() (if possible)
4. AI_getBestBuild() returns a cached value
5. The cache is set by calling CvCityAI::AI_bestPlotBuild()

One thing, which strikes me right away is the cache in AI_getBestBuild() is hardcoded to one plot radius.

The cache generation has the issue that it view the plots individually. If it makes sense to clear a forest, it will not detect that it is the only forest left. On the other hand, if we increase the AI value of lumber to make it keep the forest, it will not detect that there is nothing but forests around the colony.

The bonus resource is handled a bit oddly. It is less likely to remove a forest if there is a bonus. However some bonuses needs the forest to be cleared to be useful. Logically the best improvement for a bonus plot would be to build the best improvement possible for the yield, which is supplied by the bonus. I don't see any code like that. At the same time prime timber isn't mentioned either. It can clear forests with those bonuses.

One thing we could consider would be to make the AI use feeder service together with lumber. That way it can actually figure out how to supply lumber to colonies placed on the prairie. The design of feeder service is actually quite AI friendly.

I will try to check with the code tomorrow, now that you've directed me where to look

Added the comment about military professions (as per Nightingale's recommendation) to the first posting.

About enhancing AI Feature removal and Improvement construction choices, I'd agree that the best overall approach is likely to iterate through the possible Feature+Improvement combos for that tile (making sure to consider effects of Bonus, River, Road (and Tech/Civics for M:C) which apparently isn't the case in vanilla) and aim to improve to whatever gives the best overall "value" (which I'd guess is the intent of AI_getBestBuild() ).

How that best value is determined is up for discussion; but perhaps something along the lines of taking a weighted average of the values of the top 3 yields produced in highest amounts by that Feature+Improvement combo.

This could be adjustable by the existing tags to be fully configurable however modders would like (for example if you did want to make a special case for yields like Food or Lumber):
<iAIWeightPercent>
<iAIBaseValue>
<iNativeBaseValue>

The yields with the highest price * production could be weighted a bit more, since the best choice for producing from the tile will ususally be the yield with the highest price * production, and the other choices will often be less optimal and be selected relatively rarely. For example, if a given tile configuration (such as Forest+Logging Camp) gives 3 Food, 7 Timber, and 1 Ore, then its value estimate could be
7 * (timber price) * (timber ai weight)
3 * (food price) * (food ai weight) / 2
1 * (ore price) * (ore ai weight) / 3

It might be possible to incorporate an adjustment for the Yields currently demanded & used in production by that colony, but OTOH that seems likely to have bad long term side effects. (ie at any given moment that would be highly dependent on what colonists happen to be working in which professions at that point in time, which might not necessarily be optimal choice given the potentials of the local terrain). For example, if there happen to be a few Weavers producing Cloth the AI could be fooled into dedicating land to Cotton production to meet this demand, and ignoring the potential to instead take best advantage of the local terrain. So the best overall approach is likely to independently optimize the value of each tile to its "highest and best use", and then the AI can appropriately plan citizens/professions/buildings to take best advantage of the local terrains. (in fact this kind of approach is how economic and development planning actually takes place in most capitalist or even non-capitalist societies).

You could also try to add an adjustment factoring in production of surrounding tiles, if you want to ensure some diversity to local production. However as NG points out, if the AI can use imports appropriately then it would be a perfectly valid choice to chop all forests in a central colony and dedicate the land to whatever can be produced best there, if you can import lumber from the frontier. If local yield prices can adjust based on relative supply/demand, then the impacts of supply/demand and yields being consumed in production would also get automatically taken into account when planning improvements / builds, because it already will have been naturally incorporated into local prices. This would make an ideal way for the AI to naturally take into account consumption and uses of each type of yield when planning production.

About improving the military AI, I don't know much about this but it could be worth looking at BtS Better AI to see if there's anything that could be ported over.

I was wondering if anyone has made any progress in "healing the AI."

The "feature removal" thing has haunted every game of RaR I've played. It might seem a minor annoyance at first glance, but it effectively prevents rival colonies from developing at all beyond a certain point. If you play testgames on small maps, you will see that they eventually reach a complete standstill, where almost nothing is produced anymore, and nothing happens.

I think the AI is probably thrown off balance by the addition of all the new resources, which will return much higher values for unforested grasslands (suitable for Food, Hemp, Tobacco, Barley, Horses and Cattle) than for forested grasslands (suitable only for Lumber and Fur). That is also why Taiga is left alone, as well as Jungles on Marsh.

I haven't played TAC much. Do any of you know if the problem was already present back then?

agnat86 said:

I was wondering if anyone has made any progress in "healing the AI."

Click to expand...

I am working on it, but since I am just so weak in my coding skills, it is very frustrating and sometimes I just stop out of disappointment.
But at the moment I am trying to improve the combat logic at least a little bit.

agnat86 said:

The "feature removal" thing has haunted every game of RaR I've played. It might seem a minor annoyance at first glance, but it effectively prevents rival colonies from developing at all beyond a certain point. If you play testgames on small maps, you will see that they eventually reach a complete standstill, where almost nothing is produced anymore, and nothing happens.

I think the AI is probably thrown off balance by the addition of all the new resources, which will return much higher values for unforested grasslands (suitable for Food, Hemp, Tobacco, Barley, Horses and Cattle) than for forested grasslands (suitable only for Lumber and Fur). That is also why Taiga is left alone, as well as Jungles on Marsh.

I haven't played TAC much. Do any of you know if the problem was already present back then?

Click to expand...

The main idea of RaR, to put in so many things hasn't made it easier for the AI.

In fact, in TAC many games were much more interesting in the early to the middle game, as far as I see it. This of course is also caused by less native tribes in TAC (and therefore, in turn each of them being stronger). Furthermore, in TAC you had wars amongst the natives which gave some additional diplomatic options.

For both, TAC and RaR there is one general pattern, though: as soon as you've managed to survive into the middle game, you will win. Period.
There may be differences from game to game which founding fathers you get, and how early you can concentrate on building your army and navy, but that's it...