[AI behavior] Improvement decisions

[Padmewan]

We are running into some interesting AI issues related to workers not building certain improvements in our mod (see sig). Like the Moon mod, our world starts with 0 food on all terrain, so the AI is very hungry for food in the beginning. There are two improvements that can provide food:

1. Hothouse, +1 food, +1 with default civic
2. Terraform, +2 food, -1 hammer

The tech for Terraform comes within 2 techs of Hothouse.

In my playtesting I am finding that the AI refuses to build the Hothouse, but will (over)build the Terraform. I have a few theories for this:

(a) The AI "sees" the Terraform coming up, and since it considers it "better" than the Hothouse, refuses to build now and waits for that "better" improvement.

(b) This is because the AI is programmed NOT to destroy existing improvements. Thus, even though it SHOULD build a hothouse now and then decide later whether to replace it with a Terraform, the programmers removed this thought pattern.

(c) The AI doesn't see the +1 food from the civic and sees the Hothouse as only +1 food, not +2 food.

I know there are experiments I can run to verify/disprove at least (a) and (c), but can anyone who has playtested extensively and/or is familiar with the AI programming chip in with some explanations for what we are observing in AI behavior?

(Has anyone ever seen the AI replace an elephant camp with a cottage in vanilla Civ after Ivory goes obsolete?)

 

[NikG]

After having gone through hundres of lines of code in the SDK, I may have a solution, I am not 100 % sure, but I think I have it. There are just 1000 functions calls to this and that, and trying to determine the function in which a function calls bla bla....

It is a combination of a and b.

When the AI determines what to improve and with what, it does not take the civic into account. Only when it assign new citizen it will take civic into account.

Hope it helped.

 

[Padmewan]

That is very helpful; thank you!

I suppose to account for not considering civic changes, leaderheads with a particular leaning towards a civic are specially coded to also prefer that improvement. This suggests that a starting civic should not offer any bonuses, and that we shouldn't rely on the AI "knowing" about civic changes. (We had wanted different lifeforms to get different values from different improvements, but this will be much harder to model now without delving into the SDK. Oh well).

 

[dsplaisted]

That is very helpful; thank you!

I suppose to account for not considering civic changes, leaderheads with a particular leaning towards a civic are specially coded to also prefer that improvement. This suggests that a starting civic should not offer any bonuses, and that we shouldn't rely on the AI "knowing" about civic changes. (We had wanted different lifeforms to get different values from different improvements, but this will be much harder to model now without delving into the SDK. Oh well).

Do you guys not have anyone who can do C++ SDK work?

 

[Padmewan]

Do you guys not have anyone who can do C++ SDK work?

No, not really. I was willing to take it up at some point if necessary, but Python's been challenging enough. Are you volunteering?

 

[woodelf]

Do you guys not have anyone who can do C++ SDK work?

I'll throw in my "are you volunteering?" as well.

 

[Chalid]

I actually plan to review and partially rewrite the SDK-Improvementcode sometime in the next two weeks and want to add in a dependence on the state religion (thats for FfH), so if i'am done with this i'll let you know.

 

[woodelf]

I actually plan to review and partially rewrite the SDK-Improvementcode sometime in the next two weeks and want to add in a dependence on the state religion (thats for FfH), so if i'am done with this i'll let you know.

War machines and SDK coding! We need to lure you away from FfH somehow....

 

[NikG]

If needed I can quickly fix this problem so the AI would consider tile with civic bonus. WOuld take me 10 minutes. Can then send the DLL to you.

 

[Chalid]

I think this should take you 0 Minutes ... its already in...

look at:

CvPlayer::processCivics
CvPlot::calculateImprovementYieldChange

 

[dsplaisted]

No, not really. I was willing to take it up at some point if necessary, but Python's been challenging enough. Are you volunteering?

Well, not at the moment. Right now I don't really have the time. I am working on a pluggable AI system which will allow you to override the default AI behavior but still have the option of using the original behavior easily. Then of course I am going to Brazil for two weeks starting next week. After both of those things are done I might be available to help you with the odd SDK improvement, but it looks like there might already be people willing to do that for you.

 

[NikG]

Chalid yes, but the way I have interprented the code, the AI doesnt take civics changes into account when determine which plot to improve, something like xxx::calculateNaturalYield or something cant remember, I am at work right now, so I cant check it. Here it doesnt calculate any possible civic changes, and and this is called when trying to determine which to plot to improve. However I am not 100 % sure, so try check it out.

 

[Chalid]

I have checked it out. I considers Civics for sure.

It finally comes to CvPlot::calculateNaturalYield + CvPlot::calculateImprovementYieldChange

And the calulateImprovementYieldChange gets a ImprovementModifier-Variable from CyPlayer - when i remember correctly - that is changed in CvPlayer:rocessCivics.

Nevertheless it only calculates the best field with the civic it is using at the moment, and so all the choises get probably invalid on civic changes and of course civic Changes in the near Future are not considered.

 

[Padmewan]

Krikkitone has suggested a workaround that could work if the AI does account for civics and tech changes, which is to link the newer, better improvement to a tech advance. E.g. the improvement provides +0 yield at first, but with a new tech that happens to be the same tech that enables its building, it produces +3 yield or whatever, making it at first less attractive but later more attractive than the earlier improvement.

As a workaround this should let us move forward without twiddling with SDK... Though I appreciate all of this deconstruction, agree with everyone that the Improvement AI can be tweaked, and hope you'll continue to help us SDK innocents as we hack our way through this wonderful game...

 

[NikG]

Okay Chalid, thanks for update. Will check it my self tomorrow, going to bed now. But were does it calls CvPlot::calculateNaturalYield and CvPlot::calculateImprovementYieldChange to add them together..?

 

[Chalid]

Okay Chalid, thanks for update. Will check it my self tomorrow, going to bed now. But were does it calls CvPlot::calculateNaturalYield and CvPlot::calculateImprovementYieldChange to add them together..?

CvCityAI::AI_bestPlotBuild

aiFinalYields[iJ] = (pPlot->calculateNatureYield(((YieldTypes)iJ), getTeam(), bIgnoreFeature) + pPlot->calculateImprovementYieldChange(eFinalImprovement, ((YieldTypes)iJ), getOwnerINLINE(), true));

It comes to this line when the improvement is valid and of some value to the AI. Some value means that it
a) does change the tile yield in a positive way
b) spread irrigation when irrigation is needed
c) make a bonus available


Interestingly the different yields are assigned the following weights:
Food: 19 (when there is abundance) up to 48 (on absolut shortage and emphasisis, emphasisis depends on the natural yield of the field)
Production: 35 or 15 (emphasised or not)
Commerce: 8 or 7 (emphasised or not)

So the Ai doesn't really appreciate commerce gain when it can get food or production instead.

 

[Padmewan]

Thanks for all the to-ing and fro-ing, guys. From further experimentation, I can confirm that the AI takes into account (a) civics and (b) tech advances. So the AI still refuses to build a +1 food improvement when at Tech X a +1 food improvement appears that gets a +1 food bonus from Tech X. Our attempt to trick the AI was foiled!

However, the AI will build the +1 food improvement to gain access to a resource. I'm not sure if the AI will then replace that improvement with the better improvement (which also gains access to the resource) -- its little AI logic may stop at the first improvement it sees that offers access to a bonus and just stop there.

Interestingly the different yields are assigned the following weights:
Food: 19 (when there is abundance) up to 48 (on absolut shortage and emphasisis, emphasisis depends on the natural yield of the field)
Production: 35 or 15 (emphasised or not)
Commerce: 8 or 7 (emphasised or not)

Very interesting! In our food-starved mod, in which the value of production is relatively higher than in vanilla, this might be why our AI is gagging.

Would this weighting change if, e.g., it taked 3 or 4 food to support a pop point rather than 2? Just seeking ways to outsmart the AI without resorting to deep SDK changes...

 

[Padmewan]

Interestingly the different yields are assigned the following weights:
Food: 19 (when there is abundance) up to 48 (on absolut shortage and emphasisis, emphasisis depends on the natural yield of the field)
Production: 35 or 15 (emphasised or not)
Commerce: 8 or 7 (emphasised or not)

My guess is that these values are multiplied by the potential tile yield? Therefore, when deciding what to do with an irrigated grass tile, using averages in the Classical era:

Farm
food: 3 x 35 = 105
prod: 0 x 25 = 0
com: 1 x 7 = 7
TOTAL: 112

Workshop
food: 2 x 35 = 70
prod: 1 x 25 = 25
com: 1 x 7 = 7
TOTAL: 102

Hamlet
food: 2 x 35 = 70
prod:0 x 25 = 0
com: 6 x 6 = 36 [AI looks at final result]
TOTAL: 106

Wow... these numbers are close enough that they can't be an accident. When the first boost to the Hamlet series comes with Printing Press, the value of a hamlet is exactly equal to a farm on a river tile (assuming an average weighting as is assumed above).

Very interesting... do you know where those numbers come from, Chalid? Are they at all derived from the Yields XML weighting (which gives food 100, prod 120, and commerce 80)? How is the leaderhead bias for improvements accounted for -- is it a raw number added after this calculation that "tips the scale" so to speak?

This will vastly improve our ability to calibrate the AI, either by changing this default weighting or by adjusting our improvements to match these values... thanks a billion!!!!

 

[Chalid]

The amount of food needed per person is taken into consideration somewhere along the line. ..


Oh I have found which part in the code leads to the consideration of improvements that become available later.

 if (GC.getBuildInfo(eBuild).getTechPrereq() != NO_TECH)
    {
        if (!(GET_TEAM(getTeam()).isHasTech((TechTypes)GC.getBuildInfo(eBuild).getTechPrereq())))
        {
            if ((!bTestEra && !bTestVisible) || ((getCurrentEra() + 1) < GC.getTechInfo((TechTypes) GC.getBuildInfo(eBuild).getTechPrereq()).getEra()))
            {
                return false;
            }
        }
    }

It considers the improvement as soon as it comes within the next Era. So you will have to move it up two Eras. Fortunatley you can do archive this by including an additional ERA in your Erainfos without linking any techs or units or so to that ERA... so its kind of a Dummy Era.

Something like
ERA_IMPROVEMENT1
ERA_DUMMY
ERA_IMPROVEMENT2

As the AI will never get into ERA_DUMMY the improvement will only be considerd when the AI is within the ERA_IMPROVEMENT2

However, the AI will build the +1 food improvement to gain access to a resource. I'm not sure if the AI will then replace that improvement with the better improvement (which also gains access to the resource) -- its little AI logic may stop at the first improvement it sees that offers access to a bonus and just stop there.

You are absolutley right here... it will not build the second improvement when the first one grants access to the resource.

 

[Chalid]

My guess is that these values are multiplied by the potential tile yield? Therefore, when deciding what to do with an irrigated grass tile, using averages in the Classical era:

YEs they are quite close!

But there are some other factors as well. One is the additional gain by the improvement is added with weights:
food*4+production*3+commerce*2 and
19 Points are added if the improvement spreads irrigation and irrigation is needed.
and when more food is generated then needed 21 or 12 points are further added.

ery interesting... do you know where those numbers come from, Chalid? Are they at all derived from the Yields XML weighting (which gives food 100, prod 120, and commerce 80)? How is the leaderhead bias for improvements accounted for -- is it a raw number added after this calculation that "tips the scale" so to speak?

No those numbers are strictly hard coded and no XML is involved. To change them you'll have to modify the sourcecode ~ SDK usage.

The leaderhead weights are multiplied on that with (100+Weight)/100

I gladly dig into this as it helps understand the way the game and the ai is working and possibly shows where it can be improved.