Offensive Drafting

[Roland Johansen]

Don't want to be a PITA but this is not entirely accurate. Unhappy pop will still count for the diplo elections and for the trade routes output calculations ( when above 10 ), so the value of the 3 F tile in those conditions is not 0 in average, just a low value. How to compute it... well, that is far more complex, but in some situations growing unhappy even without draft or whip ( and no chance of fast ) is far better than working a hammer or commerce rich tile that is food deficient.

OFC in a real game situation, both trade routes and diplo votes should be considered in the decision of whip or draft as well, but that may be asking too much of the AI

And a higher population also increases the city upkeep cost and the civic upkeep cost. And it also lowers the unit upkeep cost. And all of these elements are dependent on the upkeep cost of the actual civics being used, the size of my empire, whether the AP and UN are available and whether a vote is about to happen and how important this vote is. It is also dependent on whether the city is getting foreign trade routes and whether it will get them when it grows. And of course, don't forget the effects of the difficulty level on all of these calculations. Hmm, have I forgotten some obscure element in favour of growing or against growing? I hope not... don't want to be corrected again...



But since we don't use all of these marginal effects in any of the other tile valuations, why should they now suddenly become crucial? Such elements can of course sometimes tip the scales, but they are typically not that important.

 

[r_rolo1]

I never said they were crucial. I just said that the value of growing in those situations in not necessarily 0 And if you were choosing between growing in unhappiness by working a 3 F tile and halting growth by working a 2 H tile ( ok, extreme example, but it happens ), getting 1 or 2 extra g in trade routes might tilt the balance , as you said.

But I digress ...

 

[Roland Johansen]

This is fun, but let's move back to trying to think of good formulas to teach the AI to draft better.

 

[PieceOfMind]

Sorry to say it, but that's a really vague argument and I think you even realised it while you were writing it. I hope you're not insulted.

I may have been vague, but really it's quite valid.

The example I had posted was not a good one because the bar was nearly full. The advantage of growing for several turns and then running deficit food is only really there when the food bar is nearer to empty than it is to full.

It's just like how one uses binary teching, putting science to 0% for a few turns or so while building libraries. The idea is that as soon as those buildings (or improvements) are built, the conversion as we go back to science for commerce is more efficient.

eg. Suppose we are at the healthy cap and have 6 tiles we could work but only 3 citizens to work them. 3 grass farms and 3 plains hills. Suppose that by taking the 3 citizens off completely the food deficit is 3. Suppose the three mines on the hills will be completed in three turns. We could try:

-Running 3 farms for 3 turns at a food surplus of 6 per turn, generating a total of 18 food. Then for 6 turns we can work the 3 mines at -3 surplus per turn, for a total of 6*12 = 72.
-Running 1 farm and 2 plains hills for 3 turns, then 1 farm and 2 plains hill mines for 6 turns. Total 3*4 + 6*8 = 60.

I think the need to put a little more emphasis on improved tiles is usually only relevant to hammer producing tiles. Most hammer heavy tiles don't make enough food to support themselves, so you usually want to be working them only when their output is best. If the AI can observe there are no workers anywhere near the city and those hammer tiles will stay unimproved for a long time, he may just have to go with the unimproved tiles for the time being. But if the mine is in the process of being built, there's not much point in working it til it's finished, unless whatever is being built is extremely urgent.

I think emphasis should only perhaps be given to improved tiles if the outputs of two tiles are judged to be very similar (like the 31 and 21 when at health cap).

Another comparison that might come close is unimproved flood plains vs. grass hill mines when at the health cap. Or maybe plains hill mines.

 

[Roland Johansen]

I now understand what you were trying to say. It wasn't clear to me at first. It's easy to misunderstand the written ideas of another person. Essentially, you want the AI to behave like a human that thinks ahead by giving it a higher preference for improved tiles. I understand that a human player can get an advantage by thinking ahead and thus the value of trying to get the AI to behave similar.

However, I don't think this would be an important improvement compared to the trouble to get it to work properly. The implications of adding value to improved tiles beyond their tile output are unclear. It would be nice if they were to result in your first situation every time, but that's not clear. It could equally well result in sub optimal behaviour in other cases. You'd need to create a formula first and then analyse many different cases to see how it works in various ingame situations. And in the end, at best it would only lead to situational one time bonuses for the AI where it encounters situations where it could do full food followed by full hammers for some 10 extra hammers.

I think it would be far more worthwhile to try and improve the AI worker behaviour as I personally never encounter these situations where full food followed by full hammers is more efficient. So I'd rather discuss the AI reimprovement problems or the AI sometimes not building enough workers to improve the terrain or the AI sometimes moving its workers extremely inefficient in big stacks to one city while there are barely any workers helping another city.

So I don't think you're idea is bad or something. Just that to get it to work well is not trivial and thus not the most efficient way of improving the AI.

I would most prefer to go back to the original discussion about drafting and leave all of the above for another thread.

 

[TheMeInTeam]

This is fun, but let's move back to trying to think of good formulas to teach the AI to draft better.

It is all moot unless the AI is in the nationhood civic. According to jdog that is the single biggest hurdle ----> getting the AI to actually use the civic, and in appropriate situations (so that isolated monty isn't drafting axes...just building knights like he normally does ). That will probably net a much larger overall improvement to drafting than actually improving draft efficiency.

I think given that AI empires tend to be pretty big on average, especially the ones massing for war, even the "soft drafting" would work pretty well for it, at least for starters. We're getting ahead of ourselves with the GT and drafting ----> the AI has to actually specialize its national wonders correctly first, then do that. Well, I don't know since I don't program, maybe specializing a GT draft city is easier but it tends to seem like that isn't the case.

 

[r_rolo1]

I agree with TMIT. The AI aparently is far less eager to change to nationhood then it is to, let's say, Slavery or Universal Suffrage and that is the real bottleneck of the whole drafting issue. It is true that drafting in general is less useful than either Slavery or US ( you can't draft infrastructure or siege ), but I think that the AI underestimates the power of the civic badly.

Other than that is the almost complete lack of logic behind the sites chosen for the natio ( and world ) wonders, but that is a completely diferent issue. But it is not uncommon to see a GT in a AI beuaro cap, that it isn't a bad idea for the civic itself, but it is not exactly the best site if you think on drafting , or even more exotic locations....

 

[PieceOfMind]

Perhaps then if offensive drafting were implemented it would have to be part of a planned strategy (just as it usually is for human players). The main advantage is the speed with which the new army can be created.

Another thing that I'm not sure has been mentioned much so far is the upgrade costs. I have a bad feeling that because AIs can upgrade so cheaply it is going to usually be more efficient for it than drafting. That may be an even bigger hurdle to getting motivated to make any drafting changes than swapping into Nationhood or worrying about watching the draft timer (which I've been focusing on).

 

[Roland Johansen]

I agree with you both, TheMeInTeam and r_rolo. The first focus should be in getting the AI to use the civic a bit more often. I got a bit enthusiastic in the discussion with PieceOfMind. Not that that discussion was useless, but let's first focus on getting the AI into nationhood and then try to improve its drafting efficiency in general and only then try to get it to place the Globe Theatre better.

Please read and criticise.

When should the AI use Nationhood?

Situation 1a: Defensive drafting: The AI led civilisation is in trouble because it is at war with a stronger coalition of civilisations on the same land mass.

The AI has access to the power rating of civilisations so it should be able to analyse the difference in power. I think it's a good idea to teach it to draft when at war with a stronger civilisation. But this could lead to bad decisions if the enemy was oversees and the war wasn't very active. I don't think it's wise for the AI to reduce its economy for drafting purposes when there is for instance no efficient way to reach eachother (pre galleons oversees war). So the below odds to switch just apply to a war with an enemy on the same land mass.

Chance to switch to nationhood each turn = max (power_enemies - 1.1 * power_friends , 0)

Reasoning: The AI doesn't have to switch to nationhood when at war, but if the enemy is significantly stronger (more than 10% stronger in this formula), then it should definitely consider it. The odds of switching each turn are relative to the power difference. So if the power difference is significant, then it will typically result in a switch to nationhood within a few turns.

Situation 1b: Defensive drafting: The AI is getting into trouble and needs more soldiers to survive.

The AI has access to the power rating of civilisations so it should be able to analyse the difference in power between the moment it started a war and its power at this instant. This is just an analysis of its own power changes, not the whole coalition.

So how about this:

As long as 1.5 * enemies power > ally power

Chance to switch to nationhood each turn = max (power_now - power_startofwar * 0.95 , 0)

Reasoning: the AI has a chance to switch to nationhood when it has lost at least 5% of its power. The chance is proportional to the loss of strength. So if it has lost a significant part of its power, then it will typically switch to nationhood within a few turns. But if the allied armies are 50% stronger than the enemy armies, then there is nothing to worry about.
(this for instance means that if you destroy an enemy stack of doom, reducing the enemies strength to below 1.5 times your own, then it will typically move quickly to nationhood)

Situation 1c: Defensive drafting: The AI led civilisation is losing cities

The AI is losing more cities than it gains measured since the start of the war.

Chance to switch to nationhood each turn = 0.33 * max (#cities lost - #cities gained, 0)

Reasoning: The AI won't necessarily switch to nationhood when it loses a single city, that could be a fluke. But if this situation endures or it loses more cities, then it will definitely move to nationhood.

Note that some of the defensive drafting measures could lead to offensive behaviour. They could switch the tide of the war. But their first idea is defensive.

Situation 2: Offensive drafting: The AI has a total war plan and is planning to attack a strong enemy.

An enemy is considered strong when it is stronger or almost as strong as the AI's civilisation. The enemy needs to have at least a power rating of 0.8 of the friendly power rating.

Chance to switch to nationhood each turn during war preparation = max ( 0.8 * power_alliedarmies - power_enemyarmies , 0)

Reasoning: When preparing a war against a strong enemy, then the AI will consider moving to nationhood to tip the scales in its favour. This is especially likely when the enemy is stronger.



The AI shouldn't move out of nationhood until the war ends. We want to avoid multiple switches of civics during a war.

So please tell me what you think about these ideas. Which ideas need some adjustments and which are fine as they are. And if you think they're all rubbish, then please tell me gently.

 

[r_rolo1]

I pretty much agree....

But atleast I would give a extra push to leaders that already have nationhood as prefered civic ( and probably a demerit to leaders that have a favourite civic that implies not running nationhood ). I would also consider inserting somewhere the value of the draftable unit vs the pop needed in here, because there is no real incentive on going drafting, no matter what is happening, if the best unit you can draft is a warrior ( and yes, you can draft warriors and also yes, it is possible to get to nationalism without BW or hunting )

 

[Roland Johansen]

Those sound as reasonable limitations and stimulants to using the nationhood civic.

 

[TheMeInTeam]

Perhaps then if offensive drafting were implemented it would have to be part of a planned strategy (just as it usually is for human players). The main advantage is the speed with which the new army can be created.

Another thing that I'm not sure has been mentioned much so far is the upgrade costs. I have a bad feeling that because AIs can upgrade so cheaply it is going to usually be more efficient for it than drafting. That may be an even bigger hurdle to getting motivated to make any drafting changes than swapping into Nationhood or worrying about watching the draft timer (which I've been focusing on).

I'm sure the AI will wind up applying its (ridiculous) upgrade bonuses to draftees as well...and I'm sure that will make it more effective at war regardless of crappy promos .

Hell in a recent game I even wound up doing that, just to max out the speed with which I could field advanced troops.

 

[mrt144]

I agree with TMIT. The AI aparently is far less eager to change to nationhood then it is to, let's say, Slavery or Universal Suffrage and that is the real bottleneck of the whole drafting issue. It is true that drafting in general is less useful than either Slavery or US ( you can't draft infrastructure or siege ), but I think that the AI underestimates the power of the civic badly.

Other than that is the almost complete lack of logic behind the sites chosen for the natio ( and world ) wonders, but that is a completely diferent issue. But it is not uncommon to see a GT in a AI beuaro cap, that it isn't a bad idea for the civic itself, but it is not exactly the best site if you think on drafting , or even more exotic locations....

The worst part about it is that they rush to research nationhood above other options like Liberalism.

 

[peter450]

I agree with you both, TheMeInTeam and r_rolo. The first focus should be in getting the AI to use the civic a bit more often. I got a bit enthusiastic in the discussion with PieceOfMind. Not that that discussion was useless, but let's first focus on getting the AI into nationhood and then try to improve its drafting efficiency in general and only then try to get it to place the Globe Theatre better.

Please read and criticise.

When should the AI use Nationhood?

Situation 1a: Defensive drafting: The AI led civilisation is in trouble because it is at war with a stronger coalition of civilisations on the same land mass.

The AI has access to the power rating of civilisations so it should be able to analyse the difference in power. I think it's a good idea to teach it to draft when at war with a stronger civilisation. But this could lead to bad decisions if the enemy was oversees and the war wasn't very active. I don't think it's wise for the AI to reduce its economy for drafting purposes when there is for instance no efficient way to reach eachother (pre galleons oversees war). So the below odds to switch just apply to a war with an enemy on the same land mass.

Chance to switch to nationhood each turn = max (power_enemies - 1.1 * power_friends , 0)

Reasoning: The AI doesn't have to switch to nationhood when at war, but if the enemy is significantly stronger (more than 10% stronger in this formula), then it should definitely consider it. The odds of switching each turn are relative to the power difference. So if the power difference is significant, then it will typically result in a switch to nationhood within a few turns.

Situation 1b: Defensive drafting: The AI is getting into trouble and needs more soldiers to survive.

The AI has access to the power rating of civilisations so it should be able to analyse the difference in power between the moment it started a war and its power at this instant. This is just an analysis of its own power changes, not the whole coalition.

So how about this:

As long as 1.5 * enemies power > ally power

Chance to switch to nationhood each turn = max (power_now - power_startofwar * 0.95 , 0)

Reasoning: the AI has a chance to switch to nationhood when it has lost at least 5% of its power. The chance is proportional to the loss of strength. So if it has lost a significant part of its power, then it will typically switch to nationhood within a few turns. But if the allied armies are 50% stronger than the enemy armies, then there is nothing to worry about.
(this for instance means that if you destroy an enemy stack of doom, reducing the enemies strength to below 1.5 times your own, then it will typically move quickly to nationhood)

Situation 1c: Defensive drafting: The AI led civilisation is losing cities

The AI is losing more cities than it gains measured since the start of the war.

Chance to switch to nationhood each turn = 0.33 * max (#cities lost - #cities gained, 0)

Reasoning: The AI won't necessarily switch to nationhood when it loses a single city, that could be a fluke. But if this situation endures or it loses more cities, then it will definitely move to nationhood.

Note that some of the defensive drafting measures could lead to offensive behaviour. They could switch the tide of the war. But their first idea is defensive.

Situation 2: Offensive drafting: The AI has a total war plan and is planning to attack a strong enemy.

An enemy is considered strong when it is stronger or almost as strong as the AI's civilisation. The enemy needs to have at least a power rating of 0.8 of the friendly power rating.

Chance to switch to nationhood each turn during war preparation = max ( 0.8 * power_alliedarmies - power_enemyarmies , 0)

Reasoning: When preparing a war against a strong enemy, then the AI will consider moving to nationhood to tip the scales in its favour. This is especially likely when the enemy is stronger.



The AI shouldn't move out of nationhood until the war ends. We want to avoid multiple switches of civics during a war.

So please tell me what you think about these ideas. Which ideas need some adjustments and which are fine as they are. And if you think they're all rubbish, then please tell me gently.

Well i think 1c is a no brainer, if it's loosing cities the priority must be to keep hold of what remaining cities it has

 

[Roland Johansen]

Well i think 1c is a no brainer, if it's loosing cities the priority must be to keep hold of what remaining cities it has

I guess that's the problem with getting a decent AI: it doesn't have a brain...

 

[jdog5000]

Thanks for the info and good luck reading all of that!

Could you please tell me where the
in the code? I'd like to have a look around that area.


Now, I'm at a loss to explain how the AI is working these tiles in this city. (this is the non-drafting case from my example in post 43)

The 10/6/4 weighting isn't the final word, there's a lot more to it when it comes to choosing the tiles. I agree that the tile working selection is odd in that example, I wonder if the AI perhaps doesn't consider military unit induced happiness when it decides what the happiness cap on a city is in this section of code ... it clearly has decided it doesn't want the city to grow any more, with that decision it's doing its best to get lots of hammers.

Anyway, the 10/6/4 bit can be found in a couple places in CvCityAI, including:

AI_bestPlotBuild: search for
iValue += (aiFinalYields[YIELD_FOOD] * 10);

AI_updateBestBuild: search for
(aiFinalYields[YIELD_FOOD]*10)

AI_getImprovementValue: search for
iValue += (aiFinalYields[YIELD_FOOD] * 10);

 

[jdog5000]

I agree with you both, TheMeInTeam and r_rolo. The first focus should be in getting the AI to use the civic a bit more often. I got a bit enthusiastic in the discussion with PieceOfMind. Not that that discussion was useless, but let's first focus on getting the AI into nationhood and then try to improve its drafting efficiency in general and only then try to get it to place the Globe Theatre better.

Please read and criticise.

When should the AI use Nationhood?

Very good ideas, I will definitely use them as a base. The AI decision making on when to change civics is set up different than
what you've suggested though ... it's all based on the "value" of the different possible civics on each turn.

So, for each civic the AI runs through a long addition of various factors and computes a value for each of the possible options. You can actually see these values in debug mode by holding down Ctrl and mousing over different player's territory. So, the AI might have on some turn:

Barbarism: 70
Vassalage: 322
Bureaucracy: 270
Nationhood: 221
Free Speech: 400

Then, the AI takes whatever civic it's currently running and multiplies its value by either 6/5 (most of the time) or 16/15 (spiritual or golden age). If another civic has a higher value than the current civic, then it switches if it can.

So, the way things will need to work is we'll need to add new considerations and multipliers to the inner workings of CvPlayerAI::AI_civicValue. Here's how it works right now for the relevant civics:

Vassalage:

iValue += -(getSingleCivicUpkeep(eCivic, true));

iValue += (kCivic.getFreeExperience() * getNumCities() * (bWarPlan ? 8 : 5) * iWarmongerPercent) / 100;

iValue += (kCivic.getBaseFreeMilitaryUnits() / 3);

iValue += ((kCivic.getFreeUnitsPopulationPercent() * getTotalPopulation()) / 200);

Bureaucracy:

iValue += -(getSingleCivicUpkeep(eCivic, true));

for (iI = 0; iI < NUM_YIELD_TYPES; iI++)
{
		iTempValue = 0;
		iTempValue += ((kCivic.getCapitalYieldModifier(iI) * 3) / 4);
		CvCity* pCapital = getCapitalCity(); 
		if (pCapital) 
		{
			iTempValue += ((kCivic.getCapitalYieldModifier(iI) * pCapital->getBaseYieldRate((YieldTypes)iI)) / 80); 
		}

		if (iI == YIELD_PRODUCTION) 
		{ 
			iTempValue *= ((AI_avoidScience()) ? 6 : 2); 
		} 
		else if (iI == YIELD_COMMERCE) 
		{ 
			iTempValue *= ((AI_avoidScience()) ? 1 : 2); 
		} 

		iValue += iTempValue;
}

Nationhood:

iValue += -(getSingleCivicUpkeep(eCivic, true));

iValue += (getWorldSizeMaxConscript(eCivic) * ((bWarPlan) ? (20 + getNumCities()) : ((8 + getNumCities()) / 2)));

for (iI = 0; iI < NUM_COMMERCE_TYPES; iI++)
{
		iTempValue = 0;

		iTempValue += ((kCivic.getCommerceModifier(iI) * getNumCities()) / 3);
		
		if (iI == COMMERCE_ESPIONAGE)
		{
			iTempValue *= AI_getEspionageWeight();
			iTempValue /= 500;
		}

		iTempValue *= AI_commerceWeight((CommerceTypes)iI);

		iTempValue /= 100;

		iValue += iTempValue;
}

for (iI = 0; iI < GC.getNumBuildingClassInfos(); iI++)
{
		if (kCivic.getBuildingHappinessChanges(iI) != 0)
		{
			iValue += (kCivic.getBuildingHappinessChanges(iI) * getBuildingClassCount((BuildingClassTypes)iI) * 3);
		}
}

Free Speech:

iValue += -(getSingleCivicUpkeep(eCivic, true));

for (iI = 0; iI < NUM_COMMERCE_TYPES; iI++)
{
		iTempValue = 0;

		iTempValue += ((kCivic.getCommerceModifier(iI) * getNumCities()) / 3);

		iTempValue *= AI_commerceWeight((CommerceTypes)iI);

		if ((iI == COMMERCE_CULTURE) && bCultureVictory2)
		{
		    iTempValue *= 2;
		    if (bCultureVictory3)
		    {
		        iTempValue *= 2;		        
		    }
		}
		iTempValue /= 100;

		iValue += iTempValue;
}

for (iI = 0; iI < NUM_YIELD_TYPES; iI++)
{
		iTempValue = 0;

		for (iJ = 0; iJ < GC.getNumImprovementInfos(); iJ++)
		{
			iTempValue += (AI_averageYieldMultiplier((YieldTypes)iI) * (kCivic.getImprovementYieldChanges(iJ, iI) * (getImprovementCount((ImprovementTypes)iJ) + getNumCities() * 2))) / 100;
		}

		if (iI == YIELD_COMMERCE) 
		{ 
			iTempValue *= ((AI_avoidScience()) ? 1 : 2); 
		} 

		iValue += iTempValue;
}

One thing you'll notice about these factors is that they do not depend on variables which fluctuate often, it's mostly set XML variables. When they do depend on things like number of cities, population, etc, they scale linearly ... if the player has 10 cities, the loss of one city won't have a large effect on civic values. This last point is important, because we don't want the AI to change civics a lot if it's not spiritual.

 

[r_rolo1]

It is just my eyes or the nationhood code above has nothing making the AI aware that it can draft ? That would explain a lot indeed....

P.S How is the slavery code regarding this issue? It would be interesting to compare, given that the AI aparently takes the in consideration when it can change to slavery...

 

[jdog5000]

Sorry, accidentally skipped the one line handling drafting ... double checked the other sections, everything is there now (the code isn't broken down like this in the actual function, I'm pulling out the parts which add something other than 0 for each civic).

Since you asked, here's what the whipping part looks like:

for (iI = 0; iI < GC.getNumHurryInfos(); iI++)
{
		if (kCivic.isHurry(iI))
		{
			iTempValue = 0;

			iTempValue += (GC.getHurryInfo((HurryTypes)iI).getProductionPerPopulation() * getNumCities() * (bWarPlan ? 2 : 1)) / 5;
			iValue += iTempValue;
		}
}

The AI will basically always switch to Slavery though because there's no other viable option ... Nationhood has to compete with other useful civics.

 

[jdog5000]

Here's a rough human readable translation of how all these values work for the civics relevant to this discussion:

Vassalage

iValue -= (16 * Population)/100;
iValue -= (60 * NumCities)/100;

if( at or planning war )
{

iValue += (2 * 8 * NumCities * iWarMongerPercent)/100;​

}
else
{

iValue += (2 * 5 * NumCities * iWarMongerPercent)/100;​

}

iValue += (5 / 3);

iValue += ((10 * Population) / 200);

Where iWarMongerPercent is based off a leader personality variable, with values between 166 (Ghengis) and 50 (Ghandi).



Bureaucracy

iValue -= (16 * Population)/100;
iValue -= (60 * NumCities)/100;

iValue += 2*(50 * 3)/4;
iValue += 2*(50 * CapitalHammers)/80;

iValue += 2*(50 *3)/4;
iValue += 2*(50 * CapitalCommerce)/80;



Nationhood

iValue -= (0 * Population)/100;
iValue -= (0 * NumCities)/100;

if( at or planning war )
{

iValue += (3 * NumCities) + 60;​

}
else
{

iValue += (3 * NumCities)/2 + 12;​

}

iTempValue = (25 * NumCities)/3;
iTempValue = (iTempValue * iEspionageLeaderWeight)/500;
iTempValue = (iTempValue * iEspionageCommerceWeight)/100;

iValue += iTempValue;

iValue += 6 * NumBarracks;

Where iEspionageLeaderWeight is from the leader xml and is between 60 and 130, while iEspionageCommerceWeight is typically between 20 and 50.



Free Speech

iValue -= (8 * Population)/100;
iValue -= (40 * NumCities)/100;

iTempValue = (100 * NumCities) / 3;
if( going for Cultural Victory )
{

iTempValue = iTempValue * (2 or 4);​

}
iTempValue = (iTempValue * iCultureCommerceWeight)/100;

iValue += iTempValue;

iValue += (8 * NumCities * iCommerceYieldMultiplier)/100;
iValue += (4 * NumTowns * iCommerceYieldMultiplier)/100;

Where iCultureCommerceWeight is typically 7 and iCommerceYieldMultiplier is between 100 and 200 (depends on how many libraries, banks, etc the AI has built).