My new task for dune wars will be helping the ai in deciding where to found a new city. If done right, this could not only boost ai competition in general, but also give better starting locations.
I can't do this on my own. It will need a lot of balancing and testing before we will have any satisfiable results. You can all help by trying out the new CvGameCoreDLLs that will be posted in this thread and by giving feedback.

Here is the initial reaction by davidlallen on my restarted plans to focus on that topic:

Excellent! I wish that the Arrakis mapscript used the same code as AI colony placement during the game. Was it you who suggested that perhaps the whole huge function AI_foundValue could be replaced with something much simpler customized for DW? I think that is the right approach. I never got back to my project of studying and adding comments to the existing function. Do you think it is worth a thread, to propose and review a simple function for city placement? For example, +50 points per adjacent ocean/coast plot, +100 points for each water bonus, etc?

Thanks David.

My current plan is still trying to improve the existing function. After all, the ai decisions aren't completly bad. Only if that is failing I would like to try to create a new one. We would have to make sure, not hardcoding too much things into it, making creation a challenge and examination of the current function necessary.

I've put attention, that some ai travels long road before it found its capital. Once when i played short game (unfinished) with IX, BG built her capital few turns later, on decent spot 3 tiles from my capital. The land she came from was really crappy. Too pity i just built few soldiers and killed her.

awesome dude,

good one.

My understanding is that when choosing a city site, the AI evaluates particular sites based (primarily) on bonus resources and tile yields within the BFC of that site (and to some extent on the value of those tiles with basic improvements?). It calculates a value/score for each site, and then builds a city on the site with the highest score.
Is that right?

If so, as I have said elsewhere, I think the single largest gain for city placement would be for the AI to eliminate from its score calculation for a particular site A any tiles (especially bonus resources) within the BFC of A that are also within the BFC of any other city owned by that player (or maybe by any player).
Time and time again I see the AI's build cities too close together, because they build one city and then build a second city, the second city chooses a site so as to include in its BFC resources tiles that are already being worked by the first city.

In vanilla, this is just poor, but in Dunewars this is a huge penalty, because cities are so dependent on a few particular water-yielding tiles in order to be able to grow at all.

The second thing I would suggest would be a minimum water value. If a site does not have at least 2 (or maybe 3) water-yielding tiles (non-adjacent mesas, groundwater, dew-collector resource, polar anchor grass, polar) within its BFC (and not in the BFC of another nearby city) then it should not build a city there.

ICS is not a good strategy in this mod.

I've put attention, that some ai travels long road before it found its capital.
Maybe BG started on mesa?
Having starting locations on mesa is very strange and from my understanding shouldn't happen at all. :confused: It is non foundable peak terrain, normally returning a value of 0, not suited for a game start. Maybe it's related to that other peak weirdness we are currently dealing with?

My understanding is that when choosing a city site, the AI evaluates particular sites based (primarily) on bonus resources and tile yields within the BFC of that site (and to some extent on the value of those tiles with basic improvements?).
Found value is calculated by dozens of arguments. A few examples:
-bonus resources in bfc
-yields on tiles in bfc
-potential improvements and their influence on tiles in bfc
-foreign/own culture present in bfc
-distance to next foreign/own/captial city
-area calculations (number of own/foreign/coastal cities in that area)
-a lot more...

It calculates a value/score for each site, and then builds a city on the site with the highest score.
Is that right?

Correct (but it calculates a value for each plot)

If so, as I have said elsewhere, I think the single largest gain for city placement would be for the AI to eliminate from its score calculation for a particular site A any tiles (especially bonus resources) within the BFC of A that are also within the BFC of any other city owned by that player (or maybe by any player).
Time and time again I see the AI's build cities too close together, because they build one city and then build a second city, the second city chooses a site so as to include in its BFC resources tiles that are already being worked by the first city.

This should be possible. Of course it only works for city sites (A) vs. existing cities.

The second thing I would suggest would be a minimum water value. If a site does not have at least 2 (or maybe 3) water-yielding tiles (non-adjacent mesas, groundwater, dew-collector resource, polar anchor grass, polar) within its BFC (and not in the BFC of another nearby city) then it should not build a city there.
Non-adjacent mesas will be a pain to do, but otherwise I don't see much problems.
We just have to make sure not having too much hardcoded/specific stuff in it. We don't want to rewrite it everytime someone changes the terrain xmls...

ICS is not a good strategy in this mod.
What is ICS?

There were not only peaks, just bad resourceless terrain mixed of mesas and rugged i think, some sneaky landmass.

Correct (but it calculates a value for each plot)

I think we mean the same thing here. Plot = site = tile I think.

Of course it only works for city sites (A) vs. existing cities.

Yes, but I think thats the biggest problem. And it works iteratively to spread cities out; every city is built in relation to existing cities.

Non-adjacent mesas will be a pain to do, but otherwise I don't see much problems.

Alternative fudge would just be to have mesas count as 2/3 of a water source.

We just have to make sure not having too much hardcoded/specific stuff in it. We don't want to rewrite it everytime someone changes the terrain xmls...

Understood, but the terrain xmls seem fairly fixed to me, no?

What is ICS?
Infinite City Sprawl (or Sleaze). Basically packing in lots of small cities in very tightly.
The two "main" strategies in previous versions of civ were ICS and REX (Rapid EXpansion).

I've put attention, that some ai travels long road before it found its capital.

This would surely be fixed by merging the civ start position function with the city founding function, right?

I think we mean the same thing here. Plot = site = tile I think.
City site always have a nice found value, something not necessary true for plots/tiles...

Yes, but I think thats the biggest problem. And it works iteratively to spread cities out; every city is built in relation to existing cities.
True, but it won't work very well when we have multiple settlers with founding missions on map.

Alternative fudge would just be to have mesas count as 2/3 of a water source.

Agreed.

Understood, but the terrain xmls seem fairly fixed to me, no?
No more yield balancing? :)

This would surely be fixed by merging the civ start position function with the city founding function, right?
It is already merged. :eek: Starting location are valued by the same ai_foundvalue funtion as normal city sites (but with a special argument).

City site always have a nice found value, something not necessary true for plots/tiles...

I think we are just using different terminology; I did not mean anything specific when I used the "site" term - just a potential place where you might consider building.

True, but it won't work very well when we have multiple settlers with founding missions on map.

Yeah, I thought of that - do settlers pick a location when they are built, and then not update that? Maybe there is a way to make all settlers pick new city found locations whenever another settler of your founds a city? But still, I doubt that many cities are foudned by multiple settlers at the same time.

No more yield balancing?
Probably, but the rough ratios between the well, windtrap and dew collector will likely remain unchanged. And isn't that what matters for this specific hardcap on spots with not enough water?

It is already merged. Starting location are valued by the same ai_foundvalue funtion as normal city sites (but with a special argument).
If it were compeltely merged, then every civ's start position would already be a decent site to found a city, in which case you would not observe AI players wandernig with their settler unit without founding, because they would found where they started.

I have also observed AI's wandernig; in a couple of games they have never founded, and been stuck with just a settler or worker or slave somewhere with no cities.

I think we are just using different terminology; I did not mean anything specific when I used the "site" term - just a potential place where you might consider building.
Ah ok. For me 'site' is a 100% civ4 term I read for the first time in my life inside the cvgamecoredll. It only appears in combination with 'city'. :)

Yeah, I thought of that - do settlers pick a location when they are built, and then not update that?
Afaik location are picked when a settler is starting its journey. Otherwise I remember rival settlers immediately changing their destination when I founded a city in the area they headed for. Maybe it's updated each turn...

If it were compeltely merged, then every civ's start position would already be a decent site to found a city, in which case you would not observe AI players wandernig with their settler unit without founding, because they would found where they started.

AI players wandering with their first settler only happens, if ai can't found a city on its starting tile. If that happens it should calculate a new city site and move its settler to that tile.
I remember testing a few days ago to return all peak terrain with a found value of 0 (should prevent ai from viewing this tile as a good city site (and starting location)). But the ai/human player still started on this plot type. For some reason AI foundvalue on mesa was completley ignored. :confused:

There were not only peaks, just bad resourceless terrain mixed of mesas and rugged i think, some sneaky landmass.

This emphasizes one point I made earlier. The code which the arrakis mapscript uses to choose starting plots has a completely different way of computing which are good plots. So the mapscript thought, for whatever silly reason, that the BG was at a good starting point, but the AI for founding a city did not think so, and the BG settler had to walk a long way to find a good spot. We need to solve both, or make the mapscript use koma's new scheme when he gets it, in order to really see the result.

I suspect that the start position finder liked it because it saw the water value of the mesa tiles. But this discrepancy is what I meant by merging the functions; using the same plot evaluator for both.

So the mapscript thought, for whatever silly reason, that the BG was at a good starting point, but the AI for founding a city did not think so, and the BG settler had to walk a long way to find a good spot.

When an ai has zero cities, it will immediately found one:


void CvUnitAI::AI_settleMove()
{
PROFILE_FUNC();

if (GET_PLAYER(getOwnerINLINE()).getNumCities() == 0)
{
if (canFound(plot()))
{
getGroup()->pushMission(MISSION_FOUND);
return;
}
}

I guess that means the BG settler must have been standing in a block of mesa, and moved down off it before founding.

I have noticed, however, that tiny arrakis mapscripts often give starting points that are very close to each other. In this case, the settler does not have to walk to get near me; it starts out near me. I posted a screenshot on the mapscript thread a couple of days ago.

I guess that means the BG settler must have been standing in a block of mesa,
I agree.

This emphasizes one point I made earlier. The code which the arrakis mapscript uses to choose starting plots has a completely different way of computing which are good plots. So the mapscript thought, for whatever silly reason, that the BG was at a good starting point
I checked Arrakis.py and indeed it has its own starting plot value function mostly making distance checks (which seems to fail often).

What is the general plan with map scripts? Will we continue to support both (dunepilago and arrakis) or only one of them?

What is the general plan with map scripts? Will we continue to support both (dunepilago and arrakis) or only one of them?

Good question. I think we are close to eliminating dunipelago/archipelago (they are copies of each other) and only distributing arrakis; but we are not there yet. I would be happy if you updated the arrakis mapscript with the better scheme and left archipelago alone. We could drop it, maybe for 1.7.

Well, I personally would prefer leaving the Duneipegalo there... as long as it is still playable, why cut it?
More is better.

But the Arrakis is very playable these days so I have mostly been testing that.

I removed the special starting plot function in arrakis.py. I think it's better now. I haven't seen a single mesa start. :)

Well, I personally would prefer leaving the Duneipegalo there... as long as it is still playable, why cut it?

For one thing, it doubles the time it takes me to test a release. Because of the difference in availability of ice, I try to do all the autoplays once on arrakis and once on archipelago. If it's useful, fine, but cutting it would make releases a little easier.

I removed the special starting plot function in arrakis.py. I think it's better now. I haven't seen a single mesa start.

Great, I will add that in.

EDIT: it is fine, but next time you change the mapscript, please bump the version number in the comment block, and add #koma13 by your change.

For one thing, it doubles the time it takes me to test a release. Because of the difference in availability of ice, I try to do all the autoplays once on arrakis and once on archipelago. If it's useful, fine, but cutting it would make releases a little easier.

I still think it is maybe worth keeping Archipelago, just for variety. It would be nice to force Arrakis to be the default option though.

I checked Arrakis.py and indeed it has its own starting plot value function mostly making distance checks (which seems to fail often).

I knew this was the case. If we don't specify a starting plot function then presumably both mapscripts will use the new logic you are making.

The reason that I had re-created the starting plot code was for PerfectWorld. It had large areas of bad land, and the default code only really looked at local values, so you could start on a good site, but have nowhere to expand. This code helped to alleviate that problem by taking into consideration possible later city sites. All this is probably unnecessary in Arrakis.py.

I autoplayed 3 games and printed out some nice statistics about our current situation.

Map settings:

Map script: Arrakis.py
Map size: Standard
Game speed: Quick
Players: 7
Pangaea style

Game options:
only "no barbarian" is checked

Victories:
All but last two

Statistics always taken on turn 250/330

http://forums.civfanatics.com/attachment.php?attachmentid=232380&d=1256575272

As you can see we have problems with the average population (only 10). There are also a lot of small cities, even in late game. Some of them were maybe just founded recently. :dunno:
Be careful with "Cities not growing"-line. No growth doesn't automatically indicate a bad location, but also could result from misplaced improvements.

Mesa tiles per city: We already guessed it, the amount of mesa tiles is one of the key factors to get a good city location. Similar situation in the "Food bonus per city"-line.

Desert tiles per city: This is interesting. Having a lot of desert tiles inside a bfc dramatically reduce the quality of a city site. I think this is a good argument for outsourcing spice harvest outside of the bfc. :)

Mesa shared/Food bonus shared: As you can see there isn't any relation between tiles shared with other cities and the size of a city. This disproves Ahriman claiming shared tiles are a major reason for cities not growing.

Another problem I run into while coding the statistics is, that spice is counted as a regular commerce bonus, making it completly overrated.

So my first steps in improving ai_foundvalue will be:
- writing a true algorithm to count mesa tiles available for wind traps
- dramatically increasing values for peaks (=mesa)
- rejecting all city sites with less than 2 food bonuses
- excluding spice bonus from found value
- giving a small penalty for desert(=water) tiles
- removing all checks related to coastal/area
- increasing the malus for founding cities too close
- rejecting city home plots habouring bonuses

If you want print out statistics for yourself, here is a modified Dunewars.py. Copy it to "DuneWars/Assets/Python". It is working with 1.6.1 and 1.6.2. In turn 250 a file "betterCityAI.log" will created in your bts folder.

I've noticed that the AI wastes bonuses by building cities on top of them, rather than close to them and then building the improvement. Can we also stop it from doing that?

I've noticed that the AI wastes bonuses by building cities on top of them, rather than close to them and then building the improvement. Can we also stop it from doing that?

True, I noticed that too. It is easy to avoid.

I autoplayed 3 games and printed out some nice statistics about our current situation.

Unfortunately your screenshot is not displayed; I see an empty box which says, "directupload.net: hotlinking not allowed".

Unfortunately your screenshot is not displayed; I see an empty box which says, "directupload.net: hotlinking not allowed".

Try again.

So my first steps in improving ai_foundvalue will be:
- writing a true algorithm to count mesa tiles available for wind traps
- rejecting all city sites with less than 2 food bonuses

I am not sure that a true algorithm for mesas is worth the trouble. I have thought about it, and it is clearly possible, but I suggest that an approximation is enough. I would suggest simply counting the mesa tiles and dividing by 4.

Fury Road also has very limited food resources available. When I first wrote the mapscript, there were many cases where the starting location was very poor. So what I did for the starting location was add food bonuses in the mapscript, rather than rejecting the start location or displacing it a long distance. So for start locations, maybe consider adding food bonuses instead of rejecting or displacing the location. This may be why the mapscript sometimes generates start locations for two civs which are very close together.

I am not sure that a true algorithm for mesas is worth the trouble. I have thought about it, and it is clearly possible, but I suggest that an approximation is enough. I would suggest simply counting the mesa tiles and dividing by 4.


I think it is worth the trouble. Having 4 mesa tiles clumped together is something completly different from having 4 single mesa tiles.

Fury Road also has very limited food resources available. When I first wrote the mapscript, there were many cases where the starting location was very poor. So what I did for the starting location was add food bonuses in the mapscript, rather than rejecting the start location or displacing it a long distance. So for start locations, maybe consider adding food bonuses instead of rejecting or displacing the location. This may be why the mapscript sometimes generates start locations for two civs which are very close together.

Well, I am currently working on general city sites, not starting locations. Adding new bonuses isn't an option...

Having 4 mesa tiles clumped together is something completly different from having 4 single mesa tiles.

That is true ... but you can no longer have clumps of 4 mesa tiles. I am not saying it is completely unimportant, but you could leave the exact algorithm for later so that we can start using the new code sooner.

One difference between vanilla and DW is that vanilla has alot of invisible resources, so if the AI assigns value to things like aluminum in ancient times, it's kinda cheating, so they have this code to avoid that.

In DW you have these resources that develop through the ages, so the AI won't see it's full potential.

Good point, but a different one than I was thinking of. I suppose that considering the value of crystal, which is initially invisible, would cause selection of one city site over another. But water (food) yield for a particular improvement now increases quite a lot during the game due to tech. My point was I don't think we need to model this increasing yield in order to get the right city sites.

Good point, but a different one than I was thinking of. I suppose that considering the value of crystal, which is initially invisible, would cause selection of one city site over another. But water (food) yield for a particular improvement now increases quite a lot during the game due to tech. My point was I don't think we need to model this increasing yield in order to get the right city sites.

Well, if it's picking the wrong sites, I figure there has to be something that is competing with the food. I think the AI actually assigns numeric values according to the yield types, and it's fairly simple, like too simple to reflect what is really needed. Basically, a plot gets a bonus to it's value if it can feed itself, but if you have one plot that can feed itself and a whole bunch of production plots that can't, it still gets factored in even though you can't work all those production tiles due to lack of food.

Those yield values are defined somewhere, you might get better results if you emphasize food. The default emphasizes production actually.

I don't really understand the S/M/L. Are there averages for cities in particular population ranges?

Desert tiles per city: This is interesting. Having a lot of desert tiles inside a bfc dramatically reduce the quality of a city site. I think this is a good argument for outsourcing spice harvest outside of the bfc.

I disagree with your interpretation. Obviously, more desert tiles means smaller city size, because fewer water sources - but it may have more spice access. But that does not necessarily mean worse city, or that the city shouldnt' have been built.
All it says is that.... cities with fewer land tiles have less water. Which is obvious.
And it certainly isn't an argument for pushing spice harvesting outside culture.


Mesa shared/Food bonus shared: As you can see there isn't any relation between tiles shared with other cities and the size of a city. This disproves Ahriman claiming shared tiles are a major reason for cities not growing.

I diagree with your interpretation.
Across your 3 games, looking at the totals, roughly 1/3 of mesa tilse and 1/5 of water resources are shared with other cities.

That is a large amount. This means that an AI city is getting very roughly 25% less water income per city than would a human player who did not overlap water sources.

Which suggests that overlapping resources is a big reason why AI cities are smaller.

Another problem I run into while coding the statistics is, that spice is counted as a regular commerce bonus, making it completly overrated.

Yes, I have commented on this before. Spice should be ignored when choosnig city placement, because of its transient nature. This is one fo the big reasons why the AI places cities too close to the coast, to try to maximize spice access.

I would suggest simply counting the mesa tiles and dividing by 4.
Why divide by 4? I would multiply by 2/3. On average, 3 mesas are surely worth 2 dew collectors. Remember that mesa has a base +1 water yield....

And like David says, we don't get 4-mesa clumps.

Those yield values are defined somewhere, you might get better results if you emphasize food.

This is the most important factor IMO.

IWhy divide by 4? I would multiply by 2/3. On average, 3 mesas are surely worth 2 dew collectors. Remember that mesa has a base +1 water yield....

Koma13 had proposed spending some time to implement code which correctly interprets the windtrap spacing rule and finds the optimal location/number of windtraps for a given city site. I was proposing that instead of solving this hard placement problem, simply approximate the number of possible windtrap sites as (number of mesa plots) / 4. I was not discussing any tradeoff relative to dew collectors, just counting the number of windtraps.

The 1/4 number is what you would get, if you had the one plot spacing and a region which was 100% solid mesa. (Try drawing a tic-tac-toe type board and filling it in; it comes to exactly 1/4.) But I thought of that before declumping(?). I guess the best approximation counting declumping is 1/2. Draw solid vertical lines of mesa. Now densely populate windtraps. You can put one windtrap every two plots of mesa, so, 1/2. Does that make sense?

Any "optimal" solution has to take into account edge effects; if two city BFC touch, and there is a windtrap for one city at a border plot, then the other city has fewer possible windtrap sites. I think optimally solving this is hard, which is why I recommended the approximation. It reduces development time with minimal loss of accuracy.

Ok, thx all for the feedback. Let's see how this will turn out... :p

I was proposing that instead of solving this hard placement problem, simply approximate the number of possible windtrap sites as (number of mesa plots) / 4. I was not discussing any tradeoff relative to dew collectors, just counting the number of windtraps

I undersatnd that, but you can already get more than 1 windtrap per 4 mesas, on all mapscripts.
You can at most get 2 per 3, and on average probably 1 per 2. At worst you can get 1 per 3 (an L shape mesa).
If this is purely for windtraps per mesa (ignoring the base yield of the mesa) then 1 per 2 is probably right.
If its for mesa tiles vs other water sources, then 2/3 is probably right.

Implicitly how you value mesa tiles (and how many windtraps you credit them with) affects the tradeoff you consider vs say dew collector resources: should I found in location A, or should I shift right 1 tile to location B, losing 1 dew collector but gaining 3 mesas. The answer to this should unambiguously be "yes", but I worry with your method of 1 windtrap per 4 mesa that the AI's answer could be "no".

I agree with the approximation (fully optimal is hard), but 1/2 is better than 1/4.

I thought a little bit about how to make a simple replacement for the AI_foundValue function which decides where initial cities and later settler sites should be. As we have discussed the existing function is over a thousand lines of mostly undocumented code. Here is what I propose to compute the value of a plot.

Score computation

If a plot is a valid city site (not in enemy cultural borders, not too near another city, not ocean, not mesa, *not on a visible resource*) then loop over all the plots in its BFC to total up the plot values.

In the BFC loop, skip a plot if it belongs to any existing city BFC or enemy cultural borders. Otherwise a plot's value is the sum of one number based on resource plus one number based on terrain:

Resource:
50 if it has a groundwater resource
30 if it has a water plant resource (sword grass, etc)
10 if any other visible resource except spice

Terrain:
30 if mesa
5 if polar
3 if graben or polar desert waste
2 if rock or desert waste
1 if rugged

If there is any visible crystal in the BFC, add 100. If there is any nitrate in the BFC, add 100. If there is any ice in the BFC, add 100.

Are there other things to consider? This seems pretty simple, perhaps too simple.

Starting locations

For starting locations on arrakis, we need to make sure they are equally spaced around the circle, not too close to the center. So we can do the following steps.

1. Assign each plot to a sector, where the sector is shaped like a pie slice, where the span of the pie slice is 360 degrees divided by the number of civs. So for 9 civs, each one gets a 40 degree slice. Exclude the polar region.

2. For each sector, count up the *total* score based on resources and terrain.

3. Sort the sectors by lowest score first. Picking a site in one sector will eliminate a number of possible sites in other sectors because they are too close; so we want to pick sites in the poorest sectors first.

4. Initially we want to try a minimum separation between start sites of 16 plots.

5. For each sector in score order, remove potential sites which are too close to a chosen site in an earlier sector, then pick the best site. If a sector has no potential sites due to distance, scrap all the sites for all the sectors and restart, decreasing the minimum separation by 2.

Does this seem reasonable for start position selection?

Colony sites

For later colony sites, there is an additional complication that we should not consider two possible sites which are adjacent to each other. That is, if we have one possible site, we must temporarily pretend there is a city there, and recalculate the scores for other sites with this assumption. This is one of the things which leads to extra complexity in the current AI_foundValue, and I have not carefully thought out the best way to do this yet. Distance from the current capitol should also be a factor.

Ok, here are my observations from tests I made in the last weeks:

1)
The value of AI_foundvalue doesn't teach the ai where to found cities :eek:. It teachs the ai where to found cities first. Think about the consequences, especially for ai autoplay results!

2)
As a human player I often have to decide whether I want to found one good city or two okayish cities. The ai isn't capable of this. It will always decide having one good city.

3)
The only thing preventing ai in vanilla civ4 of wasting too much terrain inbetween cities is the distance check resulting in cites placed close together

4)
AI city placement in vanilla civ4 isn't more clever than in dw. The "only" difference is, that in dw there are much more bad possible city sites (because of selective yield distribution)

1) Not quite sure I entirely understand this. Do you mean that the AI will found first using this value, and then still fill in all availble empty space with junk cities?

2) As a human player in this mod, building few good cities tends to be a better strategy than in vanilla, so this is not a huge problem. I would much prefer that the AI err towards building fewer, better cities than the status quo, where it founds many, bad cities.

3) Ugh. Unfortunate. Can you add some kind of block that will prevent the AI from building a city in a location unless its foundvalue is at least X?

4) Yes, this is true.... but it still means that AI city placement is more important here than in vanilla.

The value of AI_foundvalue doesn't teach the ai where to found cities :eek:. It teachs the ai where to found cities first. Think about the consequences, especially for ai autoplay results!

I assume that if there are several city sites, one with foundValue=100, another with 150, and another with 200, then the first settler will be sent to the 200 site. It means that the distance function must be calibrated carefully so that it trades off correctly between a moderate location nearby, and an excellent location further away.

Or, did you mean something more than that?

I'm not sure if this is better off in the mapscript thread or the city placement thread. I have written the new start plot selection code for arrakis.py using my description above. I have not tested it very much; there isn't any convenient way to run dozens or hundreds of maps and store the results.

If you are interested, please download the attachment and unzip it over your previous version of dune wars/public maps/arrakis.py. I'd be interested in any feedback. Note: since this code is entirely inside the mapscript, (a) it doesn't affect the archipelago mapscript, (b) it doesn't affect placement of the second and later cities of any civ, (c) it doesn't change where the settler "blue circles" or BBAI debug "colored circles" go.

After some testing and feedback, if it seems OK, then I will try to rewrite it inside the sdk in C code instead of python, and replace AI_foundValue. But I need to investigate more about how the rest of the sdk handles overlapping of suggested sites.

Dumb question: will this affect just starting locations for civs, or also AI city placement of 2nd, 3rd, etc. cities?
Just to know what to look for when testing.

will this affect just starting locations for civs, or also AI city placement of 2nd, 3rd, etc. cities?

(b) it doesn't affect placement of the second and later cities of any civ

tencharsneeded

Comments on the revised Arrakis initial city placement. Overall looks pretty good.

1. Bonus plots should not be starting plots - one of my number one gripes at the moment is the number of times the AI founds cities on top of resources negating the bonus they could get from them. Capitol cities on bonuses still happens in your revised Arrakis. Hopefully that is an easy fix to make.

2.

30 if mesa
...
50 if it has a groundwater resource
30 if it has a water plant resource (sword grass, etc)


I feel that groundwater might be weighted too high relative to the other water producing tiles (mesa/dew collector resources). I think that, since we reworked the early game, the yield from groundwater is not higher than the other two sources in the early stages. Aquabores give +5 water, but when you get Water Discipline, Dew Collectors provide +3 water and Wind Traps +4 water, so even if you are weighting proportionately then mesa should be higher. However, I'd probably advocate giving them all more equal weights since getting a capitol city that performs well in the early stages is probably more important than having one that performs well later. So maybe, groundwater = 45, mesa = 40 and water plants = 35.

So maybe, groundwater = 45, mesa = 40 and water plants = 35.

I did a quick test with these values and the results look pretty good. The starting plots usually seem to have 3-5 good water producing tiles. If we could just eliminate the starting plots that are on top of bonuses then we'd have a big improvement on the current code.

I did a quick test with these values and the results look pretty good.

Thanks for the feedback! The reason mesa has a lower value is due to the windtrap separation requirement. This is the discussion we had a few posts back about weighting mesa by 1/2 or 1/4. I think 1/2 is the right value; *on average*, if you have 6 mesa plots in your BFC, you can probably build three windtraps. However, if you have three groundwater resources, you can build three wells.

I will try leaving groundwater and mesa the same but bump up water plants to 40. Then each *usable* mesa is worth 60, groundwater 50, and water plant 40.

If we could just eliminate the starting plots that are on top of bonuses then we'd have a big improvement on the current code.

This is really annoying. The mapscript specifically excludes any plot with a resource from being a starting plot. At the end of the mapscript I added a print statement which verifies this. But then in the game start python, I added the same print statement, and it shows that *somebody* has added resources into my start plots after the start plots were chosen. There are routines normalizeAdd* which are called after the start plots; but in arrakis.py these routines are stubbed out. So I am having a hard time finding who is adding these, and why.

Since these are added after the fact, they do not affect the starting site location; therefore, I have added a few lines in the game start python which removes them. This works but it is a "hack" fix. I will dig further to see if I can find out who is putting in these resources.

So I am having a hard time finding who is adding these, and why.

I think I know exactly why this is. This hack was put into CvEventManager.py (by me) to resolve the Insect bonuses not being placed on the Arrakis.py map.

def onGameStart(self, argsList):
# Place random stuff on map START
if (gc.getGame().getGameTurnYear() == gc.getDefineINT("START_YEAR")):
CyMapGenerator().eraseBonuses()
CyMapGenerator().eraseGoodies()
CyMapGenerator().addBonuses()
CyMapGenerator().addGoodies()
# Place random stuff on map END

This is not really the best solution. If we make sure Arrakis.py uses the default game bonus placement (i.e. placement according to Civ4BonusInfos.xml rules) then the hack won't be needed.

Ack! Your code is erasing all the bonuses that were used to choose where to put the city sites! Then any good result we are seeing from the new placement algorithm is totally coincidence.

Let us both remove your code, and then re-evaluate the results from scratch. At least now I understand where the bonuses are coming from.

I removed that code from cveventmanager and ran a couple of tests. I see spice honey, etching resin and baradye present on the map. I am not quite sure why you needed to add that code. Is it possible the order of events was:

1. Previous bonus placement values in bonusinfos.xml in 1.4.x
2. Players observed no spice honey, etc
3. Added code into cveventmanager
4. New bonus placement values in bonusinfos.xml in 1.5.x

And then #4 made #3 un-needed? Otherwise I am confused.

As long as it is working now it doesn't really matter.

Bumping this thread because I have news.

In the last couple of days I have studied the infamous AI_foundValue code, and come to mostly understand what it does. So, using the design from this post (http://forums.civfanatics.com/showpost.php?p=8608213&postcount=37), I have implemented a new version. This totally changes the rules for the AI founding colonies. It does not change initial map positions on arrakis; this is still controlled by the arrakis mapscript. But both of them use the same algorithm.

I have added a scaling term for distance from the capital city: a site less than 10 plots from the capital has its score doubled, a site more than 20 plots away has its score halved, and sites more than 30 plots away are discarded.

I did two autoplays on the same map to test it out, one with the old colony selector and one with the new. I am surprised, and a little disappointed, that the total population, total commerce and scores with the new selector are *worse* than the old selector. We had been expecting that the poor placement of AI colonies before was a major cause for the gap between AI stats and human player stats, but that may not be the case.

There is one obvious difference between the old selector and the new selector. In the old selector, it was very clear that the AI would rush to place colonies on the polar cap. In the new placement, the AI never places a colony there, ever. I do not quite understand the reason. Perhaps there is a bug, but it is certainly not obvious.

In the design linked above, each polar plot is worth 5 points, and if there is ice there, it is worth 100 points plus 10 per ice resource. These seem like high values. I have tried doubling the point value of polar plots to 10 each, but that did not change anything. I will try 10x the value, which is silly, to make sure it is at least possible.

If I can get a version where the scores at least match the previous, I will put up a patch release like 1.7.0.3.

sweet david, looking forward to test it.

I think that the distance modifier might be too drastic. That may explain the rejection of the polar terrain.

EDIT: for example, you don't want every tile within 10 to be more valueable than every tile more than 10.

EDIT2: You might have more luck with a distance modifier 'formula' rather than discrete step values. For example say every unit of distance decreases the value by 1.5 percent or something.

Glad you're learning more and have an alternative implemented, but unfortunate that it doesn't seem to be working well yet.

In terms of parameter values, I would say:
i) Do not count the tile that the city would be built on.
ii) Desert waste should probably be worse than rugged.
iii) I would drop the value of mesa a little, maybe to 25.
iv) Can you put in a minimum found, where the AI never founds a city with value below say 85?
v) The distance thing seems problematic; the AI will *never* found a city more than 30 tiles away from its capital?
Maybe you can change this to an algorithm that looks at tiles away from existing culture, rather than from the capital?

The emphasis on crystal and nitrate might be too high. I would lower them from 100 -> 20 or 25.
It is not important that those resources be in the BFC, all you really need is to have them in your cultural borders as a strategic resource.
It is much better to have them one tile outside your BFC (but still claimed by culture fairly easily), but have the city get good water income, than to have them in the city BFC but with worse water income.

A very useful test would be if you can take a given map, and then run the AI's on it with the default BTS city placement and then again with your script, noting when cities are founded where, by each player. Maybe post some screenshots.
And then maybe compare to a human's dotmap.
If you can make it easy to switch between the old/new versions, I may be able to do some testing.

Looknig at that data (and posting some screenshots) coudl be a big help in diagnosing any weaknesses in your new script. Are AI cities still too close together? Do they not found enough cities? Do they no longer use scout thopters and suspensor transports to settle other islands? Do they found junk cities close to the capital rather than good cities a bit further away? etc.

I think my previous data collection was wrong. The results seem pretty good actually. I ran two of the games from the PYL thread and the results show about a 30% improvement in total population. I attached a spreadsheet which shows a comparison.

Right now there is no good way to switch between the new and old. A long time ago, in DW terms, koma13 added a hack so that we could easily adjust the point value threshold, below which the AI won't found any colonies. Because DW terrain ranks poorly by the default system, we had to lower the threshold a lot or else the AI would never found colonies. That is good, but my new system has a much different threshold from the old system, and it is controlled by the single XML variable which koma13 added. So, to switch from one to the other, you need to copy a different dll and also adjust this XML variable. (I think I got them out of sync before, leading to incorrect data collection.)

Probably what I should do is remove the hack variable and make two cvgamecoredll.dll files, one for the new site selector and one for the old. That way, by copying the proper dll file, you could switch between them. Or, it may be there is no real value in keeping the old one.

If I post the new and old ones, would anybody actually do a side by side comparison? Or should I just post the new one?

Looks very promising. I say just upload the new one if you are confidence that it's an improvement. We know the old version is pretty hopeless, so it should be fairly obvious to see if the new one is an improvement just by looking at where the AI is placing cities and saying "Would I have put that city there?". From the spreadsheet it looks like there is an improvement across the board in Tiles, Commerce, Cities, Population and Score. I would be very keen to try out a game since this could make the AI much more challenging.

Keeping the threshold value in the XML seems like a good idea in case we need to tweak it for some reason.

If I post the new and old ones, would anybody actually do a side by side comparison? Or should I just post the new one?

I could run a brief test if you can make it easy to switch between (let me know which files to backup), and post in the next few hours.

But if you're confident the new one is superior, then just go ahead and post that.

Honestly though, looking at the Excel sheet, the difference is just so amazingly vast that it doesn't look like it needs further testing. That difference is *huge*, it looks like you have solved the problem.
This will probably massively boost AI performance.

We might want to do some minor further tweaking (with some of the points from posts 53 and 54) if needed.

Great work David.

The AI works seems to be really paying off. From initial autoplaying the Fremen seem to be performing quite strongly in the latest patch. In fact, they may even need some nerfing.

Edit: there still seems to be a fair amount of BFC overlap in the AI city placement. Can we do something about that?

Edit: there still seems to be a fair amount of BFC overlap in the AI city placement. Can we do something about that?

Ahriman has complained about that for the vanilla placement. The fact is, the vanilla placement does account for it although I am a little unclear on the exact details. The new placement specifically counts all terrain and resource inside another city BFC as worth *zero*. Still, for both vanilla and new placement, sometimes the best available site is one which overlaps another BFC slightly.

If you can find an example where there is a new non-overlapping site which is really better than an overlapped site chosen by the new routine, that is great feedback. Please post a screenshot with your suggested different location and why. We can each go through the pseudocode I wrote in the post I linked, to see why it has chosen that site over another. Then we can work out better pseudocode.

The AI is doing much better at city placement IMO.

Here, the AI is spacing its cities much further apart, and much more like what a human player will (optimally) do.
The vanilla AI I really think tends to double count bonus resources. This AI does much much better here. The degree of overlap is very small.

And the pole still gets colonized.

Take a look at some screenshots:


http://i894.photobucket.com/albums/ac149/Ahriman_pics/placement1.jpg
http://i894.photobucket.com/albums/ac149/Ahriman_pics/placement2.jpg
http://i894.photobucket.com/albums/ac149/Ahriman_pics/placement3.jpg


Honestly, those are pretty close to the city sites I would have chosen.

I am impressed.

Honestly, those are pretty close to the city sites I would have chosen. I am impressed.

Awesome. Thanks!

This indeed looks very interesting. Is the code for this available somewhere?

This indeed looks very interesting. Is the code for this available somewhere?

It is in the 1.7.0.4 local patch (http://forums.civfanatics.com/showpost.php?p=8743458&postcount=835), including the sources. (See also post #841 for a fix to the patch in case your c:\\ directory is not writeable.)

It is probably obvious, but the changed file is CvPlayerAI.cpp, function AI_foundValue. No changes were needed anywhere else. At some point soon, I will write up what I have learned into this thread (http://forums.civfanatics.com/showthread.php?t=342364).

thanks a lot. I see you managed to completly rewrite the function. That's definetly a great step forward. :goodjob:

Thanks! What I did was:

1. Succeed at understanding the calling sequence around AI_foundValue
2. Fail at understanding the actual details of the 1000 line uncommented function
3. Throw away the existing function and write a new one of about 80 lines.

Since feedback so far seems positive, it seems that the other 900 lines of code were checking a number of things which are not really so important. I will write more about #1 soon in the thread I linked.

it seems that the other 900 lines of code were checking a number of things which are not really so important.

Not quite.... I would guess that the 1000 line code was probably supposed to work more generally. It probably looks at tile yields and improvements and increases from techs and such to decide which things are more valuable, and it cares about river access and coastal access and all sorts of other things that don't matter for Dunewars.

Whereas your code *only* works at Dunewars (and does so very well). If you suddenly massively increased the tile yield of rock and removed the yields from mesa, the code wouldn't work well anymore.

The original code is probably designed to be much more robust.

The original code is probably designed to be much more
robust.

That is probably true. But I was not able to really understand the original code.

I had one more thought on this issue; maybe we should consider allowing cities to be built on saltpan. Often the AI does a pretty poor job around saltpans, where a city site on the pan would have been far superior.

Salt is a terrain. The game disallows founding cities on salt. The AI colony value code assigns it a value of 0 when it is in the BFC. The AI does not check to see if you can desalinate or if you will be able to do so soon.

Should it assign a constant nonzero value, say half of graben? Or, perhaps it should stay zero unless you can desalinate and then it should be same as graben?

The game disallows founding cities on salt

This is what I am suggesting changing. Surely this is just a parameter somewhere.

I think that giving salt 0 value in the placement algorithm is fine, or you could give it 1, I don't think it will matter much. But sometimes as an AI or a human player (particularly on larger maps with big saltpans) the optimum city cites (in order to capture groundwater/plants/mesa) is a salt tile. A human player can desalinate that first (if its within culture) and then build (or delay settling until that happens), but the AI can't.

It is the bonuses and mesa that drive optimal city placement (where the water is), and so its unfortunate if the actual good city site that combines good accses to bonuses within the BFC is inaccessible to the AI because you can't build cities on saltpan.

1. Should the game allow founding cities on salt? This is pretty crummy terrain, and if anybody (human or AI) founded there, the city would not grow very well. I would say leave it at "no".

2. The AI may or may not see the benefit of desalination, period. I have not tested this. The actual terrain conversion is in python, so invisible to the AI, but I made desalination "look like" it was improving the terrain itself. Do you ever see AI workers desalinating?

3. You point out a human planning thing, converting a site which is "good, except lousy due to salt" and then building a city there. The planning could start from early game, the conversion could start once you achieve desalination.

4. Even if #2 works, the AI cannot plan ahead like a human for #3. You are suggesting a worker action to "desalinate to *create* a good city site", and leaving a gap in the established cities to allow that later. This kind of planning seems beyond the AI. Maybe, somehow, establishing a weight on performing desalination at certain sites, to prefer desalinating those? Then the city planner would notice later that these sites became good, and build there? The problem would be leaving a big enough gap in the city placement beforehand.

1. Should the game allow founding cities on salt? This is pretty crummy terrain, and if anybody (human or AI) founded there, the city would not grow very well. I would say leave it at "no".

This is not necessarily true. Very often, there is a thin band of salt, maybe only 3-4 tiles in total. A salt tile might have only 2-3 other salt tiles in its BFC, and might be otherwise a great city site.

This is exactly my point: a city founded on a salt tile may well grow much better than a city on an adjacent graben tile, because the former site might have more bonus resources within its BFC.

Your city placement algorithm might correctly identify that a particular salt tile would be a great city site, but because you can't found there the AI founds an inferior city in an adjacent tile instead.

The releveant comparison from vanilla is; jungle is a bad tile with low yields and unhealth, that is eventually convertible to good yields. But vanilla still lets you build a city on a jungle tile, partly to avoid having the AI make really bad city placement because it can't place on jungle.

If needed, I will use Worldbuilder to demonstrate an example tonight.
As an extreme case, imagine a situation where there is a single salt tile, surrounded by 4 groundwater tiles at 1n1w, 1n1e, 1s1w and 1s1e. Buidling on that salt tile would allow all 4 groundwaters inside the BFC. But being unable to build on that salt tile means that you can at best capture 2 of the groundwaters. So any other tile is far inferior.

You are suggesting a worker action to "desalinate to *create* a good city site", and leaving a gap in the established cities to allow that later. This kind of planning seems beyond the AI.
I agree that this is beyond the AI, this is not what I am suggesting.
What I am literally suggesting is that you just remove the block that prevents a city being founded on salt terrain type.

That way, the normal AI placement algorithm you have developed will be allowed to develop optimally.
No need for any kind of forward planning or anythnig, just better AI city placement with higher growth cities.

No need to make an example, I see what you mean. I have no objection to this change. It just has nothing to do with the AI. The AI would continue to value these tiles at zero. Both the human and the AI could build cities there if they wanted.

Does anybody else have an opinion about allowing cities on salt?

I don't mind us enabling cities on salt. It would be very rare to see good city sites in the middle of big salt pans anyway since no bonus resources are placed on salt. This would mostly be for the salt pans that only cover a small handful of tiles as Ahriman says.

I don't mind able to build on salt at all. Often I regenerate a start because I have 2-3 salts on my starting point and if I didn't build on site I would have to lose a turn. Some sites (see our current Fremen SG with former Harkonnen cities) are just plain bad, missing groundwater by 1 tile because of salt.

In the current SG, the initial city placement should be "good" but the colony city placement may be "bad". I added the new code for colony city placement in 1.7.0.4. Are you playing locally with 1.7.0.4 (not in the SG)? I think all city placement should be good in that. If you find bad starts, please send a screenshot. If you are using 1.7.0.4 or later and you see bad colony placement, please send a screenshot along with some thoughts on what you think a better nearby site would have been. This will be helpful feedback to improve the placement algorithm.

This logic will need to be updated to take account on the new Peak features I added 1.7.0.5. They should have zero plot value. This shouldn't affect placement by the current 1.7.0.4 logic too much since the Peak tiles are infrequent, but it should be coded in at some point.