Cause of BTS All-forest One-resource Starts

SevenSpirits

Immortal?
Joined
Jul 7, 2007
Messages
512
I was just looking through the code that normalizes (tries to improve) starting positions, and noticed some stupidity in it.

You know how in BTS you sometimes get starts where there is one food resource and all the rest are forests? It turns out that it's not because it considers chopping to be that powerful. It just gets stuck and can't give you any more resources!

forestydf6.png

Example start: Note forests everywhere they are legal, and exactly three hills.

Here's a summary of how it normalizes:

1) Add Rivers: If you don't have fresh water yet... Checks if it can add a lake. If so, tries to add a river in a way that might provide fresh water.
If it sees it won't be able to add a lake, it adds the river in a smart way instead.

2) Remove Peaks: Replaces peaks with hills.

3) Add Lakes: If you don't have fresh water, adds a lake.

4) Remove Bad Features: Removes jungle and maybe ice.

5) Remove Bad Terrain: Replaces crap like deserts and tundra with better land.

6) Add Food Bonuses: Makes sure you have at least one food resource or two seafood resources. Also if you have no other resources with 2+ food on the tile and no flood plains, it gives you another food resource.

7) Add Good Terrain: Essentially tries to make sure you have four grassland forests.

8) Add Extras: OK, this is the important part. This tries to up your city value until you have at least 80% of the best city value. It does this in three steps, and stops when/if it reaches 80% value, or when it runs out of stuff to do:
a. Add a forest to every plot that can have one.
b. Add certain kinds of resources until you have four or more, with seafood counting as 2/3 of a resource.
c. Add hills until you have three hills.

So here's what happens: Coming into Add Extras, it sees you have a crappy start, so it spams forests everywhere in your BFC. You already have the one food resource from step 6, but after that it can't add more resources because the forests block it! It's safe to assume the hills thing won't matter much, so at this point it has no further way to raise the city value, even if it is way below 80% of the best start. It basically screws itself and paints itself into a corner. It's a totally stupid way to try to boost the city value.

The forests used to be only added with a random chance, but that condition has been commented out. This is probably what changed going into BTS.

This is why the two most frequent kinds of starts in BTS are the ridiculous forest start and the super seafood start. (Forests can't block seafood!) All the other good starts you get were good before the normalizer's 7th step, because they have stuff like Flood Plains or good resources like gold.
 
8) Add Extras: OK, this is the important part. This tries to up your city value until you have at least 80% of the best city value. It does this in three steps, and stops when/if it reaches 80% value, or when it runs out of stuff to do:
a. Add a forest to every plot that can have one.
b. Add certain kinds of resources until you have four or more, with seafood counting as 2/3 of a resource.
c. Add hills until you have three hills.

So here's what happens: Coming into Add Extras, it sees you have a crappy start, so it spams forests everywhere in your BFC. You already have the one food resource from step 6, but after that it can't add more resources because the forests block it! It's safe to assume the hills thing won't matter much, so at this point it has no further way to raise the city value, even if it is way below 80% of the best start. It basically screws itself and paints itself into a corner. It's a totally stupid way to try to boost the city value.

The forests used to be only added with a random chance, but that condition has been commented out. This is probably what changed going into BTS.

This is why the two most frequent kinds of starts in BTS are the ridiculous forest start and the super seafood start. (Forests can't block seafood!) All the other good starts you get were good before the normalizer's 7th step, because they have stuff like Flood Plains or good resources like gold.

Interesting and a good find. :goodjob:

While it is nice that you informed other players of this part of the code, informing us will not change it (except maybe if you convince Bhruic that he should change it in his unofficial patch). If you want this behaviour of the map generator to change, then you should try to write a good bug report in the bug report section.

It's not a clear bug like a game crash would be, so to support your case that it is a bug you could add arguments like
a) it reduces the variety in different starting positions making the game starts less interesting
b) the algorithm fails to normalise the starting positions. You won't reach the 80% mark by just adding forests

If you were to do this, could you also make an argument for adding floodplains to rivers that are added in this part of the code? The upgrading code sometimes adds rivers but doesn't add floodplains when the river goes through a desert. It's the only way that you can get desert river tiles.
 
Wow, interesting. I knew something was different about the starts in BTS, and I haven't ever been much of a fan of them.
 
puting 8)b. before 8)a. should fix the problem

I'd prefer a mix of both forests and food resources to upgrade the starting position. Something like:
Repeat until starting position is at the required level
(-add forest
-Check if quality starting position is ok
-add resource
-Check if quality starting position is ok)
 
Yeah, I was originally going to post this in the Bhruic's patch thread, but realized it would be very hard to tell what to actually fix. It's not really a bug, it's just a bad design decision.

There are some other things there that annoy me too. The "smart" river-adding function has a comment by blake that he didn't want to use it because the player shouldn't always start with fresh water. But then, when they use the stupid (old) version and it fails, they place a lake!

Another point of interest is that there are some actual bugs in the code (though they don't do much), so maybe we actually can get this stuff looked over by Firaxis.

In part 8b, it tries to count the number of water tiles. Then if more than half the tiles are water tiles, it will try harder to place land resources instead of sea resources. In addition, if there are already 3+ sea resources, it will be willing to remove forest to place them. That's what it's supposed to do anyway...

What it actually does is is counts the number of tiles total. For some odd reason this always turns out to be more than half the number of tiles, so it always emphasizes land resources.

On the other hand, the removal of forests doesn't work! The condition wants to check if there is already a resource in the plot, and only try to place one if there isn't. Due to a mistake, it's only willing to place one if there IS one there. This is never. ;) So the forests don't get removed.

But say that was fixed. Then the removal of forests would still be buggy: first it removes the forest, then it does a random check to see if it should remove the forest and place the resource. So if this actually worked, it would also cause mass deforestation. :lol:
 
Oh yeah, there's also a bug in step 6 (adding food bonuses). Look at the picture in my first post. Notice how there's only one food resource? Notice how there are no other plots that produce more than 2 food, and no plots (other than the cow) with a resource that produces at least two food (e.g. a grassland copper)?

According to my description of step 6, this shouldn't happen. It should see the dearth of good food tiles and add a second food resource.

It screws up because the cows were already there, and so it double-counts them. It sees them both as a food resource, AND as an "existing at-least-two-food resource". Oops! If the cow wasn't originally there, it would have counted zero "decent food plot"s and zero food resources. Thus it would have added two food resources. So it was apparently bad to choose a city site near a cow. :crazyeye:
 
@SevenSpirits

I myself have started two bug reports about game features that weren't working properly. They weren't real bugs in the sense that they stopped the game from functioning, they were more game balance issues. Not only were they changed in the next patch (3.13), but they were changed in almost the exact way as I had proposed.

Contrary to common believe in some of the complaint threads in this forum, some Firaxians are actually reading the bug report forum and they can be persuaded by a good argument. So if you would write a good bug report about the features in the city founding algorithm that aren't functioning well, then I do think there is a good chance that it will be changed in the next patch. Not only that, but if you do a good job with suggesting ways how it could be patched, then they might even patch it exactly like you would want it to be patched.

Elements of a good bug/game balance report:
-Pick a descriptive title for the bug report thread (maybe something like 'Starting spot enhancement algorithm has some issues')
-General introduction of the issue at hand including an argument why this is an important part of the game,
-Description of the bug/game balance issue/coding mistake and explanation why this isn't the way it is supposed to work,
-Suggestion how it could be improved and why this is an improvement.

It's better not to use words like stupid, sloppy and such. Insulting a programmer is not the typical way to convince him/her that you have something interesting to report. The better structured your report and the better the arguments, the better the chance that you could convince a Firaxian reader.

I really do hope that you write a bug report as you do seem to know what the issues are. You however can't expect a Firaxian to have seen this thread. They won't be reading every thread in the entire forum.

Whatever you decide, it is interesting to know the causes of some of the stranger game starting positions.
 
Points taken... the trick is just to figure out what is a good change. I'm honestly not sure myself.

How about just reintroducing the random check for adding forests?


About the implications of knowing this, it's very interesting. First, if you see a lot of forest when you spawn, you probably have a subpar start. If you see a start with multiple forestable tiles without forest, you have a good one, possibly a great one. If you can see you have fewer than three hills, you also know you have a decent start (at least 80% of the best one), even if you have lots of forest.
 
Points taken... the trick is just to figure out what is a good change. I'm honestly not sure myself.

How about just reintroducing the random check for adding forests?


About the implications of knowing this, it's very interesting. First, if you see a lot of forest when you spawn, you probably have a subpar start. If you see a start with multiple forestable tiles without forest, you have a good one, possibly a great one. If you can see you have fewer than three hills, you also know you have a decent start (at least 80% of the best one), even if you have lots of forest.

You don't have to go into detail about how Firaxian programmers should code it. Just a comment that with the present code you won't get extra food resources and the code seems to suggest that it should be possible to get additional food resources. A section of the code goes unused. It's less boring and results in more variation in starting positions if the code sometimes adds a food resource instead of 6 forest tiles. Variation in starting positions is one of the cornerstones of the replayability of this game.
 
What a great find/post - I hope bhuric considers adding a fix if one can decided upon.

Maybe add a thread to the bug forum summarizing what you found and either link to this thread or duplicate your original post (along with rolands suggestions).

8) Add Extras: OK, this is the important part. This tries to up your city value until you have at least 80% of the best city value. It does this in three steps, and stops when/if it reaches 80% value, or when it runs out of stuff to do:
a. Add a forest to every plot that can have one.
b. Add certain kinds of resources until you have four or more, with seafood counting as 2/3 of a resource.
c. Add hills until you have three hills.

The forests used to be only added with a random chance, but that condition has been commented out. This is probably what changed going into BTS.

puting 8)b. before 8)a. should fix the problem

If they decide not to add the random factor back in for forest, maybe changing abc to bca?
 
I was just looking through the code that normalizes (tries to improve) starting positions, and noticed some stupidity in it.

Any chance you could tell us where? There are way to many source files to the CvGameCoreDLL.dll for me to guess.

Even if neither Bh nor the Firaxians decide to change it, it might be fun to edit and recompile on our own.
 
Any chance you could tell us where? There are way to many source files to the CvGameCoreDLL.dll for me to guess.

Even if neither Bh nor the Firaxians decide to change it, it might be fun to edit and recompile on our own.

It's in CvGame.cpp. The function normalizeStartingPlots() calls a bunch of other ones, like normalizeAddRiver(). The ones that merit changing are normalizeAddFoodBonuses() and normalizeAddExtras().

By the way, does anyone know how to print some debug information from the dll? I have only done that in python.

I tried using chipotle + hold down alt to check the starting plot values but they are very weird and I don't trust that source. I want to directly print out the value when it finishes adding extras.
 
Never mind, I figured the debug printing out.

It turns out that what I got from holding down alt WAS correct, which I find quite surprising. Here's data from an example map I generated (hemispheres low sea level):
Target (4/5 of best): 169
Player: G Value: 212
Player: P Value: 177
Player: H Value: 159
Player: J Value: 136
Player: M Value: 127
Player: S Value: 88
Player: M Value: 85

So only the second-best start position was within the 80%-of-first goal!

Another one:
Target (4/5 of best): 265
Player: P Value: 332
Player: H Value: 273
Player: S Value: 140
Player: G Value: 128
Player: M Value: 114
Player: M Value: 102
Player: J Value: 84

Same thing. And the worst place has 1/4 the value of the best!

The other two maps I generated (before I added the debugging code) had even more extreme values, with the best being 6 or 8 times the worst.

HOWEVER... these low value ones are not the one resource starts I was worrying about! That 84 value one had 20 land tiles including Pig, Marble, Iron, Aluminum and a Flood Plains. (Obviously the rest was forest.) Meanwhile, the second best one had Cow, Wine, Oil, Flood Plains and plenty of tiles without forest. It had comparable but inferior land and resources (IMO) and more than triple the score!!

Land-wise, I feel that 84 start was fine. I don't think that fixing the forest problem would make any noticeable difference.

So what's the difference? I think it was that the 273 start was farther away from the others. They were arranged kind of like this:

...000000
..0000x00
..00000000
.000000000
00a00b00c0
.000000000

(Continent not to scale.)
The x was the 273.
The a was the 84.

What does this tell us? The starting city valuation thinks that room to expand is really important compared to your actual terrain and resources.

Let's say that we change the normalizer so it actually does make the starts within 80% of the best. I think starts like the 84 (or the 44 I got earlier) would end up with ridiculous amounts of resources. It would make the terrain unrealistic and ugly, and it would also be kind of unfair because the starts would be really good!

So I think that changing the normalizer won't be enough. I'm going to try to figure out how the starting plots are initially chosen. I think if we can get them to be more evenly distributed, the distance won't factor as much into the city valuation, and we'll be able to get properly normalized cities without adding tons of bonuses.
 
It depends on what method you are using to try and get a valuation for each city location. It can be somewhat confusing unless you've done a fairly detailed analysis of it. For instance, there is no valuation for "room to expand" like you are theorizing.

You say that "changing the normalizer won't be enough". Enough for what? The main function of normalizing is to ensure that starting locations provide roughly similar starting bonuses. The valuation of the location isn't really relevant as long as that is true. The only "problem" would appear to be that it's over-emphasizing forests. Fixing that problem should be sufficient.

Bh
 
It depends on what method you are using to try and get a valuation for each city location. It can be somewhat confusing unless you've done a fairly detailed analysis of it. For instance, there is no valuation for "room to expand" like you are theorizing.
I admit I still don't understand all the code, but it sure looks like a big part of it is stuff like the number of cities in that area and the distance to the nearest existing city/startlocation. That's what I mean by "room to expand".

If it's not that, then what is it? In the example I gave, remember, the land was almost identical yet the values differed by a factor of three!

You say that "changing the normalizer won't be enough". Enough for what? The main function of normalizing is to ensure that starting locations provide roughly similar starting bonuses. The valuation of the location isn't really relevant as long as that is true. The only "problem" would appear to be that it's over-emphasizing forests. Fixing that problem should be sufficient.

That's only true if you think the valuation function is poor. If the valuation is any good, than being off by a factor of 6 AFTER normalizing is a problem. It also kind of changes what the normalization step does, because in my testing experience the maximum boost the normalization step can bring is frequently less than the boost needed to reach 80%. That means that the bottom, say, 50% of civs are all getting the exact same boost, even if they started with radically different values.

If you think the valuation function is poor then wouldn't it be good to fix it?

Speaking just from personal experience, I find that the starting location chooser has an unhealthy love of sticking you on a peninsula. I assume this is because it causes you to be farther away from the other civs. This is not a good thing! First, it's better to be closer to the other civs and have land behind you, than to need to expand into easily contestable areas. Second, those peninsulas make it hard to place cities close to your capital. It's a real issue with playing on maps that have lots of those kind of land forms.
 
If it's not that, then what is it? In the example I gave, remember, the land was almost identical yet the values differed by a factor of three!

You'd need to post the saves for me to be able to answer that. I mean, I could speculate, but that's all it would be.

That's only true if you think the valuation function is poor. If the valuation is any good, than being off by a factor of 6 AFTER normalizing is a problem.

I disagree. There's no reason that all the cities should have roughly the same value. Part of the valuation system, for example, is weighting based on how much the AI "values" different resources. A city with Coal in its starting BFC would be more valuable than a city with Pigs. But I don't think there's any doubt in people's minds that (assuming a 4000BC start) the pigs are a much better starting resource than the coal is.

If you think the valuation function is poor then wouldn't it be good to fix it?

Considering I've been doing that for the past 3 weeks, then of course I'll answer "yes". But I'm not improving it because I think it reflects on the starting location, but because of poor choices for subsequent city locations (in some cases).

Speaking just from personal experience, I find that the starting location chooser has an unhealthy love of sticking you on a peninsula. I assume this is because it causes you to be farther away from the other civs. This is not a good thing! First, it's better to be closer to the other civs and have land behind you, than to need to expand into easily contestable areas. Second, those peninsulas make it hard to place cities close to your capital. It's a real issue with playing on maps that have lots of those kind of land forms.

That sounds a lot like "the phone always rings in the shower" to me. I've generated hundreds of maps both with the default city placement system, and the modifications I've been making (as part of testing). While it's certainly true that sometimes the starting city location isn't wonderful, it's just as frequent that it is. With the majority of cases being solidly in the middle (ie, generically good).

Bh
 
I disagree. There's no reason that all the cities should have roughly the same value. Part of the valuation system, for example, is weighting based on how much the AI "values" different resources. A city with Coal in its starting BFC would be more valuable than a city with Pigs. But I don't think there's any doubt in people's minds that (assuming a 4000BC start) the pigs are a much better starting resource than the coal is.
I think this is a good example of where the valuation should be changed. Personally I think resource values should be divided by 1.5^(number of era away of the resource's use). Maybe not 1.5, but something like that.

That sounds a lot like "the phone always rings in the shower" to me. I've generated hundreds of maps both with the default city placement system, and the modifications I've been making (as part of testing). While it's certainly true that sometimes the starting city location isn't wonderful, it's just as frequent that it is. With the majority of cases being solidly in the middle (ie, generically good).
I should have said that better - I'm not complaining about the majority of cases, which I agree are fine. But all the times where my starting location has been really, really atrocious are due to it not realizing the drawbacks of peninsulas.
 
Back
Top Bottom