Arrakis.py map script discussion

[keldath]

Greeneyedzombie hi,

just a quastion - are you playing 1.3? or the reduced?

 

[Greeneyedzombie]

Both. but mostly reduced, since that seems to be the future dune.

 

[Deliverator]

I always get ctd's after 100-200 turns with this mapscript.

Me too. Sometimes earlier.

 

[cephalo]

I really don't think the mapscript would cause a crash 100 turns into the game. Are you guys playing on Huge? Do you not get crashes on Archipelago?

 

[Deliverator]

I was playing Large. I can't recreate the crash on Archipelago. It may be that this mapscript just makes whatever circumstance is causing the crash more likely. I've noticed turns are really quite slow too. Perhaps it's something to do with spice blows and vast expanse of deep desert in this mapscript?

 

[davidlallen]

I was trying out the latest Arrakis.py and I've got a CTD. Here are two saves. I'm using dune reduced 1.4.4. Can people see if they can autoplay 100 turns or so without a crash?

In fact, I get a crash as soon as I load either of your games, or as soon as the first turn finishes. This is because of unset variables such as sXMLNationalism. This is caused by having the Revolutions option turned on. Did you know that you were playing with these options?

It is time to actually fix revolutions so it doesn't crash. The trick is that it refers to individual tech names, and we are in the process of rewriting the tech tree. So the tech tree will have to stabilize a little first. Once keldath posts 1.4.5, I will update revdefs.py so it will not crash. Still, removing revolutions would be a better idea.

 

[cephalo]

I was playing Large. I can't recreate the crash on Archipelago. It may be that this mapscript just makes whatever circumstance is causing the crash more likely. I've noticed turns are really quite slow too. Perhaps it's something to do with spice blows and vast expanse of deep desert in this mapscript?

This map does have alot more deep desert than Archipelago. If there's alot of stuff happening to each deep desert tile, it's possible that it's just too much. If that is the case, you would find the problem much more pronounced on a huge map, and less pronounced on a smaller map. Sorry I can't test these things, I've been very busy lately.

 

[davidlallen]

I have tested the relative runtime of arrakis vs archipelago. I have found two things.

1. The number of squares on the arrakis map is much larger. For example a large arrakis is 128x128, a large archipelago is 104x64. That is 2.2x as many squares, therefore we should "expect" the runtime to be 2.2x slower. That is not precisely true, but that probably accounts for longer runtime of pathfinding, checking every plot on the map, etc. If the runtime were 2.2x slower, we would probably have to accept that. I am not sure if the arrakis map *needs* to be that much larger; maybe 104x104 instead of 128x128 would still be enough, and it would only be 1.6x larger.

2. Still, I have found one part of the python which is increasing runtime. On a large map with no python changes, arrakis takes almost 3x longer than archipelago. When I turn off the routine which upgrades the terrain based on Reservoir of Liet, it is only 2.1x slower. Therefore, the terrain upgrading routine is causing the slowdown. I will investigate this to find a more efficient way to do it.

So, it is definitely slower; we can recover some by making the map size closer to archipelago, and some more if I rewrite the terrain upgrader.

This does not shed any light on the occasional crashes, but at least Deliverator's are caused by the revolutions game options.

 

[Ahriman]

Therefore, the terrain upgrading routine is causing the slowdown.

I'd love to see the terrain upgrader happen more slowly (ie lower probability chance of change per turn) and to eventually be able to cycle other terrain types in too, so rugged/Dunes don't remain completely useless.

 

[davidlallen]

I'd love to see the terrain upgrader happen more slowly (ie lower probability chance of change per turn) and to eventually be able to cycle other terrain types in too, so rugged/Dunes don't remain completely useless.

In 1.4.2 the terrain upgrades much more slowly than 1.4 or 1.4.1. Do you still find it too fast? As I mentioned earlier, to upgrade multiple terrain types I need more infrastructure to remember the original terrain type so it can downgrade to the original level if the catchbasin or improvement is destroyed. I'll do it, with lower priority.

 

[Ahriman]

In 1.4.2 the terrain upgrades much more slowly than 1.4 or 1.4.1. Do you still find it too fast? As I mentioned earlier, to upgrade multiple terrain types I need more infrastructure to remember the original terrain type so it can downgrade to the original level if the catchbasin or improvement is destroyed.

I think its slightly too fast, but I probably need to play more before being certain.

Agree that its not high priority.

 

[cephalo]

What exactly does the Reservoir of Liet do? Does it need to examine the whole planet every turn?

 

[davidlallen]

You can read about the catchbasin and reservoir of liet in the civilopedia, dune wars concept tab, terraforming topic. I added highlights of this for the catchbasin xml help, but it is missing for the reservoir. It does look at the whole map once, but I am not sure which part is slow yet. In the city's BFC, it destroys spice and allows territory with improvements to upgrade its terrain type. Also, as part of the check, it looks to see if any previously upgraded territory has to be downgraded; for example, if a catchbasin was destroyed when a city was attacked, any territory which it upgraded needs to be downgraded. Since there is a random % chance per turn to downgrade, the whole map has to be checked.

I will have to cache this information somehow, but I haven't studied it very carefully yet.

 

[Ahriman]

Is there any way that you could store cities which have ever had a catchbasin or reservoir, and then check only the tiles in the BFCs of those cities? That would save you from having to do a global tile scan.

Another idea:
The AI often does really bad city placement for its capital, which cripples its development. I think the biggest single reason for this is that the AI is enticed by the spice resource tiles and so tries to include those in its BFC, when those are really only transient.

So is it possible to adjust the mapscript so that the spice resource is not generated during the map generation at all, and then create on turn 1 a ton of spice blows all over the map (by a one-off event?) that will then blow up on the next few turns seeding the planet with spice?
Slightly unrealistic, but the player will barely notice, and it could lead to much better AI capital placement.

Alternatively, is there any way to get the AI to ignore the spice resource when evaulating tiles for city placement?

 

[davidlallen]

Is there any way that you could store cities which have ever had a catchbasin or reservoir, and then check only the tiles in the BFCs of those cities? That would save you from having to do a global tile scan.

Thanks for the suggestion. The problem here is that the terraforming degrades when a catchbasin is *destroyed* or also if an improvement on the square is pillaged. So the entire map must be scanned. I will have to study it a little more.

The AI often does really bad city placement for its capital, which cripples its development. I think the biggest single reason for this is that the AI is enticed by the spice resource tiles and so tries to include those in its BFC, when those are really only transient.

The initial city locations are chosen in the mapscript. The spice is added later, in the GameStart function. So I don't think poor initial city location can be "blamed" on spice. I don't really know how the vanilla mapscripts (including our archipelago) actually select initial city locations. Maybe cephalo can comment on how his new mapscript selects initial city locations. I would be surprised if the vanilla routines could choose wisely for DW; hills and peaks have a large value due to windtraps, but you should also take into account the one tile spacing requirement to be really optimal.

 

[Ahriman]

Thanks for the suggestion. The problem here is that the terraforming degrades when a catchbasin is *destroyed* or also if an improvement on the square is pillaged. So the entire map must be scanned. I will have to study it a little more.

Note quite what I meant; I didn't mean store the location of cities that *currently* have a catchb/reservoir; that wouldn't work because you need to downgrade for cities where the building no longer exists (eg because of change of civic), I meant make a store of the location of cities that have *ever* had a catb/reservoir.
So have some array that is empty at game start, and whenever a catchb/res is built, take the tile location of that city and add it to the array. And tiles are never removed from the array.

[store the tile location rather than the city because the city could get razed and leave terraformed tiles around it]

So all you ever have to do is scan the tiles of the BFC surrounding each member of the array each turn, and check if they are terraformed but do not have an improvement, and if so then downgrade them.
This should work, because it is impossible for a tile to ever have been terraformed if it was not in the BFC of a city that once had a catchb/res. You never need to degrade any tile that *wasn't* in the BFC of a tile listed in the array.

The spice is added later, in the GameStart function. So I don't think poor initial city location can be "blamed" on spice.

Are you sure? I thought that the mapscript chooses the location for the starting settler, not the location for their capital; couldn't the AI move 1 tile in any direction before settling on the first turn? Or do they always settle on their starting tile?
If they always settle on their starting tile then you are correct, but if they begin with their starting units and *then* try to choose an "optimal" city site on the first turn then they can be encouraged by spice to move towards the coast and settle there, rather than inland slightly with more potential for hills.

 

[koma13]

Or do they always settle on their starting tile?

This. The only reason why ai would move before founding a city is if starting plot doesn't allow to found one.

 

[davidlallen]

Perhaps I can do the terraforming check only in the BFC of every city; that may be a sufficient speedup.

I thought that the mapscript chooses the location for the starting settler, not the location for their capital

OK, you are right. jdog5000, the Better AI guy, has pointed out that when you are in ctrl-z mode, you can see colored circles which are acceptable settler locations for the AI. Koma has looked into this a little to change the minimum acceptable value to make distant colonies easier. But this routine would need to be customized to value hill/peak highly. Perhaps at the same time, we could investigate whether spice was exerting a big influence on the choice.

(EDIT: koma knows better, see his immediately previous post.)

But putting the spice slightly later doesn't really help this part of the code, because every city (not just the first one) is chosen by this code. Also, the spice decays pretty slowly now, so valuing plots with spice "now" may not be such a bad idea.

 

[Ahriman]

The capital city placement is much more important than later ones.
But if the AI always settles on its starting tile then the no-spice-in-mapscript idea would do nothing.

 

[cephalo]

Perhaps I can do the terraforming check only in the BFC of every city; that may be a sufficient speedup.

Of course, the city might be razed, but even so, this seems like a great candidate for some pre-processing. Any time a tile get's upgraded, put it in a list for possible downgrading, then just check that list.

EDIT: and of course remove it from the list upon downgrading.