Empirical test - AI gets more resources

Beamup

Higgs boson
Joined
Jun 8, 2001
Messages
1,318
Location
Boston
I've been getting the impression in my games that the AI gets more strategic resources than the player. So, being a scientist by training and inclination, I decided to answer that question empirically. What I did was this:
1. Start new game on a huge, small-land-mass archipelago map (to facilitate assigning resources to individual civs), 7 enemy civs, with all other settings default.
2. Utilize "multi" trick to see all resources on map.
3. Count resources present on home island of each civ. If multiple civs are present on an island, divide the resources among them - e.g. if 2 civs share an island with 1 Iron tile, they are each considered to have 0.5 Iron tiles.
4. Repeat steps 1-3 9 times on Chieftan, and 3 times on Deity.
The above was performed under 1.16 with no modifications of any kind.

First of all, the results confirmed that luxuries are distributed so that each civ has access to exactly one - on multi-civ islands, there are as many luxuries as civs. I did not count how many of each luxury there were, so I cannot speak to the possibility that the AI gets more of its luxuries than the player gets of theirs. Maybe later.

Secondly, in none of these games was it possible to build the Ironworks anywhere in the world.

Here are the results for strategic resources, broken down by resource and difficulty.
"Overall" = counts both difficulties, weighted equally
"Average" = average number possessed
"Prob" = probability of having at least one tile on the island
Format: ResourceDifficulty: Average for player/Average for AI/Prob for player/Prob for AI
Resources and difficulties are abbreviated to the first letter for brevity.

AC: .99---.81---.67---.70
AD: .67---1.0---.67---.86
AO: .83---.90---.67---.78

CC: .70---.80---.67---.70
CD: .00---.95---.00---.95
CO: .35---.88---.34---.73

HC: .82---1.1---.67---.68
HD: .67---1.3---.67---.86
HO: .75---1.2---.67---.75

IC: .72---1.1---.67---.73
ID: 0.0---1.1---0.0---1.0
IO: .36---1.1---.34---.87

OC: .39---.80---.63---.75
OD: 0.0---.86---0.0---.95
OO: .20---.83---.32---.85

RC: .64---.73---.78---.63
RD: 0.0---1.0---0.0---.90
RO: .32---.87---.39---.77

SC: .64---.97---.56---.75
SD: .33---1.0---.33---.81
SO: .49---.99---.45---.78

UC: .42---.70---.56---.73
UD: 0.0---.76---0.0---.81
UO: .21---.73---.28---.77

Average number of resources lacked (i.e. none on the home island):
C: 2.9---2.3
D: 6.3---.90

Clearly, the computer was, in this experiment, substantially more likely to have any given resource than the player was. Also, the discrepancy is much greater at Deity than at Chieftain. In fact, it is possible that at Chieftain there is no difference - the discrepancies there are small, though almost all in the same direction.
Now, each of these individual overall differences turns out to be about a .75-sigma result, for those of you who know something about statistics. However, the combination of 6 out of 8 .75-sigma results is statistically significant, though not overwhelmingly so with this amount of data. Contrastingly, the comparison between how many resources are lacking on Deity is a 4-sigma result. Yes, FOUR.
Unfortunately, counting up all of the resource tiles is fairly time-consuming, so increasing the amount of data will be a lengthy process.

Conclusions:
The AI gets more strategic resources than the player.
This discrepancy depends on difficulty in the manner one would expect.
The difference in whether you have a resource AT ALL is bigger than the difference in how many you have - apparently if you have any of a given resource, you're likely to have several.
More data are needed to solidify these results - aside from the "lacking" comparison, they should be considered preliminary but highly indicative.

Anyone else wishing to help with this experiment is welcome to carry out the same procedure I used - at this point, it would be most useful to simply gather more data on Chieftain and Deity, using the same settings, in order to solidify these results, rather than to try and consider more variables.

All original data available upon request.
 
12 tests isn't much of a sample.

Something you might try is making a random map with the editor, setting up sites so you can tell easily which is which, then analyzing to see what sites have the best resources. Load the scenario and restart... jeez, I dunno how many samples you'd need. Fewer on a smaller map, I suppose. I'd want at least 3 starts for each starting site, but I really have no idea if that would be significant. Been too long since I took stats. Anyway, that would be less of a hassle as you wouldn't have to analyze the same map again and again.
 
The clumping effect is intentional, it was supposed to encourage players "cornering the market" of a given resource or luxury and thus being able to play the role of power broker.

I've never heard anything from the programmers to suggest that the AI is intentionally given more access to resources, either. If this ends up being the case it's likely a bug or just coincidence.

Dan
 
Iron: Yes, 12 isn't terribly much (personally, for truly conclusive results, I'd want 100), which is why I stressed that the results are not totally conclusive. I just had no time for more. However, it is sufficient to draw some preliminary conclusions. That's why I quoted the sigma figures, so people could tell just how significant these are. BTW, the reason I posted the results at this stage was mainly to try and recruit help in doing more experiments!

Dan: I knew the clumping of luxuries was intentional, I just meant to confirm that it was really doing that, and in an extraordinarily uniform manner (possibly too uniform for my tastes). Or do you mean that strategics are supposed to clump, too? Because in my tests they don't do so to any significant extent. In only one of the cases I tried was more than a third of any resource located on the same continent - nowhere near enough to do any sort of brokering.
 
Top Bottom