Is using random Civs really random?

[Timothy001]

After testing for 25 games, I would say yes. Using standard settings, I played 25 games. I did not finish them all, but I did keep track of the civs (6 per game) and City States (after game 4, 9 per game).
At no time did I influence the results. No Mods were used. The first 20 games were played under 1.0.0.257. The last 5 under 1.0.0.262
Civ Results:
7: Arabia, Khemer
6: America, Brazil, Kongo, Korea, Nubia, Rome, Zulu
5: Australia, Cree France, Georgia, Indonesia, Japan, Mapuche, Norway, Russia, Spain
4: Aztec, Egypt, German, Greece (Pericles), Scotland, Sumeria
3: China, Persia,
2: England, India (Ghandi), Macedon, Mongolia, Scythia, The Netherlands
1: Greece (Gorgo), India (Chandy), Poland

City States:
11: Geneva
10: Preslav, Valetta
8: Antananarivo, Armagh, Babylon, Buenos Aires, Granada, La Venta, Lisbon
7: Auckland, Carthage, Muscat
6: Bandar Brunei, Jerusalem, Palenque, Stockholm, Toronto
5: Antioch, Brussels, Hattusa, Hong Kong, Mohenjo-Daro, Nan Madol, Vilinius, Yerevan, Zanzibar
4: Kabul, Kandy
2: Kumasi
Also twice a City State was wiped out (Prob another City State)

Longest Streak was 4 consecutive games
Khemer (19-22), & Zulu (13-16)

The last Civ Seen was Poland in Game 22
The last City State was Amsterdam in Game 21

 

[Gorbles]

Multiply the sample group by 10, or 100, and this would be a data set. As it stands, 25 games from a single machine with a specific hardware and software configuration means very little. That said, testing 250 games for the sake of testing RNG faction generation is yeah, a heckuva task, haha. I appreciate that you've gone to the effort for even 25, but let me do my thing on (P)RNG so it doesn't look like I'm just dumping on this for no reason.

Let's get the easy one out of the way. The only case in which PRNG would be observably repetitive would be if they're recreating the generator every time (or using the same seed, or both). For example, in Java, if you create a Random() object, and invoke next(), if you recreate that object and call next() again it's very likely that you'll get a similar (if not the same) result, as you're defaulting on the seed and you're resetting the iteration (through the available numbers). If you're generating a new seed every time, this . . . varies? I think the inherent start bias in pseudorandom calculations (this is dependent on the algorithm used and the environment in which it is run) means you don't get the same results, but you get a limited set of starting results for the iterator. The best approach is to keep that Random() object (with a set seed) for that purpose you need it for (let's say for this case, rolling n factions in a game of (n + 1) players).

So let's assume Firaxis are doing the second or third approach. The first one is very unlikely. The second one would explain, assuming your results could be extrapolated over X00 games, the relative bias in terms of faction selection. However, so does the third one. Now, they're not programming this in Java, but pseudorandom algorithms are kind of an industry standard. They might be using a more complicated one than most languages default to (i.e. the Mersenne Twister), but these algorithms are in general a known quantity. So even with the third approach; ensuring the same random pool for all faction choices, and making sure the approach is as "random" as possible by not resetting the Random() object, you could see the kind of results you do. The number of choices the generator is having to make are too small. The pool of results is small even if the choice is large, and in pseudorandom generators, it not being truly random, there is a pattern (however obscured) in how a particular seed generates the next number in the sequence.

However, despite all this, I think there could simply be external factors on the faction generation. There may be logic in the game (depending on when factions are chosen in world generation) that could depend on the other factions already chosen, the player faction (obviously already chosen), geographical features generated, and so on. This would also greatly constrain the end results.

 

[Timothy001]

Multiply the sample group by 10, or 100, and this would be a data set. As it stands, 25 games from a single machine with a specific hardware and software configuration means very little. That said, testing 250 games for the sake of testing RNG faction generation is yeah, a heckuva task, haha. I appreciate that you've gone to the effort for even 25, but let me do my thing on (P)RNG so it doesn't look like I'm just dumping on this for no reason.

Let's get the easy one out of the way. The only case in which PRNG would be observably repetitive would be if they're recreating the generator every time (or using the same seed, or both). For example, in Java, if you create a Random() object, and invoke next(), if you recreate that object and call next() again it's very likely that you'll get a similar (if not the same) result, as you're defaulting on the seed and you're resetting the iteration (through the available numbers). If you're generating a new seed every time, this . . . varies? I think the inherent start bias in pseudorandom calculations (this is dependent on the algorithm used and the environment in which it is run) means you don't get the same results, but you get a limited set of starting results for the iterator. The best approach is to keep that Random() object (with a set seed) for that purpose you need it for (let's say for this case, rolling n factions in a game of (n + 1) players).

So let's assume Firaxis are doing the second or third approach. The first one is very unlikely. The second one would explain, assuming your results could be extrapolated over X00 games, the relative bias in terms of faction selection. However, so does the third one. Now, they're not programming this in Java, but pseudorandom algorithms are kind of an industry standard. They might be using a more complicated one than most languages default to (i.e. the Mersenne Twister), but these algorithms are in general a known quantity. So even with the third approach; ensuring the same random pool for all faction choices, and making sure the approach is as "random" as possible by not resetting the Random() object, you could see the kind of results you do. The number of choices the generator is having to make are too small. The pool of results is small even if the choice is large, and in pseudorandom generators, it not being truly random, there is a pattern (however obscured) in how a particular seed generates the next number in the sequence.

However, despite all this, I think there could simply be external factors on the faction generation. There may be logic in the game (depending on when factions are chosen in world generation) that could depend on the other factions already chosen, the player faction (obviously already chosen), geographical features generated, and so on. This would also greatly constrain the end results.

I understand that this is not a really good sample size. I've been working on the data since it was brought up on these forums, that people were a) seeing the same leaders, or b) not seeing leaders at all. I will continue to save the data as I play games. WHo knows, maybe a year I'll have 100 to 200 games on record. I simply keep excel open in the background, and switch to it between turns when I discover something new.

 

[Rotomagus]

it's not random and Moderator Action: <SNIP> the pseudo-scietific approach, Gorbles, no one wants to know this. Now. For all Firaxis civ games it's the same. Best possible synergies for your chosen civ will almost never come up. If you go for a culturally strong civ, you will always have the other cultural civs on the map. Same with all other approaches. I play Russia mostly. With 90% probability there will be gorgo on the map and vice versa. As an example. No fun here again too.

Moderator Action: Please do not use inappropriate language on the forums. Removed the offensive word. leif
Please read the forum rules: http://forums.civfanatics.com/showthread.php?t=422889

 

[Trav'ling Canuck]

After testing for 25 games, I would say yes. Using standard settings, I played 25 games. I did not finish them all, but I did keep track of the civs (6 per game) and City States (after game 4, 9 per game).
At no time did I influence the results. No Mods were used. The first 20 games were played under 1.0.0.257. The last 5 under 1.0.0.262
Civ Results:
7: Arabia, Khemer
6: America, Brazil, Kongo, Korea, Nubia, Rome, Zulu
5: Australia, Cree France, Georgia, Indonesia, Japan, Mapuche, Norway, Russia, Spain
4: Aztec, Egypt, German, Greece (Pericles), Scotland, Sumeria
3: China, Persia,
2: England, India (Ghandi), Macedon, Mongolia, Scythia, The Netherlands
1: Greece (Gorgo), India (Chandy), Poland

City States:
11: Geneva
10: Preslav, Valetta
8: Antananarivo, Armagh, Babylon, Buenos Aires, Granada, La Venta, Lisbon
7: Auckland, Carthage, Muscat
6: Bandar Brunei, Jerusalem, Palenque, Stockholm, Toronto
5: Antioch, Brussels, Hattusa, Hong Kong, Mohenjo-Daro, Nan Madol, Vilinius, Yerevan, Zanzibar
4: Kabul, Kandy
3: Amsterdam
2: Kumasi
Also twice a City State was wiped out (Prob another City State)

Longest Streak was 4 consecutive games
Khemer (19-22), & Zulu (13-16)

The last Civ Seen was Poland in Game 22
The last City State was Amsterdam in Game 21

Thanks for sharing.

I believe that on random, getting Pericles precludes Gorgo and vice versa, and same for Gandhi/Chandragupta. That's consistent with your observations, with three of those four leaders falling on the low end of the bell curve, and the fourth falling on the median.

I see nothing in this distribution to suggest that the selection of random leaders is anything but random. Which is not to preclude that there isn't a bias that would become apparent in a larger data set, but I don't see any reason to spend the time to generate that extra data. If that's not a bell curve starting to form, it's close enough to one for gameplay purposes for me.

 

[Timothy001]

In no game did I have both Greece's or India's.
As for extra data, I just keep the program running in the background. A simple click of the windows key brings it up. I type the info, then back to the game, which was doing CPU turns.

 

[Aristos]

Multiply the sample group by 10, or 100, and this would be a data set. As it stands, 25 games from a single machine with a specific hardware and software configuration means very little. That said, testing 250 games for the sake of testing RNG faction generation is yeah, a heckuva task, haha. I appreciate that you've gone to the effort for even 25, but let me do my thing on (P)RNG so it doesn't look like I'm just dumping on this for no reason.

Not really. You use Firetuner and the magic revealall, and it could be done in manageable time (even if not at once).

 

[Gorbles]

Not really. You use Firetuner and the magic revealall, and it could be done in manageable time (even if not at once).

My bad, I was assuming manual running. Nothing really preventing anyone, then

 

[TheMeInTeam]

As it stands, 25 games from a single machine with a specific hardware and software configuration means very little.

While it's not a rigorous sample, you could in principle catch really egregious things or at least begin to suspect them. It's already unlikely you'd roll the same civ 25 times in a row for example, not impossible but enough that if it happened you'd probably want to raise an eyebrow and look at the code.

This actually happened in some of the earlier EU 4 patches. People were suspect of the stated claim discovery percentage. I had a few of us do 10 trials and compare results with same claim in same situation. Once we got to 30, 40 etc with way more discoveries than suggested (closer to 30% than sub-2%) Paradox just found and fixed the bug. There was actually another time it lied with displayed odds too (terrain), but that was egregious and much better pinned down by tests (sub-50% displayed odds, 80 same outcomes in a row!).

I guess back then they still cared that the UI lies to the player, in contrast to now where they obviously and objectively do not care to a significant degree (> 2 year old bugs that can be fixed with simple text edits and no mechanical changes persist).

In this case we're not seeing those kind of extremely skewed results where a small sample would suffice to doubt the RNG is doing what it should.

 

[Aristos]

My bad, I was assuming manual running. Nothing really preventing anyone, then

Maybe we can form a little group and share the testing, that way we can also disregard hardware bias as you suggested... I am also curious about the real randomness of the random setup, as I also am under the impression that there are some patterns that are not very random...