[BTS] Extremely Unlucky Battles

[BrendanG]

Edit: I did more tests with more battles and concluded that the AI did not cheat. I was just suspicious because of extremely unlucky RNG immediately after raising the difficulty level.

I was defending a city from a surprise attack by the AI, and noticed that I lost almost all the battles. I decided to load back, abandon the city, and then attack it the next turn, and wrote down the battle odds and whether I won or lost. This was prince difficulty level.

I thought I caught the AI cheating before on a different post, but someone explained to me that I didn't understand the battle mechanics. So this time I held my mouse over the enemy unit and let the game tell me what my battle odds were. I loaded back to attack rather than defend so that I could see the battle odds before the battle took place.

These were the results, using % of victory, (W) for win and (L) for lost:
99.4% (L)
50% (L)
75.8% (W)
88.5% (L)
69.8% (L)
82.5% (W)
56.8% (L)
Those units above were the ones I wanted to send to attack, because I thought they had a good chance of winning. But I thought for the sake of record-keeping, I would attack with all of my units
36.5% (W)
70.6% (L)
36.5 (L)

Edit: I deleted some convoluted math I came up with in my original post because I couldn't remember at the time how to calculate these odds. My original math was more complicated than it needed to be and wasn't 100% right, but I got quite close to the real answer (I compared the odds to something simpler which I knew how to calculate), which is that there was only a 2.5% chance that I would get so unlucky, or a 5% chance that either the AI or myself would get so unlucky. My earlier estimates were 3 and 6 percent.


Edit: I deleted some complaints I had about what I suspected to be AI cheating and suggested that I would test more battles (which I later did) to prove it. I ended up analyzing 80 battles with excel, and the actual result was barely outside 1 standard deviation, which is not all that unusual (something like 35% chance of that happening). Considering the standard deviation was only 3.25 out of 80 battles, and my results were only slightly unlucky, I don't think the developers would go out of their way to program in such a small advantage to the AI.


So my current conclusion is that I made a bit of a fool of myself guessing about whether the AI was cheating, and then I went and proved myself wrong. I was just suspicious because I got extremely unlucky battle odds first thing after raising the difficulty level.

 

[lymond]

RNG is RNG

 

[Kaitzilla]

These were the results, using % of victory, (W) for win and (L) for lost:
99.4% (L)
50% (L)
75.8% (W)
88.5% (L)
69.8% (L)
82.5% (W)
56.8% (L)
Those units above were the ones I wanted to send to attack, because I thought they had a good chance of winning. But I thought for the sake of record-keeping, I would attack with all of my units
36.5% (W)
70.6% (L)
36.5 (L)

Technically I ought to be able to describe mathematically exactly how lucky I was using combinatorial logic or some similar thing

Just convert the percents to decimals and add them up to get the expected number of wins I think.
0.994+0.5+0.758+0.885+0.698+0.825+0.568+0.365+0.706+0.365 = 6.664 Wins out of 10 battles with the odds you stated.
You got 3.

Not sure how to do proper statistics to tell you the odds of only getting 3 wins.

Just the other day in BOTM 173 I had an extreme run of luck conquering an AI without losing a single unit. (No catapults used)
I won 17 battles and should only have won 10.377 out of 17 battles.

1) Jaguar 61% WON
2) Axe 61% WON (Cordoba Captured)
3) Horse Archer 26.2% Retreated
4) Horse Archer 39% WON
5) Horse Archer 69% Retreated
6) Horse Archer 74.3% WON
7) Horse Archer 97% WON (Seville captured)
8) Horse Archer 50% WON
9) Chariot 96.2% WON (Sword + Axe pair eliminated)
10) Horse Archer 20.9% WON (Sword on forest?)
11) Horse Archer 64.7% WON
12) Horse Archer 63.8% WON
13 ) Horse Archer 69% WON (Barcelona captured)
14) Chariot 36.5% WON (defending against Axe I think after pillaging Bronze)
15) Horse Archer 99.1% WON (Revenge for losing 2 Workers)
16) Horse Archer 24.3% WON
17) Horse Archer 27.9% WON (Isabella capital shrine city captured)
18) Axe 76.5% WON
19) Axe 76.5% WON (Spain destroyed )

I looked over the odds a bit in this other thread.
https://forums.civfanatics.com/thre...p-for-me-to-play.620140/page-15#post-15267420

I've even lost 100% battles and a 99.9% chance Great General before, but it is super rare.

 

[BrendanG]

Okay, so I did do more battles like I said I might in a follow-up comment. I also tried to remember how I did statistics at university, and refreshed myself on wikipedia, and I think I figured out how to properly calculate the odds this time.

Kaitzilla is right that I ought to have won 6.664 of the 10 battles. I also calculated that the standard deviation is about 1.5 (meaning it would be normal to get 6.664 plus or minus 1.5 wins). So I got more than 2 standard deviations off. As I recall from Gaussian integrals, that means there was about a 2.5% or less chance that I would get that unlucky, or 5% or less that the AI or me would get lucky, which is close to the 3% and 6% I estimated earlier.

I then kept playing and wrote down the odds and results for 70 more battles, bringing the total up to 80. I used excel to help me with the math. I got that I ought to have won 67.4 battles out of 80, and I actually won 64. The standard deviation is 3.25. So I got just a hair outside 1 standard deviation, which is a little unlucky, but nothing terribly unusual.


So my conclusion is that the AI does not cheat and that I was wrong. However, I was correct to be suspicious of the first 10 battles I fought being so unlucky (less than 5% chance of happening like that) immediately after raising the difficulty level.


And about the game where I took the data: the AI was annoying and managed to capture 3 of my cities, but I eventually took them all back and 1 more of his cities. I would have kept pushing, but another AI attacked me too, so I made peace.

 

[s.bernbaum]

Also, don't forget that statistical results apply to large numbers, not individual instances. If you flip a coin once, there is a 50% chance of heads or tails but whichever one you get, if you regard that as the test, will look like 100%. Do it ten times and it will still look like something is wrong because you most likely won't get five of each. Do it 100 times and it will start to look like 50% may be correct. Do it a million times and the variance will be so small that the average will be 50%.

 

[Fippy]

I did some reading on rng in games, and it seems widely accepted that they are not "true" rng.
Comps have their limits, or crashes would never happen
So i'm not expecting the same randomness of throwing coins manually, rare streaks could happen more often than they should.
Losing at 100% supports this theory, ofc you would never lose a real 100% outcome (have 100 coins, all are heads but you get tails..not possible ~~).

But this happens with both good and bad luck, no AI cheating involved..just "pseudo" rng programming.
True rng must be very hard to program.

 

[6K Man]

I sometimes think this game would be better if there was no way to see the combat odds, replacing the win % with a qualitative indicator ("Good odds", "Bad odds", "Toss-up"), or nothing at all.

So many people have looked at the combat odds here; I'd be very surprised if someone was able to demonstrate an error in the combat engine. Most complaints about the combat engine are resolved by a misunderstanding of probabilities or problems with perception (i.e. loss aversion syndrome). Good to hear that you got past those pitfalls and resolved the issue for yourself.

 

[Powerfaker]

While you cannot avoid getting hit by extremely unlucky outcome, I think it is possible to raise the survival rate of your stack as a whole a little bit in your favour by sending in your units in the ''right'' order. By that I mean that you let the previous fight decide whether you send in a unit with low odds or high odds.

Some of the things you can think of:
When a unit with >95% odds wins with his hitpoints in the red, he took more damage than was to expect. So the next fight you might get relative less damage. This might be a good time to send in a 60% odds unit and save your next 95% odds unit for a tougher moment.
Opposite, when a unit with low odds surprisingly wins a fight, you may want to pick a high odds unit (probably not your precious general), fodder or a unit with flanking (which I suspect to reverse bad fortune to good fortune and vice versa)
Not that any of this guarantees anything, but in the long run I think you save more units. It's like better to have 10 units surviving in the red than 5 in the green.

 

[6K Man]

While you cannot avoid getting hit by extremely unlucky outcome, I think it is possible to raise the survival rate of your stack as a whole a little bit in your favour by sending in your units in the ''right'' order.

I'm with you up to this point. If I have a 9XP unit and a 5XP unit with the same promotions, I'll send the 5XP one in first if the odds aren't good. And I'll sometimes send in units with 8XP at 96% odds (so if they win they get to 10XP), units with 9XP at 98%+ odds, and so on.

Some of the things you can think of:
When a unit with >95% odds wins with his hitpoints in the red, he took more damage than was to expect. So the next fight you might get relative less damage. This might be a good time to send in a 60% odds unit and save your next 95% odds unit for a tougher moment.
Opposite, when a unit with low odds surprisingly wins a fight, you may want to pick a high odds unit (probably not your precious general), fodder or a unit with flanking (which I suspect to reverse bad fortune to good fortune and vice versa)
Not that any of this guarantees anything, but in the long run I think you save more units. It's like better to have 10 units surviving in the red than 5 in the green.

I'm sorry, but that's voodoo. There's no reason why you should get a good result after a bad one or after a string of bad ones, or vice versa. If you flip a coin six times and it comes up heads six times, the odds of a head on the 7th flip are still 50:50.

 

[Powerfaker]

@6K Man
It may be 95% or 99% voodoo but not entirely. Loosing a battle at high odds means you've noticed 6 or 8 (or something like that) coin flips of ''all heads'' in a row within the fighting mechanism. It makes sense to expect some balance of some kind. But I agree rng is rng. I wouldn't expect a huge gain from this approach.

I'm working on a test. So I'll come back on this. (maybe to prove I was wrong, who knows)

 

[Lennier]

Powefaker: You won’t get any gain from that approach. The next coin flip doesn’t remember the previous ones.

 

[rah]

But this isn't a coin flip, it's a RNG. Some random number generators use a seed number like the one in CIV. The next number is based on the previous number selected. This is why you have the option to use the same seed when you reload. If you don't it changes the seed number. So if you start with the same seed, the next 10 number selected will always be the same 10 numbers. So some might say that it actually does remember the last roll.

 

[Lennier]

Technically correct; but you can't predict whether the next battle would be won or lost at 99% odds based on what happened on the last battle. So the rng "giving balance" after you win a low-odds battle by having you loose the next battle is not what happens. I.e., you can't predict the outcome of the current battle based on the outcome of the previous battle(s).

 

[rah]

Never said it could, just said that it technically remembers. I design with them frequently.

 

[lymond]

I admit to having a very cursory understanding of RNG, but did not realize that the next seed number is based on the previous seed number. Is that really true? In what way? Is the former seed number factored in the new seed number's calculation, or is the the former number excluded from new calculation. In other words you can't get the same seed twice in row?

If prior seed in some way does impact new seed, could it not go either way - more positive or more negative result than the prior.

I'm also not a expert numbers guy in terms of probability and statistics. But is not the RNG completely separate from the combat odds. I mean the seed has nothing to do with the odds, I assume. So I fail to see what one has to do with the other or why the result of one turn affects the results of the next turn. I've always assumed random was random.

At a very basic level, I've always assumed that RNG was similar to a roll of the dice (however many sided) that produces a number each turn and each action/result. Said number is plugged into some formula for producing results of a certain event/action like a combat. The result of said formula, which may be iterative based on rounds of a single combat, includes the other variable factors (strength, defense, terrain, etc.) that affect the odds, which ends in a final result - loss or defeat or in some cases withdrawal.

 

[ArchGhost]

Well RNG in a game like this is somehow seeded off the processor's clock timing. It's not technically random as you could recreate the state of the seed, the resultant sets of outcomes are deterministic regardless of the algorithms used to generate them.

And if that algorithm, such as for the combat system, kicks out multiple sets of results off that one seed, even though they can change somewhat, they are always within that same set from that seed. I think this is more along what rah is saying, but I don't code.

I think those more concerned with true randomness vs. determinism look toward physical or natural processes instead due to their volatility. Random.org uses perturbations in the atmosphere or something.

As for how sensitive something like the combat system in Civ4 is to this or the quality of its RNG...couldn't say.

I just know that it sucks to lose various advantaged rounds on the first try repeatedly throughout the same game, just to win completely BS low odds when they don't matter. Either our brains are pretty biased, or the "odds" are pretty poorly calculated. I like to think of them as lies, mostly.

 

[MAvL]

I admit to having a very cursory understanding of RNG, but did not realize that the next seed number is based on the previous seed number. Is that really true? In what way?

Typically, PseudoRNG are arithmetico geometrical sequences modulo :

u_{n+1} = a * u_n + b [c]

Where u_n is the seed, u_{n+1} is the next seed, a and b are big constant, and c is a very big one. Usually, a and c are prime. So the seed typically cycle through every numbers between 0 and c-1 (excluded) in an erratic way.

You dont use the seed as is, you typically pass it through a modulo no bigger than big (order of magnitude of a, so very small compared to c), so you effectively cannot discern it from a dice roll for reasonably small samples (which can be very large for practical use).

But is not the RNG completely separate from the combat odds. I mean the seed has nothing to do with the odds, I assume

So yeah, the seed should be completely separate from combat odds. RNG, it depends on what you call RNG (a RNG can provide only the seed, or both the seed and the seed passed through a modulo, and that number can be directly used and is separate enough from the seed), but you got the right idea.

 

[6K Man]

I think the discussion of how a RNG works compared with a roll of a physical die, is a bit of a red herring.

The RNG in Civ4 is not truly “random”, as lay people like myself understand the word. Unless you check the “new random seed on reload” box when starting a custom game, you’ll get the same results from the same actions, if you repeat them in the same order from the same savepoint. But when one views a series of numbers put out by the RNG for the first time, they will appear to be random, and for most practical purposes they ARE random. They may be predetermined, but the observer has no way of seeing or predicting the next number in advance.

With that said, a RNG isn’t programmed to make a good result more likely after a series of bad results. Or the converse. Unless, of course, the RNG in Civ4 *WAS* specifically programmed to favour bad/good results in certain circumstances, but there’s been no hint of that in any of the Civ4 reading I have done over the last 10+ years.