Underdawg
King
20 posts in and no sight of it yet?
Combat odds - AI cheating?
- Thread starter Simple Simon
- Start date
Gozert
Chieftain
Maybe my 'n' is still too small @ roughly 300! Personally I think that the probabilities should be roughly correct for +/- 40 fights on small maps, and maybe ~200 on large. Otherwise, luck plays such a big role in the game!
I'm sorry, but that is not how probability works.
Simple Simon
Simpleton
I'm sorry, but you can well define random number generators with more or less 'spread', so that you averages are consistently hit at different 'n'. Basically, you define how broad the standard deviation is.
It isn't a matter of how the random number generator works, it's a matter of the underlying laws of probability.
Winning or not winning is a binomial distribution. If you fight 40 fights, it is pretty easy to calculate the EXACT chance of winning any number of battles. The RNG will be close enough to attaining the true mean that you can use the rules of the binomial.
For large samples, people usually approximate the binomial with the normal; of course there is always debate of how large is large. If the probability of winning is p, then the variance for N trials is N * P * (1-p) and the standard deviation is the square root of that quantity.
In practice, using the normal for a binomial for 40 trials is Ok, and for several hundred is qutie good. (Note that these 'good' and 'not good' are of course based on the need for accuracy. For a gamer, these are OK. If you are building lunar landing modules, you need more precision).
So, if your chance of winning a battle is .8, and you have 100 trials, your expected number of wins is 80. Your standard deviation is 4; so, you should be pretty confident that you will win roughly 72-88 battles, and a range of 68-92 is very sure. But more precision that that isn't possible even if the RNG were absolutely perfect.
To get the chance of winning exactly 80 battles, you need the true binomial. In this example, it's about 10%.
People here talk too much about the RNG not being 'really' random. Even if there is a slight bias, and the probabiltiy should be .8 but because of flaws in the RNG process its actually .79 (a fairly large flaw in practice), it will take thousands of trials to see the difference. The variation you see is from true randomness of a binomial process, not from a poorly programmed RNG. I see people doing these trials, winning 75 times, and saying, 'Look, the RNG is broken!' No, it isn't.
Best wishes,
Breunor
Simple Simon
Simpleton
Breunor: I do not know how the RNG is programed. And frankly, I do not care. You can well produce apparently 'random' numbers (and so far I have never seen truly random numbers in any game) and fiddle with the standard deviation. If this is not the case in CIV (as you post implies), then I have bad karma. If it has been fiddled with (as I expect), then the fiddling sucks.
I remember finding a significant difference (p<0.001) in fights in Civ 3 between how the RNG was supposed to work and the results. Later it was shown that the combat system worked slightly differently, giving other results based on a good RNG......
I remember finding a significant difference (p<0.001) in fights in Civ 3 between how the RNG was supposed to work and the results. Later it was shown that the combat system worked slightly differently, giving other results based on a good RNG......
Simon, you may be correct, but I think that is very improbable.
I believe that the dozens of people here who have performed methodical tests (and have a far greater understanding of probability than you) are the ones that are correct.
Civ4 does not cheat with the RNG.
I believe that the dozens of people here who have performed methodical tests (and have a far greater understanding of probability than you) are the ones that are correct.
Civ4 does not cheat with the RNG.
Especially since, as I have shown, that bad results are many times more likely to come from a run of bad luck than they are from a poor RNG unless you have thousands of tests.
If your evidence is simply that your results show you win less than the mean, what criteria are you using to say it is from a bad RNG rather than bad luck? Clearly, the mathematics support that bad luck is far, far far more likely (again, until you get to literally thousands of trials). Even on a few hundred trials, it isn't even close.
So, again, I ask, why are you concluding it is a bad RNG?
To be clear, an RNG that is 1% off is VERY bad, we could do better than that with techniques around in the '60's and early 70's.
Besty wishes,
Breunor
). Do I care? No. I've had a win at .2% before along with plenty of wins at odds below 50% (though I don't feel there have been too many or too few wins).Airefuego
King
As you say Simon, you may also not be getting your "good karma" as you may only be attacking with odds in your favour, so you never get "unlikely" wins, only "unlikely" losses.
I suggest bringing more "cannon fodder" units to your battles - you know, axes with only 3 XP, etc. A lot of players send in these disposable units first, even at around 30-50% probability, to give their more valuable units a better chance when they attack second. Do this and you may find that you start winning a few "lucky" victories with your hapless fodder!
This is not just to make you feel happier while playing, it's also gonna help you win wars
I suggest bringing more "cannon fodder" units to your battles - you know, axes with only 3 XP, etc. A lot of players send in these disposable units first, even at around 30-50% probability, to give their more valuable units a better chance when they attack second. Do this and you may find that you start winning a few "lucky" victories with your hapless fodder!
This is not just to make you feel happier while playing, it's also gonna help you win wars
MrCynical
Deity
Simple Simon said:
The combat odds as specified in game are accurate - numerous tests on significant sample sizes have been carried out, and this has been shown to be the case. Your OP doesn't really contain much useful data for the simple reason you haven't listed a sample size. Try a few tests yourself in worldbuilder if you still don't believe it.
These threads always come down to the same point; the average human has a very poor understanding of probability, and also a selective memory. This is exacerbated by the AI's tendency to attack at lower odds than a human player is willing to.
The combat odds have been shown to be accurate (there was an error in the very earliest versions of vanilla Civ 4 involving first strikes, but that was corrected in an early patch).
Wow I'm suprised this thread survived, anyway, Simple Simon, go back to your days in Middle School (even in second grade I've seen one probabilty problem) where you learn its the chance of it happening. It does matter if its 99.9 repeating there is that still niegh (nearly) impossible victory. Unless it says 100% there is always a chance of winning/losing.
reverend oats
King
:suicide: You get one every week...
MyOtherName
Emperor
- Joined
- Dec 7, 2004
- Messages
- 1,526
Technically, even something with 0% odds has a chance of happening. You just don't expect to ever see it, no matter how often you try.
This would not happen if the 0% had 0.000.... and so on with no other number other than 0 following it. That's why the game shows us 99.9%. It's representing that the defenders chance is 0.1%. One could say this is a 0% chance and still have a chance of happening.
And on the subject of Randomness, Computers can't truly be random. Argue that statement all you want but, it just can't happen.
grumbler
Prince
Psyringe
Scout
Simple Simon: First of all, thanks for putting the question mark in the thread title, and thanks for actually keeping a list instead of making totally unsubstantiated claims based on nothing but anecdotal evidence. We get this a lot here, and that's probably why your post met a somewhat harsher reaction than it deserved. Many CivFanatics are simply fed up with people making unsubstantiated claims about alleged AI cheating when in fact they either didn't understand the game rules, or didn't understand how probability works, etc.
You kept a list to test whether the AI *really* doesn't cheat. That's pretty similar to what we did when Civ4 came out. We made testbeds, we ran experiments. We wanted to be certain that there really weren't hidden AI cheats. By doing that, we even found bugs (in the combat odds calculation as well as in the AI city site choosing algorithm), but these bugs have long been fixed. We also *did* find some undocumented AI advantages, but these are really minor (e.g. 10 hammer start bonus for building first warrior).
Apart from these bugs, in all these years, no reproducible experiment has ever shown an AI combat bias. Plus, the AI source code is open for scrutiny and doesn't contain such a bias either.
Therefore, I tend to see the result of your list as random deviation (especially because your sample size wasn't that large).
Feel free to prove us wrong. You can easily enter the worldbuilder and create a situation where you can perform 1,000 combats with 80% odds of winning with only a couple of clicks. Use the "stack attack" option, turn off all animations, and use the "Statistics" screem to count your losses. Turn off random seed preservation, reload, an repeat the experiment a number of times. Then report your findings and upload your testbed so that we can reproduce them.
It's not too difficult to do (I've done it myself, but for vanilla Civ and for 50% odds), and it's a good method to either convince us that, despite all previous results, something's wrong with combat, or to convince yourself that it isn't.
You kept a list to test whether the AI *really* doesn't cheat. That's pretty similar to what we did when Civ4 came out. We made testbeds, we ran experiments. We wanted to be certain that there really weren't hidden AI cheats. By doing that, we even found bugs (in the combat odds calculation as well as in the AI city site choosing algorithm), but these bugs have long been fixed. We also *did* find some undocumented AI advantages, but these are really minor (e.g. 10 hammer start bonus for building first warrior).
Apart from these bugs, in all these years, no reproducible experiment has ever shown an AI combat bias. Plus, the AI source code is open for scrutiny and doesn't contain such a bias either.
Therefore, I tend to see the result of your list as random deviation (especially because your sample size wasn't that large).
Feel free to prove us wrong. You can easily enter the worldbuilder and create a situation where you can perform 1,000 combats with 80% odds of winning with only a couple of clicks. Use the "stack attack" option, turn off all animations, and use the "Statistics" screem to count your losses. Turn off random seed preservation, reload, an repeat the experiment a number of times. Then report your findings and upload your testbed so that we can reproduce them.
It's not too difficult to do (I've done it myself, but for vanilla Civ and for 50% odds), and it's a good method to either convince us that, despite all previous results, something's wrong with combat, or to convince yourself that it isn't.
Except unlike Civ III, with Civ IV they gave us the source code with all of the combat resolution in it. So if you really want to know if, how and when the AI 'cheats' all you have to do is look at the source code.
And, unfortunately, your +/- 300 fights in a game are irrelevant for tracking combat odds unless every single one of them was identical. The same types of units, same promotions on those units and the same terrain for the defender in every single fight. If you have three fights, all with a 99.9% chance to win and one is a swordsman attacking an axeman, the next is an archer attacking a warrior and the third is a chariot attacking a swordsman on a hill, losing all three doesn't mean the odds are off, it means you have bad luck. Now, if you have 100 battles with your 100 0 xp archers attacking your opponents 100 0 xp warriors, all of whom are on identical terrain (let's say plains) and you have an 80% chance to win it would be reasonable to assume that you would lose 20 of them, maybe 19 and maybe 21 but it'll be close. But, if even one of those is a repeat fight, say to avenge one of the 20 losses then you didn't run through 100 battles under identical circumstances.
So, looking at it that way if you have a unit attacking another unit and you have 80% odds in your favor it is entirely possible to lose that battle. Hell, 20% is a good chance, if I had a 20% chance to win the lottery you can be certain I'd be buying a crapload of tickets. Even a 10% chance is substantial if you think about it. The point is that unless you repeat the battle so that EVERY variable is IDENTICAL no amount of tracking means anything. In other words, if you lost 200 out of 300 different battles with 90% odds... it sucks to be you, you have horrible luck but the AI still isn't cheating and the odds display still isn't wrong or misleading you are misunderstanding it.
Edit: Perhaps for my mod I'll display erroneous odds that are reduced by 10-15%
And, unfortunately, your +/- 300 fights in a game are irrelevant for tracking combat odds unless every single one of them was identical. The same types of units, same promotions on those units and the same terrain for the defender in every single fight. If you have three fights, all with a 99.9% chance to win and one is a swordsman attacking an axeman, the next is an archer attacking a warrior and the third is a chariot attacking a swordsman on a hill, losing all three doesn't mean the odds are off, it means you have bad luck. Now, if you have 100 battles with your 100 0 xp archers attacking your opponents 100 0 xp warriors, all of whom are on identical terrain (let's say plains) and you have an 80% chance to win it would be reasonable to assume that you would lose 20 of them, maybe 19 and maybe 21 but it'll be close. But, if even one of those is a repeat fight, say to avenge one of the 20 losses then you didn't run through 100 battles under identical circumstances.
So, looking at it that way if you have a unit attacking another unit and you have 80% odds in your favor it is entirely possible to lose that battle. Hell, 20% is a good chance, if I had a 20% chance to win the lottery you can be certain I'd be buying a crapload of tickets. Even a 10% chance is substantial if you think about it. The point is that unless you repeat the battle so that EVERY variable is IDENTICAL no amount of tracking means anything. In other words, if you lost 200 out of 300 different battles with 90% odds... it sucks to be you, you have horrible luck but the AI still isn't cheating and the odds display still isn't wrong or misleading you are misunderstanding it.
Edit: Perhaps for my mod I'll display erroneous odds that are reduced by 10-15%
Simple Simon
Simpleton
I did just that in the last two games. Not that I won many of those fights, not even a surprising number, but I did see some pretty weird stuff, e.g. a city attack with 21 fights in all, in which I lost ALL fights at >50% and won ALL at <50%
OK, I'll accept everyone's word for it that the RNG is a true RNG, and that it is doing as it should!
@Psyringe: Thanks! It is great to hear that some people ran extensive tests, and it is also great that some of them do not immediately yell 'use the search', but rather point out the facts in a friendly manner.
I did extensive tests in Civ 3, and found that the RNG is quite good, but maybe not suitable for small maps. Due to the small number of units you have a small draw from the RNG, so luck became such a big factor in the game that I never played small maps. Losing two out of 5 units early in the game at 0.5% odds can be a real fun and game stopper. CIV seems better, because of the changes to the combat system, but it is still annoying to have a highly promoted unit (e.g. combat CR3 sword) killed early on by a 0.1 health archer in a defenseless city.
I guess I'll have to live with the RNG and the combat system as it is, and simply remember to tweak the odds my way by playing large maps.
Simple Simon
Simpleton
you think you're a funny one, hu? What's that supposed to mean - that I am a liar? If you'd asked politely I would post saves - but since you're being an arrogant a**: Go play your own games and keep your eyes peeled - but probably you are anyways not able to distinguish between different unit types, and haven't found out about promotions, so you can't have these things happening to you
Moderator Action: Flaming - warned.
Please read the forum rules: http://forums.civfanatics.com/showthread.php?t=422889
