As promised, a Civ III combat test map

[cephyn]

Originally posted by hwinkels
For each battle, a number is calculated for the defending unit (I have no idea how that number is calculated). Then, for each whack that the attacker takes against the defender, a random number is generated in the range 0 .. 1023. If the resulting number is greater than the number calculated for the defender, then the attacker scores a "hit" and the defender's Hit Points get lowered by 1. Otherwise, the defender scores the "hit" and the attacker's Hit Points are lowered by 1. First one to 0 loses the battle. All this obviously depends on exactly how the defender's threshhold value is calculated by the game. If there's a problem with battle results, then that is most likely the area where the cause of the problem lies. It's not in the RNG.

OK here's my question -- does the defender get 1 number per battle, or 1 number per round? ie, does he get a new random number each time a hit point is lost, or does he work with only 1 random number?

If the Defender gets only 1 number, it would explain how some battles seem so lopsided -- the defender simply has a realy high number that im rolling against over and over (like Def has a 1000 and I'm trying to beat that -- of COURSE my cavalry isn't going to scratch that pikeman.)

 

[Hurricane]

I think this is how it works:

Imagine you are attacking with a Swordsman against a fortified Spearman. The Civ3 combat calculator shows a defence strength of 2.7 for the spearman, i.e. it has a 45.1% chance of winning. If the calculator is right the number generated for the battle should be 462.

Each random number generated (1 per round of battle) is compared against this number 462. If it is higher, the attaker wins the round, otherwise the defender.

So the number you are talking about is (i hope) not at all randomly generated, but according to some set of rules. However, if these rules are screwed up in some way (so that it sometimes becomes incredible high or low), strange results will occur.

 

[etj4Eagle]

Good post hwinkels. I am glad that someone was able to final take a good objective look at the issue, instead of us having to rely on the subjective end results. The only question I have, is it possible for someone's particular system to effect that random number calculation? Since the problem with screwy combat results (or other things) appears to only occur to a subset of the players. I myself have not seen any of these extreamly improrable events (maybe I am just lucky or maybe the random numbers behave better on my system?).

 

[DarkwingGT]

Excellent post Hwinkels. I'm glad that you could answer not only my questions about the RNG, but also add insight and solid evidence to back up your answers.

This of course leads to the next question, since the RNG seems not to be at fault, how is the defense number calculated? If it is a hard and set formula, then everytime we use a cavalry versus a pikemen in the mountains for instance, it should always provide the same defense number, and statistically speaking, provided the same matchup in terms of units and terrain, no matter where on the board we should roughly get the same results.

Unfortunatly my last question to Firaxis still remains, is the random number modified in any way once it is returned from the RNG.

EDIT: BTW, if it is a hard and set formula, an option to display battle probabilities would be incredibly nice. As in if you turn it on, a window will appear on the screen. And whenever you hover over a unit that's not yours, it can show you the probability of winning. Or maybe show it in the unit screen in the bottom right. And naturally you'd have to have control of an attacking unit to see the information. I point to Fantasy General as a great example of that.

 

[etj4Eagle]

Originally posted by DarkwingGT


EDIT: BTW, if it is a hard and set formula, an option to display battle probabilities would be incredibly nice. As in if you turn it on, a window will appear on the screen. And whenever you hover over a unit that's not yours, it can show you the probability of winning. Or maybe show it in the unit screen in the bottom right. And naturally you'd have to have control of an attacking unit to see the information. I point to Fantasy General as a great example of that.

SMAC had a similar setup, and it was really nice. It also would yell at you if you tried to engage in a battle with bad odds.

 

[DarkwingGT]

I loved SMAC. (Hehe, I'd hate to have that taken out of context)

I wonder why Firaxis scrapped so many of the nice features of SMAC. I guess they don't want to homogenize their products, but it seems that CivIII could really have benefited from some of the SMAC niceties.

I still want my "Do or Die" mode. And back to the topic, Dan, would it be difficult to add the % of winning a battle in a patch? Maybe an expansion pack?

 

[etj4Eagle]

Originally posted by DarkwingGT
I loved SMAC. (Hehe, I'd hate to have that taken out of context)

I wonder why Firaxis scrapped so many of the nice features of SMAC. I guess they don't want to homogenize their products, but it seems that CivIII could really have benefited from some of the SMAC niceties.

I still want my "Do or Die" mode. And back to the topic, Dan, would it be difficult to add the % of winning a battle in a patch? Maybe an expansion pack?

I am guessing that they felt some of the features would not work well in this game or were not appropriate for the audiance. I believe Dan has said that the Civ series have a broader target audience than SMAC. (Of course this is not to say that I don't miss those features, I think a game combining the best of both worlds would be great). There was also some mention that they did not have the code base, or could not use the code base from SMAC.

However I do agree with you that I would like to see that % added. Doesn't have to be pretty (maybe as an option to toggle on). But I think that would end many of these complaints when people realize that they only had a 70% chance to win the battle.

 

[hwinkels]

A couple of answers:

The defender's threshhold value is calculated once per battle. I haven't seen it change during a battle.

I should have said if the random number is greater than or equal to the defender's threshhold then the attacker scores the "hit". So ties go to the attacker.

When the RNG is called for battle, there's an extra parameter passed to the routine that is a string pointer to "Normal Combat Rand", "Air Comabat Rand", "Enemy Unit Bombard Rand", etc. In the retail game, this parameter is never used. There's also an unreferenced subroutine that references "logfile.txt". There are probably debug builds that the developers use that can log these things, but alas, they're not available to us! It's a pain to patch the .EXE since it's protected with SafeDisc. I'm trying to do it through a tricky trainer-type program, but even if it works it's not something that could be distributed as a simple patch. SafeDisc effectively prevents any user-made patches from being used.

Long story short: it would be extremely difficult for users to patch their Civilization.exe files to enable logging of battle results.

The actual formula for calculating the defender's combat threshhold is no doubt quite complicated. There are things that go into it that we know about like terrain type, fortification, city walls, across-a-river, base unit strength, regular-veteran-elite, etc. Then there may be other factors we don't about. In any case, it's way beyond my ability to figure out! :crazyeyes

I'm making progress on my substitute RNG. I could try to make it check for the "Combat Rand" strings and do some logging. Trainers change the data a program uses. Changing the code is harder (but not impossible).

 

[Grey Fox]

Can you re-compile the EXE-file without the source-code?

Can you "Break" out the code or what?

Seems like there is no safety if thats true...

Might be the reason why Cracks comes so fast to games... must be a pain in the ass for game-developers... (Especially as I might be one in the future... )

 

[Peteus]

Originally posted by cephyn
Riddle me this, though, I thought the save/reload reset of the RNG could be accomplished by exiting the game entirely, and then going back in -- therefore, to get a new seed, shouldn't all we have to do is load up the sav game, sav it AGAIN, bounce out to windows and then back in, and load up the re-saved game? If this doesn't work, why am I wrong/confused/babbling?


Also, should we be tracking this HP by HP or overall battle? For example, is it sufficient that the tank always wins, but does so with only 1 hp left more often than not? Obviously, I think *we* need to try and work this out a little better, because it is NOT sufficient to say "tank wins 20 to 0" when in fact, it lost almost half the 'battles' (assuming the hypothetical case where it always wins with 1 hp left)

A) Someone (Dan, I think, maybe Soren) stated long ago that exiting the game would reset the random seed. I'm pretty sure that it does no such thing though, as I've even been able to post saves for other users who, sure enough, witnessed the exact same battle results/promotion.

B) Tracking the loss of each hp would be more significant as far as the way the random seeds are falling and their impact. We know that more hp is good - the point is how hard is it for one unit to hit another. The result of the entire combat is really a somewhat different question.

For those who asked about reordering the attacks to scramble the combat results, that's a good approach if you assume that the random number generator is working fine and what you are really looking for is the probability of a spearman defending against an attack. You have an essentially pre-determined set of 500 (or whatever) "dice rolls", and you can choose to apply any 25 of those to your tank vs speaman attacks, generating different results each time for a larger sample.
If your question is about the generator itself though (i.e. you're concerned that it's generating numbers between 1 to 1023 but never getting above 900, or some such, then using different (and equally flawed) sets of rolls won't prove much.

Did that make sense?! :crazyeyes

 

[Lt. 'Killer' M.]

Haven`t finished my tests with the test map - lots of work to do. But one strange thing from what I`ve seen by now: trippled HP, AI Spearmen attack Warrior (50:50), 10 out of 10 lose! Most only gave 1 or 2 damage to the warrior (remeber I trippled HP). if you assemble one of Zachriels attack forces and then go for a 2 Warriors in a city - you lose. Ouch!

I`m currently doing test for "strings" - i.e. does the generator favor one side for a long time. Seems to me it`s rather stringy. Same for the promotion/GL thing. Either you get lots, or zip for a long while......

 

[hwinkels]

Killer, any chance you could post that saved game? I'd like to check that out.

I finally got my substitute RNG in the game. There was no speed difference at all that I could see, so I'm backing off my previous statement that the developers screwed up by using floating-point ops. It doesn't seem to matter speed-wise.

Now I want to see if I can get it to do some logging so that effort won't be wasted.

 

[Lt. 'Killer' M.]

hwinkels: first of all: goodjob:

second: It`s Dan normal test map; I`ll try to post my moded bic file. But I essentially just went to "experience" in the rules editor and changes 2 to 6 (conscript), 3 tp 9 /regular) and so on...

then i ran the test map. I used the lower ten of each unit type to attack in that try, then ended turn and let the AI attack. I started with the warrior, then wroker my way upwards to tanks. The AI may fight battles in a different order though so you may see something different.....

ok, fingers crossed that the upload works.....

 

[robinm]

Ok , my take on Combar RNG:

1. This is anecdotal, I have not tested it or opened up the .exe.
2. It could all just be psycological, not a real phenomenaI admit.
3. I have a strong statistics background.


quote:
--------------------------------------------------------------------------------
Originally posted by hwinkels
For each battle, a number is calculated for the defending unit (I have no idea how that number is calculated). Then, for each whack that the attacker takes against the defender, a random number is generated in the range 0 .. 1023. If the resulting number is greater than the number calculated for the defender, then the attacker scores a "hit" and the defender's Hit Points get lowered by 1. Otherwise, the defender scores the "hit" and the attacker's Hit Points are lowered by 1. First one to 0 loses the battle. All this obviously depends on exactly how the defender's threshhold value is calculated by the game. If there's a problem with battle results, then that is most likely the area where the cause of the problem lies. It's not in the RNG.


--------------------------------------------------------------------------------

Fine so far. I agree with this.

I can also accept that in a long running test the Pseudo-RNG will produce a histogram of results approaching a flat distribution.

However I think Killer may have a good idea in the "stringyness" of the outcome. I think the attacker does roll 1 dice vs the defenders defense total for each round , and I think the defence total is largely based on the rules we half understand ( Units def , terrain , forts etc. etc. ) + a random factor that is calculated once per battle. This would make the defender feel "lucky" or "unlucky" for the duration of the entire battle.

In close battle you won't feel it so much if you look at just the battle outcome - eg elite warrior vs elite warrior, the % W/L of the battle will be as expected over a big sample, but if you look at the hp of the survivor I think you'd find the winner tended to have more hp's that statistically expected. I feel that there are battles occuring where one side or the other muches through every hp of the opponent without a scratch too often. On average this will not bias the battle % w/l against attacker or defender, but it increased the chance of what looks like a freak result.

Now if we consider a very unbalanced combat this effect should become more visible (all sums rounded to 3 dp

eg. spearmen , plains , nonfortified 1hp remaining vs 3 hp cavalry.
Now the rules as we know then give the following : attacker =6
defender = 2 * 1.1 (for the plains... I still am not sure about this 10% def bonus on flat land, is it real ?)
So, P(attacker wins any 1 given round) = 6 / ( 6+2.2) = 0.731

The cavalry must loose 3 rounds in a row to loose the battle.
P(loose a round = 1-0.731) = 0.209
so P(loose 5 rounds in a row = 0.209 ^ 3 = 0.019

Not very likely. If however the randon addition to the defense total exists, then the spearman might be feeling lucky , unlucky or average, as spearmen go. Lets make an assumption that the randon bonus can be * or / by up to 2

So lets take cases : total defense for spearmen = 1.1 , 2.2, 3.3,4.4

1.1 : P (cavalry looses 1 round ) = 0.155
P (cavalry looses 5 in a row ) = 0.004

So the "unlucky spearman" dies and we couldn't see the difference.

2.2 we know already, P (cavalry looses 3 in a row ) = 0.019

3.3 P (cavalry looses 1 round )= 0.355
P (cavalry looses 3 rounds in a row) = 0.045
4.4 P (cavalry looses 1 round )= 0.423
P (cavalry looses 3 rounds in a row) = 0.076

To summarise : % chance for wim by spearmen
def 1.1 = 0.4%
def 2.2 = 2%
def 3.3 = 4.5%
def 4.4 = 7.5%

So the skewing of the defense total by a random factor for the entire battle would actaully increase the overall battle odds in favour of the weaker unit , as its average, and unlucky scenariors both give it vanishingly small odds, but the lucky scenarios actaully give it an occasional chance of a victory which would stick in the mind of a player as "freak".

I feel these "freaks" do happen often enough to suggest the random defense modifier. Mostly I feel this as a result of evenly matched battles where the winner (attacker or defender , player or AI ) seems to win with no damage too often.

Again : sums : elite warrior vs elite warrior, plains no fortification.
I'm even going to forget the 10% plains bonus for simplicity :
attack = 1 defense = 1

P (a given side wins 1 round ) = 0.5
P (a given side wins 5 in a row ) = 0.037

3.7% of the time !!
Ask yourself... do you feel it happens more that this ?

Robin

 

[Loopy]

Long take robinm, but the big flaw is you're assuming that the RNG is used for the defender. There's no reason to think that that's the case. With your pseudo-case, the easiest way to do it (with what we know so far) would be to have the RNG generate one number. If it's higher than 749 (.731*1024) the spearman wins, if it's lower the cavalry wins. That look up percentage is a very easy calculation done in one line of code. Apply Occam's Razor to coding and that's how Firaxis did it. No additional lines for the AI to cheat, combat bonuses based on the phase of the moon, or rewritten "stringy" RNG that somehow generates non-idenpendent sets of random univariate numbers, keeping track of HP and previous results to influence it's RNG.

HP differences from conscript to elite have no effect on battle outcomes. If you win a battle with an elite unit with no damage, a conscipt unit of the same type will win with no damage. This is something that can be readily seen during gameplay. Combat works as described -- there's no voodoo involved.

 

[robinm]

Loopy : Quite possibly so... like I said it could be all in my head...


I suppose I could do some real stats if I could be bothered...... but I'm too busy.... _playing_ CIV
cheers

 

[Callas]

I'm not sure I regard this map as a valid test of the combat system, because it does not fully reproduce the Civ3 game environment.

The units here were created artifically and placed on the map.

If there is code in the game to artificially boost a unit *at the time of its creation in a city* then that code will not be being used, and so the results in game will differ to the results found from this map.

--
Callas

 

[Maven]

The important percentage number would then be the defense threshold for any given battle, compared to 1024. It would be interesting to see a chart of those threshold numbers. I expect the relevant factors would be attacking/defending unit types, and combat modifiers like fortified, rivers, terrain types, cities, and city walls. Just logging the results of the routine to calculate the threshold would be interesting.

The other question is the "stringyness" of the RNG. Does one of the RNG tests address this issue? If not, it shouldn't be too hard to build a test for one, now that we know the formula for the RNG. I suggest testing it against the infamous "Elite Tank vs Spearman" matchup. If we could somehow find the actual defense threshold for that setup. Maybe a breakpoint wouldn't be too hard for just one setup. I expect on a fortified spearman in forest, it should be around 161. Then you just run the RNG a huge number of times, and see how many strings of 5 lower than the threshold show up.

 

[hwinkels]

Here's what the documentation from George Marsaglia's (geo@stat.fsu.edu) Diehard (Version: DOS, Jan 7, 1997) says in part. I edited it for brevity; anyone interested in the nitty-gritty details can do an internet search for DIEHARD and find plenty of deatils!

This is the RUNS test. It counts runs up, and runs down, in a sequence... This example shows how runs are counted: .123, .357, .789, .425, .224, .416, .9 contains an up-run of length 3, a down-run of length 2 and an up-run of (at least) 2, depending on the next values. ... Runs are counted for sequences of length 10,000. This is done ten times. Then repeated.

In addition, the docs state...

With the exception of the RUNS test, which is a `standard' test and the only one of the standard tests I have found to be very effective, all tests here are those I have developed.

Interesting that some posters zeroed in on this aspect and that it is a significant consideration. Well done. The tests operate on 32-bit random numbers and you will note that the maximum battle value is 1023 - which is only 15 bits. The standard way to deal with this is just to keep adding bits and let the program examine them 32-bits at a time. I'll look for other runs tests and see what else I can find. There should be more out there if this is considered a "standard" test.

I tend to agree with Maven that the generation of the defender's threshold might be the more important factor. It seems like we want some absolutes, i.e. random numbers be damned, if a spearman goes up against a tank he should lose 100% of the time. Instead, with random numbers involved, there's always the chance that the spearman might actually win. This seems to be the case for any matchup where the only criteria is hit points. If a tank attacks a spearman and the spearman's threshold is set to 1, there's still a small chance that the attack rounds are going to result with a few 0's in a row and the tank is going to lose. Since ties go to the attacker, setting the spearman's threshold to 0 would guarantee that the tank wins, when attacking. BUT if the spearman attacks the tank, the only way to guarantee the tank wins is to make the tank's threshold 1024 or higher. I suppose you could do it this way, but it looks like this additional complexity was left out and they resorted to a hit-points-only battle model. I want to investigate how the defender's threshold gets calculated a little more.

 

[etj4Eagle]

Originally posted by Callas
I'm not sure I regard this map as a valid test of the combat system, because it does not fully reproduce the Civ3 game environment.

The units here were created artifically and placed on the map.

If there is code in the game to artificially boost a unit *at the time of its creation in a city* then that code will not be being used, and so the results in game will differ to the results found from this map.

--
Callas

This is assuming that there are intrinsic "cheats" in the game that create super units. You are making a wild assumption that this is the case and therefore dismissing the test map because of that. There has yet to be any verified reporting of anything of the like occuring. As others have pointing out if you have terrible luck in taking out a certain unit, if you reload the game and try attacking a different unit you will have the same problem. Also the developers have stated that there are no special cheats for the computer in the battle code.

The cause of the probem is one of three factors:
1) streaky random numbers (possibly system dependent?)
2) not fully understood combat precentage calculation/maybe a bug?
3) pyschology, we just think the problem is worse than it really is