I do not care what you say. (Part 2)

@Dirk - it's the displayed combat odds that are being questioned here as much as or more than the RNG.

@PieceOfMind - reading the first page closely I believe Bigv32 was attacking Washington Axemen with Boudica Axemen (which do get C1 in WB), for which the odds calculator displays 68.1%. For that, following the same calculations I get

For his first 100 fights:
win: ( 38 - 68.1 ) ^ 2 / 68.1 = 906.1 / 68.1 = 13.305
lose: ( 62 - 31.9 ) ^ 2 / 31.9 = 906.1 / 31.9 = 28.404
X^2 = 41.709
Q < 0.01% (The calculator computes Q = 0.01% for X^2 = 15).

For the 200 fights together:
win: ( 101 - 136.2 ) ^ 2 / 136.2 = 1239.04 / 136.2 = 9.097
lose: ( 99 - 63.8 ) ^ 2 / 63.8 = 1239.04 / 63.8 = 19.421
X^2 = 28.518
Q < 0.01%
 
re RNG:
RNG.png


re OP's test: not worth to discuss any further as long as his Grasslands give a dubious 10% defense bonus.
 
@Jet, I seem to remember from his first thread that the OP questioned the RNG. I don't think you have to do any trials to check the displayed combat odds? I don't know precisely how it's done but it's all in PieceOfMind's combat calculator. RNG is the only variable.

Thx DanF, RNG seems to work correct as expected.
 
@Dirk - He questioned both. Maybe the "or more" part of what I said was wrong.

@Dan - The fact that his grassland assumption was wrong doesn't refute his claims.
 
If the OP doesn't care what you say then he doesn't care what you say so why are you bothering to say it? Why did the OP bother posting it in the first place?
 
@Jet, I see nothing about displayed combat odds in the OP's threads It's not totally clear what he claims anyway, he does a test with axes where he gets the grassland defense wrong and then? What annoys me about all this is that he speculates on the grounds of insufficient data and incorrect assumptions and is suggesting as a result that the game programmers got it wrong, oh well.
 
As I have said on previous occasions, apart from 2 specific instances, there is nothing wrong with the combat odds shown in game, assuming the fully patched game.

The odds calculation has been verified by at least 2 mathematicians on these boards. When I developed the odds for ACO I did it completely from scratch, and lo and behold they agree perfectly with the game's calculator.

The two specific occasions where the odds are not correct are 1) when barb free wins are involved, and 2) when a siege unit can damage the defender exactly down to the combat limit. Due to a possible coding error, the siege unit has to win one more hit to do 0 damage to the defender to successfully withdraw. The odds calculator however assumes this strike is not necessary, thus showing slightly incorrect odds when it does occur. Details are here.

This all means that the only thing that can be questioned is the RNG directly, and that's why that is all I have spoken about in this thread (except in this post).

DanF, your test demonstrates the RNG performs well in one of the most important ways (convergence to mean proportion). However there are other ways RNGs can be inadequate, and being streaky is one of them. It's a lot more difficult to test though.

In an ideal world I'd love to have access to the source code for Soren's RNG but I cannot find it. I'm not even sure it's accessible from the code. If however I can repeatedly call it like you did, that may be good enough. Do you have instructions for how a dummy could carry out a simple test like the one you just did, because I'm not sure where you called it from?
 
@Dirk - it's the displayed combat odds that are being questioned here as much as or more than the RNG.

@PieceOfMind - reading the first page closely I believe Bigv32 was attacking Washington Axemen with Boudica Axemen (which do get C1 in WB), for which the odds calculator displays 68.1%. For that, following the same calculations I get

For his first 100 fights:
win: ( 38 - 68.1 ) ^ 2 / 68.1 = 906.1 / 68.1 = 13.305
lose: ( 62 - 31.9 ) ^ 2 / 31.9 = 906.1 / 31.9 = 28.404
X^2 = 41.709
Q < 0.01% (The calculator computes Q = 0.01% for X^2 = 15).

For the 200 fights together:
win: ( 101 - 136.2 ) ^ 2 / 136.2 = 1239.04 / 136.2 = 9.097
lose: ( 99 - 63.8 ) ^ 2 / 63.8 = 1239.04 / 63.8 = 19.421
X^2 = 28.518
Q < 0.01%

To be honest, I highly doubt any of the assumptions going into those calculations were correct. Given the incomplete data, even my calcuation was relatively pointless. However I assumed what was the more likely case. Really I don't think he won 38 out of 100 battles at 68% odds.

I suspect that when he lined up his units the enemy units probably had 10% fortification or 10% from a combat 1 promo or something like that, so he added combat 1 to all his units. I'm only guessing here but he probably did look at the odds and make sure they were 50:50. In that case the 63 and 38 results were not extremely unusual.
 
I did a quick WB test of 200 Boudica Axes vs 200 unfortified/open Washington Axes, so that the predicted odds displayed 68.1%, and got 148 wins (74%), which passes the test PieceOfMind gave (which looks reasonable to me, although I'm not fluent in significance testing) with a Q of 7.34%. Granted, the observations were not independent of the random seed, since they all came from the same game, but for the purpose of testing the rest of the system, the result supports PieceOfMind's hypothesis that something is wrong with the data in the original post.

In other words, Bigv32, my test failed to reproduce your results.
 
DanF, your test demonstrates the RNG performs well in one of the most important ways (convergence to mean proportion). However there are other ways RNGs can be inadequate, and being streaky is one of them. It's a lot more difficult to test though.

In an ideal world I'd love to have access to the source code for Soren's RNG but I cannot find it. I'm not even sure it's accessible from the code. If however I can repeatedly call it like you did, that may be good enough. Do you have instructions for how a dummy could carry out a simple test like the one you just did, because I'm not sure where you called it from?

Soren's code:
Code:
[SIZE="3"]unsigned short CvRandom::get(unsigned short usNum, const TCHAR* pszLog)
{
	...

	m_ulRandomSeed = ((RANDOM_A * m_ulRandomSeed) + RANDOM_C);

	unsigned short us = ((unsigned short)((((m_ulRandomSeed >> RANDOM_SHIFT) & MAX_UNSIGNED_SHORT) * ((unsigned long)usNum)) / (MAX_UNSIGNED_SHORT + 1)));

	return us;
}
with the defines 
#define RANDOM_A      (1103515245)
#define RANDOM_C      (12345)
#define RANDOM_SHIFT  (16)[/SIZE]

I can't say anything about its quality, I think you are the expert here. If you want to test things like I did above, use the python console (<Shift + "~">) with the game running in debug mode (set CheatCode = chipotle in ini). Happy testing! :)
 
Actually my point is more that the data given in the OP's post is mostly complete. It is just his description of how he set it up that is lacking. He did say the odds were 50:50, and the results of 38 and 63 make this very likely.

My argument is that the OP has not used this data to construct a proper argument. Instead he seems to be ignoring (possibly unintentionally) important points relevant to RNGs, invalidating his hypothesis.

My hypothesis, if you'd call it that, is that the RNG is adequate for the purpose of being used throughout this game. It is not perfect, as no PRNG can be, but I have yet to see any signficant evidence (i.e. not just anecdotal) that would differentiate it from a pure RNG (one that is perfectly random). As I briefly mentioned earlier, if an argument put forward to discredit a RNG could equally be used to discredit a pure RNG, then something is not right.

The cardinal sin that is committed by the more naive when examining RNGs is to consider results in a retrospective fashion. This is done pretty frequently on the boards. People will say, "You won't believe my luck! I lost two 99% battles in a row!". The person or another poster soon after will calculate that the odds of such a thing happening were 1 in 10,000 and say he/she was very unlucky. Technically the RNG was just doing its job and there was no luck involved whatsoever.

Examining a random sequence retrospectively, looking for patterns (e.g. battles lost in a row) is actually a flawed way to examine the quality of a RNG. Proposing tests that would strengthen or weaken a particular hypothesis, and then carrying out those tests is the more reliable method (I was about so say the superior :lol: Attacko rubbing off on me).

If one tossed a coin 100 times and then examined the resulting string of results, it is very likely one would be able to find some string that seemed 'unrandom' somehow. For example, TTTTTTTT, or THTHTHTH, or TTHHTTHH, or THHTTTTHHT etc. If you looked hard enough you would usually find something you consider odd that had less than 1 in 10 chance of occuring. If you could do this in every random sequence of 100 coin tosses, does it prove anything? All it proves is the human brain's remarkable ability to detect patterns quickly (arguably there is an obvious evolutionary reason for this which I won't discuss here), and that the way our brains are programmed to examine data is not conducive to detecting absence of true randomness.

Testing Civ4's RNG for correctness in monobit frequency (that is, over time it produces as many "wins" and "losses" as it is supposed to) is almost pointless. This is because a PRNG that did not adequately meet this condition is likely going to fail in most other regards. Also, a PRNG that did not even pass this test would be very easy to pick with relatively simple, cheap, tests. I would highly doubt Soren would code a PRNG that was flawed in the most basic of ways as that.

What is more likely to concern users is the appearance of streaks (sometimes called 'block' frequency) or periodicity (the random sequence will repeat after an unacceptably short time). I would guess that Soren's RNG is of the Linear Congruential type (LCG = Linear Congruential Generator - look it up if you're interested) - one of the most primitive yet decent PRNGs out there. LCGs to my knowledge can be prone to periodicity though certainly not on a scale that any of us would detect when playing the game. LCGs must have a maximum period but once again if I were to guess, I'd say Soren's RNG wouldn't have a period any lower than 10^6 or 10^7.

EDIT... I see DanF has given me the source code. Thanks!:goodjob:
 
DanF, that is great.

Just a couple things..

1) Do you know how the CvRandom() function is related to getSorenRand() or getSorenRandNum()?

2) Would you happen to know how big MAX_UNSIGNED_SHORT is?

3) Also, honestly I'm not exactly sure what is going on with all those operations in the calculation for the unsigned short. e.g. the & operating on the MAX_UNSIGNED_SHORT. I'm also not sure what the function's parameters are outside of the function's context. (usNum and TCHAR* pszLog)
 
MAX_UNSIGNED_SHORT is 65535.

Dunno about Q1.
 
You can still get very unlucky even in a perfectly random system...that's why it's called luck ;). That doesn't make it "not actually random" or "unfair", it just means you got screwed!

In practice RNG screwjobs are mostly relevant early in the game where getting unusual results is more damaging (IE it's a larger % of your military). In a stack battle involving collateral siege initiative and 50 units on either side, RNG luck isn't going to be a deciding factor...it might vary units lost by a couple and that's it.

But when a barb warrior gets "lucky" vs your archer, that can actually have a significant impact.

None of this matters re: rng though. This thread just shows more proof that it's functioning as intended and fairly. If anything is at fault, it would be the gameplay mechanics or player decision making allowing random chance to become an overriding factor in success.
 
:lol: DanF, do you understand what awesome power you have unleashed? :p

By the way, that does in fact look exactly like a LCG. The constants used are probably the most commonly used. I would imagine it's the same RNG that has been used since Civ1. Even MS Excel probably uses that RNG.
 
Yeah it's an LCG alright.

Python uses the Mersenne Twister algorithm which has a much larger period but that one is faster.

EDIT: That method returns an integer in the range 0 .. (usNum-1) by the way, and m_ulRandomSeed will be an unsigned long member variable of the class CvRandom.
 
Yeah it's an LCG alright.

Python uses the Mersenne Twister algorithm which has a much larger period but that one is faster.

Why do you say but? Aren't they both good things? Probably the only advantage of the LCG is its small memory cost, right?
 
Mersenne Twister is pretty fast and only uses a couple of hundred bytes of storage (edit: more like a K or 2 since it stores a buffer of 400 odd 4 byte numbers or something).

I've seen clumpy results with LCG random number generators before.
 
Hey, getting back to the math jokes:

Two statisticians are hunting in the woods. Seeing a deer, they both fire their rifles at it. One misses 10 feet to the left; the other misses ten feet to the right. They both high-five each other and yell "We got it!"
 
Back
Top Bottom