(what if) The rules should change!

Matrix · Apr 17, 2001

Hi GOTM players!

While I was working on the results of the GOTM III, I was dissatisfied about the results, because some players finished very early, but by no means have a high GOTM score. It also surprises me in a negative way that Cactus Pete ended so low in the GOTM II, while he made a magnificent prestation. Despite my conviction in that GOTM scoring system, I began to have doubts again.

So I began to experiment.

I kept up my score in the GOTM III of every turn. It looks approximately like an exponential progression. But that doesn't really matter much.

The point is the multiplier in my formula:

A=score
B=factor
C=finishing turn
D=max turns

GOTM score=A(1+B(D-C)/D)/100

The multiplier=(1+B(D-C)/D)

I didn't want to make the GOTM score too depentent on the normal score. So I raised B from 3 to 7. But that doesn't matter. The only thing I change is to decrease the effect of 1+, which makes the people who ended in 2020 have points too. If 1+ wasn't there, the score would almost be the same, expept that people who ended in/near 2020 wouldn't have a score. So that makes the this formula simply insufficient.

Then I realized the multiplier doesn't change lineair in time at all!! At least, not when you look at the right change. Let me explain: the multiplier should make the score drop every turn by the same amount! But look: (max turns=520)
Turn 50: multiplier=7.32
Turn 60: multiplier=7.20
7.20/7.32=0.98
So the multiplier drops by 2%.

Turn 500: multiplier=1.27
Turn 510: multiplier=1.13
1.13/1.27=0.89
So the multiplier drops by 11%.

Do you understand what I mean? The only way to make the multiplier drops constantly every turn is to make it drop exponentially. Puzzling with that idea, I came with a new formula:

GOTM score=A*B^((D-C)/D)

And what happened? All the - according to me - 'good' players came at the top of the list in the GOTM II. Also, when I used this formula in my results, it appeared that the GOTM score stayed constant in time (dropped when I did nothing, rose when I did well), while it kept rising when I used the old one.

The only problem with this system is to decide what B should be.

The problem
Should the new score be correct (I want as much opinions as possible ofcourse), we already have a results standing and medals rewarded. I can assure you that the medals will go to other players now (except Shadowdale; there isn't any system that makes him drop

). So I can't make that decision alone. It's your Game of the Month too.

To be honest, I think Kev en goodbye_mr_bond should be noble and thank for the honour but let people who deserve it more have the medals, should they loose their medal (only one guy will loose his medal).

Let's hear from you!

------------------
<IMG SRC="http://home.hetnet.nl/~maartencl/tmp/MatrixBW.gif" border=0>
Studying chemistry means: having fun, drinking beer, having more fun, drinking more beer, hang above the toilet and have a good night sleep!

And each time Pedro says: "Aaaaaaaaaaaaaaarrggh", I feel so good!!!

shadowdale · Apr 17, 2001

I like the idea!!!
And it's not just because I'm still getting a medal, but because it seems unfair that people who finish really quick don't get rewarded for their efforts!!!

------------------
Veni Vidi Vici.

Coolbook:
Håkan Eriksson, Stormerne, vladmir_illych_lenin, Cunobelin Of Hippo, vanillacube.

Jabah · Apr 17, 2001

I am in favor of anything that is not rewarding too much people who spend hours optimizing their score while the issue of the game is obvious (i.e. when winning is just a question of turns)...
I do sometime like to go for a very high score, but not every month ...
About the formula, I have no idea, but looking at the way you experimented (which quite like like what I suggested for the top GOTMII), if this formula managed to prevent your score from going up doing just the tremendious end of the game, fine with me.

Jabah

stormerne · Apr 17, 2001

I have strong views on this (which I've emailed to you Matrix). Do not read on if you're squeamish...

I don't mind changing the scoring formula provided:
*1* B in the new scoring system of the order of 40;
*2* The formula is applied to GOTM #4 onwards and NOT retrospectively to GOTM #2 or #3.

*1*
What happens if you make B much greater than 40 is that the ONLY way you can get a decent score is to finish early. Even if you play well but play long, you will be worse off. Is that good?

The existing scoring system dramatically reduces the effect of endgame perfecting when there is just one pet AI city left. This is fine. The new scoring system effectively penalises anyone who does that if B is high (Matrix has shown me examples of B = 100 and B = 750). This in my view is bad.

If B = 40 then the order remains largely unchanged except that KaaK, 17th in GOTM #2 suddenly goes to 10th. With B = 750 all hell lets loose and, having seen the graphs, it strikes me as rewarding only one kind of player.

It bears mentioning that a B of 20 gives results very similar to the existing scoring system.

I am against any scoring system which reduces the number of viable strategies a player can use. If Matrix's suggestions are taken, and his suggested B values are taken, we will all have to take over the world by horseback or not bother playing. I for one will no longer play GOTM if that is the case.

*2*
Is it fair that players who have deliberately chosen a particular strategy for GOTM #3, because they know what the scoring system is, should have a completely different scoring applied to their game after they have finished? No - I don't think so. To my mind, that is not fair play.

If you change the system, that system should apply ONLY from then on. All rankings and all awards already given should remain.

I would not like it if someone told me what "noble" was. (That's an absolute term is it? Hmm, news to me.) Kev and goodbye_mr_bond may not object, but I would. And it's rather debatable as to who "deserves it more". And perhaps these people would have played differently if they'd known in advance about the scoring changes.

------------------
<IMG SRC="http://www.anglo-saxon.demon.co.uk/stormerne/stormerne.gif" border=0>

Kev · Apr 17, 2001

I know that I am speaking from the standpoint of someone who will more than likely be adversely affected by scoring changes. However, I don't think that it's fair to change the scoring formula AFTER people have played the game. I would venture a guess that many people try to "play to the formula" where in essence they will play the way that they think will earn them the better score under the rules that are given.

Granted, some people will not change their style for any reason - some people love to go for an early finish while some love the space race. That's great and I totally respect that, but I think that several of us thought in GOTM #2 that the best strategy would be go for a high score and then try to end when things were maxed out. That this would give the best GOTM score (in fact, as I keep whining about, I didn't even realize the multiplier was 7 until well into the game. I probably would have closed out my game about 70 or 80 years sooner). I would daresay that many people have used this maxim while playing GOTM #3 as well - I know that I have.

If we had known of the new scoring system that you are suggesting ahead of time, I think that I would have played like many would end up playing: Kill fast and finish early. I didn't, however, because I felt it wouldn't give me a very good score. To say that the people who finished early in GOTM #2 are better than others is unfair in this case - they merely chose to play a different style. These "early" players were aware of how the scoring was to be done before they started (presumably), and had the choice to tweak their style if they wished for a higher score.

The only analogy I can think of is to say that in an (American) football game that after a game is played it is decided that field goals should be worth 5 points instead of 3 points and this would therefore change the winner of the game. This wouldn't be fair to the other team who may have tried to kick more field goals during the game if they had known ahead of time.

I hope I'm being clear in what I'm writing. I will agree to play whatever the scoring method may be and enjoy it no matter what. I think that a system such as the one suggested by Matrix would surely help those who move quickly, but keep in mind that players very well may decide to change their style and those who score highest would likely never reach past the 1700's. Not that Shadowdale would reach that far in any case

Perhaps we may want to switch the scoring system each month to promote different styles of play. That would be fine as long as each player went into the game knowing how things were going to be scored. However, I know that we are striving for consistency in a scoring method that will be fair for all styles. Given the complexity of the game, I think we all realize the daunting task that Matrix faces and none of us have any problems if the formula changes from time to time. I would just ask that we know as we begin our games.

OK, that's my 2 cents for what it's worth.

------------------
Diplomacy - the art of
saying "Good Doggie"
until you can find a rock

Marko Polo · Apr 17, 2001

Thanks Matrix for making those re-evalutions. It's vital that the GOTM factor is being reduced the same amount each turn. I'm sure that no one here objects that.

I also agree with Stormerne that the existing HOF and medals should remain intact if we are going to change the rules. Stormerne made a very good point on that in his post.

So, what's the situation now with GOTM score calculation? I've understood that we need a system that tries to reward all different game endings from early world conquering to late game big space ship landing on AC. It's also quite clear that the majority of players wants to avoid tedium of artifically lengthening the game with pet AI city to inflate the score.

So my suggestion is: use the revised new formula of Matrix's with a balanced 'B' factor - and (suprise, suprise!!) set a upper limit to population score.

And to please them who really enjoy to boost their cities, we can set that upper limit relatively high. Unless we do so, the guy who ends the game in 2019 with 15000 score will always win. No matter what. Most of us can boost the score over 10000 if we really wanted, but this ISN'T the point of GOTM by definition! We have the official HOF for those guys!

TimTheEnchanter · Apr 17, 2001

I agree that changes to the scoring should only affect future games. Just add a note to the earlier scores indicating that they were done with a different scoring system and describe what it was. Don't change the rules after the fact. People posted that they were specifically watching their GOTM score to know when to stop based on that scoring system.

It sounds like an "exponential" scoring system might be a good way to bring a better balance between unhindered (pet city) growth/perfection and neglectful conquest. The trouble is finding the right balance point. Different people play differently and will have different shaped curves.

One other point to consider...The right factor for the GOTM II King game may be way out of whack for an emperor or deity game. Make sure you evaluate more than just GOTM II.

Try to evaluate as much data as possible before changing the scoring. We can muddle through with the current system until we have a thoroughly researched and well thought-out reason for changing it.

------------------
DEATH awaits you all...with nasty, big, pointy teeth.

Matrix · Apr 18, 2001

Ok. Current results stay. Future GOTM's will have different rules (GOTM 3 still excluded).

Indeed B=750 is far too high. I'll settle for 50. But I disagree with a different scoring system every month. I endeavour to one best system, where different kind of players will be awarded equally.

Anyway, anyhow, I was also planning to make an extra list for the fastest players and players with the highest score. Should I include GOTM II for that as well?

------------------
<IMG SRC="http://home.hetnet.nl/~maartencl/tmp/MatrixBW.gif" border=0>
Studying chemistry means: having fun, drinking beer, having more fun, drinking more beer, hang above the toilet and have a good night sleep!

And each time Pedro says: "Aaaaaaaaaaaaaaarrggh", I feel so good!!!

stormerne · Apr 18, 2001

Just so everyone gets a flavour of this: Matrix's suggestion with a B=50 means that your GOTM score will drop by 1% every turn unless you do something to grow your underlying Civ score.

------------------
<IMG SRC="http://www.anglo-saxon.demon.co.uk/stormerne/stormerne.gif" border=0>

[This message has been edited by stormerne (edited April 17, 2001).]

shadowdale · Apr 17, 2001

I can probably live with what ever scoring system you guys come up with - as long as you don't change it midgame and as long as I know more or less what to do.
I still prefer to play the long game of Civilization and not the short brutal conquest kind of game - but I can see that they should be better rewarded!!!

------------------
Veni Vidi Vici.

Coolbook:
Håkan Eriksson, Stormerne, vladmir_illych_lenin, Cunobelin Of Hippo, vanillacube.

CornMaster · Apr 17, 2001

I would like to speak on behalf of everyone who doesn't give a rats *** :

"We don't give a rats *** ."

Thank You

------------------
<IMG SRC="http://images.honesty.com/imagedata/h/207/85/32078598.gif" border=0> I AM CANADIAN! <IMG SRC="http://images.honesty.com/imagedata/h/207/85/32078598.gif" border=0>
CivFanatics Moderator and Tech Support
CivFanatics Civ 2 Ladder
My Civ 2 Scenario Page.

Cactus Pete · Apr 17, 2001

I certainly appreciate the effort to reduce the benefits of caravan tricks, superfluous wonder building, pet cities, and/or delayed launches. They seem tangential, if not contrary, to the essence and spirit of the game.

In my last post in the GOTM #2 Results thread, I pointed out that the factor (D - C)/D [maximum turns minus finishing turn divided by maximum turns] does not work equally well at each game level. Because games are both more difficult and shorter at the higher levels, the difference between the earliest and latest finishes will be compressed, the (D - C)/D factor will therefore be smaller, and the effect of finishing early will be diluted at the higher levels (and why should it be?).

Matrix's new formula, which is certainly a real improvement, still uses this factor. Before we jump right into another formula shouldn't we at least consider avoiding this problem?

I haven't worked this through -- having just come upon this thread -- but my first thought is to let the players' submissions tell us how meaningful a given finishing date is. We might be able to accomplish this by devising a formula built around an approach such as:

1. Calculating both the mean number of turns taken by all the month's GOTM entrants and the standard deviation in the number of turns played

2. Then calculating each player's z score (That is, the number of standard deviations his finishing turn is above or below the mean turn of all the entrants)

3. Taking each player's z score (and about half would be negative, of course) and multiplying it by. . . Oh, I don't know. As a first approximation, how about. . . the mean civilization score of all the entrants

4. Finally, add (or subtract) this bonus (or deduction) from the player's own civilization score to arrive at his GOTM score.

If someone doesn't quickly point out a fatal flaw in this suggestion and make me feel foolish, the next step would be for Matrix to try it, or something like it, on the data from GOTMII and see what happens. (I'm certainly not proposing changing the GOTMII results.)

shadowdale · Apr 17, 2001

I would like to say to everyone that doesn't give a rats ***

"WE don't give a rats ***" <IMG SRC="http://forums.civfanatics.com/ubb/biggrin.gif" border=0>

<IMG SRC="http://forums.civfanatics.com/ubb/biggrin.gif" border=0> <IMG SRC="http://forums.civfanatics.com/ubb/biggrin.gif" border=0>

<IMG SRC="http://forums.civfanatics.com/ubb/snipersmilie.gif" border=0>

------------------
Veni Vidi Vici.

Coolbook:
Håkan Eriksson, Stormerne, vladmir_illych_lenin, Cunobelin Of Hippo, vanillacube.

[This message has been edited by shadowdale (edited April 17, 2001).]

Matrix · Apr 17, 2001

So what you're saying, Cactus Pete, is that that factor should depend on the results.
You know, that is actually a very good idea! I don't know whether I will use your specific suggestion for the exact calculation, but I'll certainly look into it.

Off-topic: I've attached a scanned piece of paper with noted of the GOTM II, Quake II servers, my study (chemistry), which was originally the front page of a report I made of an experiment. I just thought it might be funny for you to see.

------------------
<IMG SRC="http://home.hetnet.nl/~maartencl/tmp/MatrixBW.gif" border=0>
Studying chemistry means: having fun, drinking beer, having more fun, drinking more beer, hang above the toilet and have a good night sleep!

And each time Pedro says: "Aaaaaaaaaaaaaaarrggh", I feel so good!!!

stormerne · Apr 17, 2001

Good contribution Cactus Pete. Thanks!

The only flaw I can see straightaway - and it could be fatal - is that players will not know beforehand what they are aiming for, since Matrix would be calculating the scoring factors after the results are in. I think people do need to know before they start as it really does affect which strategy some people choose.

Perhaps this kind of approach could be used as an analytical tool over several months to see if fixed rules could be sensibly applied and if so what they might be.

------------------
<IMG SRC="http://www.anglo-saxon.demon.co.uk/stormerne/stormerne.gif" border=0>

goodbye_mr_bond · Apr 18, 2001

Originally posted by stormerne:
Just so everyone gets a flavour of this: Matrix's suggestion with a B=50 means that your GOTM score will drop by 1% every turn unless you do something to grow your underlying Civ score.

This seems fine to me, Stormene. It means that simple conquest *only* is not preferable to infrastructure-building, but that you should really do both at the same time. That's the whole point of the game for me. I don't see why a quick horseman finish is any less tedious than a slow over-developed finish. Other than the time-factor, obviously.

But more than that I'm afraid I can't contribute, and hereby give you (Stormene) my proxy on whatever scoring decisions are to be made. I'm hopelessly out of my depth with the math here, I'm afraid. I'm a literature guy

not a

. (

)

Cactus Pete · Apr 18, 2001

Stormerne,

The uncertainty factor is something I had considered. I suspect that after a game or two the uncertainty will only be at the margins. Consider the possibility that a little unpredictability might be a good thing. Players might be a bit more inclined to play the game the way they find most enjoyable.

oryx · Apr 18, 2001

I have yet another scoring proposal to float. I've been trying to follow this discussion, but it all seems very technical and hard (for me) to understand. So I'd like to propose something a bit simpler for the future contests.

Each game would be ranked in two categories each getting equal weight.
1. Standard Score (Wonders, population, etc) (The early finish bonus score will be ignored for this purpose)
2. Date of Finish (Conquer the world, Spaceship) (Retirements and AI Space ships will count as 2020 A.D.)
Points will be assigned according to the rank order in each category. Because each GOTM will have a different number of entries, percentiles will be assigned to standardize scoring between GOTM's. First place gets 100 points; last place gets 0 points and in between ranks get 0 to 100 points. The final score (maximum of 200) is the total of the 2 scores above.

I believe this scoring system will have the following advantages.
1. It retains the philosophy of balancing the perfectionist versus conquerors strategies.
2. It will be easier to understand conceptually (at least for me)
3. Running up huge populations will only be partially rewarded, because only your ranking will matter.

I have tried out this scoring method for GOTM 2 in an excel spreadsheet if anybody wants to see this.

For global ranking GOTM 2 and 3 results could be converted to a percentile score, based on the rank achieved scoring the original way.

------------------
"There is no tiddle-taddle nor pibble-pabble in Pompey's camp."

TimTheEnchanter · Apr 18, 2001

Originally posted by Cactus Pete:
Stormerne,

The uncertainty factor is something I had considered. I suspect that after a game or two the uncertainty will only be at the margins. Consider the possibility that a little unpredictability might be a good thing. Players might be a bit more inclined to play the game the way they find most enjoyable.

This was my exact reaction when I read it: By making it based on Std. Dev., that means everyone does their own thing without artificially playing to a set scoring system and the GOTM score just has to do with how you do relative to the others that month.

HOWEVER, I think there is a couple problems with standard deviations. We are probably not going to have a standard bell curve. Because of the 2020 limit, there will be a large concentration of times toward the high end, with a much longer "tail" for early finishes. This might give very early finishes a HUGE bonus because they maybe many standard deviations from the mean, while those who go all the way to the end may not be "penalized" as much because they will be closer to the mean. It's been 14 years since I took my one stats class, so if I'm way off base, you guys know what to do to me (

)

I still think the best approach is to keep the current scoring system for another month or so, continue to solicit ideas and evaluate them with the data from the first several GOTMs and then make a decision when looking at a larger set of data.

------------------
DEATH awaits you all...with nasty, big, pointy teeth.

Cactus Pete · Apr 18, 2001

Extreme non-normality of the distribution of the number of turns played is a potential problem. I suspect, however, that to a large extent it would be a self-correcting problem. To use the above example, if there were a concentration of players finishing late, then late finishing would be unlikely to be reinforced by a high score (unless one of the late scores was far superior to its kin, and that would presumably cause many of the kin to try a different strategy next game).

I like the simplicity and conceptual clarity of Oryx's approach. I'd have no complaint if it were chosen.

(what if) The rules should change!

CFC Dinosaur

CFC Drunkard

Emperor

is just a

Hired Goon

Chieftain

I...am...an Enchanter!

CFC Dinosaur

is just a

CFC Drunkard

Romulan Commander

Deity

CFC Drunkard

CFC Dinosaur

is just a

Ancient

Deity

Warlord

I...am...an Enchanter!

Deity

Similar threads