Old GOTM Scoring Discussions

[Matrix]

Cartouche Bee, Beard Rinker, though you both post on topic, you were answering each other on and on and that's very irritating!

Anyway, from experience of the Civ2GOTM, I can tell there's only one thing that can penalize milking: take the square root of the score, then give a bonus for early finish.

But as I've said many times before: I'm very reserved concerning a special scoring system for the GOTM. For Civ2 is was nescessary. This scoring system isn't that flawd.

But if someone of you can tell me how the in-game scoring system works, please do. All I'm interested is what the score is without the early finish bonus and how that early finish bonus works. Only then I will consider introducing a different scoring system.

 

[ainwood]

IIRC, the internal score works by giving you one point for each tile in your control, plus one for every specialist / content person, and two for every happy person. No other factors are taken into account. You score for each turn is summed, and divided by the total amount of turns that you have had. I believe that the score-per-turn is multiplied by a factor depending on difficulty level.
I can’t remember the exact details of the bonus system, but I believe that it is a straight mulitplier based on turns left and difficulty level. There is a utility (Sir Plebs???) that explains this quite well.
I think that the real “problem” with the scoring system is that the final bonus for an early finish pales into comparison with what you can get from just playing out the turns, whether you “milk” for maximum points or not.
Hence my idea that the bonus should be a proportion of the theoretical maximum points you could accumulate by milking.

 

[Ribannah]

IMHO the problem with the scoring system is that it tries to compare things that are incomparable.

Instead we could simply give a score of 100 to the fastest finish in each category of winning (including OCC) and rate others according to the turns left till 2050 divided by the turns the fastest player in the same category had left.

This way we would not have to find ways to compensate for different maps, civs, difficulty etc. and originality would be rewarded. Milking would, of course, disappear completely.

 

[cracker]

I would like to see the option of submitting a GOTM, but checking a flag to have the score be unranked in the weighted global rankings.

This would encourage people to play for the medals and submit the games even if they do not want to milk.

Special challenges like the OCC type game illustrate a great example where the unranked flag would increase submissions.

For the record, the scoring multiplier for early finish is not balanced properly. Even as we purge out the dogpiles and other past events that have driven uber high scores, we will find that balance between milking and the turn bonus is out of line.

There are a number of options for fixing the scoring problem.

In teh past I have proposed and described a feature called "tournament score" where firaxis could code in a page in teh histograph that would load a text equation form a simple text file and convert it into an equation similar to the power, points, and culture that currently display. This would simple add a fourth option to the drop down box on the histograph. The equation would use a short list of internal variable names that Firaxis could supply and support via a page of the civ3 website.

Techniacl the corps-de-hacquer (gramphos, et al) could provide this tournament score function externally using teh same coding base that supports the C3MT and/or mapstat.

 

[Beard Rinker]

How score is calculated has been posted before but I will summarize it here to save the trouble of finding it.

GameScore = FinishBonus + PerTurnScore

FinishBonus = YearsLeft * DifficulyLevel

PerTurnScore = Average of InternalScore for every turn * DifficultyLevel

InternalScore = Happy Citizens * 2 + Content Citizens + Specialists + ScoringTiles

Specialists = Entertainer, Tax Collector or Researcher citizen.

ScoringTiles = Number of tiles in your territory except sea tiles.

DifficultyLevel = 1 for chieften through 6 for deity


Some properties of the game's scoring formula:

• Potential score increases with difficulty level. Using difficulty level as a score multiplier is probably intended to compensate for the added difficulty with each level. In reality most players can can win just as quickly on any level up to their level of compentence. The higher difficulties translate into higher scores.
• Potential score increases with the number of scoring tiles. This has an unbalancing effect as the finish bonus remains constant regardless of map size. It may take a little longer to win on a larger map, but not significantly. What this means is for the average player, milking a game to 2050 is the best strategy for high score on large maps whereas quick conquest is the best strategy on small maps. For top players, milking is the best option on all but tiny maps.
• Scores from different games are not comparable. The main factor is map size and difficulty level but other factors like world age, water level, land configuration and starting position to name a few.
• Scoring is unbalanced. As stated above, the way the finish bonus is calculated has an unbalancing effect on how to achieve the best score. It is either fast conquest on a tiny or small map, or milk the game to 2050 on a standard size map and up. Space, cultural, and diplomatic are never the best option for achieving the best score.

As it stands now, the GOTM is a one dimensional milking contest. Most players like myself are not particularly concerned with their score and play simply because there is a large base of people playing the same game. It would be nice however if you could score well with a well played game in any victory condition.

Before considering changing the scoring system, we should state what we would like to achieve. If it is simply to discorage milking games, then a simple modification might be possible. However, I think most players would like a scoring system where a well played game with any victory condition could win.

I have listing these objectives of a scoring formula in this thread, third post down but this was for the tournament and may not completely apply. One other objective that might be added is the score should be calculated from a minimum of game parameters and use existing tools available today. I'm sure Matrix does not want to retrieve dozens of values from each game and we cannot count on the players submitting more than the most basic information (i.e. name, score, finish date & condition, saved game).

Edit: Corrected scoring formula.

 

[Beard Rinker]

Ribannah

I like the concept of rating people only against others with the same finish type. Your suggestion is not quite complete though as it does not include score. A game that takes slightly longer but has a significantly higher score should be rated higher. Also if the top score is 100 and the top player always gets that, how do you compare the top scores between categories. Another problem is this may be too difficult to administer.

I'm not so sure the different victory types are not comparable. Score is made from your accumulated per turn score and your finish bonus. These tend to work against each other; the longer you play, the higher your score and the faster you finish the higher your finish bonus. If per turn score and finish bonus were equally weighted then theoretically you could make up for lost finish bonus with added per turn score. In other words any victory condition could get the highest score, it would depend on how well the game was played.

However, if this balance cannot be achieved then your approach would work better.


Cracker

Retrieving various scoring statistics in an automated fashion would be the ultimate scoring solution. These tools are not available today and if they were there would be allot of work in determining what statistics to use.

 

[Cartouche Bee]

Beard,
FWIW, a content citizens also earns a point.

Matrix,
My suggestion was merely to compensate for the inadequate bonus awarded by the game (inadequate in that the milked games score more than the early finishes) and not a suggestion as the perfect scoring system. It is obvious to me that playing the game to 2050AD earns more points than early finishers in GOTM style games (unless of course we all of a sudden start playing tiny maps). It is also obvious to me that an early finisher needs some sort of early bonus to better reflect the accomplishment in the overall ratings. I know in my milked games, approximately when they could have been won early on (that fairly early win capability is part of a good milking, therefore a milking is a blend of early win and empire building), perhaps seeing both sides gives a different perspective that others. The current scoring system favors the builder and that may indeed be the best goal for GOTM style games anyway.

 

[MPF]

I like this discussion and some good points are being addressed but I miss debate on how we let the sort victory affect the score.

I feel that some victories are easier then others. In my last GOTM (number X) I have a score that will certainly not rock this friendly competition but achieved it for the first time with a cultural victory of which I am very pleased. It takes more effort so I think this should get a better award. So here goes.

Just an idea so dont nag me about the multipliers, the are just random choosen, because they only serve to illustrate how we can award some types of victory combined with an early finish bonus. Any suggestions are greatly accepted. If anyone feels like calculating what a good ratio is between these fine. I'm just suggesting the idea so maybe we can adjust the score outcome and keep diversity.

1,0 * (score + turns left) for DOM and Conquest victory as they can always be achieved by hacking through your opponents.
1,5 * (score + turns left) for space ship victory as this needs tech development so an really early finish is off.
2,0 * (score + turns left) for diplomatic victory as this needs good strategy (even early in the game) to achieve.
2,5 * (score + turns left) for cultural victory as this needs good strategy and big investment in culture in the game to achieve.

MPF

 

[Beard Rinker]

This formula I've proposed uses the average score and finish date as a benchmark for computing scores. It does not need a map size or any other type of modifier as the average will always compensate for that.

The game score and finish date tend to work against each other; the longer you play, the higher your score and the faster you finish the higher your finish bonus. If weighted equally there is no modifier required as they balance each other off.

The following are examples of games that would receive the same score using the formula I proposed.

Example 1:
Score and finish date in month a are exactly average. Score and finish date in month b are exactly average. The scoring formula would give these two games the same score even though month a's game might be an entirely different configuration from month b. The actual game score and finish date could be drastically different.

Example 2:
Score for player a is double the average and finish date is exactly average. Score for player b is average and the finish date has double the average number of years left. These two players would receive an identical score.

Example 3:
Player a's game is a milking game and is 4.5 times the average. His finish date is 2050 so he receives no finish bonus. Player b's game is a fast conquest and finished with 4 times the average number of years left. His score was 1/2 the average. These two players would receive an identical score.

MPF
The extra time it took to achieve your cultural victory is not wasted. The longer you play, the higher your game score. This should compensate for the lower finish bonus for taking longer.

 

[MPF]

Originally posted by Beard Rinker
This formula I've proposed uses the average score and finish date as a benchmark for computing scores. It does not need a map size or any other type of modifier as the average will always compensate for that.

The game score and finish date tend to work against each other; the longer you play, the higher your score and the faster you finish the higher your finish bonus. If weighted equally there is no modifier required as they balance each other off.

The following are examples of games that would receive the same score using the formula I proposed.

Example 1:
Score and finish date in month a are exactly average. Score and finish date in month b are exactly average. The scoring formula would give these two games the same score even though month a's game might be an entirely different configuration from month b. The actual game score and finish date could be drastically different.

Example 2:
Score for player a is double the average and finish date is exactly average. Score for player b is average and the finish date has double the average number of years left. These two players would receive an identical score.

Example 3:
Player a's game is a milking game and is 4.5 times the average. His finish date is 2050 so he receives no finish bonus. Player b's game is a fast conquest and finished with 4 times the average number of years left. His score was 1/2 the average. These two players would receive an identical score.

MPF
The extra time it took to achieve your cultural victory is not wasted. The longer you play, the higher your game score. This should compensate for the lower finish bonus for taking longer.

True but investing in culture means not investing in conquest (i.e. militairy units) and growth. Since the number of tiles and number of citizens really influence youre score aiming for culture tends to cut youre score especialy for not so advanced players like me. You're method seems to generate high score by early finish (not possible for diplo or cultural finish) and long played games. It does not address type of finish to stimulate diversity and type of gameplay.

MPF

 

[Beard Rinker]

Originally posted by MPF
True but investing in culture means not investing in conquest (i.e. militairy units) and growth. Since the number of tiles and number of citizens really influence youre score aiming for culture tends to cut youre score especialy for not so advanced players like me.

One of the ugly truths of this game when you use only game score and finish date is to score well you must get to the domination limit ASAP. This is regardless of the victory type you are trying to achieve and is even true of milking games.

I too am going for my first cultural victory this month. My game was a conquest game then I switched to cultural once I reached the domination limit. That was around 1250 ad and I expect to win around 1800. That gives me around 650 years to milk the game while waiting to win. Throughout the game I have tried to grab any extra points where I can find them even at the expense of each phase taking a little longer. i.e. Keeping citizens as happy as possible, expanding borders when possible. This is what I refer to as a balanced game. There were phases where expansion, conquest or building were the main focus, but never at the exclusion of all else.

That is about as balanced a game as you can get when you use game score and finish date as the only assesment of how well you've done. If my scoring formula was used, this game should do pretty well. If only the game score is used or the finish bonus was enhanced, this game would probably be mediocre. Using the game score exclusively, I should tediously milk it for the extra 250 years. If the finish bonus was enhanced, I should have skipped any non esential building and happyness improvements and won the game as soon as possible.

Originally posted by MPF
You're method seems to generate high score by early finish (not possible for diplo or cultural finish) and long played games. It does not address type of finish to stimulate diversity and type of gameplay.

If you are referring to the spreadsheet of actual GOTM data I posted, yes you are right. Keep in mind however that in these games, the way to get the best score was to milk the game to 2050. As a result, many players did just that. I would speculate that if the scoring formula I proposed was used, not nearly as many players would milk their games. The effect would be a lower average game score and higher average number of years left. This should benifit games like diplomatic, space and cultural which take a little longer.

It does appear however that early finishes are favored even after factoring this in. This formula was originally designed for the tournament and one of their requirements was a rabid obsession with early finishes. I modified it slightly so instead of using years left it uses turns left. This achieves a better balance between milking and fast finish but the other victory type still seem to be at a disadvantage. A good test would be if one of the top players played one of these victory conditions and finshed fast while milking the game as much as possible. For example, SirPleb won the fastest space finish in GOTM 8. I wonder what SirPleb's score would be using this formula if it were more balanced and had a milking component to it.

Using turns left instead of years left the results appear a little better to me. I added the remaining GOTM data to the spreadsheet so it now contains GOTM 1 - 8. The only score above 120 is Cartouche Bee's GOTM 8 worker dogpile game. Of the scores over 100 (there are 7) 4 are early finishes and 3 are milking games.

For reference, the formula now looks as follows:

Scoring Formula

GOTMScore = 50 * (FinishBonus + ScoreRatio) / 2

FinishBonus = TurnsLeft / AverageTurnsLeft (this is the change)
ScoreRatio = PlayersScore / AverageScore

TurnsLeft = 540 - turns taken
AverageYearsLeft = Average number of years left in all winning games.
PlayersScore = GameScore - (2050 - FinishYear) * DifficultyLevel
GameScore = Actual game score.
AverageScore = Average pre-bonus score of all winning games.
DificultyLevel = 1 for Chieftan through 6 for Diety.

FinishBonus is 0 when the player does not meet a victory condition



I have attached a spreadsheet containing GOTM 1 - 8 data. It computes a normalized score using the formula above.

Edit: I have added GOTM 9 to this spreadsheet and attached to the a post four below this one.

 

[Creepster]

Not that I want to further confuse the issue, but there is another area of the score that needs to be added in/fixed. The points you receive for future techs. In the current GOTM I have 7 future techs and am hardly reciving anything for them. I will probably have 20 to 30 future techs by the end of the game, but they will only count for about 5 or 10 ponits by the look of things. I feel that this is extreemly under valued in the scoring system.

There are a host of variables that should have been included in the scoring formuale, and while I understand that you are trying to keep it simple, and to balance the game out between milkers and faster finishes if you are oing to try to fix it then it should be done right. What that is exactly I am not totally sure, but it should take into account all of the variables (GNP in terms of money per turn and production per turn, money in the treasury, total culture, population size, technological advancement, amount of territory, happy people, content people, specialists, Number of improvements in a city, and time to finish) There are probably a few more that should be added, but all of these should be weighed against the time that you completed the game in. This is no easy task and a program woul;d have to be written to export all of the numbers to determine a true score.
The reason all of these other varaiables are important is that someone who finishes early, but has no solidly built empire should not score higher than somone who finishes a few turns later, but has a much better empire. There has to be a balance between empire buiding and a fast finish. That is one of the reasons I may or may not continue with the tournament. The whole game is based upon a fast finish. I don't reall care for that style of game. I like to build up my cities and grow a bit as I am expanding.

Just to sum it up, if a new scoring system is going to happen it has to be a bit broader than what you are suggesting. I don't believe that a simple formulae based upon turns and game score will be adequate. Too many things are missed by doing that.

 

[Beard Rinker]

Creepster,

The scoring formula I proposed does a fairly good job of rewarding a balanced game. An indicator of this would be games played at the same skill level should score similarly regardless of what month they were played and what the finish type was. This should be particulary true of games played with a below average to above average skill level. The top games and bottom games have a more variability in their results.

My games are a good example of this. My finish condition is all over the map, consisting of balanced games, milked games and fast space finishes. I don't play allot of civ games other than the GOTM games so my skill level is still noticably better with each successive game. I would expect a good scoring formula to show a consistent yet better score for each successive game and it does. My scores using the scoring formula were as follows:
GOTM4 38 Rookie milking game
GOTM5 53 Milking game
GOTM6 59 Fastest space finish
Missed 7 & 8
GOTM9 62 Balanced game

Compare this to the normalized score currently used:
GOTM4 29
GOTM5 61
GOTM6 22
GOTM9 30

As you can see, my scores are all over the map. Also note that my rookie attempt at milking in game 4 is rated the same as a much better played but not milked game 9.


This is only one player's example but this is ultimately what I am trying to achieve with the scoring formula. Games are rewarded for the skill in which they are played and not simply a measure of how well you can milk a game.


The main problem with developing a scoring forumla that includes more than game score and finish date is time. It may be months before there is a utility that can export the appropriate game statistics. Once it is available, it will take a long time to decide what should or should not be included in a scoring formula and what the weighting of each component should be. The formula I've proposed is available now and is far better than simply using game score.

Ultimately, it would be preferrable to include a wide range of game statistics. Aside from rewarding balanced play, it should also be less suseptible to exploits. For example, if final treasury is part of the score and someone figures out how to exploit this, they would still not necessarily get a good score as it is one component of many.

 

[Creepster]

I agree that your scoring system would be easier to implement, and that it would be a lot quicker. I also like the idea of being able to compare games from month to month. Both of those are very valuable.

I just don't agree with the formuale you are proposing. I would like to see a more detailed realistic formulae, but that would require some one to write the program and test it. It would also take a while to get agreement on just what the formulae should be as well, and based on what I have seen in this duiscussion and the tournament I am not sure an agreement could be reached. At least not in a good timeline.

There is a lot to be said for using the current scoring system despite its problems. It does allow for us to measure up against one another and it does allow us to generate a lot of discussion.

In fairness I have not spent enough time looking and analyzing your numbers. I will do that this weekend to try to get a better feel for what you are proposing.

 

[Beard Rinker]

Creepster,

The formula is four posts up.

The attached spreadsheet now contains data from game 1 through 9 and uses the scoring formla I proposed.

I noticed your scores were also pretty consistent and better each month you participated.

 

[Creepster]

I still need to spend a little more time looking at the results, but it does seem to balance things out a bit more.

For example xarin had a cultural win early in the game 1655AD. He now moved up quite a bit to number 10 from 23, which was close to my somewhat milked game that I won in 1926.

Just looking at it quickly that does seem to help level the game a bit from a milked game to a quick win. Looking at CB and cracker helps to show it even more. A quick domination win vs a milked game have more parity now.

Thanks for the updated spreadsheet. This helps out in seeing the results of your formulae.

 

[Matrix]

Some of you need to be able to express themselves a bit more compressed. I can't read all this; I've got more things to do!

Originally posted by Beard Rinker
How score is calculated has been posted before but I will summarize it here to save the trouble of finding it.

GameScore = FinishBonus + PerTurnScore
FinishBonus = YearsLeft * DifficulyLevel
PerTurnScore = Average of InternalScore for every turn * DifficultyLevel
InternalScore = Happy Citizens * 2 + Content Citizens + Specialists + ScoringTiles
Specialists = Entertainer, Tax Collector or Researcher citizen.
ScoringTiles = Number of tiles in your territory except sea tiles.
DifficultyLevel = 1 for chieften through 6 for deity

Beard Rinker, thanks for this! No further elaboration was nescessary. I got a mind myself too. Waste of your time.

Anyway, I've worked months on a good GOTM score for Civ2. If we decide to make our own GOTM scoring system, I'd like to learn from that.

Again, there's no way we can punish milking if we don't square root the score before an early finish bonus is given, since the score also grows exponentially (or quadratically) when milking. So that's what we will do then. It might be a good idea to leave it with square rooting the basic score. That's easily done with Excel.

What are your points of view about that?

 

[Cartouche Bee]

Originally posted by Matrix
Some of you need to be able to express themselves a bit more compressed. I can't read all this; I've got more things to do!


Beard Rinker, thanks for this! No further elaboration was nescessary. I got a mind myself too. Waste of your time.

Anyway, I've worked months on a good GOTM score for Civ2. If we decide to make our own GOTM scoring system, I'd like to learn from that.

Again, there's no way we can punish milking if we don't square root the score before an early finish bonus is given, since the score also grows exponentially (or quadratically) when milking. So that's what we will do then. It might be a good idea to leave it with square rooting the basic score. That's easily done with Excel.

What are your points of view about that?

Actually the score grows the quickest mid game and slowest in the beginning and the end.

 

[Matrix]

Ok, but that doesn't matter because the score still grows faster than the early finish bonus drops.

 

[Beard Rinker]

Matrix,

&(*(d.~/\sp

I tried compressing my post but that was the result.


CB is right, score does not increase exponentially in civ 3. The increase in score per turn increases up to the point where your internal score is maxed, at that point the increase in score per turn decreases to the end of the game.

In short, I don't think taking the square root of the score would give us the desired results.

As you know, the problem in the built in score stems from the imbalance between the finish bonus and per turn score. In a nutshell, what I have done with the proposed scoring formula is to separate the finish bonus and per turn component of your score, rank them in a similar fashion to your normalized score formula, and recombine them with equal weighting.

My premise is that if per turn score and finish bonus are weighted equally, you should be able to do just as well with any victory condition. The extra time it takes for a spaceship, cultural or diplomatic victory are compensated for by the extra per turn score you can accumulate.

From looking at the GOTM 1 - 9 data, this formula seems to accomplish this, particularly with the players who are not going for fastest finish or highest score. First place seems to go to either a fast finish or milking game but I suspect these players always either milk their games to 2050 or finish as quick as possible, even at the expense of some potential per turn score. To score well with a space, cultural or diplomatic victory you must finish fast and maximize score.