Old GOTM Scoring Discussions

[Aeson]

Here is a slightly modified modifier list. Added the 60/80 categories, and updated some others:

- base conquest to 200AD from 10AD
- deity diplomatic to -300 from -100
- emperor diplomatic to -100 from -200
- emperor domination to +200 from +400
- deity domination to +300 from +400
- large conquest and domination to +200 from +300
- low difficulty domination factors increase
- some domination land form modifiers softened.
- all 'tech' modifiers for diplo and space removed for < Monarch

- Spaceship base date was lowered by 20 turns to make for a better comparison with Diplomatic (40 turn spread instead of 60)

Updated the GOTM6-9 examples with the new values.

Base victory condition 'best' dates: (standard continents map, monarch)

Conquest       : 200AD
Culture (100k) : 1500AD
Culture (20k)  : 1700AD
Diplomatic     : 1300AD
Domination     : 500AD
Spaceship      : 1500AD

Difficulty        Chi   War   Reg   Mon   Emp   Dei

Conquest       : -300, -200, -100,    0, +400, +600
Culture (100k) : -100,  -50,  -25,    0, +100, +200
Culture (20k)  :  -50,  -25,    0,    0,  +75, +100
Diplomatic     :    0,    0,    0,    0, -100, -300
Domination     : -400, -300, -150,    0, +200, +300
Spaceship      :    0,    0,    0,    0, -100, -300

Map Modifiers    Tiny Small Stand Large  Huge    Arch 60/80  Cont 60/80  Pang

Conquest       : -200, -100,    0, +200, +400    +200, +100,    0, -100, -200
Culture (100k) : +300, +200,    0, -100, -200    +100,  +50,    0,  -50, -100
Culture (20k)  :    0,    0,    0,    0,    0       0,    0,    0,    0,    0
Diplomatic     :    0,    0,    0,  -50, -100       0,    0,    0,    0,    0
Domination     : -200, -100,    0, +200, +400    +100,  +50,    0, -100, -200
Spaceship      :    0,    0,    0,  -50, -100    +100,  +50,    0,  -50, -100

 

[Cartouche Bee]

Hi Aeson,

You have a 60 turn spread between diplomatic and spaceship. I think a 40 turn spread would be a bit closer. It looked to me that spaceship victories were rated a bit high overall. The map modifiers should be the same or similar cause you have to get to the modern age for either victory type. (I know you have alot of data in the tournament to base this on but..)

I definitely favor a system that is based on the map/difficulty that is played rather than just how well people played it. We have a good idea of the performance that can be achieved and that is a better scale for everyone to improve performance against.

CB

 

[Aeson]

Is taking the base Spaceship number down by 20 turns (to 1500AD) what you are suggesting? It does make GOTM08 results look better. The Spaceship takes 9 extra techs, and at 4 per, that does make a 40 turn spread more reasonable. Good point!

 

[Cartouche Bee]

Yep, that sounds pretty good to me.

 

[Beard Rinker]

I think the formula presented by Aeson could work. Good job Aeson.

It acheives a pretty good balance between victory types and with more tweeking, this could potentially be a very good balance. If Matrix still applies his normalizing formula then that gives us the month to month comparison also.

My main concern is the amount of tweeking that will be required. I see this as an on-going process that will likely result in a slightly different formula each month.

These results were achieved in part from some after the fact tweeking. While this is a good way to perfect the formula, this should not be done with an actual GOTM. Once the condition best years have been released, they must remain unchanged even if the results show they are not quite right. The modifiers can be changed for the next month, but changing them after the fact may cause considerable disagreement.

The result of this is the actual results will not be as good as Aeson's sample spreadsheet. However, it should be as good or better within a few months once these modifiers are more refined.

One observation I made about the results is it seems to reward games where the player went for a specific victory condition as fast as possible over a more balanced approach. As an example, im GOTM 6, I finished with a fast space victory and got 7986 points using this forumla. In my GOTM 9, I finished with a balanced domination victory receiving 5446 points. GOTM 9 was played with more skill yet scored substantially lower.

This is not as big a problem as it seems. Had this formula been used, I would have finished with a cultural victory instead. My guess is I would have finished around 1850 with a game score around 7500 giving me 7551 points. Still a bit lower than GOTM 6 but within the ball park. With this formula, you must choose the finish type that best matches your game. For example, finishing with a conquest victory after a period of milking would be a bad choice.

 

[Aeson]

Thanks Beard.

I agree that any scoring system released needs to be set, even with it's flaws. That may mean that some victory conditions will be better than others, at least on some maps. It's complex enough that there probably won't ever be a 'sure' thing like milking currently is on the majority of maps, and balanced enough that the victory conditions which are favored aren't overwhelmingly so. In any case, the player will have the data to make base their choices before the game is played. Other factors like starting location can be assessed very early in the game by the player.

A few months of unofficially testing to finalize the modifiers would definitely be necessary.

The modifiers that need the most work are the low difficulty ones, as I have very little experience on Chieftain->Regent, and the conquest ones, as there isn't much data in the GOTM's since the inception of the early finish awards in that area. A lot of the 'tech rate' based modifiers are probably a bit off due to inexperience with 1.29f on my part.

The attached file is the regular division season 1 results from all 4 games. It does show the weakness of the scoring as far as comparing map to map. The Domination game stuck the player on a rather small island on a pangaea map. Domination/Conquest seems to be the harder victory conditions for lower difficulty players though, so that could be part of the discrepancy in the results.

EDIT: added a more in-depth set of modifiers for the MaxScore/MaxTurnScore equation. Also, the map/difficulty modifiers have been further modified. Mostly changes to the 'tech rate' type of modifiers, and keeping Spaceship and Diplomatic even, as they are affect almost exactly the same.

MaxScore/MaxTurnScore %

                   Water  Level

Landform        80      70      60

Archipelago   0.500   0.517   0.533
Continents    0.550   0.566   0.583
Pangaea       0.600   0.617   0.633
					

                                Size

Difficulty      Tiny    Small   Stand   Large   Huge

Chieftain       0.016   0.014   0.012   0.010   0.008
Warlord         0.014   0.012   0.010   0.008   0.006
Monarch         0.012   0.010   0.008   0.006   0.004
Emperor         0.010   0.008   0.006   0.004   0.002
Deity           0.008   0.006   0.004   0.002   0.000


Base victory condition 'best' dates: 
(standard, continents, monarch)

Conquest       : 300AD
Culture (100k) : 1500AD
Culture (20k)  : 1700AD
Diplomatic     : 1300AD
Domination     : 500AD
Spaceship      : 1500AD

Difficulty        Chi   War   Reg   Mon   Emp   Dei

Conquest       : -300, -200, -100,    0, +400, +600
Culture (100k) : -100,  -50,  -25,    0, +100, +200
Culture (20k)  :  -50,  -25,    0,    0,  +75, +100
Diplomatic     : -200, -100,  -50,    0, -100, -300
Domination     : -300, -200, -100,    0, +200, +300
Spaceship      : -200, -100,  -50,    0, -100, -300

Map Modifiers    Tiny Small Stand Large  Huge    Arch 60/80  Cont 60/80  Pang

Conquest       : -200, -100,    0, +200, +400    +200, +100,    0, -100, -200
Culture (100k) : +200, +100,    0, -100, -200    +100,  +50,    0,  -50, -100
Culture (20k)  :    0,    0,    0,    0,    0       0,    0,    0,    0,    0
Diplomatic     : -100,  -50,    0,  -50, -100     +50,  +25,    0,  -25,  -50
Domination     : -200, -100,    0, +200, +400    +200, +100,    0, -100, -200
Spaceship      : -100,  -50,    0,  -50, -100     +50,  +25,    0,  -25,  -50

 

[Yndy]

Aeson,

Do you think we should include a "defeat bonus" of let's say 1/4 point per year played after 10AD?
It may be limited to 2000 AD to avoid its use by the ones milking the game.
I will do some statistics calculations on your sheets to check "best" finish year but not before Sunday.

 

[thefrenchzulu]

I have read a few posts on this thread and realised that I probably need a Master in Statistics to participate in GOTM's. The scoring adjustments just seem too complicated.

In the Gotm10, my first, I'm busy milking. I'm by far not the best, but there is only so much you can really do to increase your score.

I think to keep it simple you need to modify the early win bonus.

A possible solution is to calculate the increment pre-domination turn score achieved by some of the best Milkers. (I think it is some kind of sigma-curve, for those with knowledge of population dynamics) Once you have calculated the average turn score, you can adjust the early win bonus to reflect the increased turns in the late game.

Something like:
Turns to end x average milking increment score x difficulty level

That should solve the milking issue.

Adjusting the score for type of wins is useless. I can win most of my games anyway I want to. It doesn't stop me from being the ultimate warmonger.

 

[Aeson]

Originally posted by Yndy
Do you think we should include a "defeat bonus" of let's say 1/4 point per year played after 10AD?

So this would only be given to those who actually lose? I'm not sure what this is meant to accomplish. Players are rewarded with extra points the longer they can hold on through the game scoring already.

 

[Aeson]

Originally posted by thefrenchzulu
I have read a few posts on this thread and realised that I probably need a Master in Statistics to participate in GOTM's. The scoring adjustments just seem too complicated.

It really isn't that complicated. Basically each game is released with some guidelines as to when a 'good' victory will occur, or with what score. Any game approaching that date or score will be near 10k points.

Example Game:

Aztecs
Emperor
Standard
Continents 60%

MaxScore: 15963

Conquest: 600AD
Cultural: 1550AD
Diplomatic: 1175AD
Domination: 600AD
Spaceship: 1325AD

 

[thefrenchzulu]

OK, I don't have the Masters in Statistics, but I think I know what you are trying to achieve.

1. See who is the best player
2. Safe us from milking to get a good score.

If you equate the early win bonus to the additional score you would achieve through milking, the best player will be the one with the earliest win!

(The problem I have with your approach, is that as player, I don't have the stats you have behind these figures. I can basically win anyway I want, but I cannot replay the game for every scenario to calculate the best probable score.)

 

[Beard Rinker]

Edit: I noticed after posting this that you are already aware of this problem.

Originally posted by Aeson
- Later victory conditions (Culture, Diplomacy, Spaceship) should always score more in the score section (more population), and so should perhaps have a general modifier to the base ConditionBestDate to compensate.

This post presents a slightly different approach to solving it.


Aeson,

I think I've found a small problem in your scoring formula.

You compute score from percentage of how close a player got to a perfectly milked game and from a percentage of how close a player got to finishing as close as possible to the earliest finish date for their victory condition.

For milked games the best you could hope for, assuming the computed max score is correct, is to acheive the maximum milked score. This would give you a score of 10000.

For other victory types, the best you could hope for is to finish the game by the earliest possible finish date. You would also receive a small bonus for your game score. This would give you a finish bonus of 10000 + a small game score component.

To me, this suggests that the best strategy is to go for the fastest finish of the victory condition with the latest 'best' date. You can get the full finish bonus and your game score component would be higher than other victory types. For example, in GOTM 9, the latest 'best' date is 1600 for the cultural victory. Compare that to the 'best' conquest or domination date of 1100. I would expect that a player winning a cultural victory in 1600 would have a higher per turn game score than a player winning a domination or conquest victory in 1100. The result is the cultural victory would get a higher score.


Stepping back for a moment, I think what you are trying to achieve is to compensate each victory condition for lost milking time. What you are doing is awarding them the equivalent of a fully milked game on top of the game score they have already achieved.

Another problem is the implied assumption that any game won by the condition best date could be milked for the maximum game score. This is not true especially with spaceship and diplomatic victories. If you can get to the point where you can win by the best date you could not at that point turn it into a milking game and acheive the maximum milked score.


This is compensated for to some degree using the modifiers you have presented but the problem lies in the underlying structure of the formula. Perhaps another set of modifiers to compensate for this might fix it.

With each victory type, your score should be at a certain percentage of the maximum milked score if you acheive the victory type by the earliest date. This percentage should be removed from your score to equalize this imbalance. For example, SirPleb achieved a space victory in GOTM 8 with 3939 points in 1545. Assuming the actual game results reflect the true 'best' space race victory and max score, I would put the 'best' space race at 1545 and max possible score at 10850. SirPleb's base score is 1919 or 17% of the max score giving him a bonus of 17% that is not possible to acheive through a purely milked game. Allot of assumptions, but I hope you get my point.

By modifying the game score by a condition score modifier, this imbalance can be removed. The effected portion of the formula would look as follows:

PlayerScore = (GameScore - GameBonus - ConditionScoreModifier) / MaxMilkedScore * ScoreAdjustment

Condition score modifier is the expected score that a player would have if they won on the condition best date.
Conquest : ?? * MaxMilkedScore
Culture (100k) : ?? * MaxMilkedScore
Culture (20k) : ?? * MaxMilkedScore
Diplomatic : ?? * MaxMilkedScore
Domination : ?? * MaxMilkedScore
Spaceship : 0.17 * MaxMilkedScore

I'm sure you have a better feel for what these values should be.


The effect of this modification would be lower scores than what you are getting. In theory, a score of 10000 would be a perfect game and it would be very unusual to beat that.

A benifit of this adjustment is it should be easier to adjust the modifiers. If the condition best dates and max milked score mean exactly that, then it should be readily apparent if these values are wrong. For example, if many players beat a condition best date or max milked score then it would suggest the best values are too low. Conversly, if some of the best players cannot achieve the condition best values then it suggests the best values are too high.

 

[Beard Rinker]

Originally posted by Aeson
Also, it might be best to convert entirely into turns.

I agree, using years adds a slight imbalance between victory types.

 

[Beard Rinker]

Continuing the thought about a condition score modifier...


When a game is released, it would also have a score with each condition best date. This would represent the score you could expect to have if you could finish by the condition best date. For example:

Aztecs
Emperor
Standard
Continents 60%

MaxScore: 15963

Conquest: 600AD 1863 pts
Cultural: 1550AD 3193 pts
Diplomatic: 1175AD 2469 pts
Domination: 600AD 1863 pts
Spaceship: 1325AD 2714 pts.

Points stated are the per-turn game score without the built in finish bonus. They could easily be stated either way.

This would give players some concrete numbers on what is considered a perfect game for each of the six victory conditions.



I think the condition score modifier is still not quite right. There probably should be different values for each difficulty level as well. A good approximation would be an adjustment up or down depending on the number of turns for the condition best date. For example:

Condition Score Modifier
Conquest : ??
Culture (100k) : ??
Culture (20k) : ??
Diplomatic : ??
Domination : ??
Spaceship : 0.17

ConditionScore = ConditionScoreModifier * MaxMilkedScore * ConditionBestTurns / ConditionBaseTurns

 

[Aeson]

I've been thinking about the problem as well. I think your solution makes a lot of sense. It's more appropriate for the modification to come from the score side like you are saying, than from the turns side like I initially posted about.

ConditionScore = ConditionScoreModifier * MaxMilkedScore * ConditionBestTurns / ConditionBaseTurns

I'll have to try it on some examples, but it looks good. The only problem that might arise is that by including the turns figures in with the scoring, it could double the significance of any inbalances in the 'best' tables. The alternative is to add another set of tables though, which would probably just make everything harder to balance.

PlayerScore = (GameScore - GameBonus - ConditionScoreModifier) / MaxMilkedScore * ScoreAdjustment

This I'm not so sure about. If the ConditionScoreModifier is meant to be the 'best' score at the condition date, then wouldn't this always return a negative number? I just woke up, so I'm having trouble wrapping my mind around it. If a player beats the 'best' date they get over 10k, but it's offset by likely falling short of the condition score? Of course if the 'best' date is set right this shouldn't happen. Sorry for talking to myself here.

I'll work on converting everything to turns. Starting on a calculator as well. I'll have it so you can manually input the terrain info from mapstat first, and then add the functionality to get the stats from the saves later.

 

[Beard Rinker]

Aeson,

I mixed some terms up in my previous posts as I was kind of developing the idea as I went. To clarify some terms:

ConditionScoreModifier - This is the percentage of the max milked score you should acheive if you acheive victory by the 'best' date. There is a value for each victory type.

ConditionScore - This is the expected score you should acheive if you acheive victory by the 'best' date. This value is subtracted from you final per-turn game score. It is calculated using the formula you quoted in your post.

That changes the variable names in the PlayerScore formula as follows:

PlayerScore = (GameScore - GameBonus - ConditionScore) / MaxMilkedScore * ScoreAdjustment



Yes it is possible to have a negative value for score portion. This is probably possible even if you finish after the 'best' date but not desirable. I think the condition score should be an expected score if you finish by the best date instead of being considered the best score you can acheive if you finish by the best date. In my limited experience with going for the fastest possible finish, there is little you can do to improve your score that doesn't detract from your goal of finishing as soon as possible. It would be a shame to hit one of the best dates and be penalized for score.

If balanced right, it leaves more ways of achieving the perfect game: finish by the best date for the victory condition, milk the game for the maximum possible score or finish slightly after the best date but with a substantially better score than the condition expected score.

 

[Yndy]

Originally posted by Beard Rinker

If balanced right, it leaves more ways of achieving the perfect game: finish by the best date for the victory condition, milk the game for the maximum possible score or finish slightly after the best date but with a substantially better score than the condition expected score.

This is exactly what one would want but bear in mind that we do not want to discourage milking, but only to give other victories a chance against milked games.

I, for one, would agree that a game played 15 hours milked for 5 hours should have a score slightly higher than a fast win, even if it took 13 hours of which 3 planning.

What I mean is make the extra time spent by milking worth something.

The way I see it a perfect scoring system would have in the top 10 players, 6 early winners and 4 milkers.

What dou think?

 

[Beard Rinker]

Originally posted by Yndy

I, for one, would agree that a game played 15 hours milked for 5 hours should have a score slightly higher than a fast win, even if it took 13 hours of which 3 planning.

It depends on how well it was played. I could milk a game for 100 hours and probably not get the top score.

If the formula works as designed, a game that is finished fast and milked effectively along the way should score best. Next best would be a perfectly milked game or a game finished in the absolute minumum game turns but not milked at all.

These differences though are slight, and probably smaller than error in calculating the best finish date and maximum possible score. In other words, your best bet is to assess the map and play the game that fits it best.

 

[Beard Rinker]

Originally posted by Aeson
EDIT: added a more in-depth set of modifiers for the MaxScore/MaxTurnScore equation.

I think you are referring to the Landform/WaterLevel and Difficulty/Size modifiers in that post.

Anyway, I could not find any reference as to how you use them. I am assuming this applies to this formula:

MaxMilkedScore = MaxTurnScore * Landform

Is the formula now

MaxMilkedScore = MaxTurnScore * (Landform + SizeModifier)

 

[Aeson]

The way I see it a perfect scoring system would have in the top 10 players, 6 early winners and 4 milkers.

Yndy,

If you look at the results from this formula, they are pretty much like what you are talking about. In GOTM 6-9 there were 16 milking games in the top 10, from 2-6 per game. The rest were early finishes, with a good representation of victory types that were actually played in each.

MaxMilkedScore = MaxTurnScore * (Landform + SizeModifier)

Beard,

Yes, this is correct. Originally it was just .65, .55, .45 for Pangaea, Continents, and Archipelago. I should have added the formula with the more indepth numbers, but forgot. The size/difficulty modifiers I toned down a bit as well.

MaxScore/MaxTurnScore %

                   Water  Level

Landform        80      70      60

Archipelago   0.500   0.517   0.533
Continents    0.550   0.566   0.583
Pangaea       0.600   0.617   0.633
					

                                Size

Difficulty      Tiny    Small   Stand   Large   Huge

Chieftain       0.0135  0.0120  0.0105  0.0090  0.0075
Warlord         0.0120  0.0105  0.0090  0.0075  0.0060
Regent          0.0105  0.0090  0.0075  0.0060  0.0045
Monarch         0.0090  0.0075  0.0060  0.0045  0.0030
Emperor         0.0075  0.0060  0.0045  0.0030  0.0015
Deity           0.0060  0.0045  0.0030  0.0015  0.0000

Here is an early version of the calculator (PC only) which handles most of the calculations. The suggestion about the condition scores hasn't been implemented yet though. I haven't figured out what the %'s should be.

The calculator shows all the modifiers implemented, and allows changing them to see the way each affects score. For now it requires that the terrain info be input by hand from MapStat. Haven't gotten around to working on the decompression or reading of save files yet.