A proposal for a slight modification of Jason scoring

[akots]

... No respect to self-appointed gurus.

They play many SGs, so their participation is very high. They get their points there in their own way.

 

[civ_steve]

Wow! The Chart looks pretty good ... Thanks, AlanH!!

I'd agree with Klarius - a faster falloff from the Top Score would lower the general averages (we're adding up to 7 players with 100 points per game!) and accentuate the effect of getting the fastest time per victory condition. I'd like to see the first 50 or 100 turns or so decay the same for all victory conditions, then maybe see a linear decay to minimum value.

Between the two Displays, I'm leaning towards the Fastest_Player version. One of my desires with this alternative GP listing is to encourage participation in the "other than Conquest and Domination" category of Victories. It seems that the Jason Best Date is consistently beaten for Conq. and Dom, but rarely for Diplomacy or 20K. This would take away any argument about Best Date effect, and give players more reason to play the other VC's without concern about competing against Best Date: your competition is the other Players!

I understand where you're coming from, but I've looked at the numbers of people playing different victory conditions. When you get very few players going for culture or diplo in a game you really don't have a valid basis for comparative scoring against the results of 10 or 20 times more domination or conquest players. As Aeson says, a middle-of-the-road player's score in the date-based table will depend almost totally on good or bad luck. Did SirPleb choose this victory condition this month?

True, but most of the Awards go to players who are in the top 20 for that game's score. There might be one (and occasionally two) award(s) that go to a player who scores below 35th place or so. Such a player would only win an award on rare occasions so even with a small sample this GP Listing would feature top players who consistently submit competitive games. And the fact that some middle-level players are picking up serious GP points by playing less popular VC's should lead some Domination players to switch.

I'd love to believe that market forces would prevail, but this isn't a big market. Fewer players will enter 400 turn games than 200 turns - fact of real life. We'll never have more than maybe 10 players per late victory condition unless we set a Sponsored Victory a la Medal Play.

Call me a capitalist! (And I like the Commercial trait best of all!) I think part of the reason why not as many 400 turn games are submitted is because of the requirement to play a Domination Game first to score highly. I believe more people will play a 400 turn game if they can play a limited game the way they wish to play and be rewarded for it.
I'm not at all recommending changing the current GPR, nor am I suggesting that this is a rigorous way to compare Players. I would suggest that this type of Listing should promote more variety in Game play and submittal and give the non-Domination player the feeling they can play a different style of game and be ranked based on how quickly they achieved the Victory Condition, not on how large and happy their empire is.

 

[delmar]

Statistical reality. Tell me how many players you would trust to deliver benchmark performance across all victory conditions in all game situations. 10? 20? Then work out how they will spread over 7 victory conditions, remembering they won't make random choices and they may not all play every game.

Unfortunately I can't work this out with the conditions you stated , but...

Even if they chose randomly you'd be surprised how often statistically you would get none of them choosing one of the conditions.

I worked this out and I am surprised. For the curious, 10 players missing 1, 2, or 3 out of 7 conditions has 90%, 60%, and 12% chance respectively, assuming random choices (and that I got the math right ). Same for 20 players is 30%, 2% and 0.04%. Anyone with more live brain cells than I care to confirm this?

And of course I agree that they won't make random choices.

So how about taking the minimum of the Jason prediction and the actual best?

Btw, both of the formula currently visible on the web page you put together are dependent on the actual best date, no?

 

[Darkness]

Did SirPleb choose this victory condition this month?

IMHO this is the most important argument against using fastest dates as the determining factor! Your game's ranking will be much more dependant on the victory condition the top players (and not only SirPleb) pick.

Let's leave it like it is now. Jason score determines the results of the games, and there are the awards for the fastest finishes in every category. There's something for everyone this way.
And now Alan can give you 'fast finish date' lovers a GR alongside the original GPR... Everybody happy, right?

 

[AlanH]

I worked this out and I am surprised. For the curious, 10 players missing 1, 2, or 3 out of 7 conditions has 90%, 60%, and 12% chance respectively, assuming random choices (and that I got the math right ). Same for 20 players is 30%, 2% and 0.04%. Anyone with more live brain cells than I care to confirm this?

I was too lazy to do it when I posted, and I'm too busy now, bit the figures don't surprise me at all. It's one of those counter-intuitive things like the probablility that two kids in a class of 20 will have the same birthday. And your numbers probably assume that all those players actually play.

And of course I agree that they won't make random choices.

... which can only (and very significantly) increase the chances of one or two or three victory conditions not getting a consistent fast finisher.

So how about taking the minimum of the Jason prediction and the actual best?

After I've done my QSC work today I'll add that option to the popup, plus one or two that spread the score out more.

Btw, both of the formula currently visible on the web page you put together are dependent on the actual best date, no?

Yes, but the second one only uses it as a multiplication factor to make the fastest finish relative to Jason score 100%. Otherwise a date faster than Jason scores more than 100%.

 

[AlanH]

And the fact that some middle-level players are picking up serious GP points by playing less popular VC's should lead some Domination players to switch.
.....
Call me a capitalist! (And I like the Commercial trait best of all!) I think part of the reason why not as many 400 turn games are submitted is because of the requirement to play a Domination Game first to score highly. I believe more people will play a 400 turn game if they can play a limited game the way they wish to play and be rewarded for it.
I'm not at all recommending changing the current GPR, nor am I suggesting that this is a rigorous way to compare Players. I would suggest that this type of Listing should promote more variety in Game play and submittal and give the non-Domination player the feeling they can play a different style of game and be ranked based on how quickly they achieved the Victory Condition, not on how large and happy their empire is.

I like the Commercial trait too . But a market is driven by rewards. We already have more rewards for fast finishing than we do for high Jason scoring - six per game plus the Eptathlon vs three medals. There are no rewards for position in the ranking tables other than personal satisfaction.

And if there are two ranking tables players will choose which one to focus on. There's no guarantee that top quality players will switch allegiance to the speed demon table, and it would be interesting to see which one gets their focus. Another market experiment!

 

[delmar]

It's one of those counter-intuitive things like the probablility that two kids in a class of 20 will have the same birthday.

Yeah, I was thinking the same thing. And yes, I assumed that all play (ie. chose something).

Btw, both of the formula currently visible on the web page you put together are dependent on the actual best date, no?

Yes, but the second one only uses it as a multiplication factor to make the fastest finish relative to Jason score 100%. Otherwise a date faster than Jason scores more than 100%.

What you say is true but it doesn't change the fact that my score will be higher or lower depending on what the best player in my category achieved. Or am I missing something here? To be clear, I am talking about this formula:

Option 2: score = 100 x (540 - (player_turns - jason_best_turns)) / (540 - (fastest_player_turns - jason_best_turns))

Any way I look at this, if I am the fastest player in a category then I will get 100 points.

Another quote from the web page:

In both cases a Histograhic victory is awarded a score equal to the ratio of the game score to the best histographic score achieved.

This would create the same situation for the Histogrpahic victory as using actual dates for the others (ie. if I am the only one going for histographic then I score like SirPleb). I think histographic victories shoud be compared to the top Jason-score across all categories. And I mean the complete Jason-score as used today for GOTM ranking.

Finally:

So how about taking the minimum of the Jason prediction and the actual best?

After I've done my QSC work today I'll add that option to the popup, plus one or two that spread the score out more.

I don't know how much effort it is for you to do these things but I want to clarify that I don't expect you to immedietly implement every idea I mention here. I throw in these ideas to see what others think about them, not because I am convinced that they will work great. Of course if you personally feel that an idea is worth the effort then there's no issue.

 

[AlanH]

Any way I look at this, if I am the fastest player in a category then I will get 100 points.

Of course. The alternative is to have some games with a top score of 150% and others with top score of 50%. That's an option as well, but makes the month-on-month ranking even more volatile..

This would create the same situation for the Histogrpahic victory as using actual dates for the others (ie. if I am the only one going for histographic then I score like SirPleb). I think histographic victories shoud be compared to the top Jason-score across all categories. And I mean the complete Jason-score as used today for GOTM ranking.

I assume you mean in the case where Jason dates are being used for the other victories? Same argument applies as above.

I don't know how much effort it is for you to do these things but I want to clarify that I don't expect you to immedietly implement every idea I mention here. I throw in these ideas to see what others think about them, not because I am convinced that they will work great. Of course if you personally feel that an idea is worth the effort then there's no issue.

I'm looking across the range of responses and applying my own bias, I'm not reacting to an individual request unless I or someone else also thinks it has merit. Then there's the small issue of time

 

[delmar]

Of course. The alternative is to have some games with a top score of 150% and others with top score of 50%. That's an option as well, but makes the month-on-month ranking even more volatile..

OK, I thought Option 1 was supposed to be a system where the scores depend on the actual fastest finish in the same category and Option 2 was supposed to be the system where the scores depend on the Jason-date for the same category. Just to be clear, when I select "Fastest Player" or "Jason Best Date" on the web page, am I choosing between Option 1 and Option 2? If yes, then this is a bit misleading IMHO.

Anyway, since we don't have a formula that doesn't depend on the fastest player, I would like to suggest that we create one. We could get it by susbtituting min(jason_best_turns,fastest_player_turns) in place of "fastest_player_turns" in Option 1. I know we discussed this earlier, I mention this just to make sure my suggestion is clear. In this system, the top score would be less than 100% in a given category if the fastest player finishes later than the Jason best date, but the theoretical maximum score would be 100%.

I assume you mean in the case where Jason dates are being used for the other victories? Same argument applies as above.

Yes, I thought one of the primary options was to use the Jason dates as a measuring stick for non-histograph victories, and then using the best actual histograph victory as a measruing stick for histograph victories would not be consistent.

 

[Aeson]

Anyway, since we don't have a formula that doesn't depend on the fastest player, I would like to suggest that we create one. We could get it by susbtituting min(jason_best_turns,fastest_player_turns) in place of "fastest_player_turns" in Option 1. I know we discussed this earlier, I mention this just to make sure my suggestion is clear. In this system, the top score would be less than 100% in a given category if the fastest player finishes later than the Jason best date, but the theoretical maximum score would be 100%.

This should work best. Cuts the variation by who plays what quite a bit. Instead of "best" to "whatever the lone submission did", it varies from "best" to "best date".

 

[AlanH]

OK, I thought Option 1 was supposed to be a system where the scores depend on the actual fastest finish in the same category and Option 2 was supposed to be the system where the scores depend on the Jason-date for the same category. Just to be clear, when I select "Fastest Player" or "Jason Best Date" on the web page, am I choosing between Option 1 and Option 2? If yes, then this is a bit misleading IMHO.

Sorry you find it confusing. There are two formulae shown on the page in a section talking abbout two calculation options. One uses Jason dates the other doesn't. There are two options in the popup. One includes the word Jason the other doesn't. As this obviously isn't clear enough I'll see if I can find a different way to explain it.

Anyway, since we don't have a formula that doesn't depend on the fastest player, I would like to suggest that we create one. We could get it by susbtituting min(jason_best_turns,fastest_player_turns) in place of "fastest_player_turns" in Option 1. I know we discussed this earlier, I mention this just to make sure my suggestion is clear. In this system, the top score would be less than 100% in a given category if the fastest player finishes later than the Jason best date, but the theoretical maximum score would be 100%.

I've agreed to try this.

I do need to do the scaling for the Jason option in a different way to make it just a single scaling factor, independent of victory condition. But I would maintain that it is necessary to scale the highest score to 100% in order to be able to add multiple games together to give a cumulative ranking. The scale factor in the case of the Jason score option has to be related to the fastest finish relative to jason across all victory conditions.

Now that I've surfaced from an intensive immersion in QSC and handed it off for QA I've come back to this. I've put together the fastest dates vs. Jason dates comparison. I've listed all the dates for all the games and all the victories except histograph and calculated the differences in turns between Jason and fastest. I've provided the average turns difference over all games for each condition.

Before you look at the results I recommend you jot down what you think is the order of those averages ....

Spoiler ..... then go look at: :

this page

 

[solenoozerec]

One thing that I understood (did I? ) is that no one managed to get a diplo victory faster than it was predicted. There could be two reasons for that:
1. Best dates are not predicted correctly for Diplo victories.
2. Diplo is the least popular victory type among best players.

 

[delmar]

I've put together the fastest dates vs. Jason dates comparison. I've listed all the dates for all the games and all the victories except histograph and calculated the differences in turns between Jason and fastest. I've provided the average turns difference over all games for each condition.

Great stuff!

I will be curious to see what folks think about this. My interpretation at first sight:

• 20K estimate is perfect
• Diplo estimate is too agressive, it wasn't exceeded a single time. The closest ever real result was +10 turns by (you guessed it) SirPleb.
• Space victory estimates also too agressive, probably because they depend on the Diplo estimate in the first place.
• Conquest estimate is way too high (ie. weak). The high average is misleading. If we remove the top 5 differences (2 of which was a victory in 2050), we get an average of 13. If we remove everything above 100, the average becomes negative. I think the overall average is high because people with a potential for low conquest date often choose domination and this often causes unrealistically high fastest dates.
• Domination estimates are OK.
• 100K estimates are probably OK. The average is high because this is not popular with the top players. When SirPleb tried, he beat the snot out of the Jason-estimate, and at least 3 other players came close, too.

 

[AlanH]

@ solenoozerec : That's correct, and diplo is the worst result relative to Jason overall.

But that wasn't the most startling news I got from it.

X-posted with delmar

 

[AlanH]

Great stuff!

I will be curious to see what folks think about this. My interpretation at first sight:

• 20K estimate is perfect
• Diplo estimate is too agressive, it wasn't exceeded a single time. The closest ever real result was +10 turns by (you guessed it) SirPleb.
• Space victory estimates also too agressive, probably because they depend on the Diplo estimate in the first place.
• Conquest estimate is way too high (ie. weak). The high average is misleading. If we remove the top 5 differences (2 of which was a victory in 2050), we get an average of 13. If we remove everything above 100, the average becomes negative. I think the overall average is high because people with a potential for low conquest date often choose domination and this often causes unrealistically high fastest dates.
• Domination estimates are OK.
• 100K estimates are probably OK. The average is high because this is not popular with the top players. When SirPleb tried, he beat the snot out of the Jason-estimate, and at least 3 other players came close, too.

I still don't "get" this feeling that some dates are too aggressive. Why isn't it a reasonable proposal that those victories are not pursued enough to give optimum results?

 

[delmar]

But I would maintain that it is necessary to scale the highest score to 100% in order to be able to add multiple games together to give a cumulative ranking.

Sorry, probably I am a bit slow. Can you explain why you can't add games if the highest score is not 100%? An example maybe?

For me, the meaning of a lower-than-100% highest score is that none of the good players chose to play for that victory condition, so it is not only acceptable but straight-out desirable that the cumulative ranking of the fastest player in that category doesn't improve significantly...

 

[AlanH]

@Delmar: Suppose in game 36 you win with a date that gives you 150% of whatever we decide is the best date. Next month you win with a date that's 50% of whatever we decide is the best date. Did you play the second game 3 times worse than the first? Probably not. The best date might have been inappropriate for either or both games, in opposite directions.

 

[delmar]

I still don't "get" this feeling that some dates are too aggressive. Why isn't it a reasonable proposal that those victories are not pursued enough to give optimum results?

Fair comment. Perhaps people going for Diplomatic and Spaceship victory are devoting too many resources to increasing their territory instead of focusing on an early date. Indeed, when SirPleb achieved Jason-estimate+10, he was 4th overall (GOTM24).

But then is this also true for 20K victory? Because then the 20K estimates are too high (weak). Or people going for 20K give up any hope for a good Jason-score anyway and they do devote everything towards a fast finish?

 

[delmar]

@Delmar: Suppose in game 36 you win with a date that gives you 150% of whatever we decide is the best date. Next month you win with a date that's 50% of whatever we decide is the best date. Did you play the second game 3 times worse than the first? Probably not. The best date might have been inappropriate for either or both games, in opposite directions.

I think my position is that we have to have some faith in the best date estimate, whatever that is. If I believed that my performance is more stable than the best date estimate we can come up with, then the logical solution would be to base the scores on my performance. Or we can go back to the Jason score. I guess Aeason is already grinning...

Also, adjusting the fastest games to 100% will solve the problem you described for a maximum of 7 people. How will you ensure that the score of the second, third, etc. player will not vary a lot? Furthermore how will you ensure that my score will not vary a lot depending on whether I managed to pick that one category that the top 10 players will ignore in 90% of the cases?

 

[Denniz]

I've put together the fastest dates vs. Jason dates comparison. I've listed all the dates for all the games and all the victories except histograph and calculated the differences in turns between Jason and fastest. I've provided the average turns difference over all games for each condition.

I think it might be it might be useful to show difficulty and do the averages by difficulty. It also might be interesting to add a column to each VC with the number of players.