Ranking with CGOTM(C3C) and GOTM (PTW)

[LeSphinx]

How the ranking system is gona be with the GOTM for PTW and the one for C3C ?
LeSphinx

 

[AlanH]

We are finalising this currently. Each player's score for a month will be some combination of their one or two scores for COTM and GOTM that will not disadvantage those who can only play one of the two. The better of the two, or the average of the two are the current shortlisted options.

 

[civ_steve]

For Global Rankings, I would suggest using the better of the two scores for that month's games. This would allow someone who plays one of the games quickly to not be penalized for submitting a rushed game, and should increase participation in both contests. It would also allow players to submit one game played for score and a 2nd game played for style (OCC, 5CC, AW, or plan your own variant) without a negative impact to their rankings. That should make the Spoilers a bit more interesting!!

 

[AlanH]

Thanks, that was the sort of discussion we've been having. The only counter to it is whether it disadvantages single version players, who can't improve on a poor first game to get a better overall result.

 

[bigchief]

What about the difference in difficulty levels between the two games? This month the C3C game is at Regent, and the classic game is at Monarch. A higher level game will almost always outscore a game played on a lower level. Is there a way to level this out?

 

[AlanH]

The global rankings work on percentages of the best score and best date achieved for the game, so that should normalise games at different difficulty levels. Where this is more of a problem is that currently higher difficulty games age more slowly in the rankings. If we use the player's best result then this will age faster if it is lower difficulty. That may be reasonable, but would the player then demand that the higher difficulty game is used once its amortised score is better? And we'd have a different problem deciding how to age the average of two games.

 

[samildanach]

Its quite tricky. I don't know what the staff should do. I think the goal of the classic GOTM should be to retain as many players as possible. If thats the goal then averaging isn't a good option as players who score well in the COTM will not submit or even play the classic games. I think that there needs to be a carrot for people to continue to play the classic, such as an additional 5% boost added to classic game scores. What I mean by that is that Classic games should have a max. score of 105% where as COTM has a max. score of 100%. So the top player in a classic will get a score of 105 where as the top player in the COTM will get the usual 100.
Some sort of score boost for classic games will also help to offset the shelf-life advantage that COTM players are likely to get with Sid level games counting towards the GPR for a very long time.

 

[civ_steve]

Thanks, that was the sort of discussion we've been having. The only counter to it is whether it disadvantages single version players, who can't improve on a poor first game to get a better overall result.

Understood. It's disadvantageous either way. Best result doesn't favor the single game player who submits only one game; averaging requires two good results from the dual game player (and in only 2 weeks time per game.) I think they're about even in weighting as far as advantage goes; taking best result means more and varied submittals and involvement, which in my opinion tips the balance.

Regarding the aging factor on the Global Ranks: I've noticed that effect in the charts. Because Deity games have usually been followed by Monarch or Regent level games, there's some months where a gap appears in the far out months when a Deity game is included, but the following Regent/Monarch game isn't. That looks odd, but is understandable. I'd stick to including the Best Result, and let it age however it does; if it were possible to pick up an initially lower valued higher level game that later after aging became higher valued than an initially higher valued lower level game (whew!), well that would be nice.

 

[dojoboy]

I don't see how you can use the best result when they are clearly two different games, in no way comparable. I may lack the necessary visionary skills for this, but I don't see how it can be done fairly.

What additional work is needed to run two global ranking systems? Then we could enjoy discussions on which one we consider to be the premier league.

 

[dojoboy]

As a side note, it would be more insightful for the global rankings to reflect performance rather than participation. I've noticed Txurce has slowly dropped down the ranks since he hasn't been able to participate. Someday, I will surpass him inspite of being half the player he is.

 

[AlanH]

I don't see how you can use the best result when they are clearly two different games, in no way comparable. I may lack the necessary visionary skills for this, but I don't see how it can be done fairly

Well, that's what is done in the global rankings every month already. The top player in each game gets 100%. The other players get percentages below 100% in proportion to their relative scores and finish dates. So performance in GOTM 31 can be compared with GOTM 30 and so on, and they can be added together to create a single global ranking. In spite of the fact that they are very different games, and they may have been played to different finish conditions. Extending this to include C3C games is not the big issue. The challenge is how to deal with the fact that some players can and will play two games a month while others can or will only play one.

What additional work is needed to run two global ranking systems? Then we could enjoy discussions on which one we consider to be the premier league.

That's easy for you to say, with your laptop full of Intel-inside and C3C .... Seriously, we are trying hard to find ways to keep this community together, not to divide it.

 

[dojoboy]

Well, that's what is done in the global rankings every month already. The top player in each game gets 100%. The other players get percentages below 100% in proportion to their relative scores and finish dates. So performance in GOTM 31 can be compared with GOTM 30 and so on, and they can be added together to create a single global ranking. In spite of the fact that they are very different games, and they may have been played to different finish conditions. Extending this to include C3C games is not the big issue. The challenge is how to deal with the fact that some players can and will play two games a month while others can or will only play one.

So, the GOTM staff can set up a classic GOTM game and a COTM game that are comparable for ranking? If so, then everyone is allowed to submit one game at the end of each month. There is no benefit to a person who has access to both, based on the honor system.

But, back to the existing discussion. Can we simply take the best possible score (Jason Score) for the map, per victory condition, and grade the submissions. So, the person who scores the highest may not earn a 100% if he/she failed to reach the best mark, but say scored 97%. Then, whether he/she plays the next game, his/her ranking score remains a 97% until he/she plays again. Then those two grades are averaged together.

 

[Xerol]

Weighted average: Better score counts for 60% of the total, worse one for 40%. Or 70/30. This is the best compromise of "Average" vs. "Best".

 

[Dianthus]

So, the GOTM staff can set up a classic GOTM game and a COTM game that are comparable for ranking?

...

Can we simply take the best possible score (Jason Score) for the map, per victory condition, and grade the submissions.

That's sort of what is being done currently, except rather than guessing the best possible Jason Score it is assumed that whoever wins got the best possible Jason Score. I understand that SirPleb is going to be competing in both the Classic and C3C, so this is pretty likely to be true .

 

[Darkness]

Its quite tricky. I don't know what the staff should do. I think the goal of the classic GOTM should be to retain as many players as possible. If thats the goal then averaging isn't a good option as players who score well in the COTM will not submit or even play the classic games. I think that there needs to be a carrot for people to continue to play the classic, such as an additional 5% boost added to classic game scores. What I mean by that is that Classic games should have a max. score of 105% where as COTM has a max. score of 100%. So the top player in a classic will get a score of 105 where as the top player in the COTM will get the usual 100.
Some sort of score boost for classic games will also help to offset the shelf-life advantage that COTM players are likely to get with Sid level games counting towards the GPR for a very long time.

I disagree! I think both should be measured equally and only one should count towrds the Global Rankings each month (or an average if you've played both).
This is because many players don't have the time to play two games each month. Also why should there be a carrot for playing the classic game?
I realise there is the fear of the classic game dying because of lack of players, but there are still plenty of people playing the Civ2 GOTM, so I don't think it'll go that fast.

Another option is seperate Global Rankings for the classic GOTM and the conquest GOTM.

 

[ainwood]

Well, we're really not very keen on separate global rankings for each. Separate pantheons: "Yes", separate rankings: "No".

In the initial discussions, we felt that as long as there was still a reasonable number of people competing in both games, then a direct comparison was probably reasonable. If (for example), we don't get the best players playing the Classic game, then it may start being skewed somewhat.

At the moment, rankings are based on score relative to the highest score. I am beginning to warm to the idea of ranking against best possible dates, so the rankings then become a bit less dependent on the player base. The big concern with this is that we need to ensure that we have comparable means of calculating Jasons for each game: Aeson has given this some good thought, and we have a system. However, things like the changed tech tree, different units etc do make the games a bit different.

We will probably agree on a system an impliment it, but reserve the right to modify it at a later date. In terms of longevity / decay of results, we might consider phasing the difficulties, or even changing the decay profile.

 

[dojoboy]

In terms of longevity / decay of results, we might consider phasing the difficulties, or even changing the decay profile.

Forgive my ignorance, but how might results decay?

 

[AlanH]

The global rankings currently amortise/decay/age (choose your term) the value of each game as the months roll by. The decay rate depends on the difficulty level. They decay linearly to zero after (difficulty + 5) months, I think. That's how you get to be a better player than Txurce. Look at the way world rankings in tennis and golf operate. Previous champions may have been better players than the current world #1 but when they don't play any more their ranking goes down.

 

[dojoboy]

The global rankings currently amortise/decay/age (choose your term) the value of each game as the months roll by. The decay rate depends on the difficulty level. They decay linearly to zero after (difficulty + 5) months, I think. That's how you get to be a better player than Txurce. Look at the way world rankings in tennis and golf operate. Previous champions may have been better players than the current world #1 but when they don't play any more their ranking goes down.

I understand the reference now, thanks. Is this approach to rankings the line of least resistance or preferred? Theoretically, if Sir Pleb, or pick another former top ranked player, sat out too long, then he'd never get back to number one if the current #1 continues to play each month. Unless he/she begins to play like me.

I guess participation should have its bonuses. Perhaps, the GOTM rankings could become a "seasonal" competition where top 20 finishers earn ranking points independent of the GOTM score. But, this goes off thread topic.

 

[Xevious]

A top player sitting out a long time is the same as a new player in terms of global ranking. They would start from scratch. If that player is consistently high scoring, they will move up the ranking, taking up to ten months to move to the top. I was up to 8th I believe (can't check at the moment), and then took 6 months off, and now I'm down around 166th or so. What little is left of my old scores will roll off while I post new ones, meaning my ranking will grow slowly, especially until I get back in the groove.