The GOTM Scoring System

[drkodos]

Most of the poll options, IMHO, seem to represent personal preferences. I am a little upset over the "PC" nature of the whole "morality" thing. :thumbdown

I have to agree with you on this 100%.


There is definitely an undercurrent that flows through some posts that one type of victory is more ethical or morally centered than other victories.

@DavviddesJ: Of course the selections one makes in a poll are personal preferences, but I believe that Denniz is wholly correct that the choices themselves are somewhat slanted. There does not seem to be an objective weighting of all options.

The poll has an agenda that appears to go beyond just finding out how people feel. It seems to have an agenda to help mold & shape people's feelings about the subject at hand.


Despite the fact that I think the GotM scoring system is skewed and I voted it could be changed for the better, I am now thinking that allowing the inmates to run the asylum is a mistake. We should probably leave well enough alone, enjoy the games in whatever incarnation they appear, and be thankful that someone is delivering the whole package to us without any cost on our part.

 

[DaviddesJ]

@DavviddesJ: Of course the selections one makes in a poll are personal preferences, but I believe that Denniz is wholly correct that the choices themselves are somewhat slanted.

There's only one option that has anything to do with "morality". I don't see how that's too many. The only smaller number would be zero.

Since you hate the idea, you should be pleased it's getting few votes.

 

[Meatbuster]

Maybe the real issue is not the score, but the fact that "certain other" victory conditions are not recognized enough in the GOTM. Sure, the fastest finisher in say, cultural, gets an icon, but it's buried within tables of scores. How about, say, at the top of the page (in addition to the current score-arranged GOTM table), all people with fastest-finish icons are displayed Or maybe list the top three fastest finishers within a victory condition?

Sure this would make it a bit like the HOF, but this might give people more motivation to go for other victory conditions.

I'm not saying the old scoring table/medals should be eliminated, think of this as an additional suggestion even if GOTM decides to implement a new scoring system.

 

[ewokimpi]

It seems to me that this path (altering the scoring system) must have been trod before, in Civ III? The Jason scoring system is implemented in the Game of the Months there, I think.

Perhaps an old-timer could post a short description of the Jason scoring system, or perhaps more relelvantly, what the problem was that the Jason system was designed to solve.

Please forgive me if I'm off topic here, or if I have my facts wrong. I did not play Civ III GOTM, and am just a newbie player. Perhaps knowing something about the Jason scoring system would not be of use to this discussion.

 

[DynamicSpirit]

I have to agree with you on this 100%.
There is definitely an undercurrent that flows through some posts that one type of victory is more ethical or morally centered than other victories.

Well yes, that's what some people appear to feel.

@DavviddesJ: Of course the selections one makes in a poll are personal preferences, but I believe that Denniz is wholly correct that the choices themselves are somewhat slanted. There does not seem to be an objective weighting of all options.

So what options are missing? Maybe I've missed something, but I deliberately put everything on the poll that I recall having been seriously suggested in recent discussions as a motivation for changing the system, but also included, as the first choice, the option of not changing the system. If I've missed something, then I would be sorry. But it would be rather more helpful to say what those missing options are rather than simply to imply that there are unspecified options not listed.

The poll has an agenda that appears to go beyond just finding out how people feel. It seems to have an agenda to help mold & shape people's feelings about the subject at hand.

Well I wrote the poll and I can assure you that I had no such conscious agenda when framing the poll. I simply listed all the complaints that I was aware of having seen voiced against the current scoring system for people to vote on. Some of those complaints are ones I agree with and have voiced myself, others are ones that I disagree with but I put in the poll anyway because I know other people are concerned about them.

I've no doubt that this *discussion* will shape people's views to some extent - discussions always do, but I don't know in what direction that'll be and I don't see anything wrong with that anyway.

Despite the fact that I think the GotM scoring system is skewed and I voted it could be changed for the better, I am now thinking that allowing the inmates to run the asylum is a mistake. We should probably leave well enough alone, enjoy the games in whatever incarnation they appear, and be thankful that someone is delivering the whole package to us without any cost on our part.

To my knowledge noone here is suggesting any change in who runs the GOTMs. This discussion and the poll will no doubt give some indication of the views of GOTM participants, and will probably give rise to some specific ideas too. I'm pretty sure that the GOTM staff are already aware of the skewed nature of the current scoring system, and I'm equally sure they'll read this thread (indeed AlanH has already contributed to it) and consider whatever points are raised. Maybe, the GOTM staff will decide in the light of this discussion that some alternative scoring system would be more appropriate. That decision will of course be theirs. So I don't really understand what the point you're trying to make is.

 

[DaviddesJ]

The Jason scoring system in Civ3 addressed the problem that victory was not scaled at all by the time at which you achieved it. So, it never made any sense to achieve a victory, if you could delay the victory while continuing to grow. The Jason formula scaled the score at time of victory by an expected growth curve.

There is a similar adjustment to the Jason formula already built into the Civ4 scoring system (presumably influenced to some extent by the GOTM). It may not be as precise, but it mostly does the job.

The "problem" that cultural victories are likely to score a lot less than domination victories, just because the players going for a cultural victory don't build up so much of the stuff that contributes to score, was just as present in Civ3 as it is in Civ4. Or, if you wanted a high scoring cultural or space race victory, you would also end up conquering most of the world just to get more points. This problem was never really "solved".

 

[ewokimpi]

I'm somewhat leery of altering the scoring system for Civ IV GOTM.

If the goal is to make it possible for different victory conditions to take the top medal positions, and so make it possible for different skill sets, different techniques, to be used to take the top medal positions, then I could agree to that.

Playing GOTMs is, for me, about comparing my play to the play of those who are better than myself. And if changing the scoring scheme could favor a wider set of skills, well, that would be cool. There will be more to learn.

However, in the private Civ IV games I play, I'd like to be able to know how I'm doing relative to the "competition" games. Am I learning how to play better, or have I become stuck in a pattern, need to try something else?

If the scoring system changed significantly, how would a player be able to compare their private games to GOTM games? Download a separate .exe file to compute their score?

I would definitely hope we would not alter the scoring system so radically that GOTM games could not be compared directly to non-GOTM games. In that case, playing GOTM would be of less interest to me.

 

[ewokimpi]

Thanks, DavidDesJ.

 

[AlanH]

The Jason system was explicitly designed to remove the need to milk a game to 2050 AD to maximise the score. Civ3's in-game score calculates a cumulative average of your happy population and territory over the game, and then adds a victory bonus to reward an early victory. The expert Civ3 players found that *the* way to maximise score was to reach, but not to trigger, the domination territory limit as fast as possible, and then to maximise (milk) their happy population for the next 1000 years or so. The Jason system modifies the victory bonus to give greater rewards for early victories, with some variation according victory type, so that players didn't have to hit enter 200 times to a good score. It was certainly successful in removing the need for milking. It is now difficult to beat a fast domination Jason score with a milked 2050 AD victory. However, the argument periodically resurfaces as to whether Jason still favours fast military victories over other VCs.

Milking in Civ4 doesn't involve hundreds of turns of score building, as I understand it. The Civ4 score is calculated on the status at the end of the game, not averaged over the game, so it just requires that the player carries out some specific actions as the end of the game approaches in order to ensure that the instantaneous score at the date of victory is maximised. So a different kind of adjustment would have to be made on order to reduce or eliminate the need to juggle these factors.

Adjusting to balance different victory conditions would be another challenge. Jason tries to calculate a "best date" for each Civ3 victory condition, based on a few key map characteristics, and adjusts the early victory bonus depending on VC. It's very difficult to judge whether this is successful. Hence the continuing debate, even now.

 

[DynamicSpirit]

However, in the private Civ IV games I play, I'd like to be able to know how I'm doing relative to the "competition" games. Am I learning how to play better, or have I become stuck in a pattern, need to try something else?

If the scoring system changed significantly, how would a player be able to compare their private games to GOTM games? Download a separate .exe file to compute their score?

I would guess that if the scoring system changed, then possible solutions would be
(a) a MOD that would give the GOTM-calculated score for private games
and/or
(b) the Firaxis score is included as an extra field when GOTM results are released - just not as the field that is used to rank the results or calculate the medal winnders. (Exactly as already happens for the base score)

ISTM that's definitely a legitimate concern you raise, but I'm pretty sure it's something that's fairly easily solvable.

 

[drkodos]

So what options are missing?

Well, the poll is merely a yes or no question, but there is more than one way to say yes. That alone gives the poll a tweaking that weights a specific response. The science of polls indicates that a poll such as this, where a yes or no response is heavily weighted in favor of one of the answers is subtle persuasion to get people into thinking that yes is the best answer and it is only a matter of choice to pick one of the yes responses.


So, it is not a mmatter of what is missing, it is more a matter of too much being included in an attempt to appease too many factions, IMO.

I've no doubt that this *discussion* will shape people's views to some extent - discussions always do, but I don't know in what direction that'll be and I don't see anything wrong with that anyway.

I apologize for my accusational tone and approach. I was not intending to attack you or the poll, but I cannot help but notice certain things.

To my knowledge noone here is suggesting any change in who runs the GOTMs. This discussion and the poll will no doubt give some indication of the views of GOTM participants, and will probably give rise to some specific ideas too. I'm pretty sure that the GOTM staff are already aware of the skewed nature of the current scoring system, and I'm equally sure they'll read this thread (indeed AlanH has already contributed to it) and consider whatever points are raised. Maybe, the GOTM staff will decide in the light of this discussion that some alternative scoring system would be more appropriate. That decision will of course be theirs. So I don't really understand what the point you're trying to make is.

I guess the bottom line point I am making is this:

Letting the players decide on the scoring system and how it is implemented is likely a mistake. It opens a can of worms in the sense that IF the system is changed, then other people will have new complaints and will insist on new changes, and they will have precedent to point to and this will only empower their lobbying efforts.

This would then make the staff spend even more time on the political process of trying to create a "fair" system in order to appease the more vocal and vociferous amoung us.

I would prefer the staff to spend whatever time and energy they have to donate on creating great games. The scoring system is flawed from Firaxis end. That is where the lobbying should be aimed, at the game developers.

 

[DynamicSpirit]

The expert Civ3 players found that *the* way to maximise score was to reach, but not to trigger, the domination territory limit as fast as possible, and then to maximise (milk) their happy population for the next 1000 years or so. The Jason system modifies the victory bonus to give greater rewards for early victories, with some variation according victory type, so that players didn't have to hit enter 200 times to a good score.

Interesting. So what we're discussing here - did something comparable actually happen for Civ III (ie. the COTMs used the Firaxis score initially, but when that was found to be inadequate, a new score, the Jason score, was introduced)?

Milking in Civ4 doesn't involve hundreds of turns of score building, as I understand it. The Civ4 score is calculated on the status at the end of the game, not averaged over the game, so it just requires that the player carries out some specific actions as the end of the game approaches in order to ensure that the instantaneous score at the date of victory is maximised. So a different kind of adjustment would have to be made on order to reduce or eliminate the need to juggle these factors.

Milking the base score for the cow award probably is most effectively done via hundreds of turns of score building. (I don't think that's been done too much up to now because the knowledge of how to do it is only just becoming well known, but I'm pretty sure it'll be done a lot more often as time goes on).

Milking the final score tends I think to involve actions over a shorter timescale. That's because the Firaxis score already rewards early finishes sufficiently that you rarely gain much if anything by delaying your victory; rather, you gain by waiting until your victory is imminent and inevitable, and then suddenly upping all the things that are used to calculate your score. So Civ 4 milking at its most effective would tend to take the form of, eg. when you are close to your victory, suddenly activating vast numbers of settlers and plopping cities down to take advantage of every available bit of land - even putting cities in places that you'd never consider putting cities in if you had to actually carry on playing. You're in a way suddenly maxing your pop, in a way that would hugely damage your civ over the next few centuries, but you don't care about that coz you're about to win the game anyway. To a small extent it theoretically could also take the form of rush-building wonders, or extorting extra techs from opponents - anything to help your score on the final turn. There's probably lots of ways you can do it, but the real killer is mass-planting cities, because population is such a big contributor to your score, and land area adds a lot too, and it's possible to increase it in a very sudden step-like way (in contrast to eg. science where the progress is slow and limited by your science rate).

Adjusting to balance different victory conditions would be another challenge. Jason tries to calculate a "best date" for each Civ3 victory condition, based on a few key map characteristics, and adjusts the early victory bonus depending on VC. It's very difficult to judge whether this is successful. Hence the continuing debate, even now.

That is quite complicated. Personally I do really like the idea of balancing different victory conditions, but I'm inclined to suspect that's best done by EEO's idea: Simply scaling scores by an arbitrary factor that's based on how badly different victory conditions have been found in practice from past GOTMs to score.

 

[AlanH]

Interesting. So what we're discussing here - did something comparable actually happen for Civ III (ie. the COTMs used the Firaxis score initially, but when that was found to be inadequate, a new score, the Jason score, was introduced)?

Way before COTM. Jason scoring was introduced around GOTM 17, which was about the time Play the World was released. COTMs always used Jason, which was just adapted minimally to handle the changes in Conquests game play.

Civ4's scoring system was certainly influenced by the Jason system, as speculated by DaviddesJ. Aeson, who developed Jason, was also involved in the Civ4 beta testing.

 

[Alraun]

I suggest that Firaxis meant it to be primarilly a war game and to encourage domination & conquests as the goal.

I'd say more that they meant it to be a reflection of history and that encouraing domination & conquest over victories such as cultural & diplomatic is much better reflection of history.

 

[Shillen]

That is quite complicated. Personally I do really like the idea of balancing different victory conditions, but I'm inclined to suspect that's best done by EEO's idea: Simply scaling scores by an arbitrary factor that's based on how badly different victory conditions have been found in practice from past GOTMs to score.

This would never work. Conquering the world on a pangaea can be done in the BC ages, whereas conquering the world on a continents map takes much longer. Likewise domination on a large map takes a lot longer than domination on a small map. This would also mean that a game where the player gets to the domination limit before winning culturally would cause him to blow away the best domination score because of the multiplier for cultural victories. Same thing with a spaceship. Getting to the domination limit and maxing population would be a requirement for all victory conditions. (Of course this concept was present in the Jason scoring system, which was my biggest complaint about it.) That said, the Jason scoring system was the closest you can get to equalizing the different victory conditions. But not only is it complicated, it still had a major drawback in requiring you to maximize territory and happiness, no matter what victory condition, to score highly.

I voted yes in the poll, but it's subject to whether anyone can come up with a formula that makes sense. I haven't seen anything that will work yet. The Jason system is the best and honestly I'd rather stick with the current system than having a Jason system for Civ4.

 

[Alraun]

I think that the goal of a game called Civilization wouldn't be prize who razes dozens of cities, but is just my opinion.

Why not? I'd say it makes for an accurate representation of the history of human civilization. A sad represenation maybe, but an accurate one.

 

[DynamicSpirit]

Well, the poll is merely a yes or no question, but there is more than one way to say yes. That alone gives the poll a tweaking that weights a specific response. The science of polls indicates that a poll such as this, where a yes or no response is heavily weighted in favor of one of the answers is subtle persuasion to get people into thinking that yes is the best answer and it is only a matter of choice to pick one of the yes responses.

So, it is not a mmatter of what is missing, it is more a matter of too much being included in an attempt to appease too many factions, IMO.

OK, I understand your point now, thanks! The reason I did that was because I wanted the poll to give some indication of what people wanted any change in the scoring system to achieve. There's lots of possibilities there, but not really many different possibilities for why you might want to leave it is at is, so having more than one 'no' didn't seem to make too much sense.

I agree that could give the impression 'yes' is better, but OTOH it'll also split the 'yes' vote in a way that won't happen for the 'no' vote - and that could easily cause people to interpret the poll results in a way that's too favourable to the 'no's Sadly, totally fair polls are almost always impossible to achieve.

I apologize for my accusational tone and approach. I was not intending to attack you or the poll, but I cannot help but notice certain things.

No problem. And likewise I apologize if my response was rather brusque.

I guess the bottom line point I am making is this:

Letting the players decide on the scoring system and how it is implemented is likely a mistake. It opens a can of worms in the sense that IF the system is changed, then other people will have new complaints and will insist on new changes, and they will have precedent to point to and this will only empower their lobbying efforts.

Understood, but it does seem that there is a precedent in that a badly designed Firaxis scoring system was replaced by a custom GOTM system in Civ3, and that seems to be accepted as having improved the GOTMs there. (From what I can tell from the forums, I've never played Civ3 GOTMs)

There will always be people lobbying to make changes, but I don't think that's an argument for rejecting all lobbying, merely an argument for exercising discernment in what you change as a result of the lobbying.

 

[Déja]

What if you categorized each VC in a separate ranking? That way, it measures the best of each VC (including best loss) and then you don't have a milking domination victory kicking all the cultural victories to the curb. It also offers incentive to play the lower-scoring VCs because you'll be "competing" against a smaller group of people. Each "Medal" could then bear some icon representing the VC it was awarded for.

Then, if you wanted to, all the scores could then be scaled, ala EEO, and you could have the general ranking. Someone who can manage to win gold in each of the VCs would certainly be worthy of the title of Master Civsman (or Civswoman)

Thoughts?

 

[DynamicSpirit]

This would never work. Conquering the world on a pangaea can be done in the BC ages, whereas conquering the world on a continents map takes much longer. Likewise domination on a large map takes a lot longer than domination on a small map. This would also mean that a game where the player gets to the domination limit before winning culturally would cause him to blow away the best domination score because of the multiplier for cultural victories.

Surely, in that situation, the domination win would still score highly because you'd get the bonus for the early finish. Avoiding triggering the domination win but waiting for a later cultural win might give you a bigger multiplier, but then you lose some of the bonus for the early finish, so it's not clear you would gain.

 

[Déja]

Getting to the domination limit and maxing population would be a requirement for all victory conditions.

What if milking were eliminated by only counting cities that have been in the player's possession, out of conquering unrest, for at least X turns (where X could be 10 or 20 or whatever it would take to nerf milking by making all those crappy cities hurt the civ a bit first)