The GOTM Scoring System

[DynamicSpirit]

I'm starting this thread in response to the huge interest that the discussion about scoring seems to have attracted in the difficulty level thread, and I thought it'd be better to continue this discussion in its own thread.

Poll attached to try to gauge opinion. I've deliberately kept the options in the poll as vague-ish, conceptual options because I think the discussion of how to change the scoring system, if that's desirable, is still ongoing and there are probably ideas still to emerge.

Background
There seems to be some considerable feeling, which I share, that the current scoring system is far from ideal. Problems people have cited include the huge incentive to milking, the fact that medals can to some extent be won by milking rather than by playing the game in the way that a lot of us feel it was supposed to be played, and the fact that domination and conquest wins seem to be heavily favoured in the scoring. It's also been suggested that the scoring should favour moral values that we regard as good (eg. penalizing slavery) though I think that suggestion is more controversial.

The Issues
I think it's not too controversial to say that
(a) No scoring system is perfect, and any scoring system is likely bias the way people play to some extent, as well as resulting in some milking by people interested in maximizing their own score, but...
(b) Quite a few people think the current scoring system can be improved on considerably.
(c) The extent to which milking can impact the scores at present risks causing the gold, silver and bronze medals to become generally far less respected as indications of ability to play Civ well than they could be.

I guess if I'm wrong in those assumptions, that'll be shown up in the accompanying poll results


Suggestions so Far
Several specific suggestions for changes have been made, including:

1. Reduce the weighting given to population (me)
2. Weight the results by victory condition (EEO)
3. Increase the weighting given to fast victories (Cactus Pete)
4. Mechanisms for excluding or even giving a negative weighting to foreign-nationality population (EEO)

I'm sure there are other ideas waiting for someone to suggest them

Technical Issues
It's worth pointing out that the current system is based on simply using the Firaxis scores and so is the minimum amount of work for the GOTM staff. Any other scoring system will require some coding to implement, and I don't know how easy that is. It's obvious from the game uploading process that the GOTM staff have some software that's capable of reading game files and extracting final score and victory status from it. Of the suggestions I've listed above, 2 and 3 do not require any more information to be read from the save files, 1 requires the components of the final score to be additionally read, and 4 presumably requires detailed analysis of the game state, including what's going on in each of a player's cities.

I'm hoping someone will know what is and isn't in principle possible at a tech level, in terms of what information can be read from and therefore what analysis performed on the savegame files.

Finally
So far I don't know of any indication that the people behind the GOTM are interested in this discussion, I'm hoping they are, and I'm hoping starting this thread on the offchance isn't going to be too presumptuous. If it is or if it creates more work for the staff, I apologize.

 

[drkodos]

What is the goal of the GotM?

IF it is to determine who is the best Civers, then the scoring system could use a change because the current system does not indicate who is best.

IF the goal is to provide a playing field for comparisons between gamers and styles of play, then no change is neccessary.

Since I feel more inclined that it should be the former, I vote yes, the scoring should be changed. Therefore I voted: "YES, for some other reason."

 

[AlanH]

Some initial reactions:

I'm hoping someone will know what is and isn't in principle possible at a tech level, in terms of what information can be read from and therefore what analysis performed on the savegame files.

Assume all game data is available in a snapshot of the end-of-game state. The main limitations are on historical information, where we only have key event information from the replay data, and from the histograph data. This means we may be able to calculate averages of some parameters across the game, but not many.

IF it is to determine who is the best Civers, then the scoring system could use a change because the current system does not indicate who is best.

How are you defining "best civers"? Players who have mastered the game can maximise their score however it's defined.

 

[drkodos]

How are you defining "best civers"? Players who have mastered the game can maximise their score however it's defined.

Players that consistantly do well no matter what the parameters of play. A player that can win any type of victory condition, and tailors their game to the matters at hand, rather than aiming for a specific type of victory condition that plays into their pre-existing strengths.

For instance: If the contest is to see who wins the fastest, then those that win quickest are better than those that take longer.

Whoever acheives the goal first and does so legally is better than those that follow.


Since there is never any goal established in any GotM's I think it is folly to think that the current set-up establishes any heirarchy or pecking order. It seems the monthly "competition" is not so much a competitive event as it is a showcase for players to strut their stuff. And there is nothing wrong with that.

BUT if awards are going to be handed out (and awards do come with the suggestion that one who wins an award is better than those that do not), then it would make more sense to tighten down the parameters and the goals of the event.


Just sayin.

 

[AlanH]

Since there is never any goal established in any GotM's ...

The gold, silver and bronze medals are currently awarded for the top scores. Isn't that the goal this thread is about?

 

[DynamicSpirit]

Some initial reactions:
Assume all game data is available in a snapshot of the end-of-game state. The main limitations are on historical information, where we only have key event information from the replay data, and from the histograph data. This means we may be able to calculate averages of some parameters across the game, but not many.

Oooh, thanks for the info and the interest, AlanH. So that basically means it's possible in principle to use any info about the final game state.

• I'm guessing the replay data means you can use things like
• When cities were founded, how many cities were founded vs captured,
• How many cities were razed,
• Whether wonders were built or captured
• How many turns specific civics were run for

(I'm not commenting on the desirability or otherwise of using that info, merely checking what is in principle possible).

 

[DynamicSpirit]

The gold, silver and bronze medals are currently awarded for the top scores. Isn't that the goal this thread is about?

That was certainly my intent when I started the thread. I didn't really think about fastest etc. awards since (a) I didn't notice anyone in the difficulties thread taking issue with the awards, and (b) I personally think that the awards system is fine as it is.

Obviously if anyone feels that there are related issues with the awards that need to be discussed too, then that's also fine IMO.

 

[AlanH]

Re. the replay data: Yes, I think so. It might take a bit of effort to get some of that out accurately, but that shouldn't affect your deliberations.

 

[EEO]

DynamicSpirit,

Thanks again for initiating all this. You did a great work and it is much appreciated.

A few remarks/questions

• 4. Mechanisms for excluding or even giving a negative weighting to foreign-nationality population (EEO)

The way this suggestion is formulated could be seen as a bit xenophobic , what was the opposite of my intent and not yours I'm sure. What I proposed was to give penalty for population brought into a civilization by force (conquest). Thus, it does not apply to cultural conversion (my trend should be to give bonus in this case !).
​

• I see two levels of discussion in this thread. A conceptual level as you have initiated it and a technical level (with formulas) as it happened to be in the previous thread. Do you think we have to mix both levels or do you prefer another thread (later) for the technical level ?

• I am relatively new in the forum discussions (but not in civ) and I wonder where such a discussion will lead us, if a consensus appears somehow ...( i.e. could the staff agree and implement a new formula for GOTMs ?)

• Finally I think a complex discussion begins, and please forgive me if some of my comments look strangely formulated: my mother tongue is French and despite the fact it takes me 3 times more to write in English, the overall quality is surely 5 times less ...

 

[DaviddesJ]

I've always thought that the score for victory should include the overall state of the world (e.g., total pop of all civs), not just your own civ. E.g., winning by diplomacy in a large, well-developed, harmonious world, should count for more than winning by diplomacy after razing all of your major opponents.

This, by itself, would help a lot with the current bias toward domination/conquest.

Of course, it's hard for GOTM to go far down this path. It's really something I wish Firaxis would do.

 

[drkodos]

The gold, silver and bronze medals are currently awarded for the top scores. Isn't that the goal this thread is about?

No, because the scoring is skewed and is not an accurate representation of playing strength.

And THAT is the point of the thread.

This, by itself, would help a lot with the current bias toward domination/conquest.

I suggest that Firaxis meant it to be primarilly a war game and to encourage domination & conquests as the goal. Here is what I offer as proof:

Page 99 in the Civilization handbook, under the Heading "DIPLOMACY", states

You'll have to learn to deal with these characters: some you'll fight right away; others you will negotiate with - at least until it's time to destroy them a few centuries down the line.

Sound a bit warmongery to me.

 

[DaviddesJ]

I suggest that Firaxis meant it to be primarilly a war game and to encourage domination & conquests as the goal. Here is what I offer as proof:

I think the manual was written mostly by marketing types. I don't think it offers a lot of insight into the design decisions.

But I do agree that combat is emphasized, notwithstanding the victory conditions that don't require combat.

 

[BLubmuz]

I don't like much the weight a conquest or a domination victory has against culture, diplo or space.
But ... this is the way the system is built, and is difficult to find a way to balance.

I think that the goal of a game called Civilization wouldn't be prize who razes dozens of cities, but is just my opinion.

A side note: in global rankings, 30K scored say, at Noble, has the same weight that 30K scored at Emperor.
I know the score is raised by the level, but an adjustement in global rankings could be easy and better reflect the ability of players.

 

[Tauro]

I don't like much the weight a conquest or a domination victory has against culture, diplo or space.
But ... this is the way the system is built, and is difficult to find a way to balance.

A side note: in global rankings, 30K scored say, at Noble, has the same weight that 30K scored at Emperor.
I know the score is raised by the level, but an adjustement in global rankings could be easy and better reflect the ability of players.

Indeed.
Bravo Vicenza (from Cagliari)
I think that only the level multipliers should be fixed

 

[DaviddesJ]

A side note: in global rankings, 30K scored say, at Noble, has the same weight that 30K scored at Emperor.

Huh? The "global rankings" are only based on comparing your score to the scores that others achieve for the same game. If you multiplied all of the Emperor scores by 10, the global rankings would be exactly the same.

 

[BLubmuz]

Huh? The "global rankings" are only based on comparing your score to the scores that others achieve for the same game. If you multiplied all of the Emperor scores by 10, the global rankings would be exactly the same.

Maybe I didn't expose correctly, I was thinking about cumulative score
premise 10 is much, say 2:
a player with 30K at Emperor has 60K + 30K Noble = 90K in cumulative score.
Just this.
Of course any multiplier doesn't affect a single game comparison.
Hope been clear this time.

@Tauro
Bello vedere che c'è almeno un altro italiano qui ... ciao.

@DaviddesJ
your post #17
I know, but we're talkin' about some change, so why not???

 

[DaviddesJ]

Maybe I didn't expose correctly, I was thinking about cumulative score
premise 10 is much, say 2:
a player with 30K at Emperor has 60K + 30K Noble = 90K in cumulative score.

The GOTM doesn't add up scores from different games (with or without weights). Nor should it.

The GOTM cumulative rankings are based on how well you do relative to other players in each game. Not on the total points you score.

 

[Denniz]

I agree that the medals for score (Gold, Silver, Bronze) are somewhat weighted in favor of warmonger victory conditions. (I think the fastest finish awards are fine the way they are.)

I disagree with any changes that attempts to force any style of play upon the competition. The most acceptable solution should be finding a way to equate all playing styles and victory conditions.

Most of the poll options, IMHO, seem to represent personal preferences. I am a little upset over the "PC" nature of the whole "morality" thing. :thumbdown

Civ4 is a fun game, warts and all. We should keep in mind that it is just a game.

 

[DaviddesJ]

Most of the poll options, IMHO, seem to represent personal preferences.

Of course they are personal preferences. That's the whole idea of a poll.

I am a little upset over the "PC" nature of the whole "morality" thing. :thumbdown

You're upset that people are given the opportunity to vote for something you don't like?

Or you're upset that some people with different views from yours did choose that option?

Or you're upset that they have different views from your own?

I really don't think any of those things should upset you. What's wrong with different people having different opinions about what's important?

 

[Déja]

(I'm sorry if any of this is redundant...)

Personally, I agree that GotM shouldn't emphasize or reward a particular style of play, but should complete its mission of providing a completely level playing field for people to compete on. Presently, Civ's scoring system benefits those who "abuse" the system by milking rather than rewarding someone for building the "best" civilization.

Now, this gets tricky in that it's hard to define what the "best" civilization is, but I think that the current scoring system is unfavorably bias toward a warrior's victory. The best thing to do would be to find a way to weight the scores (or even write a special GotM mod that calculates the scores in a more balanced manner) so that any of the six victory types has a fair chance to have the best score.

As I see it, there are two directions:

#1 Adjust the scores given in Civ
#2 Write a custom scoring system to generate scores that are more representative of what makes a successful civilization game.

It's really all a matter of devising the standards by which a civ should be judged within the competition. From there, a more fair scoring system can be devised