A proposal for a slight modification of Jason scoring

[Aeson]

Maybe it isn't as drastic as I made out, but I still can't kick the feeling that the trade off is not being offset by the scoring.

When I look at history it just seems that the 20K is sorely slighted.
COTM 03 - DaveMcW wins cow and fastest 20k, but is #18 in score.
GOTM 34 - Bradleyfeanor gets a 1545 culture 20K, an astounding time, and obviously with milking, but still is #7.

There is no way to accurately model the tradeoffs required for a very fast victory. That is because within the game mechanics there is a system of diminishing returns. If you have 300 Cavalry ready to go, you might not kill things quite as fast as someone who has 600 Cavalry, but you've sunk fewer resources into your force that will get you to the Domination limit. It's not linear, because 0 Cavalry won't get you to the Domination limit at all, and 600 Cavalry won't get you there twice as fast. Likely those extra 300 Cavalry will only make a few turns difference, and in some cases won't make any difference at all.

With 20k, if you put all your available early resources into a single city, only building a second city for a Worker pump to add back to the first until it is maxed out, you aren't going to get to the domination limit as fast as if you put a lot of your early resources into a single city, but have others diverted elsewhere and have 4-5 cities at the point where theoretically you could have maxed out your 20k city size, and instead only have it at size 8-10. You won't build a Wonder or two that you might have otherwise been able to build, but you'll have 2x or 3x the resources to keep expanding with. The difference in turns to get to 20k may only be a handful, 1-2% of the overall game turns you would play, but the difference in base score can be upwards of 50% to base score at the best date.

Diminishing returns requires an exponential curve (something like: bonus = ((bestTurn/turn) * maxscore)^X + (milkCurve * maxscore)) to accurately model. Problem with a bonus of that nature is that if it is off by a little bit from a turn perspective, it will be off by a whole lot from a score perspective. The very fast 20k's for example gave up about half their scoring potential to reach those extremely fast dates. So if we design an exponential curve that gives a turn bonus of half of max score (on top of the regular "no-tradeoff" bonus) to a game that beats the victory best date by 20 turns, what happens when/if someone beats the victory best date by 30 turns? They'll end up with several times the max score bonus. Because the best dates are not accurate to within a turn (probably accurate to within ~30 turns) an exponential curve on the date bonus side of things operating off the best dates is going to cause huge variations in score. Not 9k-12k... but 9k-200k or worse. It will essentially make the competition random. Whoever guesses which best date is off the most (and can play competently) will win by a huge margin over anyone who guesses the wrong victory condition.

The current scoring system has variation in it. It's ~10% most of the time, which is far better than I hoped for considering that we've used mods and are comparing across various game versions. Luck factors in to, and an early Settler can mean another 5-10%, but you can't model luck in any case. So you end up with a max score from 10k-12k depending on how lucky the player gets and how well the map is set up for their chosen victory condition. The best players will consistantly be in that range regardless of victory condition. Whether they are at the top or bottom of that range will have more to do with luck than the victory condition they choose.

 

[klarius]

Well, this thread went off from the initial intent quite a bit.
First I want to state that I appreciate the work of Aeson very much.
Instead of all the other people just complaining, he invented a formula, which really is much better than the Firaxis garbage.
I just wanted to propose a small modification. That was neither thought to eliminate milking, nor to give the magic bullet to equalize between different victory conditions.
I just saw an unbalance between early and late victory, which could be reduced.

 

[akots]

... To me I would like to see a 1545 culture 20K given more recognition than a "690 domination" regardless of any other scoring choices. To me it is simply a more masterfully played game. ...

I don't think so. 20K is actually the easiest victory condition to play on low difficulty levels. You need to manage a single city only whereas for early build up of considerable military you have to manage properly all cities and then it is indeed very tricky to cover as much land as possible in the shortest period of time. IMO, there is 10-fold more things to optimize.

... Akots thinks 20K/100K is boring.
I think those are the most fun way to win the game. ... The current system IS fair, it's just that it rewards a different play style than the one I prefer.

The current system cannot be called "fun or boring". It is how the game mechanics and scoring work. IMO, the style should be universal. For example, you need some culture to be able to capture the enemy cities instead of razing them or taking care of constant flips or quelling endless resistance or starving them off. I usually end Domination game with about 5-10K culture in the bin. Sometimes, it can be an option to go for 100K with milking but I hate milking. May be one day I'll go if there is a small map with scientific civilization (80K culture needed). This actually is a really bad setback since those players who don't build culture at all (except what is required for border expansion) usually do better with finish dates. However, this is very personal. It might be possible to include culture in the scoring formula though don't know how.

And certainly, exponential curve would just kill the scoring system.

 

[bradleyfeanor]

...within the game mechanics there is a system of diminishing returns...etc, etc

I think the most impressive thing about all that you said, is not only that you took the care to figure it out, but that you were able to say it in a way that I understand. Fabulous achievement!

I can now see the difficulties in accurately scoring a 20k effort. Does the same line of reasoning apply to a space race victory?

 

[akots]

... Diminishing returns requires an exponential curve (something like: bonus = ((bestTurn/turn) * maxscore)^X + (milkCurve * maxscore)) to accurately model. ...

What about a sigmoidal function? It would still be more or less linear around the best turn number but will be maximized when approaching turn 1 and turn 540.

 

[denyd]

As I understand it the formula for the score is some computed value (happy faces + people + tiles) divided by the number of turns played then added to the early finish bonus.

Since your early turns (first 30 or so) don't usually contribute much to the final score, in general the fewer turns you play the lower the divsor will be and the higher the resulting score is.

/begin rant/
Adding the fewer turns played also increases the size of the bonus, in a very broad statement "The sooner you win the higher the score" and since getting to diplomatic or space requries researching at a minimum 4 turns per tech from the Middle Ages on, all the way to the modern age, usually without help from the AI (why do they want fascism anyway?), unless you've got a very densely packed bunch of cites with massive irrigation & lots of happiness improvements, your score per turn is not rising. It seems the only way to get a top 10 score expand like crazy building nothing but barracks, workers, settlers & horses. Get to Chivalry, upgrade to knights and conquer to a domination victory. Of course, sometimes you need to research all way to astronomy or (dare I say it) magnetism to get ships to allow killing off enough peaceful tribes to reach the sacred tile count. Maybe we need an award for the most peaceful win to change the "War is Good" mindset.
/end rant/

Thanks for listening and have a nice day.

 

[solenoozerec]

As I understand it the formula for the score is some computed value (happy faces + people + tiles) divided by the number of turns played then added to the early finish bonus.

My understanding is different:
There is no division by number of turns. If there were such division than score would shrink with time. This is not the case.
Score simply accomulates and when game is over, it is you score if you lost or got it to 540th turn. If you won, you get a bonus that depends on how fast you won

It seems the only way to get a top 10 score expand like crazy building nothing but barracks, workers, settlers & horses. Get to Chivalry, upgrade to knights and conquer to a domination victory.

Interesting point. It seems like this is exactly how I started to play this game after I discovered GOTM.
Last time I didn't get even to Chivalry...
It seems like I am on the path to degradation...
I even put this guy near my spoiler -

What is next? - "No research at all chalenge (NRAAC)" and a donkey?

 

[AlanH]

My understanding is different:
There is no division by number of turns. If there were such division than score would shrink with time. This is not the case.

There *is* a division by number of turns, because the base Firaxis score is the *average* score over all the turns in the game. At the end of each turn the software adds up happy faces x 2 plus content and specialist faces x 1 plus tiles. It adds this number to the cumulative total to date and divides by the number of turns to date to get your average score.

As you said, Firaxis then adds (2050 - victory date) x difficulty as a victory bonus. This bit is replaced by the Jason victory bonus.

 

[Aeson]

I can now see the difficulties in accurately scoring a 20k effort. Does the same line of reasoning apply to a space race victory?

It would to a lesser extent. A SS (or Diplo, as they are basically the same thing) victory doesn't take as much trade-off to do extremely early as a 20k. Probably only limits your population/territory by 1/4 or 1/5 instead of th 1/2 that 20k requires at the most extreme.

Most of the time you need to get a decent research capacity to at least get to Knights or Cavalry fast to get a really fast Domination date. So if you reach Domination a little slower, building fewer Knights/Cavalry in favor of Libraries/Universities to maintain the tech pace, there isn't too much tradeoff. You do have to be more careful about who you do kill though, leaving good trading partners, and warfare is limited in that you want to keep your reputation clean.

 

[Aeson]

What about a sigmoidal function? It would still be more or less linear around the best turn number but will be maximized when approaching turn 1 and turn 540.

Not sure what you mean. Are you talking about the milking curve in total, or simply to model the "diminishing returns" bonus?

Looking at sigmoidal graphs, some version could fit the milking curve very well it looks like. Gradual buildup to 30% at turn 270, a steep incline, rounding off to 100% at turn 540.

With diminishing returns the problem isn't so much fitting an equation to it, but positioning it correctly so that the large bonus portion of the curve matches up to where the large trade-offs have been made. Positioning the limit just a handful of turns earlier or later leads to very large discrepancies that have nothing to do with how well the player has actually played.

Currently the 15% linear regression from the best date to 2050 is a compromise between no trade-off bonus and a properly formed (but very dangerous) trade-off bonus. Even at 4000BC that linear bonus should never award more than 30% of max score. We could use a sigmoidal instead to further compromise between the current implementation and a properly formed one I suppose.

The main drawback to that would be that by using the linear method there isn't so much variation in score if the best dates are wrong, with a steady decrease throughout the game. With a sigmoidal method the variation in score would be more localized, and so if someone just missed the "incline" due to a poorely placed best date they'd be left behind more. Basically the same variation but in a shorter frame of turns.

 

[akots]

... With a sigmoidal method the variation in score would be more localized, and so if someone just missed the "incline" due to a poorely placed best date they'd be left behind more. Basically the same variation but in a shorter frame of turns.

There are a few variations in sigmoidal function type (by inclusion of extra parameters) that may compensate for this by "widening" the quasi-linear central part of the curve still matching with the firaxis-score-type derivative. But it might be possible to get rid of the "tailing" at the start (first 100 or so turns) and end (more close to the actual milking curve). The best date then can be determined not by some arbitrary coefficients but by fitting finish dates to the median dates achieved by all players for a particular victory condition. Though there might be not enough entires for 100K or 20K games for a good quality fit. Also, this requires that the score will be known only after the submission is closed and not in advance. That way, if somebody beats the best date by 100 turns, he will get essentially the same bonus as somebody who beats it by 90 turns. Still the score of the first player will be higher but by a tiny margin which might as well be lower than the overall score growth achieved by the second player.

However, still getting to domination limit ASAP will be required for best score.

 

[solenoozerec]

There *is* a division by number of turns, because the base Firaxis score is the *average* score over all the turns in the game. At the end of each turn the software adds up happy faces x 2 plus content and specialist faces x 1 plus tiles. It adds this number to the cumulative total to date and divides by the number of turns to date to get your average score.

As you said, Firaxis then adds (2050 - victory date) x difficulty as a victory bonus. This bit is replaced by the Jason victory bonus.

This is strange, because if it is so, than it is possible that score can shrink.
An example: At a turn x I had many cities and a score 1000. At turn x+1, almost all my cities were destroyed, I have one happy face and one city with 9 tiles. My score gain for turn x+1 should be about 10.
I always thought that my total score in scuh case would be 1010.
If it is an average, then it should be 1010 devided by x+1. If my score per turn is not increasing then at x=2 it will be 1020/(x+2) and so on. Therefore my score should go down.

I noticed that even if civ is destroyed its score remains intact.
In COTM3 my civ was shrinking, but not my score. Before my defeat I was alomost not gaining any score, but it defenetly was not shrinking.
Maybe there is a division by number of turns to that score that is added at a particular turn?
I am certainly confused

 

[Aeson]

This is strange, because if it is so, than it is possible that score can shrink.

Yep. Load up a game where you have a reasonably good score. Give away all your cities but your capitol, then watch your score start dropping.

I noticed that even if civ is destroyed its score remains intact.

It stops updating. So if a civ was killed of on turn 180, it's total score is always divided by 180 thereafter.

In COTM3 my civ was shrinking, but not my score. Before my defeat I was alomost not gaining any score, but it defenetly was not shrinking.

Your turn score before defeat must have been sightly higher than your average score still. Since you start out with 9 tiles claimed and 1 content population, and stay at very low score level for quite some time, the average is going to be low for quite some time too.

 

[Aeson]

The best date then can be determined not by some arbitrary coefficients but by fitting finish dates to the median dates achieved by all players for a particular victory condition. Though there might be not enough entires for 100K or 20K games for a good quality fit.

As you note, the problem with this is participation. We can't be sure who is going to play what, how many of them there will be, or if they will have better/worse than average luck.

And the best dates aren't determined arbitrarily. The modifiers are based on thousands of games played and analyzed. They may not be terribly accurate a prediction mechanism, but are more stable than a median would be given the small sample size. (Even if everyone in the GOTM played the same condition it would be a small sample size)

 

[solenoozerec]

Yep. Load up a game where you have a reasonably good score. Give away all your cities but your capitol, then watch your score start dropping.

I will check it, I have even better example, in COTM3 all my cities were destroyed and I was wondering with a settler not having any score gain for some period of time. Click this link to see it:

But I trust you even without checking that.
So does it mean that by trying to survive with that settler and building a two new cities after all my cities were destroyed and surviving for more 100 years I actualy was lowering my score?
Gosh, that was stupid. I just thought "do not give up no matter what".

 

[akots]

... And the best dates aren't determined arbitrarily. The modifiers are based on thousands of games played and analyzed. They may not be terribly accurate a prediction mechanism, but are more stable than a median would be given the small sample size. ...

This is certainly 100% true.

Just curious though. Did you ever adjust the best date numbers for C3C?

If yes, is it possible to post the spreadsheet in the calculator thread...

 

[Aeson]

This is certainly 100% true.

Just curious though. Did you ever adjust the best date numbers for C3C?

If yes, is it possible to post the spreadsheet in the calculator thread...

The changes to the modifiers for C3C were:

The addition of the Agricultural and Seafaring modifiers. Industrial modifiers were toned down a bit. IIRC Agricultural got the old Ind modifiers, or something close to it. Don't recall what the Seafaring modifiers ended up being.

A change to the Diplo and Spaceship modifiers to account for the change in number of tech necessary to research. Radio was taken out, and a few more techs required to build the Spaceship now. Also a miscount of turns was fixed in both versions of the modifiers.

Then the Cultural 100k modifiers by mapsize were taken out to account for the variation in-game.

AlanH would have the exact modifiers. I don't.

The changes in C3C really haven't been analyzed enough to come up with good modifiers. But because the scoring system is crossing 3 versions, playing on edited maps, with mods to the game rules even, the best dates have intentionally been mostly factored out of the equation. Changing the best date by 20-30 turns will only have a ~1-2% impact on score in most cases, so the variation is slight at worst. The majority of the score is being derived from the population/territory side of things as it relatively stable.

 

[jeffelammar]

The current system cannot be called "fun or boring". It is how the game mechanics and scoring work. IMO, the style should be universal. For example, you need some culture to be able to capture the enemy cities instead of razing them or taking care of constant flips or quelling endless resistance or starving them off. I usually end Domination game with about 5-10K culture in the bin. Sometimes, it can be an option to go for 100K with milking but I hate milking. May be one day I'll go if there is a small map with scientific civilization (80K culture needed). This actually is a really bad setback since those players who don't build culture at all (except what is required for border expansion) usually do better with finish dates. However, this is very personal. It might be possible to include culture in the scoring formula though don't know how.

And certainly, exponential curve would just kill the scoring system.

I certainly did not mean to give the impression that the current system was boring. I was just pointing out that "personal preferences" vary, so it is really impossible to satisfy everyone.

I like the idea of including Culture in the scoring system, but it would have to be done carefully.
You would get a small bonus for different values, very interersting concept and one I will definately give some thought to.
Maybe a score bonus for each city, so having 30 1000+ culture cities was worth something, but a single 20000K was worth more.

IMO, one of the great strengths of Civ3 (and its predecessors) is that they do not necessarily require war. It is often unavoidable (like real life), but you can play, have fun and "win" without fighting a single war in some cases. (OK, so not so much at the higher levels) Unfortunately the Firaxis score doesn't reflect this much.

I'm not sure we need it for GOTM, but it is still interesting.

I just want to be clear about something. I greatly respect all the work that has gone into the current formulae. They are great, and far better than the old Firaxis score. I just like the idea of trying to continue to improve it. Aeson makes good arguments against many suggestions, but that shouldn't stop us from keeping on trying. One of the improvements I would really like is some way to include game mechanics other than Population and Victory Date.

 

[AlanH]

Don't recall what the Seafaring modifiers ended up being.

Seafaring reductions are 5 turns for 100K, 10 turns for Conquest, 15 turns for Domination and 20 turns for the later victories. No reduction for 20K.

AlanH would have the exact modifiers. I don't.

The changes in C3C really haven't been analyzed enough to come up with good modifiers. But because the scoring system is crossing 3 versions, playing on edited maps, with mods to the game rules even, the best dates have intentionally been mostly factored out of the equation. Changing the best date by 20-30 turns will only have a ~1-2% impact on score in most cases, so the variation is slight at worst. The majority of the score is being derived from the population/territory side of things as it relatively stable.

I can publish them if it would be helpful to the discussion ... but given their relative insensitivity, it may simply provide a lot of detail that will be discussed endlessly to very little effect. There are over 200 date modifiers covering map size and topology, difficulty, civ traits and victory type. To take one example, we just did an exercise to adjust the GOTM 35 scores for players who had militaristic Ottomans instead of scientific, and the effect on scores was between 0.2% and 0.5%. I don't think any player's rank in the game changed as a result.

Note also that there are just two sets of date and score modifiers, for Classic and C3C. They are not tuned for each game. Where mods are made, such as moving the era when contact and map trades can occur, or shifting the techs required for space in a game, there is no change to the modifiers.

 

[Denniz]

... I just want to be clear about something. I greatly respect all the work that has gone into the current formulae.
... I just like the idea of trying to continue to improve it.

I completely agree with both statements!

...Thing is, you *need* to milk optimal the later the victory date is. In Conq/Dom you get optimal milking value for free. That's where the difference lays.

a space oddity, in a couple short sentences, has touched the core of what I was struggling to express in several paragraphs.

The discussion in this and the COTM04 AA thread has been very enlightening, despite all the math. Not having the advanced math, I tend to take a more conceptual approach. The issue seems to center around the "milking curve" used to compensate score "passed up".

From Jason Score explanation page:

DateBonus = ((540 - PlayerTurn) / (540 - BestTurn)) * Curve
Curve = (MaxScore - (((PlayerTurn / 593)^2 + (PlayerTurn / 3200))* MaxScore))

PlayerTurn: The turn which the player achieves victory.
BestTurn: The predicted 'best' turn for that victory type.
MaxScore: The predicted score for a fully milked game.
Curve: This roughly mimics the scoring progression in a milked game.

MaxScore is determined by taking the domination limit, figuring out how many citizens that many tiles can support, and then using those numbers to calculate the upper limit for TurnScore. Then a modifier based on map characteristics is applied to the TurnScore to determine what the MaxScore would be. The formula is:

MaxTurnScore = (HappyCitizens * 2) + ContentCitizens + Specialists + Territory
MaxScore = MaxTurnScore * MapModifier

HappyCitizens: An estimate based off of territory. Each tile can be worked by one citizen, so the maximum happy citizens would be one per tile, or the domination limit. Each city 'eats up' one of these citizens, so the number of cities will slightly decrease this count.
ContentCitizens: For the purposes of this prediction, all citizens are considered as happy, which would be the case in a well run 'milked' empire.
Specialists: The number of specialists is currently predicted based off the average food per tile. The support cost of the tile worker is subtracted from the average, and then the result is multiplied by the domination limit and divided by 2, as each specialist requires 2 food.

If you look at MaxTurnScore, it is made up of two components. Pop and Territory. One is constant once you reach the dom limit. The other is quite variable depending on the map, civ traits, etc. It seems to me that the curve is modeling both components when only one, the population, is an unknown.

Passed up score for territory is simply dom limit tiles * turns remaining / 540. Correct? Why not seperate that out of the curve? This should be minor change.

Second there has been a lot of great work by people like Dianthus and ainwood with utilities that extract tremendous amounts of information from save files. Instead of a generic curve to compensate for pop score, why not use the actual save file information to model the potential for the player to grow their pop to max.

1. Take the actual tiles controled to figure max pop based the actual food potential for that civ on those tiles. (tiles with cities could be counted properly in terms of how ICS lowers max happy pop).
2. Use the ending pop values to determine model how long until max pop could be reached. creating a custom curve based on the actual position.
3. Factors such as luxuries, wonders, and existing improvements that influence happiness and growth could we counted and used in the model to determine the steapness of the curve for max pop and ratio of happy/specialests over time.

Notes:
1. you would have to normalize territory down to dom limit to deal with last turn territory expansion exploits. You could simply back out tiles by type in purportion to total mix of terrain types in the empire.


I don't pretend to think the modeling the population is a simple thing, but even a crude first attempt might provide a better comparison of different players games than the current curve. More strategy to capture the best terrain and maximizing pop growth and happiness would be required and rewarded. It could bring the balance element into the scoring equation.

I will leave it to better informed (and educated ) people to tell us whether any of this is possible/practical.