VVV scoring system suggestion - Updated

[sanabas]

sql supports inline case statements: case when numgames > n then y else z end

Fair enough. I'm a maths geek, not a programmer.

anyway, re your formula i fear things where the total is based on a specific number of games. in your system there will eventually be a set of 68 games that reward the most points and if you place well in those, there will be no vvv gain in submitting other games.

Yeah, it's possible that you can't improve your score, at least until other people fill some more tables. But I suspect it will be unlikely. Given you'll need 125 deity games if you want to make your overall score as high as possible. 68 x std+ deity, covering every civ twice and every map three times. 56 x std+ sized lower levels. 30 x smaller maps, all on deity. 15 quick games, on std+ deity. 12 time games, on std+ deity.

And then double all that for score games rather than fastest finish.

It won't be too tough to have a non-zero score for all possible games. But it's extremely likely you'll have some scores that can be significantly improved. And there are still medals to go for. Or other people's #1 scores to knock off.

I'd rather that someone could eventually be looking at the tables and be unable to imporve their own score for now, rather than the most volume giving the most points. The civ 3 model (have a 1st place in every category) is far too easy to reach the best possible score. But I'd much rather have some bias towards quality over quantity. I could probably play 100 cheesy 5 minute low level duel games in less time than it takes to play half a dozen proper deity games, and it will certainly be far easier. Having the duel-spam be more valuable doesn't make sense to me. Without capping the number of games that count, spamming quick & easy games will be the way to get more points.

 

[vexing]

Without capping the number of games that count, spamming quick & easy games will be the way to get more points.

only because you're giving points for empty tables. i understand your goal to add an incentive for starting out tables, but it doesn't really fit into a competition to give points without... competition.

 

[sanabas]

It also doesn't fit to have a competition that is supposed to cover all types of games, and force anyone wanting a high score to play mostly duel and/or domination games. 2 weeks ago I counted the vanilla gold medal opportunities. There were 3 of them on large+ maps, 82 on duel. There were 2 chances for a time VC gold, 13 for diplo, 91 for domination. There were 79 chances for quick gold, and only 2 for marathon gold that weren't duel/dom.

Seems like there are 3 possible downsides to choose from:

1 - Give empty tables some reward, leading to spamming duel games as the quickest & most efficient way to improving your score. Also means that you can reach a very high score without ever playing a competitive science/diplo/culture game.

2 - Give nothing to any game that fills an empty table, leading to smaller maps, quick games, domination VC dominating the scoring opportunities. Also means that you can reach a very high score without ever playing a competitive science/diplo/culture game.

3 - Give empty tables some reward, and cap the number of eligible games, leading to the possibility that someone can reach a point where they can't improve their score. To reach that point, you'll need to post a competitive time (even if nobody else has attempted to match it yet) on a high level with every civ, map type, VC. You can spam duel games to get off to a quick start, but after 80 or so, you'll need to play other games if you want your score to keep improving.

To me, 1 & 2 are a bigger problem, they're less encouraging of a wide-ranging competition, even though they encourage competition for domination games and milking duel games for score. 3's problem is unlikely to happen, will offer a lot more opportunities for competitive players by seeding empty tables with a target to aim for, and encourages all types of games, rather than a small subset of them.

 

[Mesix]

I think a big problem is the vast number of possible tables which can be filled. Since each combination of several variables is a distinctly separate table, there are numerous tables unfilled.

Using the aforementioned Duel maps as an example:

8 difficulties
4 speeds
20 map types
34 leaders
5 victory conditions

108,800 tables available to for the duel (or any of the 6 available sized) maps!

By contrast, the Civ 4 HOF did not have separate tables for every leader. This change would effectively reduce the number of tables at each map size to 3200. The domination games which would qualify would be 640. If map type were also combined (which I believe they are in Civ 4), the number of duel/domination games would be only 32. The Duel (and likely also small) land based maps would quickly be dominated by Hun Battering Ram rushes and the issue would work itself out. It would be like the Quecha rush tables in Civ 4. Once you do a few variations it becomes disinteresting and you move onto something more fun.

 

[vexing]

To me, 1 & 2 are a bigger problem, they're less encouraging of a wide-ranging competition, even though they encourage competition for domination games and milking duel games for score. 3's problem is unlikely to happen, will offer a lot more opportunities for competitive players by seeding empty tables with a target to aim for, and encourages all types of games, rather than a small subset of them.

your analysis of the result of your system (3) is wrong though. if vanilla were under that system, the results would still be about the same:
the duel domination games that make up the majority of the competition are spread across multiple leaders, speeds, map types and difficulties, therefore 4/6ths of the total points are unaffected by your system. only the subpoints for vc and size are greatly affected.

it seems everyone agrees duel domination games are a problem, however that problem has been alleviated somewhat in g+k (aside from atilla). i think the proper course is just to deal with it directly by rewarding those games less, hence everyone's proposed map size multipliers.

 

[sanabas]

your analysis of the result of your system (3) is wrong though. if vanilla were under that system, the results would still be about the same:
the duel domination games that make up the majority of the competition are spread across multiple leaders, speeds, map types and difficulties, therefore 4/6ths of the total points are unaffected by your system. only the subpoints for vc and size are greatly affected.

It may not change the current standings if it was applied to the vanilla tables, though I suspect it might just be leader/map type that are largely unchanged. But it would certainly change how to improve your current vanilla score. Right now, the best way for tractorboy to improve his score is to try and beat some of the ~50 duel golds he has on his DL. Use my suggestion without a cap, and that's still the easiest way for him to improve. Put in a cap, and more duel spam won't help, he needs to post competitive times/scores on bigger maps.

it seems everyone agrees duel domination games are a problem, however that problem has been alleviated somewhat in g+k (aside from atilla). i think the proper course is just to deal with it directly by rewarding those games less, hence everyone's proposed map size multipliers.

Problem is, even if you reward them less, they'll still be the most time-effective path to quick points, because of the reward for a table with lots of entries. I don't think playing one type of game over & over should get you anywhere near the top of the standings.

One other thing I think is a benefit form having a cap is that the subsections will actually be different. Right now, they're pointless. Every game contributes identically to all 6 sections.

 

[vexing]

to encourage variety i previously proposed this: http://forums.civfanatics.com/showpost.php?p=12085869&postcount=3

 

[sanabas]

Interesting.

Would make all 6 sub-categories have different totals, would replace the need for a cap, would make duel spam far less effective. There'd still be some incentive to duel/dom-spam your way to the top of LoN/Map-quest, and it'd be a little helpful to inferno/tempi, but it would help far less to the other 2 sections, and for overall score.

I like it.

Each VVV sub-category would probably need 2 more columns on the display page, showing total overall score, and therefore the maximum score per civ/map-size/difficulty/speed/VC/map-type.


The only potential problem I can see is mainly for the inferno/go the distance sections, and also applies slightly to the tempi section. Because deity games will automatically be worth more than settler, large games worth more than duel, so to have all your scores at the average will require more lower level games compared to your high levels, more duel compared to std+.

I think the easiest way to adjustthat would be to look at the average without that particular modifier, and limit the score per category in that section to that average x that category's modifier.

So, for example, let's say we use Duel = 0.4, tiny = 0.6, small = 0.8, std = 1.0, large = 1.1, huge = 1.2

If I haven't fubared the formatting...

duel	tiny	small	std	large	huge	avg	total

40	24	16	34	11	25	25	150

contributes:

25	24	16	25	11	25		126

without size modifiers:

100	40	20	34	10	20.83	37.47

max contribution (37.47 x 0.4/0.6/etc):

14.99	22.48	29.98	37.47	41.22	44.97

totals:

40	24	16	34	11	25	25	150

so contributes 

duel	tiny	small	std	large	huge	avg	total

14.99	22.48	16	34	11	25		123.47

So if we take the straight average, duel, standard & huge are all capped. Tiny is not, even though there would be roughly twice as many tiny games as there are huge games to get those scores. Remove the map size weightings from the average calculation, so that it's roughly the same number of games that count on each size, and tiny is now capped, standard is now a little below the cap, and huge is barely halfway, and still offers full value for another game. It'd be even more glaring on inferno, where it would take ~5 times as many settler games as deity games to have both at the overall average.

 

[sanabas]

Another tweak that could help lessen duel spam, and encourage 'proper' games: Instead of using log (n+1) for number of games in a table, use log (n) on duel/tiny/small or king & below, and use log (n+1) for std+, emp+ games. So for duel games, settler games, etc, you'd get no points for an empty table (log 1 = 0), only start getting them when there are 2+ entries. For emp/immortal/deity on standard+ size, you'd be rewarded for filling an empty table.

 

[Mesix]

I think any changes to the scoring system should only apply to the eventual expansion of the HOF after BNW is released. Existing HOF tables for Vanilla and G&K should remain as is because it seems wrong to change the rules after the current leaders have invested so much time. If I want to catch up with them, I am happy to play by the same rules. When there is a clean slate of new tables for BNW, I will be happy to have a well thought out update to the scoring system what we can all compete in a fresh field.

 

[TractorBoy]

Another tweak that could help lessen duel spam, and encourage 'proper' games: Instead of using log (n+1) for number of games in a table, use log (n) on duel/tiny/small or king & below, and use log (n+1) for std+, emp+ games. So for duel games, settler games, etc, you'd get no points for an empty table (log 1 = 0), only start getting them when there are 2+ entries. For emp/immortal/deity on standard+ size, you'd be rewarded for filling an empty table.

I totally agree with this, you only have to take a glance at the vanilla VVV to see where things went wrong. Players were not willing to respond to a good submission unless they could beat it, for everything to run smoothly there also has to be some sort of reward for the guy that does respond. A case in point would be Vexing who posted one or two extraordinary Deity games, those efforts should not go unrewarded.

 

[sanabas]

I think any changes to the scoring system should only apply to the eventual expansion of the HOF after BNW is released. Existing HOF tables for Vanilla and G&K should remain as is because it seems wrong to change the rules after the current leaders have invested so much time. If I want to catch up with them, I am happy to play by the same rules. When there is a clean slate of new tables for BNW, I will be happy to have a well thought out update to the scoring system what we can all compete in a fresh field.

I don't have a problem with them getting aplied to the current tables. After all, it's not changing the game tables themselves, just how an overall score gets worked out. I don't think it would make much difference to the overall order. And could always show both sets of scores, anyway.

 

[Denniz]

@sanabas, could you detail your suggestion as whole again. I couldn't tell what from vexing suggestion you were including with yours. (Think of it as a final spec for me to code from.)

I was thinging to add another score column to the Vanilla and G&K VVV pages with the new scoring method and providing a toggle to change the sort. That is assuming a) I can figure out how it is supposed to work and b) I can figure out how to code it properly.

BTW, what would you guys suggest we call the new scoring formula\system? I will provide credit at the bottom of the page, so who should that be?

 

[sanabas]

No worries. Bit busy over the weekend, should put it up in detail Sunday night.

Was just looking at Vexing's suggestion for taking an average, and limiting scores in each subsection to that average. It's an alternative to putting a hard cap on number of games that qualify.

 

[Denniz]

No worries. Bit busy over the weekend, should put it up in detail Sunday night.

Was just looking at Vexing's suggestion for taking an average, and limiting scores in each subsection to that average. It's an alternative to putting a hard cap on number of games that qualify.

Both could be difficult to implement so it might be a good idea to explore both, just in case.

Anything requiring choosing which specific games are applied to a category/sub-category is complex and problematic. The best approach is to score a game by it's table results only and then pick the highest scores that qualify for each sub-category.

Applying limits or altering scores based on factors other than individual table results leads to a need to min/max different games see which scores best for the player. That level of complexity is very difficult and can be controversial.

 

[sanabas]

Fair enough. I don't think either method would be that difficult on the player's end, but I might be underestimating the controversy. Picking the highest scores, or including all scores, both lead to identical results across all 6 categories, and offer either a lot of incentive to duel-spam, or make it much more likely that you reach a point where you can't improve your overall result.


Anyway, on to the full detail...

Every entry on a table would generate a fastest finish score, and a score score. A time VC would generate both, it'd just be identical.

That score would be:

(scaling modifier) x (difficulty modifier) x (mapsize modifier) x (speed modifier) x (number of entries modifier) x (result modifier), done the same way for every single game.

Scaling modifier: purely for aesthetics about what numbers you prefer to see, if you use 100, games will mostly score between 40-500, if you use 1, mostly from 0.4-5. Won't change results at all.

Difficulty modifier:
Settler: 1.0
Chief: 1.2
Warlord: 1.4
Prince: 1.7
King: 2.0
Emp: 2.5
Immortal: 3.0
Deity: 4.0

Mapsize modifier:
Duel: 0.4
Tiny: 0.6
Small: 0.8
Std: 1.0
Lge: 1.1
Huge: 1.2

Speed modifier:
Quick: 0.8
Normal, Epic, Marathon: 1.0

Number of entries modifier, where number of entries = n:
For all duel/tiny/small or settler/chief/warlord/prince/king: log (n)
For all standard+, emperor+: log (n+1)
The way I'd do this if making a database/spreadsheet myself is to make all games be log [n + (diff x size)], where diff = 0 for king and below, 1 for emperor+, size = 0 for small and below, 1 for std+, which gives the desired formulae. Dunno if it's easier to pull variables straight from individual settings like that, or to look at which of the 48 size/diff combinations a game belongs to. I think it'd also make it easier to tweak, to scale the rewards for an empty table rather than a straight yes/no.

Results modifier:
Finish time:
Quick:
15/(15 + your turns - winning turns)

Std:
20/(20 + your turns - winning turns)

Epic:
25/(25 + your turns - winning turns)

Marathon:
30/(30 + your turns - winning turns)


In-game score (and finish time score on a time VC):
(your score/winning score)


Multiply those together, get a score for each game. Using a scaling of 100, having the best time on a std/std deity with 9 entries would give 400 points. Filling an otherwise empty std/std deity table would give 120.4 points. Having the best time on a quick/duel settler with 2 entries would give 9.6 points.


How to get the total:

Option a: Simply add all the time scores together, or the best x scores, to generate the time total. Do the same for all the score scores, to get the score total. Add them together for overall. The 6 different categories would be irrelevant, as they'd all be identical, same as now. You'd just have VVV finish time, VVV ingame score, VVV overall. For me, this is the least preferred option, by far. It means you can simply play one type of game to generate all of your score, if you prefer single landmass dom games, or you prefer high level Babylon/Spain science/diplogames, there's no reason to play anything else, beyond a handful to actually meet the VVV requirements.

Option b:
League of nations: Your best 2 (or 3, or whatever) games for each nation qualify, sum those to get your total points.

Map quest: Best 3 (or 4, or whatever) games for each map, sum those for total.

Tempi: Best 15 for each speed, sum for total.

Go the distance: Best 10 for each size, sum for total.

Machiavelli: Best 12 for each VC, sum for total.

Inferno: Best 8 for each difficulty, sum for total.

Sum all 6 totals to get overall VVV fastest time points, likewise to get overall VVV ingame score points. Sum the two to get overall VVV points.

Pros: Simple to understand, easy to see where you can imporve your overall score the quickest. Makes the individual categories relevant & different, so LoN you can do purely with pangea/gp/terra/inland sea domination if you want to, map quest you can use all Babylon science games if you want to, but a high overall total will require a variety.
Cons: It's possible to reach a point where it's no longer practical to improve your score. That will take a very large number of games, though. More likely that you reach a point where more domination games won't help, or more Attilla games won't help, or more duel score games won't help, etc. Which may upset those who prefer to play a certain type of game most of the time.

Option C:
Work out the average score for each nation in LoN, and cap the points for each nation to that average. Likewise for mapquest, machiavelli, etc. Slightly more complex for tempi/inferno/go the distance, work out the average of game total/diff modifier for inferno, cap the points to average x diff modifier. Same for the other two using mapsize & speed modifiers. So it'd be:

LoN: Total points/34 = a, any nations with more than a points only contribute a, any with less contribute full points. Sum them to get LoN total.

Map Quest: Total points/20 = b, any maps with more than b points only contribute b, any with less contribute full points. Sum them to get Map total.

Machiavelli: Total points/5 = c. All VCs capped at c, sum to get total.

Tempi: (quick points/0.8 + std points + epic points + mara points)/4 = d. Quick is capped at 0.8d, the others are capped at d.

Inferno: (settler + chief/1.2 + warlord/1.4 + ... + deity/4)/8 = e. Settler is capped at e, chief at 1.2e, deity at 4e, etc.

Go the distance: (duel/0.4 + tiny/0.6 + small/0.8 + std + large/1.1 + huge/1.2)/6 = f. Duel is capped at 0.4f, std at f, huge at 1.2f, etc.


Again, sum all 6 totals to get overall VVV fastest time points, likewise to get overall VVV ingame score points. Sum the two to get overall VVV points.

Pros: You can always improve your score by playing more games, even if you've already submitted 1000 of them. Like option b, makes each of the 6 VVV categories different, encourages a wide variety of games. For those who prefer a certain gamestyle, they don't reach a point where they can't improve their score without playing other types, they only reach a point where the other styles would be more helpful, but their preferred style still gets them rewarded, just not as much.
Cons: A little more complicated to quickly see where your score can improve, or to see how much improvement you've made between updates. Offers a bit more incentive than option B to duel-spam.


I think B or C would both be good, I'd probably prefer C, though it requires more effort to work out how to improve your score. I'm not a fan of A, both for making the categories irrelevant, and for encouraging spam/single game types.

 

[TractorBoy]

Very well put together sanabas, I especially like your explanations because they helped me to understand more clearly.

I like Option B;
If this is applied to VANILLA, some of the submission caps would have to be upped because there are less maps and less Nations. I would be very interested to see how things would pan out, some of the guys on very few points (which is the same everywhere) would shine in certain categories and that would be good for all concerned!!

 

[Mesix]

I think it would be better if the total points for any given category would max out at the same level. It could be a percentage rather than a sum. The higher score and more complete (closer to total games) your valid entries for each category, the closer your rating in that category would be to 100.

An example would be:

League of Nations: 50
Map Quest: 34
Tempi Trophy: 80
Go the Distance: 100
Machiavelli: 56
Inferno: 62

VVV: 63.7

The VVV is simply the average of all the events. Each event is a percentage of the total points available for that event. The scoring of the games and the points for the events could work exactly as Sanabas outlined or by some other method as the HOF staff decides. Having a percentage displayed makes it easy to compare and also easy to see where a player can improve. In the above example, perhaps playing each map another time through would increase the score. Perhaps the maximum number of entries has been reached, but some of the maps are very low in the results for the table. Clicking on the event to be improved will give a player detail for the entries which are counted for that event.

 

[sanabas]

I think it would be better if the total points for any given category would max out at the same level. It could be a percentage rather than a sum. The higher score and more complete (closer to total games) your valid entries for each category, the closer your rating in that category would be to 100.

Percentage of what? If you've completed the maximum number of entries, and your lowest score is a #1 entry on a duel table, is your % 100? If it's not, how do you get to 100? If it is, and you replace that lowest score with a #1 on a standard deity table, is your new % 102?

I agree having the #1 total in each be roughly equal is a good idea, but I don't see a way to set a theoretical maximum that's the same for all of them. Unless you just show everybody's total as a simple % of 1st place in each category. Which means that if you already have the lead, a better game just keeps your total at 100, but decreases everyone else's.

 

[Mesix]

It really isn't that tough.

34 civilizations.

If you allow 2 entries per civilization to count toward the League of Nations, that would be 64 games which can count towards the total. To reach 100% you would need to have a 1st place finish compared to other players who have submitted games with similar conditions (top of the table) on all 64 games! Submitting 1 game for each civilization (32 total) would count as finishing the event, but scoring the event would take into account not just the 1st submission but also 1 additional submission per civ (but not every submission, only the top 2).

Look at the Civ 4 HOF as a model. An empty table will always award 100 points. When someone else plays the game, the top player has 100 points, and the 2nd place player has something less than 100 points (based on how close the finish was). Two games count for any event which are added together, so a max of 200 is awarded (100 for each entry).

Now look at the Civ 5 HOF as a model. Submitting a game for an empty table has no value. When a 2nd entry is made, the better of the 2 gets a bronze medal. When 4 entries are made, the top 2 get silver and bronze. When 6 or more entries are made, the top three are awarded gold, silver, and bronze respectively. The only way to score points on any given table is to be one of the top 3 finishers. Regardless of how close you are to the top finisher, you will get a fixed score for silver and a fixed score for bronze and nothing for 4th place.

There should be a fixed total attainable by playing games which qualify for any given VVV event (Tempi Trophy, Machiavelli, etc.). What ever that total is, your score for that event would be x/y where x = your total score in all qualifying games and y = the max total you could achieve by being 1st place in all the tables for those games.

Using the example of 2 entries per civ for the League of nations we arrive at 64 games which count for League of Nations scoring. Let's say that a player takes the time to play every civ 1 time and submits 34 games. Let's further say that all of the entries are 1st place finishes in the table. That player will have achieved 32 top scores and 32/64 = 50% which would be their score for League of Nations. If they play 16 more games and achieve 16 more 1st place finishes, this would go up to 75%.

Now let's use the same example and say that 2 games of every civ have been played, but not all are 1st place finishes. How the score is calculated for games which are less than 1st place will determine what score they contribute to VVV. Let's use the current Gold, Silver, Bronze as a model. Getting 64 golds would be 100%. Getting 64 silvers would award 66%. Getting 64 bronze would award 33%. Getting 64 finishes which are all 4th place or lower would be 0%. If we use a model more like Civ4 HOF for scoring individual games, then each game submitted would be some fraction of 100. The score for the event would be the total of all these scores divided by 6400.

Each event might require 2 games of each setting, or some might require more. An example might be the Tempi Trophy for playing all 4 game speeds. Perhaps finishing 1 game of each speed will finish the event, but the score might consider 4 games per speed (total 16 games) or perhaps 6 games per speed (total 24 games).