QSC Tech Scoring Discussion

[cracker]

Now that we have completed three solid games with a broad number of participants across many skill levels, we have enough experience and enough data to begin an informed discussion of how we might revise the QSC tech scoring so that the QSC Scoring may better reflect a number of different strategic approaches to the game.

This is a complex issue that really has 4 or 5 smaller embedded issues that may need to be addressed.

First, we need to have a little historic perspective of the process that has been used to develop and evaluate the QSC scoring system. As originally defined, the QSC scoring was intended to be a cummulative measure of the value of all things built, purchased, or otherwise acquired by a civilization. Fundamental to this scoring system is the base point assignment of 1 QSC point for every shield, food, or gold unit. Something that costs 100 gold units to purchase would have a 100 point value under this system.

Even when the scoring system was first defined, we recognized that it would be necessary to revisit to scoring values of some elements of the game and in particular we recognized that a better system of scoring the value of technologies would be necessary.

We recognized that some technologies had significant value to certain strategies while other technologies would be strategicly worthless under those same conditions. The value of each technology may not technically be the same to all strategic approaches to the same game under the same map conditions. Under ideal circumstances we would have a definition of the perfect, most powerful tech progression and acheiving that tech progression quickly would score high in the QSC while researching other technologies of little or no strategic value would not have a positive impact on score.

Another factor that we realized that experienced players would pick up on is that the actual cost to obtain ancient age technolgies can be 25% to 50% of the actual research cost if you learn to take advantage of:

1. contact induced tech devaluation
2. free techs from huts, even when produced by your rivals
3. minimum science tech gambits
4. counter-intuitive tech research, trading, and brokering
5. gpt and resource driven tech trading
6. selective "pointy stick" research
   [/list=1]
   We also need to combine this knowledge with our growing understanding that some technologies can be decidedly less valuable because the enabled wonders and/or improvements may not play a significant role in early game winning strategies. An example of this is the relatively worthless impact of the Great Wall wonder as it is implemented in the out-of-the-box game.
   
   So the questions before us can be broken down into three major categories:
   
   1) Should we implement a change to the QSC scoring system that will reduce the scoring value of the technologies across the board by a certain percentage while still keep those scores related in some way to the actual research costs or should we define a committee of experienced QSC and GOTM players who will assign a technolgy score table that can be implemented?
   (after we have had some discussion here, I will try to consolidate some of the discussion options into a poll and collect player inputs.)
   
   2) Should we move to implement these changes immediately, even as they may impact scoring of Qsc18-Celts, or should we wait to begin implementation with Qsc19-Ottomans?
   (there is a separate poll and discussion thread for this issue)
   
   3) Should we implement changes to certain technologies and/or wonders that will increase their value in the game to make them viable strategic choices that have some value other than just being ticket punches that must be obtained enroute to the next era?
   (there is a separate poll to discuss this issue in an example applied to Construction and The Great Wall.)

 

[flexo]

1. Have to think about that one.

2. No, wait until a new game (QSC19 in this case).

3. Yes. First I think things you get for "free" shouldn't count towards your score. ie. you build pyramids, you get points for pyramids, but you also get point for granaries. That shouldn't be. I think construction is pretty good as it is, even thou the great wall wonder isn't as good as it should be (that is why I voted for a free wall in every city).

 

[RufRydyr]

1-Yes. 2 and 3-see those threads.

I'd like to see less emphasis on going deep.

I also don't like being penalized by not buying horseback riding when it's available because I'm waiting to do a massive cash upgrade.

I'd like to see a scoring system that scores the best for someone sticking to the strategy they are following. So, I don't know how that could be done. Can you award more tech points to someone that starts wars to slow the tech race down if that's their strategy for not letting the AI get to their UU or something like that? Can you award more to someone who's strategy is beelining to Monarchy to increase production by being out of despotism? Can you award more to someone who is beelining to map-making to get tech cost down? There are so many strategies to choose from and I think that someone who is following a strategy for techs should score higher than someone who isn't.

I think researching what the AI isn't researching and trading those techs for what the AI has is very effective and should be considered common sense and not a strategy. It should result in lots of points, too.

 

[flexo]

So your are suggesting different "scoring tempates" for different games depending on how you have started out? That might be an idea I guess. But then the templates have to be even out so someone doesn't just start something to get a high qsc score and then do the switch the turn after 1000 bc.

 

[SirPleb]

I did a quick and dirty Excel analysis of QSC scores vs. final scores for the past three GOTMs. I put both scores to be out of 100, then calculated variances between QSC and final standing. Then I recalculated the same stuff with the tech component of the QSC score changed to different percentages of its current value.

I found that in all three GOTMs, the relationship between QSC standings and final standings is overall improved after dividing the tech part of the QSC score by 3. GOTM16 and GOTM17 showed significant improvements, GOTM18 just a small improvement. Smaller and larger divisors did not work as well, 1/3 seemed to give the best overall fit.

So, it seems to me that a simple first step which could be taken is to reduce the tech part of the QSC score to 1/3rd of its current value.

An alternative to this which might be simpler and which I think would have nearly the same effect (possibly even a better effect) would be to stop double (and triple) counting prerequisite techs. Just count each tech for its own direct cost.

I think that more complex schemes might be very challenging to work out. Some techs seem obvious to me in their value, e.g. I'd count Horseback Riding for zero at 1000BC. But others (and perhaps even that one) are highly debatable. Part of the problem is that the various values are different for different maps, and are even more variable depending on one's strategy even on the same map. I think these issues are so complex that they're not worth addressing - a simple broad brush devaluing tech overall seems good enough to me.

So my replies to the three questions are:

1) Yes, implement a change, and just reduce the value of tech in the QSC. (Optionally also form a committee to examine a possibly better system. But don't count on that coming to a conclusion, start by making the simple change even if the committee is founded.)

2) Change immediately.

3) Don't change the behaviour of the tech tree. It works ok as it stands in the game, changing it might be better but would not be better by enough to justify having non-standard gameplay considerations.

 

[DaveMcW]

1) I fully support the system where every shield, gold, and food unit counts for 1 point. This would mean techs are valued at 100% of their research cost, wonders are valued at 100% of their build cost, and wonder effects, such as pyramids/granary, are not counted at all. (Armies are worth 400 shields, the same as a wonder).

I believe factoring net income into the QSC scoring system would allow a better evaluation of wonders and overall achievement, but apparently no one else does.

 

[hotrod0823]

I think researching what the AI isn't researching and trading those techs for what the AI has is very effective and should be considered common sense and not a strategy. It should result in lots of points, too.

It is also a "strategy" to follow the AI up the tree to get tech on the cheap and effectively slow the pace. I think your comments about researching off techs and trading are downplaying how important proper tech choices can effect the overall play of the game. A player looking for a fast launch will try to use the AI to move quickly up the tree. A conquest game with an Ancient UU maybe they want to slow the pace to have their Gallic be a powerhouse for a longer period before the AI gets guns.

That really wasn't on subject but I have to agree with Dave the tech is what it is and if you would rather have your GDP in gold and military then so be it. I prefer mine in technology so am I going to be penalized for it? Leave it as is for tech values.

Wait until Gotm19 - perhaps try out a scoring system for the Redux games

Don't change the tech tree effects at all.

 

[Yndy]

Echoing in part Sir Pleb I would also agree a decrease in the weight of the technology points as they can count for more than half of your final score under the current system.

 

[BillChin]

The committee that did the GOTM "Jason" scoring system did an excellent job. I vote to give the same committee power to change the QSC scoring. The "Jason" scoring system seems much better thought out than QSC scoring.

Sir Pleb's analysis confirms what anyone can see at a glance, that there is only marginal correlation between QSC score and final game score.

What I would like to see is an immediate change to get more correlation between QSC score and final game score. Sir Pleb's suggestion for 1/3 tech values sounds like a good start, and then an in depth analysis to tweak further. No scoring system is going to be 100%, but the current QSC system is way out of alignment.
+ Bill

 

[tao]

In QSC 17, I had a galley shipping 2 settlers to faraway islands. Obviuously, my qsc score would have been better by settling closer. Is the settler-distance from capital another criteria? But I can see new exploits popping up ....

 

[Bamspeedy]

1.
Either, a.) techs are valued at exactly the same amount that is required to research it yourself or b.) techs you have are given a small bonus (5-10%) above what the initial cost would be. (this small bonus is just to promote players to progress along the tech tree reasonably well, and help the players who are doing most of their own research). Currently, the bonuses gives techs a value of anywhere from 0-100+% over what it costs you to research(20-60% being the most common range).

If any bonus is given for tech values, the bonus should be consistent. A % that is given the same for every tech's value, none of this pyramiding value stuff. Like if a tech is worth 400 gold, it would be worth 440 QSC points, and a tech costing 100 gold would be 110 QSC points (if 10% was used). The prior system would give a tech costing 96 gold, a value of 96 points, and another tech costing 160 gold a value of 384 points.

Also, I think that any partial research that had been done should get this bonus. If someone is researching a big tech like Monarchy, and misses getting the tech by a couple of turns, they could lose out on alot of points. They might be more tempted to research a cheaper tech, just to make sure they got another tech completed before 1000 B.C., so they can pick up more bonus points.

2. Depends on how much information you can give us. I would hope for the Ottomans game, but if all you can tell us is "tech values have changed" but not tell us how it is changing or by how much, then we better wait for GOTM20, so we have more information.

 

[Ribannah]

Originally posted by DaveMcW
1) I fully support the system where every shield, gold, and food unit counts for 1 point. This would mean techs are valued at 100% of their research cost, wonders are valued at 100% of their build cost, and wonder effects, such as pyramids/granary, are not counted at all. (Armies are worth 400 shields, the same as a wonder).

Ditto.

Note that not only tech (Trade/Discount), but also food (Granaries) and shields (Whip) can be obtained cheaper than normal.

More complex schemes will (1) always be arbitrary (2) make the QSC scoring untransparent.

A small question relating to Wonders remains: should Granaries and Barracks that are disbanded upon the completion of the Pyramids / Sun Tzu, still count for points?

 

[Singularity]

Yes, tech is not necessarily a reflection to the skill of the opening gameplay. So a reduced tech score and enhancement of other elements will perhaps reflect the strength of your game.

I think territory should be valued higher, as well as improvements of the terrain. A counter to the reduced tech value would perhaps be points for scouted terrain - and a bonus for each hut popped. That way the guy who got map of the terrain can compete with the one who got mysticism from a hut. Luxury resources in your terrain also should have some form of value i think.

Reducing QSC score to a simple count of food,shields and gold invested is a bit crude in my sence. It's the utilization of the resources that matter, not the volume of it.

 

[Kemal]

As there are many different approaches to each game of civ3 (as the "grouping" of the qsc results already indicates) developing a solid scoring system that does not bias any given approach at all may prove to be very difficult to say the least.

Like others have already mentioned, an important consideration must be to leave the scoring system as transparent as possible, to make a comparison between empires as easy as possible. I personally believe that no matter what scoring system one uses, it is just not possible to fully compare a game where the strategic choices have been made based on a military plan to a game where the strategic choices have been made based on a scientific/economic plan. Is having 30 horsemen in 1000 BC better than already being a Republic in the medieval age? I cannot tell.

Since the qsc is meant as a method of players helping other players to improve on their early gameplay, thus making it important to quickly see in which aspects a certain game has done better than another, I'd say to let transparency prevail over scoring and leave the system as it is.

 

[Bamspeedy]

These will probably all be moved to another thread, since they don't deal directly with 'tech', but I just want to state my opinions.

Originally posted by Singularity

Reducing QSC score to a simple count of food,shields and gold invested is a bit crude in my sence. It's the utilization of the resources that matter, not the volume of it.

The sheer volume of food, shields, gold is the whole point of the QSC. The QSC's whole emphasis is who can micromanage the most out of the terrain. There isn't alot of stuff you can build that will increase your investment before 1000 B.C., so micromanaging will generally get you a better score. With a cut-off date, unfortunately (there is no way to avoid this), to get the highest score you would want to get your cities 'primed' to be at their peak in productivity/commerce at 1000 B.C. If the capital produces a settler shortly before 1000 B.C., the 2 population points lost won't make up for the new city, if the settler has to travel too far of a distance. Barracks, temples, etc. cost upkeep, but won't help your score unless you actually are using them before 1000 B.C. If you aren't waging war (barracks), or have the population size (temples) to utilize those buildings, then they cost you points. Perhaps we should get points for culture?

A counter to the reduced tech value would perhaps be points for scouted terrain - and a bonus for each hut popped.

Amount of explored territory would be hard to count. Everything else is pretty easy to count, but explored territory would require someone to count all the tiles on the explored map (which can easily number in the hundreds) and may require people to bee-line to mapmaking so they can trade maps and easily rack up tons of points. Goody huts, would depend solely on the player's honesty in reporting how many huts he popped. I wonder if there is any programs that could be created to give us these figures?

I think territory should be valued higher, as well as improvements of the terrain.

Agreed. ICS is strongly favored in the QSC, because ICS's strong point is early in the game. ICS gives up early territory, but makes up for this in gold/shields, because the new cities are settled quicker (but have less of a long-term potential). Giving territory more points would help balance things out. Having lots of workers is great for long-term gains in the game, but for the QSC you would want just barely enough workers to keep up with the population.

A small question relating to Wonders remains: should Granaries and Barracks that are disbanded upon the completion of the Pyramids / Sun Tzu, still count for points?

If you won't get points for all the granaries from the pyramids, then I think you should get points for the granary that you built beforehand. I think people should not sell the granary in order to get the points, though. This makes it easier to check if they did build the granary (just right-click on it and see if they can sell it). The player just needs to make a note, so that others know to check for it. I don't think the 7 gold (or whatever small amount it is) is worth getting, so they shouldn't sell it as proof that they did manually build the granary.

 

[Ribannah]

Granaries and Barracks also have maintenance costs which can be quite a burden if you have several and need to keep them in order to count for the QSC score.

 

[Bamspeedy]

But if you get the wonder, you pay no maintanence cost for it. I'm 99% sure on this, unless this is a new/old bug I am unaware of.

The only way to know if you had built the building before the wonder or not would be to right-click and see if you can sell it or not.

 

[flexo]

If you have a granary and build the pyramids the cost of maintainance is zero. I'm more than 99% certain of this. Same goes for other wonder grants. Just like when you build the wonder that pays for all marketplaces and such (that I can't recall the name of now, the deadend one after banking).

If distance from capital is a factor for city worth, then wouldn't it make milking sense to send the second or third settler you make on a trek far far away, only to settle on the last possible turn for a great score? Don't think that should be an option.

 

[Skyfish]

I agree with Sir Pleb's comment.

 

[mad-bax]

From my perspective this discussion can be reduced to two strands of thought. Luck and trading.

Luck. Goody huts and exploration that leads to early contact are both dependent on luck. Therefore lucky players will get a higher score than they deserve. GOTM17 I explored East and met Egypt b4 1000bc therefore I could trade for techs I would have had no hope of getting otherwise. This was nearly pure luck.

Trading. Although it costs X gold to research a tech, that tech can be traded for Y other techs. So, in effect you can get several techs for the expense of one. IMO getting lux and resources hooked up is a better measure of skill since having been lucky enough to find them (and/or research the tech) then getting them hooked up by the end of the timeline would require pure skill.
Unfortunately it is difficult to separate luck from skill. Better players will still be more advanced than me, even if I am lucky.

Sir Pleb says reduce the tech points to a third of their current setting. This seems fair to me.

Maybe the scoring could reflect how many techs you have compared to the other CIVS. A game where the player has a total of ten techs with a lead of two techs over the next civ would score more tech points than another who had 12 techs but 2 less then the most advanced civ. Just an idle thought.

As an aside I would say that I really could not care less what my QSC score is, except that it is the easiest way to tell if I am improving or not. If the score does not reflect the skill of the player, then the score has no value to me at all. I just like trying to reproduce the timelines. People should get points for reproducible timelines.