1. We have added the ability to collapse/expand forum categories and widgets on forum home.
    Dismiss Notice
  2. Photobucket has changed its policy concerning hotlinking images and now requires an account with a $399.00 annual fee to allow hotlink. More information is available at: this link.
    Dismiss Notice
  3. All Civ avatars are brought back and available for selection in the Avatar Gallery! There are 945 avatars total.
    Dismiss Notice
  4. To make the site more secure, we have installed SSL certificates and enabled HTTPS for both the main site and forums.
    Dismiss Notice
  5. Civ6 is released! Order now! (Amazon US | Amazon UK | Amazon CA | Amazon DE | Amazon FR)
    Dismiss Notice
  6. Dismiss Notice
  7. Forum account upgrades are available for ad-free browsing.
    Dismiss Notice

News: Game of the month for Civ V - feedback appreciated

Discussion in 'Civ5 - Game Of The Month' started by Erkon, Aug 9, 2010.

  1. wwassme

    wwassme Chieftain

    Joined:
    May 3, 2006
    Messages:
    126
    Location:
    Cedar Rapids, Iowa, USA
    To comment on a few of the issues raised:

    I plan on continuing to play GOTM and BOTM (don't play WOTM). It looks like I probably need a new computer before I move to Civ5. Plus, I feel like I'm still learning the ins and outs of Civ4. Anyway, I'm usually not an "early adopter" and prefer to wait for at least the first patch.

    I think a training series of games would be great for everyone. Although I'm not likely to participate by playing, I'd be interested in reading the spoilers. That would also give the admins sets of saves to test their information extraction programs on.

    I think banning spreadsheets, test games, etc., is unworkable. I can't see where you'd draw the lines or how you'd enforce it. If others want to spend hours calculating the optimum configuration to generate the maximum hammers, commerce, beakers and food, that's fine with me even if I don't want to take the time to do it.
     
  2. Grifftavian

    Grifftavian Chieftain

    Joined:
    Nov 15, 2005
    Messages:
    1,647
    Location:
    1 SW of Atlanta. GA USA
    I like the idea of going to three spoiler threads (early game/mid-game/late game) and opening the early game spoiler on or a day after the [civ5] GOTM begins. I believe that will be beneficial with a new game, since we'll all be starting at the bottom of the learning curve with [civ5].

    To encourage more spoiler entries, perhaps it should be specified that posting in the spoiler threads is a prerequisite for winning any GOTM Medal or Award. Or would that requirement be too restrictive, or mean too much extra work for the GOTM Staff? Regarding the "Best Spoiler" idea, is it possible to apply the Thread Rating application to individual posts? If so, perhaps doing that with the just final spoiler posts may be a good idea. :dunno:

    I agree that the sub-competition or gauntlet should be player defined. I think one way the GOTM Staff could support that would be adrianj's idea of adding a poll to the pre-game discussion thread. Perhaps a check box could be added to the GOTM submission form so players could indicate if they wanted their submission included in the Gauntlet competition. And maybe an extra set of awards for the Gauntlet competition: Steel Gauntlet/Bronze Gauntlet/Leather Gauntlet. (BTW, does anyone know if there will be an auto-logging feature with [civ5]?)

    If people played through the [civ5] demo and submitted those, would that be of any benefit to the GOTM Staff in getting a head start on the extracting the information you need to develop the [civ5] GOTM Mod? Somebody has already proposed an unofficial [civ5] Demo competition over in the [civ5] Demo thread, but if the GOTM Staff can get any useful information from Demo saves, perhaps some sort of official Demo competition is in order.

    Like several others, I agree with godotnut's competition ideas regarding future [civ5] GOTM. I especially liked his idea in the last paragraph of his post about adjusting the Adventurer or Challenger saves when the Contender save is at the higher and lower ends of the spectrum.

    I never played any of the Civ3 GOTMs, so never participated in any of the Quick Start Challenges, but that sounds like a good idea to me, and something that would be very useful and educational for learning a new game.

    Well actually, it sounds like an excellent to outstanding idea to me!!!! I'll be looking forward to those playing games.

    Miscellaneous Stuff
    I've never won anything but a Shield award, so I'm a bit partial to them, but I'd be in favor of keeping them them around for [civ5]. They're basically the GOTM version of the Wooden Spoons from the SGOTMs. Perhaps they are a backhanded compliment, but their part of the fun of the GOTM competition IMO.

    [Screwball Idea Alert] Two or three times per year (i.e., every 6th or 4th GOTM) have a GOTM with a single victory condition. Designate these games as a Major GOTM (or some other designation) and apply a x1.5 or x2 multiplier to the results of these major games when calculating the global rankings. [/Screwball Idea Alert]
     
  3. da_Vinci

    da_Vinci Gypsy Prince

    Joined:
    Jun 13, 2004
    Messages:
    4,172
    Location:
    Maryland, USA
    As perhaps the leading beneficiary of the global ranking reward of participation, I have to admit that observation is true! :lol: (says the master of the power of persistent mediocrity).

    Elo systems, as I understand them, are based on head-to-head competition analysis (or at least the chess version of them is). If that is a requirement, could Civ(n) XOTM results be converted to a set of head-to-head results?

    Yes, if we were to treat each XOTM as a round-robin tourney, where each player beats or does not beat each other player based on score and/or speed result (perhaps within some range close results are considered a draw).

    Of course, multiple victory conditions creates an issue ... perhaps we view each game as separate round-robin touneys in each VC?

    And since highest speed is ususally not highest score, perhaps on submission a game must be designated, for Elo purposes, as a speed game or a score game?

    One issue to consider in Elo systems is a potential disincentive to play/submit if that risks one's high rating, particularly if there are a lot of low rated opponents. Or a disincentive to play/submit for a poor win or a loss.

    One solution is a system that only ranks wins ... no penalty for playing and losing. A loss is a non-attempt.

    To deal with rating risk if highly rated, consider this performance rating equation:

    Spoiler :
    from http://en.wikipedia.org/wiki/Elo_rating_system

    Performance Rating is a hypothetical rating that would result from the games of a single event only. Some chess organizations use the "algorithm of 400" to calculate performance rating. According to this algorithm, performance rating for an event is calculated by taking (1) the rating of each player beaten and adding 400, (2) the rating of each player lost to and subtracting 400, (3) the rating of each player drawn, and (4) summing these figures and dividing by the number of games played. This can be expressed by the following formula:

    Performance rating = [(Total of opponents' ratings + 400 * (Wins - Losses)) / Gms]

    This is a simplification because it doesn't take account of k-factors. But it offers an easy way to get an estimate of TPR.

    With an approach of adding or subtracting 400 rating points, in theory all wins agains opponents rated more than 400 points lower is a performance rating lower than starting rating, while all losses against opponents rated more than 400 points higher is a performance rating above the starting rating.

    This might be addressed by limiting the ratings calculations to the comparisons of results of your wins against other winning players that are within some range of your initial rating (maybe +/- 400 points?).

    Finally, to initially seed the system, perhaps we assign provisional ratings to some of our current veteran players as follows (Civ 4 example, based on level usually win at)

    Wins at Noble = 1400
    Wins at Prince = 1600
    Wins at Monarch = 1800
    Wins at Emperor = 2000
    Wins at Immortal = 2200
    Wins at Diety = 2400

    Of course, if there is a non-head-to-head Elo scheme already out there, then all of this is moot ... :mischief:

    dV
     
  4. Deckhand

    Deckhand Not so nice GOTM Staff

    Joined:
    Sep 22, 2008
    Messages:
    4,696
    Location:
    California
    I think we should focus on encouraging entries and not posts. If a post is required to win an award, then some may be less inclined to participate. Becomes too much work.

    Sometimes I post in the spoiler threads and sometimes I just don't have/make the time. And I usually don't read them until after I have posted (thus haven't read either BOTM32 spoilers yet).
     
  5. Grifftavian

    Grifftavian Chieftain

    Joined:
    Nov 15, 2005
    Messages:
    1,647
    Location:
    1 SW of Atlanta. GA USA
    Just an idea I threw out since several people had written in this thread that they would like to see more people posting in the spoiler threads. In retrospect, you are most likely correct, making an award contingent upon posting in the spoiler threads could have the opposite effect. Perhaps finding some way to recognize the best spoiler write-ups for each game will be enough to encourage more people to describe their game in the spoilers.

    Regarding more entries into the GOTM competitions, I think having a notice on the Civfanatics main page might help. For instance, I didn't know that they were still having Civ II GOTMs until I saw the notice about the 100th Civ II GOTM on the main page a while back. I don't recall the last time I saw an announcement on the main page regarding a Civ III or Civ IV GOTM. I know there are links to the GOTM Page in the left hand side bars, but I think a short notice on the main page about an upcoming GOTM, with a link to the pre-game discussion thread, might help to spread the word and get some more people interested in playing and submitting. Another notice on the main page congratulating the medal winners, with links to the results thread and the spoiler threads might help to generate more interest as well.
     
  6. Dhoomstriker

    Dhoomstriker Girlie Builder

    Joined:
    Aug 12, 2006
    Messages:
    13,329
    The Current Global Rankings System is great due to the Regular Participation that it encourages
    Which, if you are Cactus Pete, can get annoying at times if you want to try and maintain the top spot.

    That said, he's a great example in favour of the current participation-based Global Rankings, as he's stuck it out for the test of time and has moved on to even become a game designer!

    To be honest, I've participated more over time due to the "persistent threat" of a "severe drop" in the Global Rankings by not playing in the current month's game (or set of games across expansion packs), than I would have participated if there wasn't this "penalty" for not participating regularly.

    If regular participation is one of the goals of these GOTM games, then the method of progressively "stagnating" the value of scores from older GOTM games is a great formula to stick by.

    What's great about the Global Rankings is that you can "look back" in time to a previous month, say, to a time period when you were more active. You can then point to your friend (or more likely, to yourself) how "great" your top 50 standing was.


    We'd probably actually reduce submissions by people being afraid of "lowering their Elo value" if they had a bad game and we'd only see the best games being submitted from some of our players. Others might just submit all of their games regardless, but a certain portion of our players would only submit if they'd had a great game.


    I love how the Global Rankings ENCOURAGE participation, particularly regular participation, even from losses and retirements.

    That's why I brought up the idea earlier of awarding "minimum" values in the Global Rankings for losses and retirements (that were not "fake" 4000 BC retirement submissions). If our goal is to maintain and increase participation, regardless of how someone scored, then the current Global Rankings system is already great, and a small tweak of giving a bit more credit for failed attempts (losses and retirements) that actually get submitted would encourage even more regular participation.

    Once you "get in the habit" of submitting regularly, win or otherwise, you'll tend to stick around a lot longer than if you only submit "your great games."
     
  7. Erkon

    Erkon Chieftain Retired Moderator GOTM Staff

    Joined:
    Nov 5, 2005
    Messages:
    3,910
    Location:
    Malmö, Sweden
    Regarding Global Ranking

    Perhaps we could try out a couple of systems once we have statistics from training series games? "We" in this case actually means "you" i.e. any player who wants a better ranking system could play around with the database :) (given that 1) the data base can be built, and 2) that the data base can be exported)

    One way to remove the fake retirements would be to have a condition that the submission must have passed the date of the earliest submitted victory? That could give an incentive to keep on struggling in the hope that someone else actually manages to win :lol:
     
  8. MacBalt

    MacBalt Learnéd Fool

    Joined:
    Aug 3, 2006
    Messages:
    677
    Location:
    Tucson, Arizona
    Real life is such a time-sink, I'll feel lucky if I'm able to keep playing just BOTM. Even now I seem barely able to keep up with one of the three xOTMs, and with Civ V coming up... and perhaps (hopefully) another try at the whole SGOTM thing... well... :undecide:

    :agree:Yes, please.

    This will do, I guess... but I bet a creative map designer could also come up with some special, er, "surprises" set up with the gauntlet crowd in mind (eh Leif? :evil:). Perhaps a designer-initiated sub-competition should not be totally ruled out -- even if it might be best to delay such fiendishly clever map designs until Civ V gets to be more of a known quantity.*

    *(Also, it would be difficult to require specific saves, spoiler-posts, or data from entrants in a player-initiated competition. For me, encouraging more high-quality spoiler posts from excellent players was one of the most appealing aspects of the whole "gauntlet" idea. :huh:)
     
  9. da_Vinci

    da_Vinci Gypsy Prince

    Joined:
    Jun 13, 2004
    Messages:
    4,172
    Location:
    Maryland, USA


    Minimum value is an interesting idea, what size of minimum do you have in mind? In score, minimum might have to be fairly large to make any impact on games with 400,000+ top scores ... ;) Losses are always worthless in the speed side, are you suggesting a minimum there as well?

    My thought would be, on score side, to give losses and retirements their base score at the time, or firaxis score, whichever is larger (or if firaxis score is always larger, base score times some factor), so that we actually reward longer survival there. And the score is still a function of player effort.

    dV
     
  10. Dhoomstriker

    Dhoomstriker Girlie Builder

    Joined:
    Aug 12, 2006
    Messages:
    13,329
    Minimum Values are for the Global Rankings, not an individual game's Score Value
    Let's remember the context of a minimum value... we are talking about the Global Rankings table. The actual score values have already been scaled against a percentage scale when they hit the Global Rankings table (I've seen values in the Global Rankings table of 0.1% through 100%, or a blank value indicating no applicable value), so a minimum value would be implemented against the Global Ranking's scaled values (3% for retirements, 5% for losses, or what-have-you, instead of seeing a value like 1.3%, for the most recent set of games in the Global Rankings table).

    These "minimum values" would get scaled down over time as scores "stagnate" from previous games, just like all other values stagnate, but I think that picking a value like 3% or 5% for the most recent set of games in the Global Rankings table will not "interfere" with the top rankings in any way (so you won't get complaints from the top players) but will encourage more regular participation from those who don't often win our XOTM games. It will also encourage more submissions from those who do win regularly but "have a bad game" that they otherwise would not have submitted.

    We learn not only from top players but also by sharing our knowledge with and teaching less-experienced players (teachers often learn just as much as their students do, by the very act of teaching). One goal here is to give some sort of recognition to the less-experienced players who seem to fit the model that the Global Rankings table is set up to encourage, which is that of rewarding regular participation. The other goal is to have our "regular players" submit every game that they play.

    As for the individual games, the "see the full results here" tables would not need to see a change at all.


    Losses vs Retirements
    As I said earlier in this thread, the goals for rewarding losses and retirements are slightly different:
    For losses, we want to encourage people to play out their games and submit, regardless of how well they do.


    For retirements, we want to encourage people to have less of an incentive to retire "just because they will keep losing Score if they continue on with the game."

    By rewarding the players that stick it out for a loss slightly higher than those who retire are rewarded, we encourage people to try and turn-around a dangerous-looking situation, which is often actually possible to do.

    We also give players more incentive to "keep playing for longer to see if they have a hope of turning-around a dangerous-looking situation" before retiring if we have a maximum value (in addition to a minimum value) for retirements. One would be less inclined to retire as soon as a war breaks out and could play on to see how they can salvage the situation if they will not score any better by retiring earlier than by retiring later. The players will be encouraged to learn more and may realise that a lot of situations that otherwise looked hopeless are not always as hopeless as they initially appear to be.


    Minimum vs Maximum
    The other idea previously raised was to actually give a MAXIMUM value for retirements--both a Minimum and a Maximum value.

    Here's where the model can get slightly more complicated in how we'd want to implement it, but if we truly want to discourage people retiring just to get a higher Score value when they are losing a war, instead of playing out their games for a bit longer to try and learn from the experience, then a historical evaluation of Global Ranking Scores against Retirement Score values could give us an idea of what kinds of numbers we see. Using the existing values, we can get a feel for what numbers might make for good "minimums" and "maximums."

    It would be a lot easier for someone with access to the database to perform a query against Retirment games to give us a few values than to manually calculate them, such as the highest Score, lowest Score, and median Score seen for Retirements. It probably would help to have a few other values for comparision's sake, such as the same values for Losses and the lowest Score value seen for a win.

    If it is not possible to get this data, then I would arbitrarily suggest the values of:
    Losses = Minimum of 5%, where the loss meets a certain set of criteria
    Retirments = Minimum of 3%, Maximum of 5%, where the retirement meets a certain set of criteria

    The "certain set of criteria" would be aimed at preventing people from abusing these minimum values for "free" Global Ranking points by not seriously participating, such as by retiring before even playing very far into the game or intentionally allowing Barbs to walk into your undefended City or Cities just to lose. The "certain set of criteria" can be as simple as using a single criterion, such as finishing date.

    Erkon suggested using the criterion where the finish date must occur after the earliest submitted victory date, which sounds like a pretty reasonable suggestion. However, depending upon how much information about finishing dates is carried over to the Global Rankings tables behind the scenes, this information may be incredibly easy or incredibly difficult to extract and apply. We'd need someone who knows about the database's structure of the Global Rankings to chime in on what's possible in this regard.

    In fact, if the Global Rankings tables do have some way of easily referencing finishing dates, then this concept could even be retroactively applied to the current Civ 4 XOTM Global Rankings tables (I don't know how the Civ 3 Global Rankings tables work but perhaps even there, too, if the model is the same). It wouldn't necessarily be a trivial matter to implement, as you'd probably want to store static values instead of constantly recalculating minimum and maximum values every time that the tables are displayed, to avoid causing a performance hit to the existing Global Rankings tables. So, that might mean creating a new "Adjusted Score" column in in the Global Ranking Score table behind the scenes that gets displayed in place of the current Score values. It might even be a lot of manual work to set up if the Global Rankings can't easily reference the historical finishing date values. Thus, I won't hold my breath to see it happen for the Civ 4 Global Ranking tables. But, now's the time to plan for any such required changes to the Global Rankings' database structure for Civ 5.


    Global Rankings: Score vs Speed
    Losses and retirements do not even get a value in the Global Rankings table for the Speed value, so I do not see a change happening there.

    However, both the Score and, by extension, the Combined table, would give a bit of a boosted reward to Losses than is currently seen.
     
  11. Erkon

    Erkon Chieftain Retired Moderator GOTM Staff

    Joined:
    Nov 5, 2005
    Messages:
    3,910
    Location:
    Malmö, Sweden
    Minimum Values for Global Ranking

    One simple mechanism would be to modify the current global ranking system so that victory submissions receive between 5 and 100 (instead of 0 to 100). Losses and retirements could get 5 points, and no-one gets less than 5. Insert threshold of your choice. Insert losses/retirement criteria of your choice. This system would work both for score and speed, right?

    The sole purpose of this mechanism would be to reward loss/retire submission by giving some points in the global ranking. However, is this a suitable and efficient mechanism? Will we see noticeably more loss/retire submissions? Or is there another way to encourage and show appreciation to those who struggle to win and fall short?
     
  12. da_Vinci

    da_Vinci Gypsy Prince

    Joined:
    Jun 13, 2004
    Messages:
    4,172
    Location:
    Maryland, USA
    I think we want to encourage three things ... 1) starting a game even at a level you may not be able to win at, 2) submiting it even if you lose, and 3) playing to an in-game finish rather than retire.

    The current system, which give some points for a loss in the score side, does 1 and 2 to some degree, the question there is whether more incentive (something on the speed side) is needed.

    Currently, the system actually incentivizes early retirement, with the way time or a losing war will decay one's Firaxis score (as opposed to base score for the time effect).

    So there are two ways to approach this ... one is a generic set of points for losses or retirements, which should reward a play to conclusion more than a play to retire. In this approach, all non-wins of the same type (loss or retire) whould score the same.

    One could assign GR points to the wins 0 to 100 as usual. Then add 15 to each. Then give 10 points to losses played to completion, and 5 points to retirements (based on a "substantial try", to be defined later).

    Second approach, which I think I like better, is to rank the losses and retirements according to the actual results in the game. So suppose we assign GR points to wins 0 to 100 as usual, then add 10. Now assign retirements and losses score GR points based on the BASE score at the end scaled between 0 and 10 for points. While this still incentivizes a retirement if being destroyed, I don't mind that as it does allow the submission to reflect the high water mark of the game (rather than the final disaster). And it incentivizes continuing for growth if you are going to survive but not win.

    On the speed side, how about ranking the losses from 0 to 10 based on duration of survival. I know, it is the opposite of speed, but it is a time-related metric. And it would further incentivize continuation.

    Thoughts?

    dV
     
  13. Rusten

    Rusten Chieftain

    Joined:
    Aug 14, 2007
    Messages:
    3,098
    Location:
    Moscow
    I'd be ok with that, but people like shiny awards. If there's no official award then it may die out like the BOTM gauntlet did.
     
  14. srad

    srad Chieftain

    Joined:
    Dec 10, 2007
    Messages:
    627
    Location:
    Heidelberg, Germany
    Guess this is right!
     
  15. Grifftavian

    Grifftavian Chieftain

    Joined:
    Nov 15, 2005
    Messages:
    1,647
    Location:
    1 SW of Atlanta. GA USA
    Never Mind.
     
  16. AlanH

    AlanH Mac addict, php monkey Moderator Supporter GOTM Staff

    Joined:
    Jan 9, 2003
    Messages:
    28,733
    Gender:
    Male
    Location:
    England
    Nice idea :)

    However, I'll still be begging for saves (and replay files, if they exist) to look at from Sept 21, as my UK copy of the game won't arrive until about a week later, and even then, there's no guarantee that it will run in my Mac's Virtual Windows Machine.

    My first priority will be to get the save parser working, so that we can process submissions automatically. I'd like to achieve this during the testing phase if possible, so that we can start to see how/whether we might implement all these fancy new scoring requests, and get the results processing under way.

    Development of a full competition mod will undoubtedly require the SDK, which will include the source code of the DLL, or whatever is its equivalent in Civ5. We'll also need to get to grips with the Lua scripting language that I believe replaces Python.

    So I urge players to lower their expectations. Given the number of unknowns to get to grips with, and the voluntary nature of our resources, it's going to take some weeks, or even months after release to get into full competition mode.
     
  17. Fluroscent

    Fluroscent Chieftain

    Joined:
    Nov 15, 2002
    Messages:
    1,179
    Location:
    Denmark
    The need for mods etc. to be created is definitely a blessing in disguise. How the rules/awards are best designed is probably going to be impossible to tell before some games have been played, and the mechanics become clearer. Let us get a few games that do not "count", just to see how it all works out.
     
  18. whb

    whb Chieftain

    Joined:
    Jan 21, 2002
    Messages:
    702
    Why make them "not count"? The global ranking will decay at a reasonable rate anyway, but if that's not good enough then just reset the global rankings and medals after the mod comes in. (I believe this is what the HOF did, where they had a beta, but didn't say things like "this doesn't count" that would simultaneously discourage participation and encourage cheating).
     
  19. LowtherCastle

    LowtherCastle Chieftain

    Joined:
    May 18, 2005
    Messages:
    20,934
    1. Focus on sharing and learning between players. That's what was great about the QUick Start Challenge. It helped players and brought the community closer. All decisions/changes should prioritize community sharing/growth.

    2. Do NOT lessen the number of overall awards. That would be too exclusive for a friendly competition.

    3. Exception to #2: Replace the Gold/Silver/Bronze medals with a fastest finish victory condition that is arbitrarily rotated from month to month. Month 1: Domination. Month 2: Cultural. Month 3: Diplo ... Month X: Domination. Month Y: Cultural. Month Z: Diplo ...

    This enables the most competitive to compete with each other if they so choose and also gives weaker players a hint at which victory conditions might be more accessible in any given month. And this eliminates milking (except for the COw award which should remain for inveterate milkers).
     
  20. da_Vinci

    da_Vinci Gypsy Prince

    Joined:
    Jun 13, 2004
    Messages:
    4,172
    Location:
    Maryland, USA
    So the only score award would be the cow, which is based on base score? Nothing based on the Firaxis score? (or whatever the Civ 5 equivalents are).

    Also, for the designated speed VC, are you suggesting a gold, silver and bronze for speed in that VC (and obviously no other speed award in that one for that game), or just the one top speed award in the designated VC? If we are dumping score altogether, then why not keep three medals in the designated, highly competitive VC?

    Now, to wait for Cactus Pete to cry out in pain, and give his rebuttal ... :mischief:

    dV
     

Share This Page