Reloading in GOTMs

[DaviddesJ]

Police cars out in the open are best at controlling driver speed; police cars hiding are best for writing tickets.

I don't think this is really true. Police cars in the open only slow traffic where the police cars are. People go just as fast everywhere else.

 

[bio_hazard]

I'm in favor of enforcing the 'no-reload' rule- the results of games that were reloaded can still be posted in the appropriate final spoiler section for each game, so there's still the participation aspect for those excluded from the awards...

Not sure I see the need for this, but *if* there is a decision that the results of reloaded games can be somehow uploaded to the staff, to simplify things I'd suggest that the Adventurer start be replaced by the reload-eligble start.

I'd further suggest that this start be world-builder enabled, so people who need extra help or play in a rushed manner can add units to that city they accidentally left undefended, get that last space-race tech that they forgot to get, etc. People who want to bend the rules in order to have an enjoyable game can do so. People who want a fair competition will chose the contender/challenger starts.

 

[Thrallia]

The reality is people cheat and that policing the cheating is what is causing so much headache for the staff and contributing the delay in the results. Regardless of what you do, people will continue to cheat. As this is just an informal game with no substantive awards, for most the cheating centers around what is enjoyable not about some kind of gain.

Rather then fighting the cheating, you can capture 90% of it by having the players self admit to it. This does a couple of things. It saves the work of the staff in checking all the games that have self admitted to it and is more inclusive of the vast majority of players out there who are not used to playing without reloading. There are not that many people who submit games and even fewer now.

that sounds like rather stupid logic if you ask me...'people will do it anyway so rather than fight it, just let them'

 

[drkodos]

I don't think this is really true. Police cars in the open only slow traffic where the police cars are. People go just as fast everywhere else.

Actually, it is probably the most effective known method of controlling driver speed. It is not perfect, but it is better than any other method that is pragmatic. Here is how it is done:

Position cars every 3 to 5 miles on heavy trafficked highways. Only put an officer in one out of every four cars, rotating officers so that drivers cannot discern which vehicles are staffed at any point. Simultaneously keep one/two vehicles highly visible, mobile and rolling in traffic at the speed limit. Contolling speed serves the consensus benefit of the community. Writing tickets serves the beneift of the authority, and oft begs the question of whether it may not actually serve to benefit the community.


The metaphor could easily be adpated and applied to these gamesm submissions and results. All that would need be done is to fully publish whatever "evidence" exists right alongside a gamer's file info. The court of public opinion will take care of whatever business need be undertaken.

It's cheap, effective, and nullifies any real gains made from protracted efforts.


Then again, I choose looking at the CIV games in rather similar fashion to the way I view chess; the post mortem is where the real learning takes place. Even the best GM's do not always play the best lines and/or variation OTB. It's in the skittles room or the analyzing stages where the real "correctness" of any strategy can be born out and "proved", if proof even exists. So, I am always more consumed with the autopsy and what can be gleaned from it rather than I am swayed by direct results. Because unlike chess, and a major point from which the games move away from each other, CIV retains a large amount of random element (luck) which can be too decisive in too many games.

I also think names should be used publically (user names). Absolutely. I can see no real reason to not use names. The very system of publishing all submitted files with all pertinent issues precludes the use of names, and from the point of transparancy, makes the most sense. There is nothing to be gained by keeping the names under some cloak covered by one's rusty dagger and everything to be gained by being as open as possible in matters such as this.


~ richard

 

[DynamicSpirit]

I think one benefit of the current system is that, by sending out the message that playing over (yes, I think that's a better term than 'reloading') is unacceptable, it encourages people to play straight through and to cope with any mistakes or bad situations that arise; that is arguably an extremely good way to become a better Civ player. It's certainly helped me hugely. Before I started playing GOTMs I was always reloading. Now I almost never do, even in my own private games. As a result of that I've learned that numerous situations that I'd previously have thought of as utterly hopeless and a cause for quitting the game are in fact nothing more than a harder-than-normal challenge (See my WOTM03 final spoiler for a good example of that).

If, as some people are suggesting, the system is changed to allow partially-replayed entries marked as such, that will inevitably send the message that playing in that way is fine, and I think it's inevitable that a fair few people who currently do follow the rules will end up replaying turns. ISTM that will significantly reduce the benefit of the GOTMs in helping people to become better Civ players.

 

[DaviddesJ]

Position cars every 3 to 5 miles on heavy trafficked highways. Only put an officer in one out of every four cars, rotating officers so that drivers cannot discern which vehicles are staffed at any point. Simultaneously keep one/two vehicles highly visible, mobile and rolling in traffic at the speed limit. Contolling speed serves the consensus benefit of the community. Writing tickets serves the beneift of the authority, and oft begs the question of whether it may not actually serve to benefit the community.

The metaphor could easily be adapted and applied to these games submissions and results.

I don't think you can apply that metaphor at all. The speed control system works because speeders fear the possibility of punishment. They wouldn't slow down at all, if they knew that the police never write any tickets.

But I personally have no problem with publishing game "results" from players who don't follow the competition rules. It's just a question of what you want the GOTM to accomplish. Traditionally, the primary purpose of the GOTM (at least as set forth by cracker and others, many years ago) is to explicitly encourage and reward a style of play where players don't take advantage of something that the computer opponents can't do, namely, the opportunity to take back any decisions they regret. It would be a pretty big change to give up that purpose.

The forms of cheating that are the most powerful are also those that are the least possible to detect or prevent. Nothing is going to prevent people from getting high GOTM rankings by cheating, if they want to. I personally don't care much about the rankings, for this among other reasons. But I do think that the GOTM has contributed to the enjoyability of Civ, over the years, precisely by illustrating that it's quite possible to do just as well against the computer, even at high difficulty levels, without having to "take back" one's unfortunate decisions. I hope that aspect continues.

 

[Ribannah]

Then again, I choose looking at the CIV games in rather similar fashion to the way I view chess; the post mortem is where the real learning takes place. Even the best GM's do not always play the best lines and/or variation OTB. It's in the skittles room or the analyzing stages where the real "correctness" of any strategy can be born out and "proved", if proof even exists.

Heh, this is a slightly optimistic view of the quality of the post-mortem analysis after a game of chess.

 

[Mastiff_of_Ar]

I have the answer!

How about we just play by the rules set forth by the people who take a great deal of time out of their life to create this game for us? Huh? Huh?

I play as lousy as anyone, but I absolutely love the GOTM and WOTM. I especially love the pregame discussion... and trying to be so careful to follow my "plan" to victory... and then seeing how everyone fared after I (usually) get my butt kicked.

I wouldn't have that without the GOTM staff. Thanks guys, again, for making a twice monthly event that's just wonderful...

 

[Denniz]

One benefit of private exclusion of submissions suspected of replaying (my favorite term ) is it gives people a chance to reform. Someone who reforms and becomes a better player may forever be tainted by their former behavior were their exclusions make public. Also, it is fairly easy for someone to make a mistake if they are not really organized. Needing to learn to be more organized in managing their save files is not a reason for public condemnation.

In addition, publicizing details of why a submission was excluded would only provide information to people who just don't want to give up the crutch of replaying.

So I think that any suggestion to make public the names of transgressors or the nature of their transgressions would be counter productive.

As for allowing people to submit and self report their replaying, isn't that a little like holding AA meetings with an open-bar?

 

[The-Hawk]

One benefit of private exclusion of submissions suspected of replaying (my favorite term ) is it gives people a chance to reform. Someone who reforms and becomes a better player may forever be tainted by their former behavior were their exclusions make public. Also, it is fairly easy for someone to make a mistake if they are not really organized. Needing to learn to be more organized in managing their save files is not a reason for public condemnation.

I agree. Also, I would think this takes some pressure off staff. If someone is gonna be publicly named, you need to be very sure about their guilt. It would be hard to "un-taint" someone if a mistake was made.

I think one-strike... you get warned. Two strikes... banned from competition (unless you can demonstrate some extenuating circumstance to the judge.... umm I mean Moderator ).

As for allowing people to submit and self report their replaying, isn't that a little like holding AA meetings with an open-bar?

 

[DynamicSpirit]

I have the answer!

How about we just play by the rules set forth by the people who take a great deal of time out of their life to create this game for us? Huh? Huh?

I play as lousy as anyone, but I absolutely love the GOTM and WOTM. I especially love the pregame discussion... and trying to be so careful to follow my "plan" to victory... and then seeing how everyone fared after I (usually) get my butt kicked.

I wouldn't have that without the GOTM staff. Thanks guys, again, for making a twice monthly event that's just wonderful...

I totally agree with that. I always absolutely look forward to each GOTM - the staff responsible for creating and maintaining it have created something that provides an awful lot of fun for me and I'm sure a lot of other people. The game rules strike me as totally reasonable, and I don't see any reason to change in that regard. I do though understand it must be really frustrating for anyone who's submission gets rejected because they forgot to save or something - I just don't see what else could reasonably be done in those cases though.

 

[AlanH]

I do though understand it must be really frustrating for anyone who's submission gets rejected because they forgot to save or something - I just don't see what else could reasonably be done in those cases though.

Hey! We're reasonable people! We have, and do, cut some slack for players who have crash problems, or have reloaded from an autosave and explain it to us. If that was all that was happening then this thread would not exist.

 

[ewokimpi]

One benefit of private exclusion of submissions suspected of replaying (my favorite term ) is it gives people a chance to reform. Someone who reforms and becomes a better player may forever be tainted by their former behavior were their exclusions make public. Also, it is fairly easy for someone to make a mistake if they are not really organized. Needing to learn to be more organized in managing their save files is not a reason for public condemnation.

Another reason to avoid public disclosure is that the replays may have been caused by occasional game crashes. I had two crashes in WOMT1, and one in WOTM2, which I documented. I communicated with the moderators this week about these games, and it appears they are acceptable for inclusion in the WOTM rankings.

Flagging my profile/these games with a public indictment of replaying would needlessly sully my name, when I am, in fact, not cheating (or whatever euphemism one wants to use).

In addition, publicizing details of why a submission was excluded would only provide information to people who just don't want to give up the crutch of replaying.

So I think that any suggestion to make public the names of transgressors or the nature of their transgressions would be counter productive.

As for allowing people to submit and self report their replaying, isn't that a little like holding AA meetings with an open-bar?

Agreed.

The point is to become better players, and learn from the best. Reloading decreases learning, and learning from "reloaders" is problematic.

 

[da_Vinci]

One benefit of private exclusion of submissions suspected of replaying (my favorite term ) is it gives people a chance to reform. Someone who reforms and becomes a better player may forever be tainted by their former behavior were their exclusions make public. Also, it is fairly easy for someone to make a mistake if they are not really organized. Needing to learn to be more organized in managing their save files is not a reason for public condemnation.

I guess there is no private exclusion for those who post in the spoiler threads, since once those great war stories are posted, but the game does not appear in the rankings, the cat is our of the bag.

The-Hawk wants to ban people on the second strike. As in ban forever? That is a pretty harsh consequence (from the perspective of Civ, not real life), and if we are going to be doling that out, we need to be sure that we have a reliable system. So let's bring the science of detection systems into the picture (my next post...)

dV

 

[da_Vinci]

We have been going through setting up systems to allow us to publish GOTM / WOTM results in a timely manner. In parallel with this, we have been improving our systems to detect people reloading.

It is great to improve your ability to detect replaying (more precise, as discussed above in the thread), as long as you don't increase collateral damage in the process (aka increase false positives). Those familiar with sensitivity, specificity, positive and negative predictive values know where I am heading with this. Once we start to assign meaningful consequences to the results of the detection system, we need to look at its performance.

But first, it appears there are five types of games submitted to GOTM:
1. Games with cheating, by players who care about score and try to be covert
2. Games with cheating, by players who don't care about score, and are perfectly comfortable coming here and telling us that they cheat
3. Games with replaying due to some inadvertant circumstance (out of players control (crash) or some oops! (like a forgot to save)), where no attempt to change outcomes has been made.
4. Games with no replaying, but there is some feature that makes the staff suspicious of replaying (perhaps frequent save and RESUME events?)
5. Games with no replaying that look clean.

So step one is to decide which of these types of games we want to include in the valid submission category.

Next, we need to decide how well our detection or diagnostic system performs. In particular, it must be able to separate a resume event (nothing replayed) from a replay event.

Suppose that our detection system can correctly identify 90% of cheating games as cheating games. That would be 90% sensitivity. 10% of cheating games go undetected, for a 10% false negative rate.

Suppose our system correctly identifies 90% of valid games as valid. That is 90% specificity. 10% of valid games get identified as cheating, the false positive rate. Not great if we are banning on strike 2.

It gets worse ... now suppose out of 110 submissions, 10 are cheaters and 100 are valid in truth. What will our 90% sensitive and 90% specific system tell us?

9 cheaters are labelled cheaters, 1 slips by. 90 valids are labeled valids, and 10 valids are labeled cheaters. And because there are 10 times more valids than cheaters in reality, of the 19 labelled cheaters, only half really are cheaters!!

So we really need a system with a false positive rate of 1% or less (99% or more specificity) to avoid penalizing the innocent. Do we have that?

It would be in everyone's best interst to reduce false positives. If there are behaviors that raise red flags that can be avoided (such as manual save within a turn, perhaps?), then players ought to know what they are. If more than two or three replays due to a crash won't be accepted, that should be known, as on the fourth crash one then knows not to submit (risk of a strike). If crash incidents need to be reported, that needs to be explicit (seems vague at present).

Yes, this is all a pain in the But if the mods choose to go to intensive enforcement with harsh consequences, I think they bring this duty on themselves. If we figure this out in detail once, and post it for all to see, it might avoid rehashing it over and over again, which seems to be the usual course of events now (and thus, more efficient in the long run?).

dV

 

[DaviddesJ]

Yes, this is all a pain in the But if the mods choose to go to intensive enforcement with harsh consequences, I think they bring this duty on themselves.

Or they could just ignore you. That would be my vote.

People have been "rehashing" this on CFC for at least the past five years. I don't personally think an additional round of "rehashing" is necessary or warranted. I certainly don't think the administrators need to justify themselves any further.

 

[da_Vinci]

Or they could just ignore you. That would be my vote.

Your candidate may well win. I just don't want to see a system evolve where people trying to act in good faith get punished.

dV

 

[DaviddesJ]

Your candidate may well win. I just don't want to see a system evolve where people trying to act in good faith get punished.

What do you mean by "evolve"? The system has worked just the same for many years. They were having these exact same arguments when I first joined CFC, in 2003. They come and go, but they haven't changed.

I'm unconvinced there is any problem. Anyone who thinks they are treated unfairly can raise the issue, either privately with the staff, or publicly. I believe they will be treated fairly either way. Until I see evidence to the contrary.

This is how it has always been.

 

[Mutineer]

To be true, it is interesting to compare one's game with others.
I consider myself resonably good player, but I allways wander reading reports, how people decided to go this way when there was no indication that word shaped this way befor hand.

I tend to do average plan that most lickly will allways work and have a back up plan if it does not.
For example:
Trying to get CS slingshot on monarch/emperor dificulty is not one of average plans. It is very lickly to fall in many cases and commit too mach resources.

But, somehow, they seems allwasy win. Well, there will be a low of big numbers and some one will be lucky enoght.

Bottom line, better detection system is allways better, as I will be more sure that people got lucky or just playing betetr then me.

On other hand no detection system is unfalible and I see the way to beat any system.

So, maximum this system could achieve is to ensure that majority of results will be valid one's. People who want to get score by any means will allways found way around any system.
One just could hope that people playing Civ is mature enogth not to do it.

I plaed a few GOTM, but never submit as I never had patience to finish game after it was decided and really not mach interested in getting fastest victory posible. On other hand In GOTMs are fun, as one meat some unuseal scenarious, some interesting twists to the formular.

 

[da_Vinci]

We have been going through setting up systems to allow us to publish GOTM / WOTM results in a timely manner. In parallel with this, we have been improving our systems to detect people reloading.

I am very sorry and very disappointed to say that I have just had to send out e-mails to a very significant number of players to advise that we will be excluding their submissions from the results when they are published. Further, a significant number of 'warning' e-mails have also been sent.

If you still continue to cheat, and get caught again, as I said, our tolerance only goes so-far.

Regards,

ainwood
On behalf of the GOTM staff, and I suspect also on behalf of the majority of the community who want to play in a fair competition.

@DaviddesJ: Evolution as evidenced in ainwood's lead post:

1. Technical changes to enhance detection of cheating (are we sure that did not come at the price of more false positives?). If ainwood could have detected all of this before the change, I'm sure we would have heard about it sooner.
2. Significant number of definite or borderline instances (and do I detect a bit of surprise at the number, so it is new?), again raising the possibility that its more false positives. More people who don't cheat raises the number of false positives.
3. If you are caught again ... (unspecified consequences) seems like you better be sure that you are not convicting the innocent.

And posts in this thread that seem to suggest that some acting in good faith are getting dinged. Seems like a sea change to me. (And you did ask ...)

dV