Adjustments to exclude specific games from GOTM results

Status
Not open for further replies.

cracker

Gil Favor's Sidekick
Joined
Mar 19, 2002
Messages
3,361
Location
Colorado, USA
Many of you may know that each month we receive hundreds of submitted game files for each QSC and GOTM game and that every one of the save game files is carefully scored and/screened to determine if the submitted games should be considered valid for inclusion in the game results. The vast majority of the game submissions easily pass the initial screening review indicating little or no problem with 85 to 90% of the game submissions.

The remaining 10-15%, some 20 to 30 games each month, are screened to determine if the initial indicators of technical problems or potentially unusual play behaviors might be signs of greater problems such is player cheating or the use of external advantage tools and/or exploits. Again the vast majority of these games are cleared and accepted without difficulties.

Each month we have an average of 6 to 8 games out of more than 200 submissions that begin to represent conditions that we might say include some use of play factors that should be prohibited. The vast majority of these games are from newcomers or inexperienced players who do not know any better or who are still trying to develop the confidence to participate in the community on their own level. These players receive counseling and assistive contact that we are expanding to include helpful welcoming services and other introductory information.

Mixed in with these few new players who are naturally developing an understanding of the simple basic rules of the GOTM, we also will usually have one or two games by experienced players that may push close to the acceptable limits and require a detailed review to help assure that all rules are being followed. In many cases, these reviews can determine that things are definitely unusual even if 100% evidence of wrong doing cannot be determined from any one single set of events in any one single game. We cannot say that it would be impossible for a galley to survive for two successive turns in the ocean without sinking, but we do know that this event does not occur every time.

Over the past six months, we have been actively reviewing the game submissions of one particular player that have been being flagged each month by two or three of our screening methods as being somewhat unusual. These reviews have indicated with certainty that something is unusual about these individual games and many of the individual game events, even though no one event or game could be ruled out as impossible. It would be unfair and unreasonable to accuse a player of wrong doing just because of one lucky sequence or one unusual set of events in one game.

However, when we begin to combine all of the statistics that are available to us, across hundreds of different players in sequences of 4 or 5 different GOTM games and then compare these statistics against the reloading, play time duration, map movement decisions, and RNG outcomes for any one player we can begin to determine if the game results of that player are reasonably obtained or if those results are obtained by cheating in any one of a number of different methods. The signature of this unfair behavior becomes more and more pronounced as we have more games to compare and base our decision upon. With only one or two games, we can only be 60% to 75% certain that something is wrong. Even under these conditions we will usually contact the player to attempt to steer them away from the suspected rules violations.

When we get three games in a pattern, then we can often determine with 85-90% certainty that something unfair is being done in the games. Depending on the game play signature of all the other players who played the same game there may be enough information at this point to determine that a players is actively cheating and then to take definitive action. When a fourth game in a pattern is received and continues to demonstrate questionable reload and game durations statistics combined with unusually favorable results this provides us with enough specific information to say with 90-95% confidence that the player is deliberately using reloading or advanced map knowledge to unfairly manipulate the game results.

By the time we reach this point, the individual players responsible for these submissions have usually received multiple private warning and counseling contacts as well as consuming literally hundreds of extra volunteer staff hours plus usually delaying the publication of game results by one or two days each month.

This is the condition that we currently find ourselves in with reaching a decision that one of our regular players who has been disciplined the past for reloading and cheating but has continued to do this on an ongoing basis during the last four GOTM and QSC games.

While I personally want to respect the individual human dignity of this individual player, I also believe that we have an obligation to the greater community of all players who participate in the GOTM. This obligation includes recognizing that a fundamental respect for your friends and your peers will not include deliberate cheating to manipulate the game results to always obtain only the most favorable outcome from every game situation. All things should average out over time and in allowing these natural rhythms to occur we will find and conquer new challenges.

I have asked the primary staff members to initiate processes to locate the affected games from this player over the past 10 months and to make preparations to remove these results for all reported game announcements. We will also make sure that any other players in previous games who may have been denied awards or recognition due to these illegitimate games will have these recognitions rightfully restored to them.

The individual player in question will also be banned from further participation in any GOTM or QSC game event or discussion and will be removed from the Community Participation (Monk Rankings) Roster as well as the Global Ranking. It will take several days to a week to correctly update all these features.

All the members of the GOTM Staff urge you all to recognize that normal players who play and enjoy their games to the best of their own personal abilities do not fall into these same categories of detectable and deliberate long term distortion of the game results. We have very few of these events that occur within the GOTM games and the people who make up virtually all the members of the GOTM community are rarely subject to the intense scrutiny that has been required for this specific case.

Our purpose is providing you with this explanation of the actions that you will observe taking place in the background, is to help you see that these processes are fairly and evenly applied over a series of months and always with the intent of trying to help the individual players resolve the apparent play issues before they might effect their position in the community or their relationships with other players. It takes an active effort for someone to violate the expected rules of play and to exploit these advantages over a four to six month period and normal players who do not use artificial foreknowledge of the map or reload to alter the expected outcomes of events will not be effected by these screening and review processes.

I would like to ask all members of the community to read this information for what it represents and not to focus too much on the individual who elected to provide us with this specific set of examples. Records of past games will be properly updated and the challenges and excitement of current and future games will continue to hold our attention. The sooner we can address any questions that you may have and move forward with the more enjoyable focus of the games, the more we will all be able to continue to add to the great acheivements that we have already made during the first five months of this record setting year.

If you have any questions or concerns on these issues during the coming weeks please feel free to contact me privately by email at gotm@civfanatics.net and I will either answer your questions personally of ask one of the primary GOTM staff members to provide you with any specific information that you may need.

Sincerely,

Cracker
 
Wow, judging from the length of your post it’s about a highly visible player.

I guess we’re going towards the Catholic Church policy of preventing recognition of ‘false’ miracles even by dismissing questionable ‘real’ ones. I’m OK with that and with you (the Staff) to judge on it (I’m also OK with the Catholic Church for that matter).

Banning someone from the event is something which I would not agree but it’s your call and if you can live with that I can too.
 
I normally do not feel like adding my voice to this kind of matters, as I doubt that I would add anything to the considerations of people who are by far more experienced in the management such an environment and in the mechanics of Civ3.

I only want to stress that, from the limited understanding of my perspective, this can hardly be considered a "questionable real miracle".
No criticism intended to your post, Yndy: I only wanted to point everyone to the contents of Cracker's announcement, and to the care he took in writing it, as an indication of the scruple and the time he (and the staff) have dedicated to this issue, and especially on the matter of treating all players fairly and equally.

My personal doubts were cleared when I was able to witness the amount of work that is going on behind the scenes, and I suggest that anyone who may justifiably have other concerns to contact Cracker as he suggests.
 
What does it take to be branded a "reloader?"

I have submitted two games of the month (19 & 20) and won both of them, but my score is rather pitiful compared to most people here. :(

But when I'm playing the GOTM I every once and a while to check my e-mail for my PBEM turns, and to do that I have to exit PTW completely to check my mail (I use dial up). The reason I have to exit is because I use MS Word as my e-mail editor and for some reason a MS Word-PTW combo causes my computer to crash. I have saved the game regularly and exited or just exited at the beginning of the turn and loaded the autosave when I came back.

I even had to load an autosave from the turn before I won GOTM 20 and redo the turn because my win was unexpected (diplomatic) and I hadn't saved the game the turn before I won...

I would hate to be branded a "reloader" when I not even sure what that is. And I would hate to stop playing my PBEM turns while I'm playing GOTM. :(
 
I understand (from other threads) that the player in question has also been banned from posting on these forums, and so has no "right of reply". Given that his/her identity has not been kept secret, and most of the community is aware of who it was, I wonder if this is a bit of a draconian thing for us as a community to have done?
 
@ltccone: I'm not officially in the loop, but I've been playing for the last four months and lurked in the forum for three months before that. I've seen this discussion before, several times.

So I feel safe in saying that you have nothing to worry about. What Cracker is discussing with respect to reloading is getting an unfavorable RNG result (like your first suicide galley sinking on its first turn, for example), reloading the game to the same turn, doing something to use up that RNG roll (I'm not sure what all does this, but it's a fairly long list of things), then trying again. Maybe this time the galley survives its first turn, but dies the second. So you reload that second turn, reset the RNG, and try again.

What you're doing is nothing like that --- you're not manipulating game results ---, and I'm almost positive it wouldn't even show up on Cracker's radar. Even if it did, the worst you would get at first is an email from Cracker asking you about it, and the option to change your emailing habits in the future to avoid further odd-looking submissions. :)

Renata
 
From my own experience, and I'm sure cracker's and the team's too, public flaming between the accused cheater and other posters is never good, not for anybody involved. Discussing bans is not allowed in other parts of the forum, either. PM is the right tool for that, and as cracker mentioned, that tool has been used extensively for the last several months.
 
Originally posted by Renata
@ltccone: I'm not officially in the loop, but I've been playing for the last four months and lurked in the forum for three months before that. I've seen this discussion before, several times.

So I feel safe in saying that you have nothing to worry about. What Cracker is discussing with respect to reloading is getting an unfavorable RNG result (like your first suicide galley sinking on its first turn, for example), reloading the game to the same turn, doing something to use up that RNG roll (I'm not sure what all does this, but it's a fairly long list of things), then trying again. Maybe this time the galley survives its first turn, but dies the second. So you reload that second turn, reset the RNG, and try again.

What you're doing is nothing like that --- you're not manipulating game results ---, and I'm almost positive it wouldn't even show up on Cracker's radar. Even if it did, the worst you would get at first is an email from Cracker asking you about it, and the option to change your emailing habits in the future to avoid further odd-looking submissions. :)

Renata

Thanks. That makes me feel better. Now I know what is meant by "reloading."
 
I am assuming that if you haven't gotten a notification that every thing is not out of the norm. My particular habits vary. I have a habit of rushing through the QSC period quickly but in the end a session may only last 4-5 turns 10 tops over an hour period or so. Not to mention the times when I just sat down and open the game only to be booted by my wife.

It would be interesting to see exactly what our individual reload counts are as an FYI.
 
Originally posted by hotrod0823
Not to mention the times when I just sat down and open the game only to be booted by my wife.
:rotfl:

Perhaps we could start a "Partner of the Month" game with a medal for the most "Civ Denial" points.

No pictures please...


Ted
 
She keeps threating to break my disks. ;). Oh the joys of 1:00AM civ.

In all seriousness I hope the community realizes that this was obviously a hard decission to make and from the sounds of things coming for a long time. Overall making these tough choices are what makes the game more enjoyable for the rest of us in the long run and I applaud crackers and the GOTM staff for addressing these issues and taking what they deem to be appropriate action.

Hotrod

[edit] Anytime a player leaves or Is asked to leave they will be missed. Theory, game play and discussion is and should be just as important as results. That is what will be missed the most NOT the final score.
 
ltccone,

Here is another piece of information to help clarify the issues and reassure you that what we are looking at here are longterm averages of behaviors of all the players around you in all the GOTM games. The process does not target individual people but clearly helps us to identify games that are substantially "not like what the normal games look like."

You see some of this data being presented to your each month as the GOTM results are published and each month our confirmed base of data gets better and allows us to make stronger determinations about how the various submitted games fit into the overall picture of what the scores and results shoul look like.

The image below, is taken exactly from the statistical summary of the Gotm18-Celts results that you saw published two weeks ago. The basic graph shows the turns-per-session results for all the game submissions for Gotm16-Rome, Gotm17-Carthage, and Gotm18-Celts.

I have overlayed on top of this image to add the turns-per-seesion data for your Gotm19 game even though we have not yet completed the full statistical analysis of the GOTM-19 data. Your tps value for Gotm19 was at 27.9

I also included Renata's Gotm19 tps value of 55.25 shown as the forward most blue telephone and just marked the column in the Gotm18 data were Renata's tps value would have fallen because I had that value handy.

The purple shield is the tps value for hotrod0823 in Gotm19 at 33 turns-per-session.



The tps values for the excluded games where not the only factor used to determine that something unfair was being included in those specific games but I have marked the positions on the graph where all four of the most recent games would have appeared in the data if they were not now excluded.

It is very unusual to see games down in the lower TPS range without being able to trace those results to some sort of known technical problem, player experience, or some sort of game altering elective player choices.

The ongoing averages across hundreds of players playing the games independently and in their own personal styles continue to reaffirm our understanding of the key game mechanics and how the successful as well as average players play the game. A player who is playing anywhere close to the way the 95% of all the other players play the game will not be likely to brush up against the screening limits except of rare and unusual occasions.
 
Originally posted by cracker
ltccone,

Here is another piece of information to help clarify the issues and reassure you that what we are looking at here are longterm averages of behaviors of all the players around you in all the GOTM games. The process does not target individual people but clearly helps us to identify games that are substantially "not like what the normal games look like."

I really appreciate how much effort you went through explaining this. It really makes a lot of sense now. :goodjob:

My tps will probably be a little lower for GOTM 20 because I finished on the weekend when I get more PBEM turns and late turns can take a long time to play. But I can see why a tps of 5 or so would raise a red flag because in a "normal" game you really play many turns in one session, especially in the ancient age "spacebar" turns.
 
I'm very sorry to hear about Ribannah. Since there are always at least two sides to every story, it's very hard to sort out what is right and what is wrong. On one hand, I do believe that Ribannah innocent because she told me so and because it was obvious that she is a very skillful player who knows almost every trick in the book. On the other hand, I also trust the GOTM staffs and their decision because I know that they didn't take this matter lightly, considering that it took them many months and countless hours reviewing and analyzing her games in comparing to every other submissions before they come up with this decision.

-------

To Ribannah,

I know that you can not post here no more, but I'm sure you can still be reading this. If you are truely innocent, I hope that you hold no ill feelings toward Cracker and the GOTM staffs because they have a tough job to do and you didn't make it any easier for them. For example, I was totally speechless about your lone archer who was responsible for the destruction of two civilizations in the GOTM18. You were either very lucky and brave or you had ESP or something because I don't think any of us would dare to make a move like that. Yes, I know...an archer against a warrior, chances are the archer will win most of the time; however, I would never dare to attack a city with only 1 archer (who happen to travel a long way from home) because even if my archer win, he may be wounded and he would have very little chance against the counter attack of another warrior which I would have no sure way of knowing about the number of defenders in that city. Although it is possible to stand on the near by hill/mountain to observe the AI movement in/out of that city to predict the defenders' strength, but I think it would be very tough to convince the jury about that.

Please don't get me wrong, I am not saying that you were cheating in any case because I have always believed that everyone is innocent until proving otherwise. I'm saying that even if I'm the world #1 defense attorney, it would be very tough for me to prove your innocent. I'm really trying very hard to defend you, but you don't leave me much to work with; therefore, your case seems hopeless.:( If you are truely innocent, I hope that you learn something from this incident and try to relax more. Go decorate the country home or something and try not to take things so seriously, especially in a computer game. I wish you well in life. You will be missed!:cry:
 
Once again the most excellent moderators have bent over backwards to be fair and have put forth tremedous effort to communicate to the forum.

I applaud their efforts and commend them for their hard work.

As for people way outside the norms, take a step back and put it into perspective. It may be just a game, but keeping everyone on the same set of rules is tremendously important. People will loose their faith and stop putting many hours into the game and the forum if integrity is lost.

Just look at other games that have failed to check the hacks/ cheats/ exploits, like Diablo2. Nobody plays the game as it was originally intended, and the discourse is generally crude and vulgar. Kids go buy hacked items and use exploits to kill other players just because they can and without regard for how much time someone might have spent building that character.

Similarly, who will spend the time to complete in GOTM if you know a reloader will be out there doing the 1 archer rush?

Thanks for all the hard work to all the moderators.

Semper Fidelis
 
This is a sad thread and probably the less said the better, we should focus on the awesome fun in GOTM as it just keeps getting better and better. But I want to say one thing. I'm sure I say this on behalf of many GOTM members and no one needs to post an agreement, I trust the staff will know how many of us feel the following:

We thank you Cracker and all the GOTM staff. You are volunteers and are here to have fun. This is clearly the most unpleasant of tasks, something you have not done lightly, something you tried your best to handle without it coming to this. That you undertake this unpleasant but necessary work on our behalf as well as all the positive aspects of GOTM deserves our sincerest appreciation and thanks.
 
The GOTM club ain't a democracy and CivFanatics ain't either. Participants have no rights, no redress. The GOTM staff acts as judge, jury, and executioner. Absolute power corrupts absolutely. Irregardless of the motives of the staff now, which I believe to be honorable and squarely behind the best interests of all, it is an undeniable fact that over time persons will be trampled for lesser and lesser offenses. And the GOTM staff will not realize that they are being heavy handed, from their perception, they will always be right.

Conviction of the accused without a presentment of proof, bannishment of the convicted, with no right to respond to the accusations, no due process, no open process, no neutral and objective third party review.

In other words this place is starting to resemble a fundamental southern Baptist church. :D

What happened today is probably good and yet fortells a dark future. Absolute power corrupts absolutely.

Edit: Cracker I posted this here rather than using the pm or email because I believe thinks pertaining to individuals rights should debated openly, even if it is just a private gaming forum.

It would be harder but better to ensure that persons accused of cheating have access to due process in an open environment.

(sneaks away aware that I have just painted a big target on my back).
 
Originally posted by whb
I understand (from other threads) that the player in question has also been banned from posting on these forums, and so has no "right of reply". Given that his/her identity has not been kept secret, and most of the community is aware of who it was, I wonder if this is a bit of a draconian thing for us as a community to have done?

Please let me assure you that this was not quick thoughtless decision taken lightly. All the staff has suffered some angst over this. Other options available were considered but even the most liberal minded of us came to the same conclusion.

Remember this was a decision arrived at over many months. Due process IMHO has been followed and a “right of reply” had been sought in all previous cases.

As an aside: A big welcome back to the GOTM, I hope you are well rested after your 14 game break.
 
I hope no one takes offense to this. Meant to be humorous, to break up the seriousness of this post. But, Here goes.

So, someone was using a corked bat. I didn't know Sammy Sosa was here too?!
 
This is a hard thread to read, as I'm sure it was a very hard thread to post. I'm certain that Cracker and the GOTM Staff conducted their investigation in this matter in as thorough and meticulous a manner as they conduct the GOTM process itself. This is very unfortunate, I wish there were another way but I do trust Cracker and the Staff in making this decision, and I appreciate the effort (and sometimes, the angst) put into the GOTM and in making it a fair and fun event.
 
Status
Not open for further replies.
Top Bottom