Adjustments to exclude specific games from GOTM results
- Thread starter cracker
- Start date
- Status
(off-topic)
Thanks .. it was a 14 game break without a home internet connection, and about a 12 game break without home pc at all (That's what moving continents can do to you!) I had been following the new-souped-up-even-more-fabulous GOTMs with considerably envy on the website. But now
I can play them with glee 
So very many thanks for all the great work you guys do!
(/off-topic)
In a token attempt to give this post an on-topic reason to exist:
You guys have done an great job explaining how the banned player's TPS was well out of the statistical norm. And I understand there was a fair amount of other damning evidence. (Good job, guys).
Can I, however, ask you to reassure everyone that Turns-Per-Session will never be used as the sole measure for assessing a game's validity -- since it isn't actually a good candidate for "likelihood" tests.
In other words, can I express my hope that we're not going to be rooting out all coffee-break gamers as suspected felons any time soon?
Bamspeedy
CheeseBob
In taking the disciplinary measures as far as this case had to go, it had more than just the 'turns-per-session counter' to reach this decision.
I also know that this decision was not based on any one or few "questionable real miracles".
IMO, It is when everything falls into place perfectly in EVERY aspect of the game, and is perfectly played game after game after game, it becomes suspicious. By 'perfectly played', I don't mean using good strategies/tactics/city placement/good diplomacy,trades and planning etc., but every move whether it seems logical or not (based on what map knowledge you have at that time, and other factors), ends up being the 'best move'.
And after 101 battles you lose only 4 units (ancient units vs. ancient units), seems fishy. Sure, that kill ratio may be possible, but when other things like suicide galleys, hut results and most other things that depend on the random number generator also have nearly as high success ratio.....
Qitai
.
Just one line.
I trust GOTM Staff.
Moderator Action: The reason Ribannah is currently banned is because we will not have any selfserving diatribes related to this issue. Posting her messages by proxy will not be tolerated.
You are welcome to choose not to participate in the games but that will be your loss because you are welcome here with your new persona. Tou have made greta strides to be a part of this community and we do not want to lose you as part of the fallout from this issue. This is an unnecessary choice to make.
You may not choose to alter the game submission, discussion, and reporting processes or develop alternative posting or scoring methods (just as Ribannah has done in the past). We can use her past conduct as a 100% certain behavior model and so far every person who has engaged in this disruptive conduct is no longer with us.
Also, do not 100% quote yourself as a means of propogating aggressive posts. They have no place here and you see no other examples where this is tolerated. -cracker
Please read the forum rules: http://forums.civfanatics.com/showthread.php?t=422889
You are welcome to choose not to participate in the games but that will be your loss because you are welcome here with your new persona. Tou have made greta strides to be a part of this community and we do not want to lose you as part of the fallout from this issue. This is an unnecessary choice to make.
You may not choose to alter the game submission, discussion, and reporting processes or develop alternative posting or scoring methods (just as Ribannah has done in the past). We can use her past conduct as a 100% certain behavior model and so far every person who has engaged in this disruptive conduct is no longer with us.
Also, do not 100% quote yourself as a means of propogating aggressive posts. They have no place here and you see no other examples where this is tolerated. -cracker
Please read the forum rules: http://forums.civfanatics.com/showthread.php?t=422889
cracker
Gil Favor's Sidekick
I can easily say this for you because it reflects the general operating philosophy.
"Turns-Per-Session numbers are not and will be the sole criteria for determining if a game is valid, with the exception that some games that have TPS numbers in the range 0.4 to less than 2.0 may be excluded without further review on the general principals of just being too far out the expected range of normal player sanity."
In this specific example, we could use at least 5 different measurement criteria to support the final decison. All with either equal or stronger indications than have presented her for the TPS numbers.
serttech2003
Warlord
- Joined
- Apr 23, 2003
- Messages
- 143
On what basis do you not believe them, yet believe someone else? Lots of people don't play the GOTM because of the cheating that goes on, yet when someone is caught .....
Oh never mind, its not worth fighting about.
Oh never mind, its not worth fighting about.
Moderator Action: email me at gotm@civfanatics.net with these issues and we will get you to a more informed position without compromising the public and private processes.
We will not be conducting any portion of a media trial here.- cracker
Please read the forum rules: http://forums.civfanatics.com/showthread.php?t=422889
We will not be conducting any portion of a media trial here.- cracker
Please read the forum rules: http://forums.civfanatics.com/showthread.php?t=422889
cracker
Gil Favor's Sidekick
In all fairness, I do not think this assertion is even remotely true. The internal data that we have compiled on the GOTM games is available nowhere else in the entire Civ3 world and it paints a picture that the games are solidly presented on a fair and representative basis.
Your chances of encountering artificial play enhancement in a PBEM or Multiplayer game as at least several orders of magnitude higher than encountering similar behavior in the GOTM.
The power to overlay 250 games and timelines that are played on the same essential map setup is an almost unbeatable tool. This is why we can detect unfair play advantages with a higher level of certainty and also why we can detect bugs in the software and gaps in the AI decision algorithms that go totally undetected for years, even by direct test efforts of the software manufacturer.
The fact that our processes include direct contact with the players who may move into grey areas of performance when those areas are detected early should even be a greater assurance to you that the validation processes work reliably and that we are working with the community and player interests held at a very high level of importance.
We need to tolerate some flexiblity here to let people express their concerns and get answers to questions that they may have. There is no need to have posts that turn a calm but concerned and rational discussion into a sequence of heated venting episodes.
If you feel you need to say something aggressive that might not really be appropriate, and if that will make you feel better, then email me with your issues and I will try to address your concerns. If you raise a point that a detailed response with information that may be useful to others, we will make sure that the information gets posted here if it is appropriate.
Qitai
.
On the basis that they are working on this voluntarily and it does not benefit them to brand anyone as a cheater if it is not for the greater good. On the basis of the hardwork they have put in to transform GOTM to what it is now. On the basis of reading their post and from it, their personality and values. On the basis that we have indeed seen Ribannah getting extreme luck on doubtful strategies. Is that enough?
Txurce
Deity
ltcoljt, you seem to see no basis for trusting the GOTM staff to impose a ban, and ask what evidence does Qitai, or anyone else for that matter, have for trusting this decision?
I would reply that those involved in the decision have 1) proven their dedication to giving us an unparallelled gaming experience, 2) put up with periodic annoying posts without being vindictive, 3) devoted a staggering amount of time to the issue, 4) have a much better understanding of the game's mechanics than most of us, 5) have software tools which allow them to apply that understanding to investigating a series of games, and 6) have had a series of ultimately unsatisfactory discussions with Ribannah in the past. That's reason enough for me to have faith in this action, without knowing a single fact.
Sirp has a question similar to yours: why doesn't the GOTM "post all the evidence against someone in the forum for all to read, and allow the person to respond openly. This will give assurances to all that the process is fair."
I see this differently. First, posting all the facts would almost certainly not give assurances to all that the process was fair. With any group as large as ours, some are bound to find it unfair, for good reasons or bad. This would then generate a divisive and depressing debate, which would counter whatever benefits would be gained by posting all the facts - and be somewhat pointless in light of the fact that the decision to ban has already been made.
Now perhaps there are some who feel that the GOTM staff shouldn't have made this decision on their own; that the facts as seen by both parties should have been presented in one thread, and then the community could have decided via a poll. This makes philosophical sense to me, but breaks down in light of the fact that all of the GOTM participants have implicitly agreed to play according to the rules as established and administered by the GOTM, in exchange for benefiting from the experience that the GOTM staff provides. It's a trade-off, similar to representative democracy. In my opinion, the benefits skew greatly in favor of the community. Given this, I think it would be best if the community would acknowledge the power they have implicitly given the GOTM staff, appreciate what they receive in return, accept that life isn't perfect and group action even less so, and move on from what is a difficult and unhappy experience for all of us.
cracker
Gil Favor's Sidekick
ltcoljt,
I do not think that you are the only person who is uneasy. This is a natural feeling that some people may have.
I just urge you to look at the larger pieces of information that you already have and then to recognize even with all of the information that we as the staff have had to review we can only be certain that Ribannah's pattern of games reflect something that should not be considered a valid set of results when we look across 6 seperate game events and comapre specific details of those games to the games of a total of 200 to 300 other players.
When we look at all of this information from the bigger picture perspective we cannot support a conclusion that these games should be considered valid events.
If you would like more specific details using some comparative examples from your own games, just email me a gotm@civfanatics.net and we will get you the information that you need.
I do not think that you are the only person who is uneasy. This is a natural feeling that some people may have.
I just urge you to look at the larger pieces of information that you already have and then to recognize even with all of the information that we as the staff have had to review we can only be certain that Ribannah's pattern of games reflect something that should not be considered a valid set of results when we look across 6 seperate game events and comapre specific details of those games to the games of a total of 200 to 300 other players.
When we look at all of this information from the bigger picture perspective we cannot support a conclusion that these games should be considered valid events.
If you would like more specific details using some comparative examples from your own games, just email me a gotm@civfanatics.net and we will get you the information that you need.
So we've just had six GOTMs in a row where according to you, "artificial play enhancement" has occurred, and yet you're telling us that it's thousands of times more likely to happen in a multiplayer game? That's barely even possible, since it is actually pretty hard to cheat in a multiplayer network game (not that it's not possible, but it'd be beyond the capabilities of many players). In a PBEM it'd be more likely, but I tend to doubt one would play 6,000 PBEM games and have cheating involved in all of them.
In fact, I would assert that cheating has likely occurred in some form or another in each and every GOTM.
According to you, Ribannah's cheating was flagrant and continuous - across six consecutive games. So if it had only occurred across five or four games, the result would have stood? If it had gone on for six or more games, but was a little less flagrant, then the results would have stood? So why shouldn't we assume that there are other players who have cheated and have simply been less flagrant about it?
After all, there are numerous methods of cheating which are incredibly difficult to detect. One can read the spoiler threads before they qualify to do so for instance. Or likewise, one can simply reload the game on occasion, when something particularly adverse happens. For example, one could play an incredibly aggressive "farmer's gambit" strategy, which is rather risky in the case of AI aggression, but where the game is reloaded in the event that the AI declares war.
As such, cheating is a big reason why I have been dubious about the GOTM in the past, although it is not as big a reason as a similiar issue: exploits. Many players use tactics that intend to exploit bugs in the game, or inadequacies in the game's design, rather than play the game the way it is intended to be played.
That Ribannah was one of the more outspoken players concerning this, it is of some concern to me also that she is now banned.
-Sirp.
cracker
Gil Favor's Sidekick
Sirp,
I think you have to recognize that in an environement where we have literally hundreds of comparative examples for any given game we can begin to develop a better understanding of what the statistical impact of various potential exploits/cheating events might be.
The undetectable preknowledge events become detectable in the fact that they alter player behavior in the immediate game and then again in other games over time. You cannot know something that you are not supposed to know and then expect your eye not to twitch in that direction at least once and a while.
We can use an example for the recent Gotm18 game results to make sure that this process is slighty better understood even though it will still contain some uncertainty. In the Gotm18 game, the start position was specifically designed to encourage settling right on the starting position without moving. While at the same time potetially providing one peice of information to help in the detection of players who might use foreknowledge of the map gained from any number of sources to alter the outcome of their game.
The image below shows these opening move positions for 87 games that were analysed in detail.
(thanks to RufRydyr for the graphic as well as the opening city placement analysis)
We expect that from any position a certain number of players can randomly make any move based on a variety of basically random choices. So seeing players move to any of the surronding squares is not unexpected. The 1's and 2's provide a basic background signature that statistically means we could expect any where between 0 and 4 players to move into any of the given swaures with no statistical inference. The 70 players who started at setteled at the starting position would have behaved exactly as expected and then the 2+1 and 2+1 plus up to 4 of the 11 would be totally within expectations. So a total of 70+2+1+2+1+4=80 out of the 87 games behaved just as expected without considering the impact of any opening worker moves.
Moving the only worker to the 11 square would reveal the fish square and following this reveal we would expect many players to make the additional move to the 11 square in a non random way. Moving the worker to the 11 square would not be a rimary startegic move because we would actually expect most players to move the worker to the hill or to one of the four bonus grassland squares surrounding the start position. Again the actual player moves confirmed these expected moves and only 5 players in the 87 game sample made opening worker moves onto the 11 square.
This process of elimination left us with a numeric expectation that 85 out of the 87 opening move sequences would be within normal exectations while 2 of those 87 games could be considered to perhaps be suspect (less than 2.3%). The unknown question at that point was which of the 4 random move games and with of the 2 suspect games would belong to which of the 11 players that actually did move to the 11 square.
Essentially on its own we could identfy no bad games by thes one move sequence alone but we could confirm that at least 97.7% of all the games would be making valid opening moves and we could also identify a list of 11 games that could potentially be used as dat points to compare across other similar situations where we could identify one specific move where know ing some hidden feature of the map in advance could have influnced the player move decisions.
All of this information set to the side, no other comparative game play event in the Civ3 world has access to the same volume and the same quality of data that we can use in the GOTM games to perform quality control and game performance analysis. These tools of comparing game events to similar events that are independently produced by players in total separate game playe experiences can tell us a great deal about how the game works and how the players respond to the game environment. As we get more and more data we can extend or ability to predict what players might be expected to do and also to determine the likelihood that players would repeatedly do the one good but unexpected thing in a series of multiple events.
This actual example was used to analyze games that were submitted to the Qsc19 event and then several games that were not included in the qsc were validated against the expected 97.7% confidence level for moves that did not fall onto the 11 count position.
I think you have to recognize that in an environement where we have literally hundreds of comparative examples for any given game we can begin to develop a better understanding of what the statistical impact of various potential exploits/cheating events might be.
The undetectable preknowledge events become detectable in the fact that they alter player behavior in the immediate game and then again in other games over time. You cannot know something that you are not supposed to know and then expect your eye not to twitch in that direction at least once and a while.
We can use an example for the recent Gotm18 game results to make sure that this process is slighty better understood even though it will still contain some uncertainty. In the Gotm18 game, the start position was specifically designed to encourage settling right on the starting position without moving. While at the same time potetially providing one peice of information to help in the detection of players who might use foreknowledge of the map gained from any number of sources to alter the outcome of their game.
The image below shows these opening move positions for 87 games that were analysed in detail.
(thanks to RufRydyr for the graphic as well as the opening city placement analysis)
We expect that from any position a certain number of players can randomly make any move based on a variety of basically random choices. So seeing players move to any of the surronding squares is not unexpected. The 1's and 2's provide a basic background signature that statistically means we could expect any where between 0 and 4 players to move into any of the given swaures with no statistical inference. The 70 players who started at setteled at the starting position would have behaved exactly as expected and then the 2+1 and 2+1 plus up to 4 of the 11 would be totally within expectations. So a total of 70+2+1+2+1+4=80 out of the 87 games behaved just as expected without considering the impact of any opening worker moves.
Moving the only worker to the 11 square would reveal the fish square and following this reveal we would expect many players to make the additional move to the 11 square in a non random way. Moving the worker to the 11 square would not be a rimary startegic move because we would actually expect most players to move the worker to the hill or to one of the four bonus grassland squares surrounding the start position. Again the actual player moves confirmed these expected moves and only 5 players in the 87 game sample made opening worker moves onto the 11 square.
This process of elimination left us with a numeric expectation that 85 out of the 87 opening move sequences would be within normal exectations while 2 of those 87 games could be considered to perhaps be suspect (less than 2.3%). The unknown question at that point was which of the 4 random move games and with of the 2 suspect games would belong to which of the 11 players that actually did move to the 11 square.
Essentially on its own we could identfy no bad games by thes one move sequence alone but we could confirm that at least 97.7% of all the games would be making valid opening moves and we could also identify a list of 11 games that could potentially be used as dat points to compare across other similar situations where we could identify one specific move where know ing some hidden feature of the map in advance could have influnced the player move decisions.
All of this information set to the side, no other comparative game play event in the Civ3 world has access to the same volume and the same quality of data that we can use in the GOTM games to perform quality control and game performance analysis. These tools of comparing game events to similar events that are independently produced by players in total separate game playe experiences can tell us a great deal about how the game works and how the players respond to the game environment. As we get more and more data we can extend or ability to predict what players might be expected to do and also to determine the likelihood that players would repeatedly do the one good but unexpected thing in a series of multiple events.
This actual example was used to analyze games that were submitted to the Qsc19 event and then several games that were not included in the qsc were validated against the expected 97.7% confidence level for moves that did not fall onto the 11 count position.
Jabah
Emperor
I really like cracker's analysis of the first city position. During the last GOTMs, I really had the felling that in EACH game I settle exactly 1 square away from the optimal position while doing what I consider logical moves (with other units), which is something that almost never happen in my private (random) games.
Having 'normal' ressources pictures (when there was still a choice) and not being able to see well under the fog (and some more advanced analyses in this particular case) were the explainantions I accepted when I started a small hot discussion during GOTM19 spoiler#1, but a few people were obviously, according to your analysis, either lucky in their weedy moves or knew better...
Jabah
Having 'normal' ressources pictures (when there was still a choice) and not being able to see well under the fog (and some more advanced analyses in this particular case) were the explainantions I accepted when I started a small hot discussion during GOTM19 spoiler#1, but a few people were obviously, according to your analysis, either lucky in their weedy moves or knew better...
Jabah
Darkness
Shadow creature
I think it's a very sad thing that once again a player is found to be a cheater....
On the basis of the facts laid out for us by cracker I can only say I think the staff made the correct decision. I know this is not a democracy and what I say won't change a thing, but I wanted to say it anyway.
Now I only hope cracker won't do the same as matrix and resign because he's desillusioned by finding a cheater, 'cause I really like the GOTM as it is right now...
Enough said about this. I gotta get back to work...
On the basis of the facts laid out for us by cracker I can only say I think the staff made the correct decision. I know this is not a democracy and what I say won't change a thing, but I wanted to say it anyway.
Now I only hope cracker won't do the same as matrix and resign because he's desillusioned by finding a cheater, 'cause I really like the GOTM as it is right now...
Enough said about this. I gotta get back to work...
Green Light
Just lurking
Originally posted by cracker
The image below shows these opening move positions for 87 games that were analysed in detail.
This image is very informative, thanks
Could we perhaps get this kind of image available from the future QSC's, its very interesting to see how people built their first 2 cities.
Sailorstick
Vanilla 1.29 - Conquest
Ban 'em Cracker. I have NEVER reloaded, even from stupid mistakes, and it makes it unfair for the countless people like myself.
Edit: I know it's a cliche, but it isn't their right to play GOTM's, it's a priviledge. (sp?) If they insist on abusing it then why don't you put the foot down?
Edit: I know it's a cliche, but it isn't their right to play GOTM's, it's a priviledge. (sp?) If they insist on abusing it then why don't you put the foot down?
- Status
Similar threads
- Sticky
- Sticky
