SGOTM13 - Separated discussion of test maps

Status
Not open for further replies.
Hey I have an idea that will help to put this argument into perspective. STW claims that no winning team has ever won without using a test map.
My survey is:

How many teams have used a test map at some point and still lost?


From my perspective, I have played on FE for 4 SGOTMs and we have 1 bronze medal. I also will make an educated guess that FE had used a test map at some point in all previous games only taking 1 medal (SGOTM12). So if we disregard the skill of our strategy in SGOTM 12 and assume that our medal was based solely on our test map, that is a 1 in 12 chance that a test map will help.

So have any teams that have no medals ever used a test map and still lost?

Clarification of the Conjecture that Test Game Use is Required for a Top Three Win:

To clarify, I'm suggesting that using a test game is a necessary condition for placing in the top three of a SGOTM. I never intended to give the impression that this was a sufficient condition for placing in the top three of a SGOTM. Obviously, not every Team that uses a test game is going to place in the top three, because more than three teams use test games.

Teams that place in the top three also have to play with a great deal of skill, great strategy and near flawless execution with few if any mistakes. I'm saying they also have to make effective use of test games as well.

No SGOTM Team with realistic aspirations to finish in the top three will forgo the significant advantage of using a test game, unless the nature of the SGOTM is such that a test game would not be fruitful to use. The exception is a SGOTM that would be too hard to model via a test game due to the poor correlation between the test game and the real that nullifies the test game's predictive advantages. The poor correlation is due primarily to the increased randomness based on the SGOTM chosen settings, especially the randomness from game elements that provide the player with little or no control to mitigate its effect.

So for example, SGOTM-13 might have been a game where a test game would be less useful, because Tribal Villages were originally included. Let's assume Barbarians are also on. A test game could still have been effectively used, but Teams can be both heavily rewarded or heavily penalized by entering Tribal Villages with little opportunity to mitigate bad outcomes. A great outcome would a free Technology, especially at epic or marathon speed. A bad outcome would a horde of Barbarians that kill the Scout and beeline your cities; the Scout is essentially dead = no/little mitigation is possible; good city defense can mitigate damage done by the Barbarians after killing the Scout, but the Barbarians might still destroy some improvements. Excellent teams don't need the luck made possible by Tribal Villages, but an early free Technology like Bronze Working may give a Very Good team enough of a boost to propel them to a win, especially in a game requiring early warfare.

Add events and quests and the game becomes even more random in a way that the player has no/little control.

Randomness and Player Control:

The Game has two types of Randomness, Good Randomness that the player can control at least somewhat and Bad Randomness that the player has no control over at all. Tribal Villages and Events/Quests are Bad Randomness, because the player is being rewarded or being penalized arbitrarily for doing the same thing: Actively entering a Tribal Village or passively receiving the news of an event that impacts the game going forward. An example of Good Randomness is the chance that a stronger unit will occasional fail to subdue a significantly weaker unit or even if it loses cause the stronger unit more damage than expected. The player can easily mitigate such unfortunate RNG outcome with sufficient additional units and choosing a more optimal combat order. On defense, the player mitigates by having a good mix of unit types and considers what if key defenders are lost and plans ahead for that possibility. Another type of Good Randomness is the AI Opponents actions are somewhat Randomized without leading the AI into an inferior position; making the AI unpredictable such that is defense remains strong is Good Randomness that proves a stronger and somewhat unpredictable AI.

Quests are also Bad Randomness, because the Player can do nothing to control which Quest is offered and when it is offered, if ever it is offered. However, the Quest itself is good; do what is required to get the reward or ignore the Quest; the player must determine whether it is better to ignore the Quest (does not help to win sooner or is impossible to achieve in time) or pursue it (will help to win sooner). Good players do not need quests and will often ignore them, since they may cost more to achieve than the benefit they provide is worth.

Sun Tzu Wu
 
Suddenly actively lurking upon the SGOTM forum/spoiler threads, I came upon this one. It's certainly an interesting discussion, given others I've seen from participants in other formats. Nothing official that I've seen shows in this thread, which means obviously teams will continue to use test games. I agree very strongly with STW's "bad randomness" as per above; literally rewarding teams at random is skill equalization and cheapens the sense of competition. Huts and events should never be enabled in competition games, for example, because they can and will be deciding factors in a turn advantage at least sometimes.

However, test maps are a different animal. Dynamic Spirit expressed concern over teams reconstructing the map as they go, putting forth a theoretical model that if the entire map were revealed, it would be like playing the game twice. That analogy ignores one critical reality:

The entire map isn't revealed turn 0. Each iteration of test map only uses information that a team has already expended resources to ascertain! When designing a test save with new elements, you're still not able to undo poor choices made to that decision point; indeed, a test save would do well to include them!

Let's say in a test save a team plans to oracle something and can, through precision micro, reasonably expect to get the oracle + key tech turn 30 (hypothetical date which is impossible in reality). They plot out every micro detail, add new tiles as they explore in the real game, try many iterations until they just barely eek out t30 oracle in the test save and then...

Oracle goes in the real game t20! Plans out the window! Maybe they run into a barb city with modern armor, or maybe there's a barb city with 4 tactical nukes in it such that players would have to sac units just to get past (maybe there are 1000 tactical nukes in it, such that players wouldn't want to get anywhere NEAR it!). Maybe a team goes for religion and then divine right is discovered t10.

My point is that test games can NOT undo actions and they can NOT replicate playing a game twice, because no matter how much reconstruction you do, you can only plot out actions going forward with a test save. This makes test saves very similar to micro spreadsheets and other models; a method to analyze the current situation and plan for (expected, not concrete!) future situations but completely incapable of doing anything beyond that function.

Essentially, it's yet another tool of micro tedium. Everyone has world builder available. Even without a guide, a person can play around with the world builder and in about 30 minutes or less figure out what the buttons do, then reconstruct the start tile by tile. That doesn't spoil the map info, however, unless they are somehow opening worldbuilder with the actual save, which isn't happening!

Thusly, test saves are one tool among many to compete well in SGOTM, and as I plan to participate in future ones again I'd advocate allowing them. I can't find a reason (competitive imbalance, cheapening the real map, cheating, etc) that would merit denying players them, and the technique has been very insightful to learning how top minds think if you read threads as a lurker.
 
STW, I just realized that you somewhat recently posted again on this matter and that you still seem to think that no SG team has ever won without test games.
I remain convinced a SGOTM Team must make use of a test game to place in the top three, unless the best Teams don't use test games or the SGOTM is of a nature that makes using test games fruitless.

Sun Tzu Wu

@LowtherCastle:
Concerning SGOTM-6, let us please just civilly agree to disagree

Sun Tzu Wu

Clarification of the Conjecture that Test Game Use is Required for a Top Three Win:

To clarify, I'm suggesting that using a test game is a necessary condition for placing in the top three of a SGOTM.

Sun Tzu Wu
I reread your posts and discovered that indeed, according to you, your position is untenable. You see, when I used "test game" in SG6, I was referring to a CIV game in which I simply opened up a game of CIV with the same speed settings and tested out how the game mechanics are. In other words, at the very best, it was what you refer to as a "type 0 test game." But...in your own words, a "type 0 test game" "is not really a test game at all":

Types of Test Maps as Originally Defined by DynamicSpirit:

I would call this a 0 dimensional test game, because it is similar to the real game only in the map script and those game and map parameters available in the Options tab of the F8 Window.

...

I like the name "type 0 test game", because the zero helps to remind me that its not really a test game at all, since it is effectively a completely different game, just on a vaguely similar map.

Sun Tzu Wu
Surprise, surprise! Guess what that means? SG6 was won by a team that did "not really [use] a test game at all."

I'm willing to bet the SG1, SG2, SG3, SG4 and SG5 were the same, btw.
 
deleted
 
Whether or not every single top 3 team has used test games or not is irrelevant. Here are other things top 3 teams have always done:

1. Built workers
2. Researched technologies in the order closest to optimal
3. Planned turns ahead with math
4. Built enough units not to die

Obviously, these things don't hurt competition; rather they are part of the competition. Even if every single team uses a test save, that is an in-game tool available to everyone who owns the game and can compete. I'm looking for unfair advantages here and I'm not seeing them.

When a competitive environment bans something, it needs (at least in theory) a reasonable basis for doing so. What is the BASIS for a potential ban on test maps? I can't see it. Unless you're somehow enforcing speed play there just isn't a way that a test map can damage a single first time playthrough...correct me if I've missed some element of them though.
 
What is the BASIS for a potential ban on test maps?
As best I can determine, the basis is an erroneous assumption that award-winning teams' test maps are used to "model" the outcomes of the real game.

OSS won gold in SG12 and we used test games to a) test our MM plans, and b) determine some mechanics of CIV (independent of any scenario per se).

As others have pointed out, there are plenty of ways to a) test MM plans without a test game, such as spreadsheets or pencil and paper. Likewise, there are plenty of ways to b) determine CIV mechanics, such as studying the code and studying articles in the War Academy.

The notion that test games are used to model the actual outcomes of the game is to me silly, simply because I find it absurd to think that a test game could provide you any useful information. If anything, I find using test games to test MM quite dangerous, becaiuse while you test your MM, the game also implants misleading suggestions of what will happen and one has to be veeeeerrrrrrry careful to ignore them. In other words, one has to be veeeeeerrrrrrry careful to NOT PAY ATTENTION TO GAME "MODELLING."

Kind of funny actually...
 
Whether or not every single top 3 team has used test games or not is irrelevant. Here are other things top 3 teams have always done:

1. Built workers
Actually, in SG12, OSS stole most of our workers... :cool: but I get your point, of course...
 
the basis is an erroneous assumption that award-winning teams' test maps are used to "model" the outcomes of the real game.

Erroneous assumption =/= legit basis ;). Though you're on my side of this argument so of course you feel that way too. I don't understand the origin of that assumption though; it's literally impossible to model the real game until you know its details, and they're impossible to know until you've played to discover them! There's no way out there...

I tried a test map approach in BOTM 45 challenger and nearly won the game. Early micro really was boosted though my patience waned after city #2. Even then it was strong; I was able to time techs to beat barbarians and even get semi optimized micro for about 70 turns.

And yes, things like wonder times/DoW/etc from test games can't be trusted.
 
Claims to have not done something in a SGTOM are pointless without irrefutable proof. Only what the BUFFY/HOF module can verify can be held up to scrutiny. Anything outside of the real SGOTM game played with BUFFY/HOF module is not subject to any creditable method of proof.

So, whether a SGTOM team used test games, spreadsheet models or anything outside of what the BUFFY/HOF module can verify is outside of realm of what can be proven. There simply isn't a trusted verification tool outside of the BUFFY/HOF module, so what we can prove is limited to that module's capabilities.

If test games are poor models of the real game, why should we ban them and why indeed should any team that (claimed) they didn't use them be held in high esteem?

Since, the SGTOM rules never actually banned test games and the SGOTM-14 rules have reinforced their legitimacy, I see no point in rehashing this debate about whether or not they should be allowed.

It is further pointless to claim that any SGOTM was won without the use of test games, because that is clearly outside of what the BUFFY/HOF module can determine.
Certainly some teams may make that claim, but will any one really believe them? Just because someone says they believe doesn't mean they really do. Without irrefutable proof, we have nothing we can really trust, unless one accepts something as (religious) faith. Faith has its place, but not in the SGOTM competitions.

Moving on to something more current: Flying camera trick and the "radar trick"

These methods of determining the height of terrain have been banned in SGOTM-14, but the BUFFY/HOF module can not detect their use. Teams have used these methods in previous SGTOM competitions when they were less widely known. Winning teams probably gained significant advantage in their SGOTM games though their use. Admission of the use of these height detection methods in SGOTM-14 will almost certainly disqualify a team's game. Denial of use in previous SGOTMs will have high value to the team's making the denial, but there's no way the team can prove their denial is the truth. This is the same as the question of test game usage; it can't be proven either way.

Rather than have faith that no player will violate the flying camera trick and/or the "radar trick", these technique should be allowed to equalize the competition between those that have no qualms about violating an unenforceable rule and those that wouldn't even consider doing it.

Clarification: I advocate complying with the SGOTM-14 ban the flying camera trick and/or the "radar trick", since we can't change the rules in the middle of a SGOTM. I do believe this ban should be reconsidered for SGOTM-15 though.

Sun Tzu Wu
 
Claims to have not done something in a SGTOM are pointless without irrefutable proof. Only what the BUFFY/HOF module can verify can be held up to scrutiny. Anything outside of the real SGOTM game played with BUFFY/HOF module is not subject to any creditable method of proof.

So, whether a SGTOM team used test games, spreadsheet models or anything outside of what the BUFFY/HOF module can verify is outside of realm of what can be proven. There simply isn't a trusted verification tool outside of the BUFFY/HOF module, so what we can prove is limited to that module's capabilities.
News for you. I know this is old-fashioned but it needs to be pointed out. This is a gentleman's game. Gentelmen uphold not only the law but the spirit of the game. Among other things, gentlemen never EVER accuse other players of cheating without clear evidence.

If test games are poor models of the real game, why should we ban them and why indeed should any team that (claimed) they didn't use them be held in high esteem?
Umm... You start by insinuating that your competitors are liars without even the slightest evidence. If you were a gentleman, which you obviously are not, it would never occur to you to go down this path. On top of that, you seem to have no concept of what a good model or a bad model might be. A PERFECT model of a Civ4 game encapulates all the information that the player has at the moment the model was constructed. It also has ZERO information about the information the player does not have. As the perfect information and the zero information get mixed together as you move into the future, the model inevitably becomes less reliable.

Furthermore, for some strange reason, you are obsessed by one particular kind of model. You have never spoken out against spreadsheets. Or against pen and paper. Or against the innate human ability to keep a model in one's own mind. What is special about test games?

Since, the SGTOM rules never actually banned test games and the SGOTM-14 rules have reinforced their legitimacy, I see no point in rehashing this debate about whether or not they should be allowed.

It is further pointless to claim that any SGOTM was won without the use of test games, because that is clearly outside of what the BUFFY/HOF module can determine.
Certainly some teams may make that claim, but will any one really believe them? Just because someone says they believe doesn't mean they really do. Without irrefutable proof, we have nothing we can really trust, unless one accepts something as (religious) faith. Faith has its place, but not in the SGOTM competitions.
Oh man. Yet more denigration of your fellow man. I submit that faith does indeed have its place in the the SGOTM competitions. Among other things, I believe that the vast majority, perhaps all (or close anyway), of the players in the game are honest and try, as best they can to follow the rules of the competition.

That is my faith. Prove me a fool.

Moving on to something more current: Flying camera trick and the "radar trick"

These methods of determining the height of terrain have been banned in SGOTM-14, but the BUFFY/HOF module can not detect their use. Teams have used these methods in previous SGTOM competitions when they were less widely known. Winning teams probably gained significant advantage in their SGOTM games though their use. Admission of the use of these height detection methods in SGOTM-14 will almost certainly disqualify a team's game. Denial of use in previous SGOTMs will have high value to the team's making the denial, but there's no way the team can prove their denial is the truth. This is the same as the question of test game usage; it can't be proven either way.
Crap. Utter crap. From one end to the other, this is crap. Vicious malicious crap on top of that. Flying camera was legal in the last game and it is not in this one. What possible evidence do you have that following the rules in the last game would lead the same person to violate them in this one?

On top of that, it is simply nonsense to claim that "Winning teams probably gained significant advantage" through their use of this trick. Probably? Probably? What kind of weasel words are that? PD won because they identified the map and focussed on the prime objective, not because Kossin played some dumb trick at the beginning. What's more, unlike you, their competitors were classy enough to acknowledge why they won.

Rather than have faith that no player will violate the flying camera trick and/or the "radar trick", these technique should be allowed to equalize the competition between those that have no qualms about violating an unenforceable rule and those that wouldn't even consider doing it.
Jeez. Learn something dude. 1) Most people are fundamentally decent. 2) It's just a game.
 
@sun Tzu Wu

You begin by calling your fellow players liars without providing any proof while at the same time very strongly demanding proof from everyone else in making their claims, and then go on to talk about "probably" when talking about flying cameras. Do you know how that looks to the casual reader? Perhaps you need to think more carefully before posting?
 
I think there's a simpler way to understand all of this.

1. The Civfanatics Community is a community. It has always been based on a spirit of sharing. Sharing knowledge of the game and sharing friendship and good times. The games are based, most fundamentally, on the honor system.

2. The community has occasionally found programming deficiencies that can be exploited in ways that people consider to be against the spirit of the game and thus the community has banned such exploits. Those bans hold, whether they can be verified and proved or not.

3. The flying camera, for example, is a widget designed to give us nifty views of REVEALED TILES. That it can be exploited to peek under the fog is, like all exploits, a programming deficiency that is obviously against the spirit of the game: Why would there be fog if you can peek under it?

Now we get to the heart of the matter.

4. To allow exploits rather than ban them, based on the reasoning that their used cannot be monitored, has one of two effects for the majority of CFC members who are honest, that is, those who prefer to play within the spirit of the game:
a) you criminalize the honest, or
b) you penalize the honest.​
That's it. If I'm against using the exploit and I'm weak, then I violate my own ethics and use them. If I'm strong, I don't use them and am thus penalized.

5. We ALL know that no one can lie to himself. We ALL know that no one can derive true satisfaction from winning while cheating. Meanwhile, if an honest person does manage to win, despite all the cheaters (if there are any), he can truly derive some satisfaction from that. So...Who cares if some people cheat? Let them rot in their own self-created dungeon.
 
Rules against activities (i.e flying camera trick and radar trick) that can not be detected/enforced actually cause more harm than good. You will never be able to measure the compliance level and thus the damage it causes to the game.

Fix the game when possible, but please don't try to fix flaws or ban their use when its impossible to do. If the GOTM staff had the technical means to detect or prevent these activities, they would not hesitate to do so. On the other hand, there's nothing worse that having rules that can not be enforced.

If you really want to play a gentleman's game, stop using the BUFFY/HOF module. Now ban random seeds and replaying turns without the rules compliance module and guess what your compliance will be. How well to you really think total voluntary compliance (no BUFFY/HOF module) will go?

Perplexingly, I don't understand how being against "unenforceable rules" means I'm accusing everyone of cheating. I have not accused any particular person (or team) of cheating, although I have pointed out how trivial it is to cheat versus an unenforceable rule. Indeed, how can anyone have a basis for accusing someone of violating an unenforceable rule whose violation is undetectable? That fact should be obvious to anyone, especially to the staff who spend some of their time enforcing the rules.

Good luck playing the game without enforcing any of the rules (no BUFFY/HOF module). If you really think that having unenforceable rules is a good idea, perhaps none of the rules should be enforced.

Sun Tzu Wu
 
STW, this is really simple. You're way overcomplicating it. The simple truth is, someone who wants to cheat can violate every single rule in the book and go undetected. Not BUFFY, not the built-in detectors, nothing can stop someone from cheating, if he knows how and has his mind bent on doing so. Nothing. Not a thing.
Not a single thing.
So in the final analysis there's really only one solution. It's called an honor system.

One more thing. For me, using the honor system and banning exploits gives me ALL the quality I can hope for. I love playing SGs. I love the honor system. I love the rules that we abide by. I've been on a team in which I, OLD GEEZER THAT I AM, ACCIDENTALLY GIFTED AWAY OUR THIRD CITY AND WE STILL WON!?!?!

So I, personally, am really, really, really NOT worried about cheaters. The only reason I pushed for banning the flying camera is to have SG14 be as close to the spirit of CIV-Bts as possible, under the honor system.
 
There is a clash of philosophies here that I do not think can be solved in this thread.

Having been a CivFanatic for over 8 years, and a member of Staff for over 2 years, I have come to agree with this:
So in the final analysis there's really only one solution. It's called an honor system.

As a flawed, commercial software product, the game we come here to play and discuss is not one made for the type of competition that Sun Tzu Wu wishes we could have. In fact, the staff wishes we could have the same thing that Sun Tzu Wu would like, an ironclad competition that leaves no doubt about the skills of the players and the results of competitions. This simply does not exist, in the Civ world or in the greater world.

We have designed a system in order that we may play this game together and enjoy the company of others who share our passion for playing Civilization, whatever the version. In doing so, we have made decisions on how to structure rules and, I hope, been open to input and the need to modify rules to reflect the state of the technology we have.

Part of this is an acceptance that we must trust the honor of people within our community to play fairly and to follow rules that we cannot always verify. My experience has been that the truth eventually comes out and when it does, we will pursue what we find in order to prevent, as much as possible, cheating, in whatever form.

I have met some really great people here at CivFanatics and am honored to call them friend and to trust them. I could not be a member of staff if I could not say this as I would probably tear my hair out to find each and every cheat. What life would that be?
 
STW, this is really simple. You're way overcomplicating it. The simple truth is, someone who wants to cheat can violate every single rule in the book and go undetected. Not BUFFY, not the built-in detectors, nothing can stop someone from cheating, if he knows how and has his mind bent on doing so. Nothing. Not a thing.
Not a single thing.
So in the final analysis there's really only one solution. It's called an honor system.

One more thing. For me, using the honor system and banning exploits gives me ALL the quality I can hope for. I love playing SGs. I love the honor system. I love the rules that we abide by. I've been on a team in which I, OLD GEEZER THAT I AM, ACCIDENTALLY GIFTED AWAY OUR THIRD CITY AND WE STILL WON!?!?!

So I, personally, am really, really, really NOT worried about cheaters. The only reason I pushed for banning the flying camera is to have SG14 be as close to the spirit of CIV-Bts as possible, under the honor system.

If the above were completely true, there would be no requirement to use the BUFFY/HOF module. However, the competitions require use of the BUFFY/HOF module. Therefore, I contend that the BUFFY/HOF module does have value in ensuring that games are played honestly. If they had no such value as LowtherCastle implies above, the staff would not require BUFFY/HOF module.

Only when the staff lifts the requirement to use the BUFFY/HOF module or any other rules compliance module will we be operating (100%) on the honor system.

Use of the BUFFY/HOF module ensures compliance with certain basic rules of not generating a new RNG seed and ensuring that players do not replay turns. I confess that I have no idea how it does this and perhaps it isn't quite 100% infallible; it doesn't have to detect 100% of this type of cheating to be an effective deterrent of such nefarious activities. The BUFFY/HOF module probably has some sort of virtually impossible to crack cryptographic code based on large prime numbers (no one should be suggesting otherwise, especially the staff).

Honor System = No Verification:

The Honor System is a system where no rules are enforced; compliance with the rules depends on the players not violating any rule, even those they are not aware of or forget about. If there are many rules such as avoiding exploits, some players may not be aware of all of them and may innocently violate them. With a 100% Honor System, no one will detect any violations, because there are no tools that can detect them using a standard Civ4 save file.

Hybrid System = Partial Verification:

The Hybrid System consists of both unenforceable rules and enforceable rules. Thus, the Hybrid System is part Honor System and part Compliance System (described below). The current SGOTM system is a Hybrid System.

Compliance System = Total Verification:

The Compliance System has only rules that are (technologically) enforceable. Unenforceable rules will never be allowed in a Compliance System. One example of technological enforcement is the BUFFY/HOF module and associated staff only tools for detecting violations and verifying compliance.

---

When LowtherCastle and leif erikson spoke above about an "honor system" above in this thread, they are really speaking of a "hybrid system" where most violations of the enforceable rules are detected by the BUFFY/HOF module and presumably some tools that only rules compliance staff have access to. The remaining rules in this system (like the bans on the flying camera trick and the radar trick) are not enforceable by any means (unenforceable). They are a burden and blight to the whole system, because the only person you can be sure that does or does not violate these unenforceable rules is yourself.

I'm not aware of any society (in history or groups of people like the CFC) that has survived the imposition of unenforceable rules. You may think such rules fix flaws in the society, but they just cause irreparable divisions of those that comply with the unenforceable rules and those who do not. The result is usually the self destruction of the society, because it is based at least in part on unenforceable rules.

The Idealistic View of an Unenforceable Rule:

Ask an idealist what level of compliance he will get on his newly proposed unenforceable rule which everyone agrees fixes a real problem. He will optimistically and triumphantly state "obviously 100%, because it fixes a problem that everyone agrees needs to be fixed".

The Pessimistic View of an Unenforceable Rule:

Ask a pessimist what level of compliance he will get on his newly proposed unenforceable rule which everyone agrees fixes a real problem. He will pessimistically state "obviously 0%, because there is no reliable mechanism to enforce compliance".

The Realistic View of an Unenforceable Rule:

Ask a Realist what level of compliance he will get on his newly proposed unenforceable rule which everyone agrees fixes a real problem. He will emphatically state "obviously it doesn't matter, because no one will ever be able to tell who is or is not violating the rule. Therefore the best guess of the compliance level is somewhere between 0% and 100% with all percentages equally likely.

Sun Tzu Wu
 
Status
Not open for further replies.
Top Bottom