SGOTM13 - Separated discussion of test maps

Status
Not open for further replies.
Joined
May 18, 2005
Messages
23,520
Originally Posted by DS
As an example of the other extreme, suppose in a GOTM we'd decided to reveal the entire map upfront, including locations of resources. Now, someone who had enough patience with worldbuilder could do:

4. Build a test game that happens to have exactly the same map in every detail as the real game.

I think everyone would agree that is completely unacceptable, since playing their test game then playing the GOTM is no different from playing the real GOTM twice through.


That's a very poor hypothetical argument, IMO. Obviously, the thing that's wrong with it is not the test play itself, but the fact that you're playing the same map repeatedly with perfect prior knowledge. In SGOTM practice, what we do is update test games as knowledge of the map is revealed in game. There is no prior knowledge being used as a basis for play decisions at any point. In fact, the practice of test mapping is the opposite of your example, in that it gives rapidly diminishing returns the further it is advanced. To take it to the extreme, it would be both insane and utterly pointless to model a perfect test game up to the turn before you launch a spaceship. The primary use is for optimizing the early game tactics. As many people have already pointed out: this is no different from using your brain, pen&paper or Excel. It's just much more fun, and (most often) quicker and easier, to do with a test map.
I agree. And there's even a more fundamental problem with the hypothetical argument: the random seed. Every team could build their own map but each one would have a different random seed and would behave differently.

This highlights the danger of test games and the fundamental reason why stw's argument is fatally flawed. Test games tell you nothing for certain about what will happen in the real game with respect to AI behavior.
 

neilmeister

Lentils have feelings too
GOTM Staff
Joined
May 4, 2007
Messages
6,511
Location
Australia
deleted
 

neilmeister

Lentils have feelings too
GOTM Staff
Joined
May 4, 2007
Messages
6,511
Location
Australia
deleted
 
Joined
Feb 12, 2006
Messages
2,648
Location
Hahndorf, Australia
No Team has won a SGOTM without using a test map

This does not prove your assertion that test games are THE essential tool in SGOTM's. It only proves that they are useful. You could just as easily argue that no team has got a wooden spoon without using a test game and according to the same logic you use thereby conclude that test games actually hinder your play and lead to low scoring results!! Your logic is faulty.

No-one is denying (as far as I can see) that test games have their uses. I would say in particular in the early game.

I think Mitchum stated it very well indeed in post #31
 

Mitchum

Deity
Joined
May 31, 2006
Messages
12,800
Location
San Diego, CA
I think Mitchum stated it very well indeed in post #31

I have to agree with you on this one... :D

Sun Tzu Wu,

It's clear from this thread that your main argument (which I boil down to: the team with the best map maker wins) has offended several people... and not just players from the top teams. Rather than defend your primary argument, you look for one tiny loose thread in a single post and start pulling it to see if you can unravel the whole argument against you. You conveniently ignore all other posts.

The basis of your argument seems to be: No winning team can prove that they won without a test game (I'm excluding MW SG6 because I think we can all agree that their use of a test game has nothing to do with the type of test game we've been discussing here). So, if all gold medals have been won by teams with test games, the team with the best WB skills wins the gold. Is this your argument? Did I miss something because I don't follow the logic here.

Rather than belittling the skill and experience of everyone on this forum by stating that WB skills = gold medal, you might want to consider a different approach if you want ANY support from our community in your attempt to eliminate test games from S-GOTM.

EDIT: Boy, the moderators are earning their pay checks this week!! :lol:
 

Sun Tzu Wu

Deity
Supporter
Joined
Mar 26, 2007
Messages
7,920
Types of Test Maps as Originally Defined by DynamicSpirit:

A couple of general observations in response to various posts here:

Test maps can mean very different things depending on how customized they are. Examples are:

1) You set up a game that has the same difficulty and map script etc. as the (S)GOTM, just to get a feel for what playing Civ is like with those parameters.

I would call this a 0 dimensional test game, because it is similar to the real game only in the map script and those game and map parameters available in the Options tab of the F8 Window. Note that some map types have sub-parameters that may not appear in the Options tab, so one would only know the values of those sub-parameters, if the scenario designer published them as part of the scenario description. So, depending on the map type, the test game might have normal islands whereas the real game might have tiny islands, assuming the scenario designer did not publish that particular map type sub-parameter.

The advantage of the type 0 test game is no "evil" World Builder editing of the underlying map is required.

The disadvantage of the type 0 test game is the starting position of the map is determined by random. The game also randomly positions the players units such that each unit's visual ranges are either adjacent or overlapping with at least one other player's unit. The visual ranges of each player's unit are modified as usual by being on a hill, having a view from a hill/shore be obstructed by an adjacent hill/forest/both, etc. There is virtually no chance that the terrain and position of the units will even remotely match those in the real game. The players' optimized progress of the test game and the players' optimized progress of the real game will likely only be the same only for turn 0 prior to ending the turn. From that point forward they will most likely rapidly diverge.

I like the name "type 0 test game", because the zero helps to remind me that its not really a test game at all, since it is effectively a completely different game, just on a vaguely similar map.

2) You do (1) AND you edit the starting area so it looks the same as in the advertised starting screenshot.

I would call this a 1 dimensional test game. because it adds the first useful dimension, the knowledge of the area near the the Settler as revealed in the real game which is quickly expanded to what the initial City and first 1-2 turns of unit movement in the real game reveal.

Probably all SGTOM Teams will also move all non-Settler units in the real game such that maximum or more important plots are revealed to provide the best BFC information that permits the locally optimal place for the Settler to build the initial City. The Settler is usually not moved. It is normally at this extremely early point in the real game that the test game is made by first creating a type 0 test game (step 1 above). Next use World Builder to modify the start area of the map to precisely match exactly what terrain and all units visible in the real game. Lacking any reason to do otherwise, the test game's Settler can be left in position and the terrain under and around be changed to match what is visible in the real game, including other units.

This type 1 test game can be used to decide where to settle the first City. Next actually settle the City on the test game "winning" plot in the matching plot in the real game. This may reveal a few more plots and will reveal the entire BFC of the new city. The type 1 test game undergoes another round of World Builder, so it is again in visually identical to the real game.

The advantage of the type 1 test game is it can now be used to plan the City with perfect precision until a second City is settled or until the enemy enters the city's BFC or a neighbors trade network connects with it via River/Sailing or Road.

A type 1 test game is usually finalized by cloning the visible aspects of the real game by the end of turn 2 or maybe turn 3. The exploration progress in the real game can be just as easily tracked in the real game, so there is no point replicating additional revealed plots in the test game, other than make it appear more like the real game. Thus, a type 1 test game is last modified by World Builder in turn 2 or turn 3. Further modification turns it into a type 2 test game as described below.

3) You do (2) AND as you explore the map, you edit your test game and perhaps replay so you're testing on as near as you can get to the same map as the real game.

I like to call this the type 2 test game, because it adds the second and final dimension, the rest of the real map as it becomes available via exploration.

The advantage of the type 2 test game is additional cities can be precisely modeled as well as unit movement over already revealed territory.

The disadvantage is every time the test game's map is modified by World Builder all of the internal, mostly hidden, state of the game is forever lost. This is particularly noticeable in Diplomacy values. The problem is World Builder doesn't have the capability to set this internal, mostly hidden, game state. Even if it did, "thank god" its impossible (due to Civ IV's design and failing that the BUFFY module) to get this internal, mostly hidden game state from the real game.

Sun Tzu Wu's request for coordinates (although he later said he didn't actually want the info) appears to imply he was planning to do (3).

Yes, I was planning a full type 3 test game, otherwise I wouldn't have been concerned about the test game's map edges coinciding with the real game's map edges.

Also, I did want to be able to do this without knowing the Settler's coordinates. I realize now that's not possible even with a GOTM staff provided test game until its BUFFY locked, since otherwise the first attempt at editing the test map would reveal the Settler's coordinates (at least approximately, while trying not to look :lol:.)

The question then is, how far is this within the spirit of the (S)GOTMs? Different people will have different opinions. Personally both (1) and (2) strike me as quite acceptable for getting a feel for the game and starting area, but on a game that is very clearly a completely different game.

A type 0 test game isn't really a test game at, since there's no editing of the map at all. There's really no justification for banning type 0 test games, because you really can't tell all players in a SGOTM that they can't play any SG, GOTM or HoF Game that happens to use the same map type, and map/game parameters.

A type 1 test game is equally as bad as a type 2 test game. The greatest benefit in using a test map is for micromanaging the initial expansion phase. Small gains early on reap huge benefits later. Due to the internal game disconnect (losing the mainly hidden internal game state with each World Builder update), type 2 test game are much less reliable and thus less useful in helping to plan for the real game.

I'm enthusiastic about banning both type 1 test games and type 2 test games.

As an example of the other extreme, suppose in a GOTM we'd decided to reveal the entire map upfront, including locations of resources. Now, someone who had enough patience with worldbuilder could do:

4. Build a test game that happens to have exactly the same map in every detail as the real game.

I think everyone would agree that is completely unacceptable, since playing this test game then playing the GOTM would be no different from playing the real GOTM twice through.

Just curious, do GOTM players use test games? Theoretically, a GOTM player could play several practice runs based on a type 1 test game and then be well practiced for the real game that is the only one that counts. Also, a GOTM player has no team mates, so his use of a test game is much easier to keep secret without even actively doing so.

Back to your interesting example. Let us call this a type 3 test game where the test game is identical to the entire map. The type 3 test game has the major advantage of needing no World Builder changes after Turn 0; this means there will be no mostly hidden internal game state resets due to World Builder invocations. However, a type 3 test game may be surprisingly not much worst than a type 2 test game, because the Random Number Generator is used by over half of Civ IV's game subsystems. The type 3 test game and real game may appear identical until opponent contact and that will be radically different, simply due to a different sequence of random numbers.

So where do you draw the line? Personally, I'm inclined to draw it at (2) (provided the revealed starting area is small), because that seems to me to mark a clear boundary: You are using no more information than is available before playing the game. I stress that's my personal opinion, not a GOTM staff opinion. (3) makes me uncomfortable, since it seems to me potentially too close to pre-playing the actual game, so personally I'd prefer people not to do that, but obviously there's no rule against it (partly because we'd have no way of enforcing such a rule, partly because I'm not sure there is a consensus on whether that's OK or not).

Would a useful way forward for this discussion (which btw I do think is worthwhile, so I'm happy that Sun Tzu Wu's request brought it up) is to try to figure out if there is a consensus on how much information from the real game it's OK to feed into a test map before the test map becomes 'unacceptable'?

I believe your concerns about type 2 test games that are used throughout the game are mostly unwarranted, because they require a lot of effort to maintain and they probably are very inaccurate due to World Builder resetting much of the test game's internal state each time the map is updated, typically each turn set.

I would definitely be in favor of banning all test games, except type 0 test games, which are simply games with the same settings, untouched by World Builder.

How can I want information to make test games easier to maintain while wanting all test games banned:

I have a similar issue with HoF rules that, gasp, allow Tribal Villages. Many very strong Deity Religious Leader Diplomatic Victory #1 games were achieved via popping a critical Technology from a Tribal Village. Though I loath Tribal Villages, it can be next to impossible to beat a #1 game that got such a free Technology via a Tribal Village, so I'm almost forced to use Tribal Villages to compete effectively.

The issue I have with SGOTM is quite similar, I'm strongly against using test games, but it seems no Team has a chance of winning without using them, so I'm reluctantly pursuing the use of test games in SGOTM-13.

Sun Tzu Wu
 

kcd_swede

Jag är Viking!½
GOTM Staff
Joined
Jun 21, 2007
Messages
7,413
Location
Stockholm's B.F.C.
What exactly is the goto mapping trick? No specifics, just a general gist.

When a unit, like a scout is selected, you hold the right mouse button down and venture off into the fog with the "goto" circle. You will see from the dashed line that forms some information about terrein. Clealry a bug, imo. We won't use it at Unusual Suspects.
 

Indiansmoke

Deity
Joined
Dec 12, 2006
Messages
4,124
Location
Athens, Greece
When a unit, like a scout is selected, you hold the right mouse button down and venture off into the fog with the "goto" circle. You will see from the dashed line that forms some information about terrein. Clealry a bug, imo. We won't use it at Unusual Suspects.

Yes and plus I saw that terrain reveal thing that plastic Ducks did with the game camera, never new you could do that but that is also a bug obviously.
 

Sun Tzu Wu

Deity
Supporter
Joined
Mar 26, 2007
Messages
7,920
SGOTM is a competition where the Team with the best test game makers and adequate players win. (No offense to past winners who play within the rules of the games.) If that is what you really want, then please understand the skills you are really rewarding (map making = World Builder skills).

Providing an official test game start would level the playing field between Teams with poor map making skills and those with expert map making skills.

Map Making, NOT the Most Important Skill in Winning SGOTM:

I now have better understanding of test maps and how much they aid SGOTM Teams. I no longer believe this statement: "Map Making, the Most Important Skill in Winning SGOTM". Please notice the word NOT in the title of this post.

Thanks to everyone who wrote to convince me that I was giving test games too much credit for SGOTM wins. I still believe that test games shave dozens of turns off a win, but of course it is the Team Members' playing skills that win the game and their game playing skill is by far the vastly more important component of the win.

Although, I now believe that test games are vastly smaller factor than before, I remain convinced a SGOTM Team must make use of a test game to place in the top three, unless the best Teams don't use test games or the SGOTM is of a nature that makes using test games fruitless.

Sun Tzu Wu
 

Sun Tzu Wu

Deity
Supporter
Joined
Mar 26, 2007
Messages
7,920
@LowtherCastle:

I apologize for the less than civil manner in which I spoke to you in this thread. It shall not happen again. Concerning SGOTM-6, let us please just civilly agree to disagree and forget it.

I regret causing the moderators extra work to remove some of my text that crossed the line.

Sun Tzu Wu
 

neilmeister

Lentils have feelings too
GOTM Staff
Joined
May 4, 2007
Messages
6,511
Location
Australia
I apologize for the less than civil manner in which I spoke to you in this thread. It shall not happen again.

I regret causing the moderators extra work to remove some of my text that crossed the line.

Me too!
 

Erkon

Deity
GOTM Staff
Retired Moderator
Joined
Nov 5, 2005
Messages
3,910
Location
Malmö, Sweden
Ahh, a heated debate over test maps! :clap: It serves as a valuable source of inspiration for potential future SGOTM maps :D. I think I have an idea of a map* that cannot easily be regenerated regardless of WB skills, but I have to play test it first :mischief:


* Yes, it would be a very, VERY, VERY evil map
:mwaha:
 

Dhoomstriker

Girlie Builder
Joined
Aug 12, 2006
Messages
13,420
I could equally use a spreadsheet or a test game to determine what are strong Worker actions, tech paths, and build orders for my first City. I could also do so using paper and a pencil--and have done so for a couple of my games in the past.

As has been stated, the relative value of a test game that "appears to match the terrain" of the real game goes down drastically outside of this Worker-action + build-order-planning concept, mostly because random-value generation affects so many conditions and AI actions within a Civ 4 game, to the point that using a test game offers very little in terms of predictability of what will happen in the future.

What actually ends up happening is that, due to convenience, players will use their test game to try out game mechanics that they could equally do in a "0 dimentional" test game. So, often, a "type 2 test game" really ends up being used as a quick way of testing out concepts that could be tried in a "0 dimentional" test game.


Anyway, if I were to come up with a hypothesis about the impact of a particular factor on the odds of winning a top spot, such as the impact of test games, then I would attempt to isolate this factor as much as possible.

If you really want to attempt to prove your assertion about the impact of test games, may I suggest that you take the following approach:

1. Pick a previous SGOTM that you have not played (or read the teams' threads about) and play it.

2. You are apparently good at discovering test saved games that teams have used in their game... I will authorize you to use every team's test games from that same SGOTM. However, you are not to use any other information (so, do your best not to read their suggestions or discussions when searching for test saved games... just search for their test games). You could even solicit the help of someone else to go and purloin the test games for you, in order to give further evidence that you have had no exposure to the ideas of other teams who played the same game.

3. Since your claim was that it is the test game that makes the player and not the player using the test game who makes the relevant observations that matter, you are also not allowed to read any observations that were obtained by other teams as a result of playing test games. You must draw your own conclusions and observations from the test games. Actually, since it is a team-based game, if you want, you can try to recruit some teammates to play and then host the game in a thread that you post over in the Succession Games Forum.

4. You could optionally make your own test games, but I think that it would be ideal if you only used the test games that other teams made and used already, to further isolate the impact to the specific subset of test games that were used. Sure, you would have access to more test games than any single team had access to, but then, certainly, if test games are the be-all, end-all, your results in the game will by far be the best of the best.

Sun Tzu Wu said:
the Team with the best test game makers and adequate players win
5. See how well you, a hinted-at, self-proclaimed "adequate" player can do compared to the results of the top teams, along with any teammates that you are able to recruit who have not already played said SGOTM. Use only the test saved games from other teams' threads but no other information from said threads.

We can't know how well you would have done without the test saved games, so we would have to accept your self-proclaimed level of playing competence and thus believe that if you do play well that you wouldn't have done so without the test games.

However, what this effort will do is it will help to isolate, although far from eliminate, other possible factors for your performance. When you complete the game with a result that competes with the Gold Medalists, then finally you'll have started to have built some compelling evidence.

"Proof by example" will not prove a rule, but without even an example to support a theory, even the most credulous of spectators will generally dismiss your ideas.
 
Joined
Feb 12, 2006
Messages
2,648
Location
Hahndorf, Australia
Ahh, a heated debate over test maps! :clap: It serves as a valuable source of inspiration for potential future SGOTM maps :D. I think I have an idea of a map* that cannot easily be regenerated regardless of WB skills, but I have to play test it first :mischief:


* Yes, it would be a very, VERY, VERY evil map
:mwaha:

Will someone make sure Erkon does not get to create the map for some future SGOTM? (unless of course I'm feeling particularly masochistic one day :p, in which I case I reserve the right to change my mind!) :D
 
Joined
May 18, 2005
Messages
23,520
Yes and plus I saw that terrain reveal thing that plastic Ducks did with the game camera, never new you could do that but that is also a bug obviously.
lol now that's embarassing.

Erkon, I thought this kind of stuff got modded out after SG7. Do you think our HOF friends can mod out these two bugs?

Ahh, a heated debate over test maps! :clap: It serves as a valuable source of inspiration for potential future SGOTM maps :D. I think I have an idea of a map* that cannot easily be regenerated regardless of WB skills, but I have to play test it first :mischief:


* Yes, it would be a very, VERY, VERY evil map
:mwaha:
Never ignore divine inspiration... :)
 

Erkon

Deity
GOTM Staff
Retired Moderator
Joined
Nov 5, 2005
Messages
3,910
Location
Malmö, Sweden
...Erkon, I thought this kind of stuff got modded out after SG7. Do you think our HOF friends can mod out these two bugs?...

No. Both features are part of the graphical engine, and it turned out that they cannot be modded (if I remember correct).
 

Brian Shanahan

Permanoob
Joined
Jul 27, 2009
Messages
3,899
Location
The house that I shaped in my heart
When a unit, like a scout is selected, you hold the right mouse button down and venture off into the fog with the "goto" circle. You will see from the dashed line that forms some information about terrein. Clealry a bug, imo. We won't use it at Unusual Suspects.

Thanks, it's quite an exploity little bug all right.
 
Joined
Feb 12, 2006
Messages
2,648
Location
Hahndorf, Australia
I t also shows you where roads and railroads are even though they are not on your map. You can tell by how fast your unit is predicted to go through terrain your map hasn't been updated for in a while. This particular thing I accidently use in private games fairly often because I have the map already, but the AI has built roads since my map was updated and to my surprise I find my unit progressing much faster. The exploit about that is that the pathfinder chooses the best route according to now existing roads and railroads, not according to your knowledge of the terrain. This doesn't happen when I'm playing SGOTM's though because I'm playing carefully enough that I don't usually just set a unit off on a five turn journey - I move each unit one turn's move each turn!
 

unclethrill

Why am I up right now?
Joined
Sep 30, 2007
Messages
2,237
Location
Germany
Hey I have an idea that will help to put this argument into perspective. STW claims that no winning team has ever won without using a test map.

My survey is:

How many teams have used a test map at some point and still lost?


From my perspective, I have played on FE for 4 SGOTMs and we have 1 bronze medal. I also will make an educated guess that FE had used a test map at some point in all previous games only taking 1 medal (SGOTM12). So if we disregard the skill of our strategy in SGOTM 12 and assume that our medal was based solely on our test map, that is a 1 in 12 chance that a test map will help.

So have any teams that have no medals ever used a test map and still lost?
 
Status
Not open for further replies.
Top Bottom