Reloading in GOTMs

[DynamicSpirit]

Is there any way to tweak the HOF mod so that it requires you save after a victory and/or 're-saves' your most recent autosave in the event of a defeat? This would seem to solve a lot of problems.

There is already an option under the HOF3 menu to create a save on exit, which may help (though I guess it would have the disadvantage of saving even if you've just opened a save to look at without playing anything? I don't use it partly for that reason). But remembering to save after your victory doesn't seem a particularly onerous task, no? It is after all fairly clear in the submission instructions that that's what you're supposed to do.

 

[Vynd]

I just wanted to chime in and say that I support what the moderators are doing. I think it is entirely reasonable for them to implement some measures that will help ensure people play by the posted rules. Those that feel that the rules or the enforcement system are too strict can always establish their own event, or just play the GOTM as "exhibition" games and participate in the forums without submitting. But the moderators are the people who are putting real time and effort into organizing the GOTM and I think it is only fair that they be allowed to operate it in the manner they see fit. If they do a really bad job of it then most folks will stop playing... but I doubt that's going to happen.

I've had occasional instances of the game crashing and having to use autosaves. Most (or all?) of the time I've mentioned it to the moderators. And the autosaves I went back too were always fairly up-to-date. I've never even received a warning. So from my perspective, it seems like all you need to do is take some reasonable precautions and you'll have no problems.

 

[Ribannah]

Please read what we are saying. Short sessions in themselves do not result in EXCLUSIONS. Thay may result in WARNINGS. A warning is a set of recommendations.

Hence my question: what happens if the recommendations are not followed?

Has anyone been excluded during this exercise because they played short sessions?

We have no way of knowing. All we know, is that it is considered important enough to issue warnings, so important even that an unspecified number of false positives is considered acceptable.

To misquote the Bard: "Methinks there are some people here who do protest to much."

Speaking for myself: I have not protested yet, I am asking questions - and not getting many answers, I might add - and I am pointing at faults and consequences. If I were to make a similar statement at this time, it would be that there are some people here who have too many illusions and are willing to sacrifice other people to maintain them.

 

[DaviddesJ]

There are some people here who have spent years whining about one aspect or another of the GOTMs. This probably doesn't get them much respect.

 

[Ribannah]

I do find it pretty strange that players can spend a lot of time getting to the end of a hard-fought GOTM, and then forget to save, though

It is really quite logical, if you take the time to think about it. You save a game because you are going to continue it later. You don't need to continue a game that you have finished.
It would therefore be very helpful if a finished game is saved automatically, or at least a reminder pops up. It is now an option, but players won't know that the option exists. It should be the default setting, IMHO.

I just wanted to chime in and say that I support what the moderators are doing. I think it is entirely reasonable for them to implement some measures that will help ensure people play by the posted rules.

I think pretty much everybody agrees on that. The question is: will the measures that are put into effect actually accomplish this?
I am also sure that this staff does not implement measures because they feel entitled to (since they put in so much work). They do it in an honest attempt to improve the gotm.

 

[bio_hazard]

I think this has already been raised. The HoF Mod for Windows already has that option - save on exit. So if you reach the end of the game and quit, and if you have that option enabled, then you get a save automatically.

[EDIT] I do find it pretty strange that players can spend a lot of time getting to the end of a hard-fought GOTM, and then forget to save, though

Not that strange, really. For one thing, this is often in the wee hours. Also, not that outrageous to assume the computer will do it for you (which it really should). Nice to know the Windows version can automatically do that. Is it a dll issue for the mac side?

 

[ainwood]

Hence my question: what happens if the recommendations are not followed?

Then we will assess the evidence we have, make a decision, and if people don't like that decision, we will listen to their explanations.

What would you have us do instead?

We have no way of knowing. All we know, is that it is considered important enough to issue warnings, so important even that an unspecified number of false positives is considered acceptable.

The "warnings" are hardly warnings; they are a note to say "We're struggling to verify the integrity of your game. Please help us out - these suggestions will help in that respect".

In response to those suggestions, I have had a number of people e-mailing me for clarification on what the issues are. I have reviewed what they have said, and in many cases agreed that their explanations tie-in perfectly with the information we have.

Your claims that '... an unspecified number of false positives is considered acceptable' is wrong. Of the games we have excluded, I am very confident that we have got the decision correct. Claims of false positives appears to be rationalisation, and an obfuscation.

Regarding short sessions, you are missing the point: Short sessions, in of themselves, do not result in exclusions. I can categorically state that we have not excluded any Civ4 games simply because people play short sessions. But think of it this way: If a game is 20 hours long, and someone plays it in two sessions, we only need to verify two sessions. If someone plays in in 30 sessions, then we need to verify 30 sessions. If people play longer sessions, it makes our job easier, and hence our request.

Speaking for myself: I have not protested yet, I am asking questions - and not getting many answers, I might add - and I am pointing at faults and consequences. If I were to make a similar statement at this time, it would be that there are some people here who have too many illusions and are willing to sacrifice other people to maintain them.

And we have people who are quite happy to sacrifice the integrity of the competition to score a few extra points or a coloured badge in what is supposed to be a fun competition. We're not 'sacrificing' anyone. We have what we consider to be reliable systems. Just because we're not telling people the intricate details of them does not make them any less reliable.

 

[da_Vinci]

Some steps to be sure a player's game is valid and included:
1.) Create a Savefile at the end of your current game session just before you exit (you can create intermediate Savefiles, and can load them later for informational purposes, but never play any turns in them)
2.) At the beginning of your next game session, load the last good Savefile created at the end of your last session and play on
3.) Only load an autosave when you have a very good reason to use one
4.) Use the most recent usable autosave
5.) Set your autosave interval to every turn (instructions are posted earlier in this thread)
6.) PM a moderator about the use of the autosave

Follow these steps and your game will likely be accepted, but don't abuse Autosaves; they are replays and loading too many autosaves will result in exclusion.

@ civ_steve: This is a really useful list. Some thoughts ...

Item #1. Couldn"t this be automated by activating save on exit in HOF mod?

Potentially add: the suggestion that on second occurance of a crash (if not the first) address the problem before continuing to play?

Possibly add: suggestion to avoid manual saves in mid-turn? Seems like those might look suspicious, and be unnecessary if autosave is every turn.

Maybe your list ought to be included in the welcome to the Civ 4 GOTM thread? I tried to read all of the info on rules and procedures when I first came to GOTM, and still did not have a sense (until this thread) that notification of a mod via PM in the event of a crash is expected.

By the way, would be curious what happened to a mod to get you one of the warnings, too

dV

 

[Gyathaar]

Possibly add: suggestion to avoid manual saves in mid-turn? Seems like those might look suspicious, and be unnecessary if autosave is every turn.

No.. in theory the more often you save the better.. incase of a crash then there is less replaying needed

 

[Ribannah]

Then we will assess the evidence we have, make a decision, and if people don't like that decision, we will listen to their explanations.

What would you have us do instead?

Gather more useful evidence, before you make a decision.
Give people a chance to explain, before you make a decision, instead of after.

Your claims that '... an unspecified number of false positives is considered acceptable' is wrong. Of the games we have excluded, I am very confident that we have got the decision correct.

I know you well enough to take it for granted that you consider the evidence with all due care, and only take action when you are very confident. The nature of indirect evidence is that in spite of all your efforts you can still be wrong (i.e. generate a false positive or negative) and that you cannot know how often that will approximately happen - while you can make an estimate if you are using direct evidence.

I can categorically state that we have not excluded any Civ4 games simply because people play short sessions.

That is good to hear. But if that is the case, then why are you measuring them in the first place?

But think of it this way: If a game is 20 hours long, and someone plays it in two sessions, we only need to verify two sessions. If someone plays in in 30 sessions, then we need to verify 30 sessions. If people play longer sessions, it makes our job easier, and hence our request.

That is only because you have chosen sessions as the basis of you evidence. In fact, if you would not be biased against more sessions, you would have more work to verify 2 long sessions than to verify 30 short ones, not less. Now if you would gather other types of evidence, the number of sessions would not affect your workload. (Besides, I thought it was automated? )

And we have people who are quite happy to sacrifice the integrity of the competition to score a few extra points or a coloured badge in what is supposed to be a fun competition.

Sadly, that is all too true. The question is, do the measures taken have any significant effect?

We have what we consider to be reliable systems. Just because we're not telling people the intricate details of them does not make them any less reliable.

I agree, in fact it potentially makes them more reliable. But the issue is: does your system reflect what you think it reflects? You are using what is indirect evidence, or in modelling terms: you are using a proxy, with the assumption that the proxy is isomorphic with what you wish to investigate. If this assumption does not hold, and I submit that it doesn't, the validity of the conclusions cannot be established.

 

[da_Vinci]

No.. in theory the more often you save the better.. incase of a crash then there is less replaying needed

That is what I initially thought when I made lots of saves during my crash crisis in GOTM 11.

Later, it occured to me that saves in mid turn might look like saves intended to preserve the position before a critical decision, for a restart if it went badly.

If all of the mods agree that more saves are better (or at least not worse), that is a welcome bit of clarity.

dV

 

[civ_steve]

Item #1. Couldn"t this be automated by activating save on exit in HOF mod?

There's already been some discussion about this option, both for and against. It can be used, but if you win you still have to select the option to 'play a few more turns' so you can save at that point. After a couple of submittals it becomes 2nd nature. And replaying from the last Autosave so you can submit your defeat will always be accepted.

Potentially add: the suggestion that on second occurance of a crash (if not the first) address the problem before continuing to play?

I would think the statement "don't abuse Autosaves or you might get excluded" would lead the average person to this conclusion.

Gyathaar has nicely responded about saving in the middle of a turn. It's not saving often that will look suspicious, it's loading AND playing an earlier save that looks suspicious (loading just to look around and get information, no moves of units, no actual play actions performed, is fine).

If it is deemed useful, the list may be edited and/or added.

Regarding my warning, these things happen! It's just a warning, and there are players who got them who are playing entirely above board games and something is or appears just a bit suspicious. It is not a big deal.

There are players who got emails that their game(s) was/were excluded. There's been a lot of discussion about 'False Positives'. In this case the player gets an email and can respond, so even if there were 'False Positives' there is a process to correct them.

 

[ainwood]

You are using what is indirect evidence

Who said we were using (only) indirect evidence?

 

[Mutineer]

Ribannah,

What is the point of your continues posts?

We are not in court, but even in court judge and jury take in account undirect evidence when deciding on guild or innnosence.

Point is, there is a process to solve posible problems. No system is perfect.

So, what is your position?
1) No attempts should be make to ensure fair competition?
2) Moderators should waster there time doing it, but explain eveyrone in detail how they are doing it, so cheaters can devise betetr ways to go around it?

3) Trust moderators who are doing job just from the love of the game to hold competition the best way they can?

This the only 3 options we have.
Choise one.

I have big beef with HOF mod, that why I did not take part in Hof competition and did not play official Hof game. You have this choise 2.

On other hand, I do appriciate what they are doing and I see no fault in running GOTM and attempt to ensure it's fairness.

For me it is not actiall Fairness that matter, I do not need avards to feel respected or to people to listen to what I am saying.

What matter to me is opportunity to compare my game and critical decisions with other people. That dissapear if people replay save when they have information I could not have when I actially made the decision.

So, I am very happy to accept Option 3 and move on.

 

[civ_steve]

Gather more useful evidence, before you make a decision.
Give people a chance to explain, before you make a decision, instead of after.

We've gathered quite useful evidence, we have decided on the criteria to mark a game excluded, and we notify the player who submitted such a game so they have a chance to respond. We are now waiting for responses and will review those responses to see if any exclusions should be overturned. After a sufficient period, we will post the 'final' results. Seems like the opportunity to explain is there.

... in spite of all your efforts you can still be wrong (i.e. generate a false positive or negative) ...

We've tended to error on the side of 'include with Warning'. Players can still submit explanations which will be evaluated.

That is good to hear. But if that is the case, then why are you measuring them in the first place? ...

... That is only because you have chosen sessions as the basis of you evidence. In fact, if you would not be biased against more sessions, you would have more work to verify 2 long sessions than to verify 30 short ones, not less. Now if you would gather other types of evidence, the number of sessions would not affect your workload. (Besides, I thought it was automated? ) ...

... I agree, in fact it potentially makes them more reliable. But the issue is: does your system reflect what you think it reflects? You are using what is indirect evidence, or in modelling terms: you are using a proxy, with the assumption that the proxy is isomorphic with what you wish to investigate. If this assumption does not hold, and I submit that it doesn't, the validity of the conclusions cannot be established.

I think its been stated several times that we will not discuss the methods being used. You should stop attempting such a discussion.

Sadly, that is all too true. The question is, do the measures taken have any significant effect?

We are committed to running as fair a competition as we can. Obviously we should take action on games that are not fair by the competitions rules. Within the competition, these actions will have significant effect. Outside ... well, this is a game for entertainment, after all!

 

[jprc]

… and we notify the player who submitted such a game so they have a chance to respond.

I received the "warning", and a reply address was provided with it. I did replied, has not received (yet) an answer, but the email did no bounced back either. It is also possible to post or PM. So yes, there is a way to respond.

I think its been stated several times that we will not discuss the methods being used. You should stop attempting such a discussion.

However, what is really missing in the "warnings" sent lately is: Which wotm/gotm do you think is suspicious?!
I sent 3 (wotm1, worm2, wotm3), and I have no idea which one the warning refers to...
This is a big issue, as I have played the wotm1 100% alone, and I am 100% positive (no false positive) that no reload/replay whatsoever has been done allowing to know what will happen ahead of it. This is a strong statement and I do know that the spirit of the wotm was kept when wotm1 was played. However, because of my environment, I do not have a classic way of playing and HOF can react to this. (c.f. end of post).
The wotm 2 and 3 were only played at 10% or so by me, as it has been played "hot seat". If I tend and want to trust my colleagues, I am unable to be 100% certain.
Knowing then which wotm is essential for offering an opportunity to answer and explain, as it is pretty uncomfortable to receive a dry "warning" while you have a job that is usually placing you on the other side of the "warnings"!

Outside ... well, this is a game for entertainment, after all!

99.99% true, and in my case, it is not!!!
I have used wotm2 and wotm3 as a direct work experience.
I used Civilization for offering my team an opportunity for relaxing together, while taking decisions on their own, thinking as a group or on their own, and then having the next one in charge of applying decisions or consequences initiated by the previous player. During some working hours, the game was then played with a hot-seat procedure.
If it is entertainment, it is also a work experience where trust is important.

I wait for the final result before reacting, as it seems that the moderators are not willing to share with players at what point of the game problems occurred (which would allow me to know who, PERHAPS, did not follow my orders). As they are also in charge of keeping the system as safe as possible, it is also understandable.

I also wait to see if wotm1 is included or excluded:
- If it is excluded, it means that, if you do a certain sequence of rightful events, the hof system has a flaw, knowing that what we have to do for experiencing this flaw is only possible in a special circumstance, and environment (0.001% of all cases?...). It would also mean that my colleagues (who used the same sequence as me in a much heavier and frequent way), can still be trusted.
- If wotm1 is not excluded, and wotm2 and/or wotm3 is/are, then this is not "a game for entertainment" as it would mean that the hof system identifies correctly and positively the sequence done in wotm1, and I can start revising my professional opinion of the way team members are applying direct orders while unobserved...

As you can see, this game can have strong professional consequences on people!...

I will not detail here the sequence I/we have to use while playing. This sequence is not a choice, but is made mandatory due to the environment in which the game is played.
If a flaw in hof, this sequence will be detailed by PM to the moderator(s), as there is no need to put public any flaw, and better to see this directly with the HOF programmer(s) for fixing the issue.

 

[Ribannah]

So, what is your position?
1) No attempts should be make to ensure fair competition?
2) Moderators should waster there time doing it, but explain eveyrone in detail how they are doing it, so cheaters can devise betetr ways to go around it?
3) Trust moderators who are doing job just from the love of the game to hold competition the best way they can?

This the only 3 options we have.
Choise one.

I am going for option #4, and offer my expertise in these matters to point at weaknesses with regard to the method used and to suggest possible improvements. I deal with stuff like this (modelling, testing, judging, arbitrating - not extracting data from savefiles) on a daily basis in the real world.

What matter to me is opportunity to compare my game and critical decisions with other people. That dissapear if people replay save when they have information I could not have when I actially made the decision.

That is what matters to me, too. The difference is that I don't see measuring sessions having a positive influence.

We are committed to running as fair a competition as we can.

I am not doubting your commitment. I am trying to teach you something.

 

[AlanH]

@jprc: We *shall* be responding to e-mails and PMs, yours included, and you *will* know which games were affected.

 

[Thrallia]

That is what matters to me, too. The difference is that I don't see measuring sessions having a positive influence.

didn't you read what they said? the largest reason they suggest longer sessions is so that they don't have to look at as many sessions for cheating. Its quite simple really, its impossible to replay something if you only have one session unless you replayed the entire game in one session. Thus, when there are more sessions, there are more opportunities for replaying. This leads them to needing to check each one closer, using whatever information they have.

And you have no idea what type of evidence they might have. Direct or indirect...they have our final save file along with every bit of information that Civ saves...and it saves a TON of data. Stop fishing for information and just accept that the staff is doing everything they can to protect the integrity of our competition and will do their best not to exclude any game unfairly.

What more do you want? I would think taht if someone HAD received an email saying their game was being excluded and it was a false positive they would have said something in this thread by now.

 

[Ribannah]

Thus, when there are more sessions, there are more opportunities for replaying.

That is not true. The opportunities to play over are increased with the number of saves, not with the number of loads. Nor, for that matter, can you conclude something with regard to the conditional probability that playing over has actually been done. There are numerous ways to play over without increasing the registered number of loads.

This leads them to needing to check each one closer, using whatever information they have.

Indeed, and that introduces a bias (technical term, it does not follow that the staff is biased).

Just accept that the staff is doing everything they can to protect the integrity of our competition and will do their best not to exclude any game unfairly.

As I have stated numerous times, I am fully accepting that. We are truly fortunate with the staff we have. That does not mean that there is no room for improvement, or that by definition other people's insights have no value and should be ignored.

I would think taht if someone HAD received an email saying their game was being excluded and it was a false positive they would have said something in this thread by now.

Perhaps they have.

Now, since I do not want to dominate this discussion, I'll sit down and listen for a while.