Multiplayer Woes

[BlueGenie]

Right, will keep that in mind and not yell at you for not being around. *wink*

We'll try and see what happens though, see if this fix helps a lot, or just a little. Will go the SVN way then too I suppose. I'll just have to compile it for the others and send it to them. *grin*

Cheers

 

[Koshling]

Right, will keep that in mind and not yell at you for not being around. *wink*

We'll try and see what happens though, see if this fix helps a lot, or just a little. Will go the SVN way then too I suppose. I'll just have to compile it for the others and send it to them. *grin*

Cheers

No need for you to compile anything, just get the latest DLL from the SVN and provide it to your friends.

 

[AIAndy]

No need for you to compile anything, just get the latest DLL from the SVN and provide it to your friends.

@Koshling: Your PM storage is full btw.
The details in answer to your last PM: The different log entries were from async calls to the ai routine for choosing a tech which were for advising the player what techs to take. That was also at least one major OOS reason because it generated a synced random number to decide if it should do beelining (without checking if the bAsync bool was set).
I have fixed that by breaking the loop in those async situations (I guess we don't want it to advise beelining for human players anyway).
Btw, I saw in that code that it forces beelining for religions in a separate loop. Is that intended with current C2C?

 

[BlueGenie]

No need for you to compile anything, just get the latest DLL from the SVN and provide it to your friends.

I know, I meant compile into a .rar file with the UserSettings set as well as the SVN version (and any files we want changed in our games).

Cheers

 

[Koshling]

@Koshling: Your PM storage is full btw.
The details in answer to your last PM: The different log entries were from async calls to the ai routine for choosing a tech which were for advising the player what techs to take. That was also at least one major OOS reason because it generated a synced random number to decide if it should do beelining (without checking if the bAsync bool was set).
I have fixed that by breaking the loop in those async situations (I guess we don't want it to advise beelining for human players anyway).
Btw, I saw in that code that it forces beelining for religions in a separate loop. Is that intended with current C2C?

It pre-dates the recent dead-end-techs-for-religions changes, so it may no longer be appropriate. I haven't revisited it since those changes were made.

 

[Thunderbrd]

I've also identified a number of OOS bugs I believe I can enlist some help from the present team to resolve. The first, an OOS on the first round, is a bit mysterious still... that was resolvable, however, by reloading.

The Second OOS bug I found was in the new leader traits. Washington's -15% commerce when played as a member of a team, tries to apply its penalty on its team member on the system that is running him, while the other team member's computer doesn't show the reduction and that causes an OOS every round you try to play with him in this fashion. (HOW this is happening is mysterious to me and beyond my programming skill but I could at least identify the problem itself.)

The third OOS bug we ran across was repetitive but strangely correctable. Any time a unit was used to modify the city in any way (add a building, butcher, join as a specialist) it would cause the other player to have to click on the offending city to resolve an emerging OOS. I suspect this error may be due to some coding changes in CvCity that aren't gelling all that well with the previous python mission codings involving these.

Can't say I know how to fix any of these problems but I figured I'd report them here.

I'll have some more to discuss tonight. I drummed up another unknown OOS issue last night and have the logs for it as well.

 

[Koshling]

I've also identified a number of OOS bugs I believe I can enlist some help from the present team to resolve. The first, an OOS on the first round, is a bit mysterious still... that was resolvable, however, by reloading.

The Second OOS bug I found was in the new leader traits. Washington's -15% commerce when played as a member of a team, tries to apply its penalty on its team member on the system that is running him, while the other team member's computer doesn't show the reduction and that causes an OOS every round you try to play with him in this fashion. (HOW this is happening is mysterious to me and beyond my programming skill but I could at least identify the problem itself.)

The third OOS bug we ran across was repetitive but strangely correctable. Any time a unit was used to modify the city in any way (add a building, butcher, join as a specialist) it would cause the other player to have to click on the offending city to resolve an emerging OOS. I suspect this error may be due to some coding changes in CvCity that aren't gelling all that well with the previous python mission codings involving these.

Can't say I know how to fix any of these problems but I figured I'd report them here.

I'll have some more to discuss tonight. I drummed up another unknown OOS issue last night and have the logs for it as well.

This is just a wild guess, but I wonder if the changes I made to city commerce and maintenance calculations, to just set a dirty flag rather than force recalculate every time (and then recalculate when the value is requested if it's dirty) might cause this. Clicking on the city would likely reheat all the values and force a recalculation. The fox would probably be to the end value out of the sun checksum or something (at
least if it was marked as dirty). Alternatively you could recalculate any dirty values at the point of the sync checksum calculation, which would happen automatically if direct access to the underlying member variable was replaced by a call t the method to get it.

Note - I'm saying all this without looking at any code, and with very little knowledge of the MP sync system, so I may be on entirely the wrong track. It's just a suggestion.

 

[Thunderbrd]

It all sounds fairly accurate but I'm not following you very well as to what you're actually suggesting. Honestly, the whole reference to the term 'dirty', and the function of a checksum in the first place has never been clear for me. Where these are mentioned, my mind glosses over it in search of what does make sense. That said, the effect of what's happening and what I've seen you've changed in the code does seem to add up to that sort of conclusion.

 

[AIAndy]

It all sounds fairly accurate but I'm not following you very well as to what you're actually suggesting. Honestly, the whole reference to the term 'dirty', and the function of a checksum in the first place has never been clear for me. Where these are mentioned, my mind glosses over it in search of what does make sense. That said, the effect of what's happening and what I've seen you've changed in the code does seem to add up to that sort of conclusion.

You use checksums to find out if a big bunch of data is still the same without transferring all of it. Instead you calculate a checksum from the data and then only transfer (or store) that checksum.
For the multiplayer synchronization that is done in CvGame::calculateSyncChecksum.
As far as I can see it uses the normal methods to access game information so it should not cause any problems that some lazy evaluation is now used.

What the dirty flags do is storing if a certain information has to be recalculated. If they are always properly set if the base information changes, then that should not really cause a desync.

 

[Thunderbrd]

Ok. Those bits are starting to at least make recognizable sense now. At this point, I should go take a look at calculatSyncChecksum and see if it makes any sense to try to read through it.

One thing I've never understood, too... if the game is so good at realizing when its out of sync, why can't it also, once an oos is detected, just go ahead and re-sync? Doesn't it select the initiating computer to be the primary operator (host) anyhow? Why shouldn't it simply self-compare the differences and align itself to whatever game data the host believes should exist? Why must we completely exit the game and restart EVERY time this happens? (having one computer leave and come back takes just as long and causes a worse OOS every time too.) Can't something be done to change this process???

 

[AIAndy]

Ok. Those bits are starting to at least make recognizable sense now. At this point, I should go take a look at calculatSyncChecksum and see if it makes any sense to try to read through it.

One thing I've never understood, too... if the game is so good at realizing when its out of sync, why can't it also, once an oos is detected, just go ahead and re-sync? Doesn't it select the initiating computer to be the primary operator (host) anyhow? Why shouldn't it simply self-compare the differences and align itself to whatever game data the host believes should exist? Why must we completely exit the game and restart EVERY time this happens? (having one computer leave and come back takes just as long and causes a worse OOS every time too.) Can't something be done to change this process???

It often only recognizes the snowball effects of the desync as the sync checksum does not include a lot of the synced data (especially not the part of it that is in Python.
But yes, it would be possible to write resyncing code. It is just a matter of effort. You'd either need to add checksum methods to every game object to recognize which object exactly has desynced (this could be done similar to how I implemented it for the info classes and you could also use that for some more detailed sync checks to catch OOS earlier) or you'd need to serialize and send a full game state. In both cases you can use the savegame serialization methods so it would actually not be that huge of an effort.

 

[Thunderbrd]

I take it such a system you say could be developed would amount to - "Oh, we have an OOS, ok, hit the button to resync on the computer we want to align ourselves to" and in a few moments, we're back in action (not eliminating or completely ignoring the OOS but making recovery from one much easier?)

I may have to learn yet more about how both checksum and serialization works - such a setup sounds extraordinarily handy and would help us immensely for our multiplayers to stay tuned when a small change in code starts inviting multitudes of difficult to debug OOS problems. It would at least buy us time to fix and allow the players to keep playing despite the problem.

 

[AIAndy]

I take it such a system you say could be developed would amount to - "Oh, we have an OOS, ok, hit the button to resync on the computer we want to align ourselves to" and in a few moments, we're back in action (not eliminating or completely ignoring the OOS but making recovery from one much easier?)

I may have to learn yet more about how both checksum and serialization works - such a setup sounds extraordinarily handy and would help us immensely for our multiplayers to stay tuned when a small change in code starts inviting multitudes of difficult to debug OOS problems. It would at least buy us time to fix and allow the players to keep playing despite the problem.

It could be automatic also. If you recognize an OOS, identify the desynced game objects and then send them in a message from the host to all players using the serialization methods for savegames (or don't identify and just send all).

 

[Thunderbrd]

I would so greatly like to see something like that implemented.

When you say 'send a message'... how does that look in code? You were saying earlier that such messages were sent to keep things in sync initially as well. Where is it doing this so I can see an example?

 

[AIAndy]

I would so greatly like to see something like that implemented.

When you say 'send a message'... how does that look in code? You were saying earlier that such messages were sent to keep things in sync initially as well. Where is it doing this so I can see an example?

CvMessageData contains the actual messages.

 

[Dancing Hoskuld]

I assume you have seen this thread about allowing the tech screen in multi player mode.

Edit of course we don't have a CvEventManager so we would need to persuade the BUG team that it would be a good idea.

 

[AIAndy]

I assume you have seen this thread about allowing the tech screen in multi player mode.

Edit of course we don't have a CvEventManager so we would need to persuade the BUG team that it would be a good idea.

What exactly do you want to have working in multiplayer?

 

[Dancing Hoskuld]

It just means that the tech screen works in multi player so that like in single player you can go to tech ie stack up a list of techs to study rather than having to select a tech each time you get one.

 

[Thunderbrd]

I've never had problems using the tech screen like that in multiplayer before... do we have a bug there?

 

[BJ69]

Hi, been trying to get a mutliplayer game of this mod working with a few friends of mine. However its not been going well at all. I was quite careful to make sure everyone had exactly the same copy of v26 with the patch. Tried 5 test games now and only in 1 of them did we manage to get past the first turn (to turn 12), the pattern is normally from the start of the game almost constant oos errors occur (though in that one to turn 12 game, it did stop for around 2 minutes), utill it either kicks the other player out of the game (we were simply trying 1v1 duel maps to see if it would work) or a crash to desktop. We've all tried clearing our cache's re-installing the mod, everything. I can also confirm that its not just our copies of Civ4 itself, as we've manged to play other mods without any oos errors at all. The only thing that's noticeably different between me and the two others i've been testing it with different is I have an older copy of civ4 complete they have the steam version.

Terrible shame as we are now being forced to use mods less rich in content, and once you've played caveman age its hard to do without it.

This is a collection of the log filse from that one 'successful' game.
http://www.sendspace.com/file/mf6f7w