Multiplayer Woes

[Aevum]

For those of you wishing to find What caused the OOS, the easiest thing is to compare the 2 OOS Logs from each computer using
WinMerge
http://winmerge.org/downloads/
It is a Free program, that lets you load say, 2 text files and will highlight the differences in red and allow you to use a great *find next difference button.
I hope no one was reading thru them by hand lol.

 

[AIAndy]

For those of you wishing to find What caused the OOS, the easiest thing is to compare the 2 OOS Logs from each computer using
WinMerge
http://winmerge.org/downloads/
It is a Free program, that lets you load say, 2 text files and will highlight the differences in red and allow you to use a great *find next difference button.
I hope no one was reading thru them by hand lol.

If you have TortoiseSVN installed, it contains TortoiseMerge which does something similar.

 

[Sn0bb]

Shouldn't this be like stickied or something?
Impossible to find it down the line. I atleast was so confused if there is any support for MP at all in this mod.
Or maybe a new thread could be created where new players may get some help on settings and features to avoid OoS errors.

 

[AIAndy]

Shouldn't this be like stickied or something?
Impossible to find it down the line. I atleast was so confused if there is any support for MP at all in this mod.
Or maybe a new thread could be created where new players may get some help on settings and features to avoid OoS errors.

I am always happy to check and fix OoS errors but I need the OoS logs.
As C2C is developed constantly new causes for OoS can be introduced or some rarer combination of options still has some.

 

[Camber]

Edit: nevermind, I found that I had an older version installed.

 

[BlueGenie]

Trying to help out here so we can play MP games sometimes in the future so we started up a 3 man game.
We are using the Advanced Civics but have not done any major changes on this MP setup.

Three issues cropped up straight away.
First of no one got the Clubman initial free unit. A Stoner Thrower and the BoHSS was all anyone got.

The second was that anyone pressing end turn the first turn got frozen out of the game, I.e the game froze completely. The others got the usual "waiting to hear from player xxx". This we can't give anything but the save game as no logs nor mini-dumps were created. If it's duplicate-able all that's needed is for someone, anyone, to end their turn in the save game I'm supplying. Don't even need to move any units, just Shift-Enter. Oh, it's set with folder name Caveman2CosmosMP.

So we restarted and loaded up the save again, to find the third issue. On entering the game we immediately got an OOS, this before anyone had made any moves.
I got the logs but the only thing different from the player with different value is the Soren Rand number. Two players got this on line 9:
Next Soren Rand Value: 6863
The last player got this:
Next Soren Rand Value: 5634
Everything else matches. (thanks for the comment about TortoiseSVN having a diff option).

As I'm not sure that will help I'm supplying the save game if someone gets a chance to check this out though I'm not sure that will help either.

This might give a clue though.
The Network file; I tried starting the game via Pitboss to join on my own to double check the frozen game on end turn (which it did too) and found that the network log kept counting up turns without anyone even logged in yet.
First it logged this:

Spoiler :

[1395339.882] DBG: Resetting the portal! (m_bConnectComplete=false)
[1395384.061] DBG: Resetting the portal! (m_bConnectComplete=false)
[1395384.326] WRN: Disconnecting from entire Peer Network!
[1395528.300] DBG: HOSTING a MULTIPLAYER, SAVED, INTERNET game!
[1395528.300] WRN: Game INet mode doesn't match player INet mode - DirectIP mode?
[1395529.298] DBG: Adding active connection-level player info for NetID 0
[1395529.298] DBG: Setting NetID 0 PRIVATE address to 192.168.0.24:2056
[1395529.314] DBG: Adding NetID 0 to Portal!
[1395529.314] DBG: Got NetworkReady from NetID 0
[1395529.314] WRN: Not responding to NetID 0 NetworkReady (state == INIT_SENT_READY)
[1395529.314] DBG: Checking if we're connect complete! PeersReady=yes :: AlreadyComplete=no
[1395529.314] DBG: Got our last outstanding network ready message! Sending Version Verification! (m_bConnectComplete=true)
[1395529.454] DBG: Got Verify Version message from NetID 0!
[1395529.454] DBG: Sending Pitboss NetID to NetID 0
[1395529.454] ERR: We are broadcasting a public host, but have not initialized public host tools!
[1395529.454] DBG: Received pitboss info message from NetID 0
[1395535.803] DBG: Sending sync start message everyone
[1395535.928] DBG: Got a sync start from NetID 0
[1395535.928] DBG: SYNCLOG: Hot turn set to 1
[1395535.928] DBG: SYNCLOG: STARTING TURN 1
[1395536.209] DBG: SYNCLOG: Received EndTurn for Player -1 (NetID 0) - Checking in for Turn 1
[1395536.209] DBG: SYNCLOG: All Checked In for Turn 1
[1395536.209] DBG: SYNCLOG: Hot turn has been reset on turn 1
[1395536.209] DBG: SYNCLOG: STARTING TURN 2
[1395536.318] DBG: SYNCLOG: Received EndTurn for Player -1 (NetID 0) - Checking in for Turn 2
[1395536.505] DBG: SYNCLOG: All Checked In for Turn 2

Then it kept going via:

Spoiler :

[1395536.505] DBG: SYNCLOG: STARTING TURN 3
[1395536.568] DBG: SYNCLOG: Received EndTurn for Player -1 (NetID 0) - Checking in for Turn 3
[1395536.817] DBG: SYNCLOG: All Checked In for Turn 3
[1395536.817] DBG: SYNCLOG: STARTING TURN 4
[1395536.880] DBG: SYNCLOG: Received EndTurn for Player -1 (NetID 0) - Checking in for Turn 4
[1395537.114] DBG: SYNCLOG: All Checked In for Turn 4
[1395537.114] DBG: SYNCLOG: STARTING TURN 5
[1395537.176] DBG: SYNCLOG: Received EndTurn for Player -1 (NetID 0) - Checking in for Turn 5
[1395537.426] DBG: SYNCLOG: All Checked In for Turn 5

And kept going until I shut Pitboss down on, it seemed, turn 967.
Interestingly though when I joined (around supposedly turn 700 or so) I was still on turn 0 and the turns did not move while I was logged in. Not sure if they did on pitboss or on the network log while I was in the game though.

Anyway I was thinking that the auto-end turn thing on no decisions code could have something to do with the freeze of the game (and the pitboss counting down turns). We did not have that option activated though, we made sure the second time we tried by everyone going into BUG and making sure it was not checked.

Cheers

 

[AIAndy]

Trying to help out here so we can play MP games sometimes in the future so we started up a 3 man game.
We are using the Advanced Civics but have not done any major changes on this MP setup.

Three issues cropped up straight away.
First of no one got the Clubman initial free unit. A Stoner Thrower and the BoHSS was all anyone got.

The second was that anyone pressing end turn the first turn got frozen out of the game, I.e the game froze completely. The others got the usual "waiting to hear from player xxx". This we can't give anything but the save game as no logs nor mini-dumps were created. If it's duplicate-able all that's needed is for someone, anyone, to end their turn in the save game I'm supplying. Don't even need to move any units, just Shift-Enter. Oh, it's set with folder name Caveman2CosmosMP.

So we restarted and loaded up the save again, to find the third issue. On entering the game we immediately got an OOS, this before anyone had made any moves.
I got the logs but the only thing different from the player with different value is the Soren Rand number. Two players got this on line 9:
Next Soren Rand Value: 6863
The last player got this:
Next Soren Rand Value: 5634
Everything else matches. (thanks for the comment about TortoiseSVN having a diff option).

The important log in that case is the random log as it records all the random numbers that are generated and what they are generated for so you can see at which point they start to differ. Often it means that some local code used the synced RNG.

As I'm not sure that will help I'm supplying the save game if someone gets a chance to check this out though I'm not sure that will help either.

This might give a clue though.
The Network file; I tried starting the game via Pitboss to join on my own to double check the frozen game on end turn (which it did too) and found that the network log kept counting up turns without anyone even logged in yet.
First it logged this:

Spoiler :

[1395339.882] DBG: Resetting the portal! (m_bConnectComplete=false)
[1395384.061] DBG: Resetting the portal! (m_bConnectComplete=false)
[1395384.326] WRN: Disconnecting from entire Peer Network!
[1395528.300] DBG: HOSTING a MULTIPLAYER, SAVED, INTERNET game!
[1395528.300] WRN: Game INet mode doesn't match player INet mode - DirectIP mode?
[1395529.298] DBG: Adding active connection-level player info for NetID 0
[1395529.298] DBG: Setting NetID 0 PRIVATE address to 192.168.0.24:2056
[1395529.314] DBG: Adding NetID 0 to Portal!
[1395529.314] DBG: Got NetworkReady from NetID 0
[1395529.314] WRN: Not responding to NetID 0 NetworkReady (state == INIT_SENT_READY)
[1395529.314] DBG: Checking if we're connect complete! PeersReady=yes :: AlreadyComplete=no
[1395529.314] DBG: Got our last outstanding network ready message! Sending Version Verification! (m_bConnectComplete=true)
[1395529.454] DBG: Got Verify Version message from NetID 0!
[1395529.454] DBG: Sending Pitboss NetID to NetID 0
[1395529.454] ERR: We are broadcasting a public host, but have not initialized public host tools!
[1395529.454] DBG: Received pitboss info message from NetID 0
[1395535.803] DBG: Sending sync start message everyone
[1395535.928] DBG: Got a sync start from NetID 0
[1395535.928] DBG: SYNCLOG: Hot turn set to 1
[1395535.928] DBG: SYNCLOG: STARTING TURN 1
[1395536.209] DBG: SYNCLOG: Received EndTurn for Player -1 (NetID 0) - Checking in for Turn 1
[1395536.209] DBG: SYNCLOG: All Checked In for Turn 1
[1395536.209] DBG: SYNCLOG: Hot turn has been reset on turn 1
[1395536.209] DBG: SYNCLOG: STARTING TURN 2
[1395536.318] DBG: SYNCLOG: Received EndTurn for Player -1 (NetID 0) - Checking in for Turn 2
[1395536.505] DBG: SYNCLOG: All Checked In for Turn 2

Then it kept going via:

Spoiler :

[1395536.505] DBG: SYNCLOG: STARTING TURN 3
[1395536.568] DBG: SYNCLOG: Received EndTurn for Player -1 (NetID 0) - Checking in for Turn 3
[1395536.817] DBG: SYNCLOG: All Checked In for Turn 3
[1395536.817] DBG: SYNCLOG: STARTING TURN 4
[1395536.880] DBG: SYNCLOG: Received EndTurn for Player -1 (NetID 0) - Checking in for Turn 4
[1395537.114] DBG: SYNCLOG: All Checked In for Turn 4
[1395537.114] DBG: SYNCLOG: STARTING TURN 5
[1395537.176] DBG: SYNCLOG: Received EndTurn for Player -1 (NetID 0) - Checking in for Turn 5
[1395537.426] DBG: SYNCLOG: All Checked In for Turn 5

And kept going until I shut Pitboss down on, it seemed, turn 967.
Interestingly though when I joined (around supposedly turn 700 or so) I was still on turn 0 and the turns did not move while I was logged in. Not sure if they did on pitboss or on the network log while I was in the game though.

Anyway I was thinking that the auto-end turn thing on no decisions code could have something to do with the freeze of the game (and the pitboss counting down turns). We did not have that option activated though, we made sure the second time we tried by everyone going into BUG and making sure it was not checked.

Cheers

I have to say that I am surprised that Pitboss worked at all as I have not yet done any work on getting it compatible nor tested it at all so there are likely some issues.
There are especially some codes that do not know how to deal with the player number that Pitboss itself uses and do not expect to get that value in function calls.

Your guess that the auto-end turn plays a role here is possible, especially considering that the BUG that runs on the Pitboss is checked for that option and might return true even if all of you do not have it activated.

If the desire is there, I can invest some time in making C2C Pitboss compatible (and I guess it is a good way to play C2C MP) but until then Direct IP is the recommended method for C2C MP.

 

[BlueGenie]

We didn't have that active. *sighs* Will do a retest tomorrow then.

The Pitboss started up with C2C and I could load the game. When I joined I was OOS right away though.

Cheers

 

[BlueGenie]

*double post for getting the unread icon to lit up again*

Did anyone know what the reason for the game freezing on pressing end of turn was, or was able to figure it out?

We did not play with pitboss, that was just when I tested the freeze on my own. We did a Direct IP connection as that is how we play.

Cheers

 

[AIAndy]

*double post for getting the unread icon to lit up again*

Did anyone know what the reason for the game freezing on pressing end of turn was, or was able to figure it out?

We did not play with pitboss, that was just when I tested the freeze on my own. We did a Direct IP connection as that is how we play.

Cheers

Ah, ok.
I'll give it a try this evening to see if I can reproduce it with two instances of C2C running on my PC.

 

[AIAndy]

Ah, ok.
I'll give it a try this evening to see if I can reproduce it with two instances of C2C running on my PC.

I have tested that on my local PC now.
With two instances of C2C running I set up a multiplayer DirectIP game and played 5 turns and I got neither an OOS in that time nor a crash or freeze.

 

[BlueGenie]

It's always easier on a single computer I think. Maybe. Because local rand is the same? But this was about the freeze.
*ponders* Test starting the same game but loading it in via pitboss and connect to it.
For me that generated the same game freeze as when we set up the MP game. And we did get a freeze whenever anyone tried ending their turn.

Could the Advanced Civics have anything to do with it freezing? We did try to run with those after all. Maybe we should attempt it without those?

Cheers

 

[AIAndy]

It's always easier on a single computer I think. Maybe. Because local rand is the same? But this was about the freeze.

The local rand is not the same as it is process specific and the two Civ4 instances run in separate processes.
The difference is of course that the messages between the two processes arrive nearly instantly. And all BUG options were set to the same value in both instances.

*ponders* Test starting the same game but loading it in via pitboss and connect to it.
For me that generated the same game freeze as when we set up the MP game. And we did get a freeze whenever anyone tried ending their turn.

Could the Advanced Civics have anything to do with it freezing? We did try to run with those after all. Maybe we should attempt it without those?

Cheers

I don't have those installed so there is a chance that that is the reason. But it might also be some game option or BUG option that you have set and I don't.

 

[BlueGenie]

Got a chance to do some quick tests on Rev3202. First we tested with cleared UserSettings. Worked fine. No freeze.
Then we tried with the same UserSettings we had last time.
On ending turn the game froze, again.

I'm uploading the UserSettings we used so you might have a chance to check what it could be that breaks ending turn. Hopefully anyway, as one should be able to use most if not all the various user settings in BUG in MP too, right?

We then tested with adding the Advanced Civics with cleared user settings too. Worked like a charm, no problems with freeze on End Turn.

We didn't play far enough for OOS's, and we didn't get any OOS's when reloading (which we did several times). Hopefully it was this tiles revealed/not revealed issue that made that happen last time.

Cheers

 

[AIAndy]

I'm uploading the UserSettings we used so you might have a chance to check what it could be that breaks ending turn. Hopefully anyway, as one should be able to use most if not all the various user settings in BUG in MP too, right?

Yes, but not all are properly synced between the players yet which means that it is better to make sure you have set them to the same value. Worst case should be on OOS though.

 

[BlueGenie]

We all had the same UserSettings files. I created a game folder (Caveman2CosmosMP folder) with all user settings adjusted and set up already, RAR'ed it, uploaded it to TeamSpeak, and they others took it down from there and just extracted it into their Mods folder. So we all had the exact same files in C2C and the same options in BUG/Domestic Advisor.
*ponders* Could it be the CustDomAdv.ini file that messes it up? I'd not think so, but one never knows.

Cheers

 

[AIAndy]

We all had the same UserSettings files. I created a game folder (Caveman2CosmosMP folder) with all user settings adjusted and set up already, RAR'ed it, uploaded it to TeamSpeak, and they others took it down from there and just extracted it into their Mods folder. So we all had the exact same files in C2C and the same options in BUG/Domestic Advisor.
*ponders* Could it be the CustDomAdv.ini file that messes it up? I'd not think so, but one never knows.

Cheers

I could reproduce it with the settings you provided and the root cause is definitely the Minimize AI Turn Times option, which is not one you should have active for simultanuous turns anyway.
Looking at the code it seems like as soon as one player ends his turn, it starts to repeatedly execute a part of code that would do something if it was the AI's turn but the AI has ended its turn far earlier so effectively nothing happens except freezing that computer.
I'll just make that option do nothing in simultaneous turn games. In the meantime just deactivate that option.

 

[BlueGenie]

*smile* Thanks AIAndy. You rock. We'll hopefully get a chance to play a bit this weekend so I'll post here if/when we run into any OOS problems.

Cheers

 

[BlueGenie]

OOS on turn one.

Logs attached.

I don't know if it's the logs you need. We have all the logs enabled so just let me know the basic name/naming on any logs you'd need. I don't see anything differently with the RandomLogger files that could assist in debugging any OOS's.

Again we are all having the exact same files all the way down to all UserSettings files.

Cheers

EDIT: On reloading the game we entered in OOS. I'm adding all the logs that were updated for us, logs from Player 0 and Player 2 as Player 1 had the same sync value as Player 0.

Cheers

 

[AIAndy]

OOS on turn one.

Logs attached.

I don't know if it's the logs you need. We have all the logs enabled so just let me know the basic name/naming on any logs you'd need. I don't see anything differently with the RandomLogger files that could assist in debugging any OOS's.

Again we are all having the exact same files all the way down to all UserSettings files.

Cheers

That is weird. Player 1 has a different rand seed at the end but the sequence of random numbers generated was exactly the same.
Was there any Random Logger - Set 1?