Cannot load multiplayer pitboss game with hybrid turn mode when two humans are at war

dmnd

Chieftain
Joined
Jun 13, 2014
Messages
4
I'm running a 12-player pitboss game of BNW on a dedicated server. Things have been relatively smooth and we're on turn 53 after about 6 weeks of play. Unfortunately, two nights ago a power outage took down the dedicated server.

While loading an autosave initially appears to work, there is a small bug that prevents us from continuing the game. To explain the bug, please bear with me as I provide context.

We are using hybrid turn mode. There are two pairs of players that are at war. At the beginning of a turn, the 8 peaceful players and India (the first warring player) all become active. The 3 remaining warring players are blocked from taking a turn. Once India completes his turn, he becomes finished and the next warring player becomes active. This has been working fine since turn 35 when war begun.

But when trying to load the game, the turn appears to begin as usual. But there is one small thing wrong. When the game loads, only the 8 warring players become active. India starts off blocked! :(

Since India and the other 3 warring civs are all blocked, none of them ever transition to active, so the turn and game are deadlocked. We tried having every other player click end turn, but we get stuck in a state with 8 finished peaceful players and the 4 warring players are still blocked.

In the past we've successfully been able to load from an autosave. The issue only occurs when two human players are at war and hence in consecutive turn mode. I verified that loading the same game at turn 34 (when no human players were at war) works properly - all players are active.

I've put a lot of work into organizing this game, and all 12 players have been at it for 6 weeks at roughly one turn per day. I'm willing to follow whatever steps are necessary to get this game working again (e.g. debug builds, debugging it myself if you send me symbols, etc), so feel free to ask anything of me if it will help. Thanks in advance!

In the above I used the terms active, blocked, and finished to describe the turn state of a warring player. I've defined them below for clarity:

  1. Blocked: player cannot issue orders yet, but can scroll around the map. To exit this state, a warring player earlier in the turn order must transition to state "finished" by finishing their turn. Only then will a "blocked" player become "active". Icon in lobby is undarkened and does not flash.
  2. Active: player can issue orders, build buildings, etc. Player presses "end turn" to exit this state. Icon flashes in the lobby.
  3. Finished: player has already completed their turn this round by clicking "end turn". Player cannot issue more orders. Icon is darkened in the lobby.

I've attached 2 saves from this game - one at turn 34, which works fine because no players are at war. The other is turn 53 which cannot be loaded because of this issue I described above.
 

Attachments

  • AutoSave_0034 BC-2640.Civ5Save
    879.1 KB · Views: 269
  • AutoSave_0053 BC-1880.Civ5Save
    984.5 KB · Views: 303
Since writing that post I discovered a lot the source code is in the SDK. Awesome!

So I debugged things and worked out that for whatever reason my save game has turn state set to inactive for the warring players. I tried to reproduce this with another game, but that one seems to generate correct savegames that load up the lobby correctly.

Anyway, since I can build my own DLL, I added in an override to the deserialization code that enforces that the first human warring player has active turn state.

If anyone else runs into this, you can find the fix here: https://github.com/dmnd/CvGameCoreSource/commit/1ba3dfb1b846d64089f82e397ad5069419dcecfb
 
It turns out that my fix didn't work. After applying it, India joined the game and could complete his turn as expected. But once he hit end turn, the other warring players remained inactive, instead of the next one becoming active. Looks like there is more broken state than just the first player being set to inactive.
 
I looked into this a bit more and have narrowed down the problem somewhat. Once India takes his turn, the server reveives the NetTurnComplete message:

Code:
[304115.781] Net RECV (3) :NetTurnComplete : Turn Complete, 3, 5/13
[304154.093] Net RECV (3) :NetCloseConnection : m_iPlayerID=3
[304154.093] Net SEND (3): size=32: NetCloseConnection : m_iPlayerID=3
[304154.093] DBG: ConnectionClosed Player(3)

Shortly after that the player disconnects. But contrasting that to a test game that seems to be working:

Code:
[883900.125] Net RECV (0) :NetTurnComplete : Turn Complete, 0, 1/3
[883900.171] DBG: UpdateMoves() : player.setEndTurn(true) called for player 0 syzygy
[883900.281] DBG: UpdateMoves() : player 0 syzygy running AutoMission (NO_MISSIONAI) on Spearman id=24576
[883900.281] DBG: UpdateMoves() : player 0 syzygy running AutoMission (NO_MISSIONAI) on Scout id=32769
[883900.281] DBG: CheckPlayerTurnDeactivate() : auto-moves complete for syzygy
[883900.281] DBG: changeNumGameTurnActive(-1) m_iNumActive=0 : setTurnActive() for player syzygy
[883900.281] DBG: changeNumGameTurnActive(1) m_iNumActive=1 : setTurnActive() for player 1 Narooke
[884667.796] Net RECV (0) :NetCloseConnection : m_iPlayerID=0
[884667.796] Net SEND (0): size=32: NetCloseConnection : m_iPlayerID=0
[884667.796] DBG: ConnectionClosed Player(0)

You can see that the `setEndTurn(true)` line isn't being called for India in the buggy game. Digging into that, it seems that somehow India has `player.hasBusyUnitOrCity()` returning true. Additionally it seems like `player.hasBusyUnitUpdatesRemaining()` returns true also because that next log message isn't printed.

Code:
// Code from CvGame.cpp:8315
// KWG: This code should go into CheckPlayerTurnDeactivate
if(!player.isEndTurn() && gDLL->HasReceivedTurnComplete(player.GetID()) && player.isHuman() /* && (isNetworkMultiPlayer() || (!isNetworkMultiPlayer() && player.GetID() != getActivePlayer())) */)
{
	if(!player.hasBusyUnitOrCity())
	{
		player.setEndTurn(true);
		if(player.isEndTurn())
		{//If the player's turn ended, indicate it in the log.  We only do so when the end turn state has changed to prevent useless log spamming in multiplayer. 
			NET_MESSAGE_DEBUG_OSTR_ALWAYS("UpdateMoves() : player.setEndTurn(true) called for player " << player.GetID() << " " << player.getName());
		}
	}
	else
	{
		if(!player.hasBusyUnitUpdatesRemaining())
		{
			NET_MESSAGE_DEBUG_OSTR_ALWAYS("Received turn complete for player "  << player.GetID() << " " << player.getName() << " but there is a busy unit. Forcing the turn to advance");
			player.setEndTurn(true);
		}
	}
}

So my theory now is that some kind of state to do with India's units is also wrong.
 
I ran into a bug that sounds very much like the one you reported, in which players get erroneously set as unable to submit turns. Here is my bug post:

http://forums.civfanatics.com/forumdisplay.php?f=390

Did you find a solution? I hate to see an 8 player game with so much time devoted to it come to an end because of a bug.

Thanks! John
 
We ran into the same issue with our game.

Played some rounds, 2 player started war, some rounds just worked out fine, but then the server crashed and we had to load a savegame. Now everybody can end their rounds, but the 2 players on war cannot. Both can log in, on the player-list they are not greyed out, so their turn is missing, but they don't have a "End Round" button, but a "Waiting for other players" button.
We even waited to have our turn timer of 2 days run out, but that did not help, the timer stays at 0 sec and the round does not end.

Did anybody find a solution to this?

We really did invest a lot time and money in setting this game up, and having to cancel it now because of of this bug, would be really disappointing.
 
We ran into the same issue with our game.

Played some rounds, 2 player started war, some rounds just worked out fine, but then the server crashed and we had to load a savegame. Now everybody can end their rounds, but the 2 players on war cannot. Both can log in, on the player-list they are not greyed out, so their turn is missing, but they don't have a "End Round" button, but a "Waiting for other players" button.
We even waited to have our turn timer of 2 days run out, but that did not help, the timer stays at 0 sec and the round does not end.

Did anybody find a solution to this?

We really did invest a lot time and money in setting this game up, and having to cancel it now because of of this bug, would be really disappointing.

If there is a solution, you will most likely not be able to continue your game, as the solution would be in the form of a modified CvGameCore DLL with at least one of hte classes using modified FStream Read() and Write() functions, which would break savegame compatibility.

I honestly don't even know if it's worth finding a fix for, either, since hybrid mode is already implemented in quite a haphazard way that is a bit of nightmare to follow. I know this because I've been messing around with a complete rewrite of Hybrid mode that effectively turns hybrid into just a modified version of simultaneous instead of a sequential-simultaneous amalgamation (players who are not at war with one another can play their turns simultaneously, so warring humans are effectively simultaneous players that get activated at different times rather instead of being sequential players); the only sequential players in Hybrid will be AI players, which is the same way simultaneous works.
 
Top Bottom