Non-host player calling InitUnit() and kill() lua functions causing re-sync in multiplayer game

LyarSoft

Chieftain
Joined
Aug 18, 2022
Messages
10
When playing with a mod giving player a button to transform a unit into other civil's unique unit (by calling lua function InitUnit() and kill() to replace the original one) in multiplayer game, the game automatically reloads whenever a non-host player press that button and end his turn, and the action is canceled, the. Log says "DBG: Received a sync request for unit () which does not exist locally!". These 2 methods seems to be calling CvLuaUnit::lKill() and CvLuaPlayer::lInitUnit() in dll, but upgrading a unit with CvUnit::DoUpgrade() which also kill and spawn a new unit of next era by calling the backend function of CvLuaUnit::lKill() and CvLuaPlayer::lInitUnit() in class CvUnit and CvPlayer dose not trigger resync in multiplayer. Does anyone know how to fix this?
 

LyarSoft

Chieftain
Joined
Aug 18, 2022
Messages
10
Well it seems that all the unit operations done in an "official" way are sent to the class CvDllNetMessageHandler of non-host players and sent by some methods in macro defined object gDLL or other dll interfaces on host. By doing this, any unit operations (and buildings maybe) are sent to all the players, therefore the operations called by non-host player can be correctly executed by host and recorded in syncronizing data structure.
My solution is to use SelectionListGameNetMessage(GameMessageTypes.GAMEMESSAGE_DO_COMMAND,...) (It's actually calling CvGame::selectionListGameNetMessage in DLL) instead of InitUnit() and kill() in lua scripts and define some new command enums such as COMMAND_KILL and COMMAND_INIT_UNIT_INPLACE (to spawning new unit at the position of an existing unit), then passing through kill() and a new method for inplace spawning in class CvUnit.
 

Serp

King
Joined
Apr 1, 2015
Messages
660
Did you find out more / a solution?
Are you sure it is caused by "init and kill"? You write "mod giving player a button" -> I already noticed in my multiplayer modpack, that every modded popup with a decision will that affects the game will cause desync.
Eg. whowards mod to "recall a tradeunit". Or he changed some stuff regarding goodyhut choice and therefore replaced the original pop up with a modded one, which exactly looks the same, but cuased (and still causes) desync whenever a human shoshone player is choosing his goodyhut reward. (I reverted his change in my custom dll to "fix" this).
There is also a very old thread about this topic (I think with whoward and someone else..) trying to find a solution:
 
Last edited:

LyarSoft

Chieftain
Joined
Aug 18, 2022
Messages
10
Did you find out more / a solution?
Are you sure it is caused by "init and kill"? You write "mod giving player a button" -> I already noticed in my multiplayer modpack, that every modded popup with a decision will that affects the game will cause desync.
Eg. whowards mod to "recall a tradeunit". Or he changed some stuff regarding goodyhut choice and therefore replaced the original pop up with a modded one, which exactly looks the same, but cuased (and still causes) desync whenever a human shoshone player is choosing his goodyhut reward. (I reverted his change in my custom dll to "fix" this).
There is also a very old thread about this topic (I think with whoward and someone else..) trying to find a solution, but as far as I know, they did not find one
My solution is nothing to do with lua-side.
It seems that most of the lua extension setter methods (or methods starting with "lChange") will cause desync , my solution is to use gDLL->sendFoundReligion to send arguments for those send methods since it is capable of sending 7 32-bit-integers and 1 string (const char* in CPP). If you notice that calling a lua method (like CvLuaPlayer:lInitUnit) causing desync, just send arguments by gDLL->sendFoundReligion or other network methods to make sure those functions are executed by all the players. My solution here is to make sync copies of luaMethods (CvLuaUnit::lKill() and CvLuaUnit::lKillSync()), and invoke sync versions in the original version if the game is a multiplayer one, which means you donnt need to modify lua code. The sync copies sends their unique command types and arguments to gDLL->sendFoundReligion(), and the network reciever method CvDllNetMessageHandler::ResponseFoundReligion decides which backend method to invoke.
If you are not clear about which lua-side method exactly cause desync, just implement network varients of all the lua methods starting with "set" or "change" in DLL.
Since the VC version of CIV5 does not support coding with variadic templates, the code will be very ugly (over 30 if-else pairs within the receiver function CvDllNetMessageHandler::ResponseFoundReligion, or switch-case maybe).
Some problems I met:
You cannot send CPP poinnters by network methods, send their unique identifiers instead. For example, CvUnit* shall be uniquely determined by GET_PLAYER(playerID/*the unit's owner's ID*/).getUnit(unitID).
All the network methods are sent on the NEXT FRAME of the frame are invoked, but lua functions will stuck the game in this frame. So it is impossible to fetch the return value of some methods like CvLuaPlayer:lInitUnit if it is sent by network methods. Just execute them locally on the caller's PC and then execute them on remote players PC. Since the message is sent to all the players including the invoker, you may create unique identifier for this invoke and do not execute them again on the invoker's PC.
 

Serp

King
Joined
Apr 1, 2015
Messages
660
uhm, ok. I'm no programer, so at least this first time reading I did not understood anything from what you wrote =/ :D
But I would be greateful if we first could clarify the initial issue... I can't imagine that simply calling "InitUnit()" alon in a lua script is already causing desync. I have a lot of mods active in multiplayer that do spawn (init) units, without desyncing. Or is it only the combination of spawn and directly kill the same unit afterwards?
Eg a "convenient start" mod that let you start with "super settles" and it will replace them after first turn with normal settlers:
Code:
       for unit in pPlayer:Units() do
            if (unit:GetUnitType() == GameInfoTypes["UNIT_SUPERSETTLER"]) then
                pSettler = pPlayer:InitUnit(GameInfoTypes["UNIT_SETTLER"], unit:GetX(), unit:GetY(), GameInfoTypes["UNITAI_SETTLE"], unit:GetFacingDirection())
                pSettler:SetEmbarked(unit:IsEmbarked())
                unit:Kill(false, -1)
            end
        end
this code does not cause desync.
So I still assume the problem is the "button" from a custom UI you pressed ?!
 

LyarSoft

Chieftain
Joined
Aug 18, 2022
Messages
10
uhm, ok. I'm no programer, so at least this first time reading I did not understood anything from what you wrote =/ :D
But I would be greateful if we first could clarify the initial issue... I can't imagine that simply calling "InitUnit()" alon in a lua script is already causing desync. I have a lot of mods active in multiplayer that do spawn (init) units, without desyncing. Or is it only the combination of spawn and directly kill the same unit afterwards?
Eg a "convenient start" mod that let you start with "super settles" and it will replace them after first turn with normal settlers:
Code:
       for unit in pPlayer:Units() do
            if (unit:GetUnitType() == GameInfoTypes["UNIT_SUPERSETTLER"]) then
                pSettler = pPlayer:InitUnit(GameInfoTypes["UNIT_SETTLER"], unit:GetX(), unit:GetY(), GameInfoTypes["UNITAI_SETTLE"], unit:GetFacingDirection())
                pSettler:SetEmbarked(unit:IsEmbarked())
                unit:Kill(false, -1)
            end
        end
this code does not cause desync.
So I still assume the problem is the "button" from a custom UI you pressed ?!
More precisely, it is not the method invoking itself causing desync.
CIV5 maintains lists (actually a CPP container named "set" that ensures no duplicate elements are inserted) of units/cities/plots and other game objects on all the players' PC. At the end of each turn the game will ask all the clients to check their lists one by one. If a unit in the Non-host player's list does not exists in the host player's, it will cause a re-sync and all the lists will be change to be identical with the host-player. This might be what the game is doing when showing you a loading screen.

Therefore, some invokes of the setter methods will not cause desync:
The code you posted might be executed by all the players since it is invoked in AI turns or invoked at the end of a turn that all the players will call them at the same time with the same initial game state. In this case, all the player execute the same setter method like SetEmbarked and InitUnit with their the same initial state of their unit lists, then the adding (InitUnit ) and changing (SetEmbarked) are executed with same arguments. If initial states and the operations on different players' PC are same, the final version of the lists are also identical, so the game will not desync.
You mentioned the button will cause desync. It is obvious that any time when the button (especially those only show when clicking a unit) is pressed, the relating function is called by only one of the players. I can not select other player's units and click their buttons.
If the button is to replace your swordsman with legion, a non-host player pressing it will delete the swordsman and add a legion in his unit list, with the host's unit list unchanged. At the end of this turn, the game will iterate through his unit list, find the legion and notice that is does not exist in the host's unit list, then a desync is triggered.
If you transform all the swordsman to legion by calling unit:Kill and Player:InitUnit at the end of each turn, the swordsman's removal and the legion's emergence are executed on host's unit list, so in the next turn the legion will not be erased.

So a solution is to ask all the players to call the setter methods.
 

Serp

King
Joined
Apr 1, 2015
Messages
660
Okay, I think I understood it:
Yes, it is the old topic regarding modded UI/buttons that do not sync (unlike games UI/buttons).
But since this old topic still has no perfect solution, it is good that you experiment and try finding alternative solutions to the NetSyncTool I linked in my first response.
And since you say that some functions, like Unit Upgrade do work, they most likely have their own way of syncing, while kill and init do not have code that syncs (I guess you already told me so in above posts).
And your solution now would be to recreate ALL lua functions and write a synced version of them...
--
Sounds interesting, but also ALOT of work, that is not really needed if you keep in mind that it can also be solved by either not using modded UI, or using workarounds like the one linked in NetSyncTools.
--
I'm also interested in a more user firendly version which can transport more parameters than the current NetSyncTools. They also mentioned "custom missions" and "Network.SendChat" as vehicle, but unfortunately they stopped developing it further.
 

LyarSoft

Chieftain
Joined
Aug 18, 2022
Messages
10
Okay, I think I understood it:
Yes, it is the old topic regarding modded UI/buttons that do not sync (unlike games UI/buttons).
But since this old topic still has no perfect solution, it is good that you experiment and try finding alternative solutions to the NetSyncTool I linked in my first response.
And since you say that some functions, like Unit Upgrade do work, they most likely have their own way of syncing, while kill and init do not have code that syncs (I guess you already told me so in above posts).
And your solution now would be to recreate ALL lua functions and write a synced version of them...
--
Sounds interesting, but also ALOT of work, that is not really needed if you keep in mind that it can also be solved by either not using modded UI, or using workarounds like the one linked in NetSyncTools.
--
I'm also interested in a more user firendly version which can transport more parameters than the current NetSyncTools. They also mentioned "custom missions" and "Network.SendChat" as vehicle, but unfortunately they stopped developing it further.
It seems that the NetSyncTool you mentioned is utilizing ICvEngineUtility1::SendSellBuilding() in ICvDLLUtility.h. This function can only send 2 32-bit-integers. Although the reciever function CvDllNetMessageHandler::ResponseSellBuilding() has a third argument PlayerTypes, it is always fixed to the player who send this message.
The Network.SendChat lua method is probably linked to the ICvEngineUtility1::sendChat in ICvDLLUtility.h. It can send a CvString (which wrapped more message like the string lengh, encoding than the simple C string "const char*") and 2 32-bit-integers. Unfortunately I didnot find the receiver method in CPP files, so it might be impossible to decode our customized data sent by this method.
The function SendFoundReligion can send much more arguments (8 32-bit-integers and 1 string as I mentioned), and the receiver method is exactly in CvDllNetMessageHandler.cpp, so I think it is the best method for sending customized data.
Throwing an event and trigger the function on all the clients after SendSellBuilding() or other network message is processed is much easier, but it does not work in my case. In the lua function of the mod I'm playing, I need to invoke player->InitUnit() and give some special promotions to it, since all the network message is sent next frame after the current lua function finished, I cannot get the reference to the new unit in that lua function.
 

LyarSoft

Chieftain
Joined
Aug 18, 2022
Messages
10
Okay, I think I understood it:
Yes, it is the old topic regarding modded UI/buttons that do not sync (unlike games UI/buttons).
But since this old topic still has no perfect solution, it is good that you experiment and try finding alternative solutions to the NetSyncTool I linked in my first response.
And since you say that some functions, like Unit Upgrade do work, they most likely have their own way of syncing, while kill and init do not have code that syncs (I guess you already told me so in above posts).
And your solution now would be to recreate ALL lua functions and write a synced version of them...
--
Sounds interesting, but also ALOT of work, that is not really needed if you keep in mind that it can also be solved by either not using modded UI, or using workarounds like the one linked in NetSyncTools.
--
I'm also interested in a more user firendly version which can transport more parameters than the current NetSyncTools. They also mentioned "custom missions" and "Network.SendChat" as vehicle, but unfortunately they stopped developing it further.
Unit Upgrade and other "official" functions dont have their own way of syncing, they are calling Send... methods on CPP side. For example upgrade is calling SendDoCommand() with argument COMMAND_UPGRADE
 

LyarSoft

Chieftain
Joined
Aug 18, 2022
Messages
10
thanks for the explanation. So your SendFoundReligion solution is already quite similar to the NetSyncTool code, although you do it in DLL code instead of lua, right?
Would be great if you could share your final result as soon as you are satisfied :)
I think reflection scheme is all I need, which allows you to invoke any function by its name. I'm working on it now. If sucess, we can call something like "player:SendExecute("InitUnit", Arguments...)" to execute the mentioned function on all the clients.
 

LyarSoft

Chieftain
Joined
Aug 18, 2022
Messages
10
thanks for the explanation. So your SendFoundReligion solution is already quite similar to the NetSyncTool code, although you do it in DLL code instead of lua, right?
Would be great if you could share your final result as soon as you are satisfied :)
I have a solution now allowing anyone to invoke any lua function on all the clients by providing its name to solve the sync problem. See my new thread for detail.
 
Top Bottom