MPMPM - Multiplayer Mod DLC-hack (Updated!)

[Poizen 87]

Wow, somehow I missed this hint. Going to delete all the print statements in the mods I have used, as I am experiencing an annoying lag at the beginning of AI turns.
Just to confirm - whit will it do exactly to the mods?

It won't do anything to the mods functionality beyond preventing them from sending dozens of lines of code to your lua.log file.

I'm rather slightly annoyed the mod makers didn't think to do it themselves- it's a bit sloppy, as none of the regular Civs produce console print statements, and it's not really useful unless you're parsing for bugs. Nonetheless, here's how I went about it:

1. Get Notepad++
2. Search for *.lua in your mods folder to bring up all lua files
3. Open them all simultaneously
4. ctrl+f and go to replace tab
5. Replace 'print' with '--print' and 'logger' with '--logger'. Use replace all in open documents

Now this may temporarily break certain mods, particularly if they have print or logger statements that occur in the middle of a line rather than the beginning. To fix this, I loaded up Civ and just let it dump all the run-time errors into lua.log, and then manually fixed them myself. This usually involved simply moving the '--' to the beginning of the line. The other common issue is dprint statements. The find and replace will change them to d--print, so do a replace d--print with --dprint in all documents. There were only 3 or 4 occurances of this in my entire 60+ civ mod-pack; I saved far more time by doing it this way rather than manually replacing every print and logger statement.

For the record, this is for performance tuning. This will not fix your modpack if you are crashing or encountering errors. What it will do is boost your performance and lower your per-turn wait times, and significantly reduce the occurance of re-sync events in multiplayer. So it's only recommended you do this after you've fine tuned your modpack and parsed it for runtime/syntax errors.

 

[Poizen 87]

Cicero: I'm using Colonialist Legacies Inuit in a mod pack, and I seem to be experiencing a strange issue. I can have my workers build the unique improvement, the Inuksuk, but it seems completing one of them in multiplayer breaks the game. Resyncs will occur every turn and crashes frequent.

Any idea if this might be related to them having an expensive function (the improvement checks all surrounding tiles for possible border expansion based on a culture 'challenge') or if custom graphics/models on unique improvements might be to blame? I'm leaning toward the former, else part of me would expect the desyncs to occur when I start building the improvement, and the graphic appears, as opposed to when it finishes.

 

[cicero225]

@cicero225:
After adding the german textdatabase function, the override files only contains the german text and no english text.
My CopyTextDatabaseGerman() function does look exactly like the en_US function. I just changed en_US into de_DE. Should I change more, e.g variable names to get more than one language, or isn't it possible to get a multi version?
I tried to do merge the english and german versions textfiles (this units_mongol file). But this seems not to work. When I try to start a game, it will crash back to desktop.

Oh that's right, I forgot the function writes to the name within the variable textFileName rather than an explicitly named file. Hmm, doing all languages would take more though then. For what it's worth though, it is possible to copy one of the languages to one of the other empty xml files in Override. (there are a lot of completely empty files). Alternatively, it's possible you missed something.

Yeah, I only have 2 sets of numbers up there. 242 and 244 iirc.

Do one of your other mods have a .dll? That would cause an error like this.

Cicero: I'm using Colonialist Legacies Inuit in a mod pack, and I seem to be experiencing a strange issue. I can have my workers build the unique improvement, the Inuksuk, but it seems completing one of them in multiplayer breaks the game. Resyncs will occur every turn and crashes frequent.

Any idea if this might be related to them having an expensive function (the improvement checks all surrounding tiles for possible border expansion based on a culture 'challenge') or if custom graphics/models on unique improvements might be to blame? I'm leaning toward the former, else part of me would expect the desyncs to occur when I start building the improvement, and the graphic appears, as opposed to when it finishes.

There's nothing obvious I see about the Inuksuks that would be a problem, and just being slow shouldn't do it every turn. Can you try (this may seem odd) having both players turn on visible yields then repeatedly screenshotting the Inuksuk at the end of each turn? Sometimes in this way it becomes obvious that one player does not have the same values/unit locations as the other player, which is an automatic desync.

 

[manekk]

Thanks Poizen

I am only after performance now - I have spent the last few weeks ensuring I use non crashing and non conflicting mods, and trying to make things work or harassing cicero for help
I will leave the print messages in the few mods that I am still not 100% certain about.

 

[Poizen 87]

There's nothing obvious I see about the Inuksuks that would be a problem, and just being slow shouldn't do it every turn. Can you try (this may seem odd) having both players turn on visible yields then repeatedly screenshotting the Inuksuk at the end of each turn? Sometimes in this way it becomes obvious that one player does not have the same values/unit locations as the other player, which is an automatic desync.

What are the event handlers that cause issues with Events and Decisions compatibility? GameEvents? If so:

Oh yeah, if that's the case than the Inuksuit are likely the culprits. Many of the "passive" abilities of all of our civs use GameEvents.PlayerDoTurn.

From the Colonialist Legacies thread when I asked about events that trigger at the beginning of a turn. some of these 'GameEvents' scripts seem to be triggering appropriately however, without causing a desync. For example Canada uses these GameEvent handlers:

[BNW] Colonialist Legacies - Canadian Dominion (v 1)\Lua\CanadianRCMPCaptureScript.lua (3 hits)
Line 33: GameEvents.UnitSetXY.Add(function(player, unit, x, y)
Line 61: GameEvents.PlayerDoTurn.Add(function(iPlayer)
Line 93: GameEvents.PlayerCityFounded.Add(function(iPlayer, iCityX, iCityY)
[BNW] Colonialist Legacies - Canadian Dominion (v 1)\Lua\CanadianRCMPHappiness.lua (1 hit)
Line 41: GameEvents.PlayerDoTurn.Add(RCMPHappinessPower)

Yet it appears this Civ works as intended even in multiplayer, without desyncing. Many of the JFD Civs, which appear to all be working, use this series of handlers as well.

Edit: I may have found the issue. Turns out the Inuit and one other Civ in my modpack use the math.random() function, which you mention on the first page is a problem.

 

[Poizen 87]

Are multiple xml files with the same name still problematic? Like if a bunch of my civs have 'DiplomacyText.xml'?

 

[cicero225]

What are the event handlers that cause issues with Events and Decisions compatibility? GameEvents? If so:



From the Colonialist Legacies thread when I asked about events that trigger at the beginning of a turn. some of these 'GameEvents' scripts seem to be triggering appropriately however, without causing a desync. For example Canada uses these GameEvent handlers:

[BNW] Colonialist Legacies - Canadian Dominion (v 1)\Lua\CanadianRCMPCaptureScript.lua (3 hits)
Line 33: GameEvents.UnitSetXY.Add(function(player, unit, x, y)
Line 61: GameEvents.PlayerDoTurn.Add(function(iPlayer)
Line 93: GameEvents.PlayerCityFounded.Add(function(iPlayer, iCityX, iCityY)
[BNW] Colonialist Legacies - Canadian Dominion (v 1)\Lua\CanadianRCMPHappiness.lua (1 hit)
Line 41: GameEvents.PlayerDoTurn.Add(RCMPHappinessPower)

Yet it appears this Civ works as intended even in multiplayer, without desyncing. Many of the JFD Civs, which appear to all be working, use this series of handlers as well.

Edit: I may have found the issue. Turns out the Inuit and one other Civ in my modpack use the math.random() function, which you mention on the first page is a problem.

Hmm, I thought I would have seen that when I looked, but I probably just missed it. Use of math.random is guaranteed desync, because different random numbers will show up on each player's computer->different things happen->desync.

Substitute in Game.Rand() for every math.random(). They have slightly different syntax.

Game.Rand( int mval, string log_message) returns a random integer between 0 and mval-1, and inserts the log_message into the lua log. I think the log_message is mandatory, but not sure.

math.random(), on the other hand, returns either a pseudorandom decimal (double precision) between 0 and 0.9999... or, if called as math.random(mval), gives a random value between 1 and mval.

It's obnoxious, but math.random(mval) need to become Game.Rand(mval,"message")+1, and math.random() needs to become Game.Rand(mval,"message")/mval , where mval is chosen sufficiently high that the change in precision won't matter (1000 is probably safe enough).

Lua stores all numeric values as double precision, so no type conversion is necessary.

Ask me if that doesn't make sense to you.

Edit: Reference links:
http://modiki.civfanatics.com/index.php/Game.Rand_(Civ5_API)
(It doesn't specify the integer range, but the civ 5 source code clearly shows 0 to n-1)
http://www.lua.org/pil/18.html

 

[cicero225]

Are multiple xml files with the same name still problematic? Like if a bunch of my civs have 'DiplomacyText.xml'?

It is only problematic if the xml filename matches something in the base game. Otherwise, it will have no effect. In this case, there is no base game file named DiplomacyText.xml, so it does not matter. In fact, I'd forgotten, but MPMPM is coded to explicitly prevent this from ever happening (if it detects that's about to happen, it renames the file. For various reasons, an XML based base game override is 100% guaranteed to be unintentional on the part of the mod creator)

Explanation:

Civ 5 xml-file loading:

1) xml files in the base game (including dlc folders) are loaded. Only xml files of certain specific names are acknowledged. This is hardcoded somewhere beyond our access.

• If the same filename is encountered more than once, priority is given to the the one in the folder structure with the highest priority value. This is assigned in a special datafile included in dlcs and expansion packs. i.e. BNW overrides G&K overrides Vanilla. MPMPM is assigned an absurdly high priority, so it overrides everything. If there are two files of the same name in the same region, one is loaded more or less at random.

2) When the continue button is pressed on the mod selection menu, everything in the mods folder assigned "UpdateModDatabase" in the mod info (the vast vast majority of mod xmls) is then loaded on top of the base game database. Unlike step 1, the actual name of the xml does not matter, and duplicates are tolerated.

3) During MPMPM game creation, the entire mods database from the mod game is saved into a series of files (in the Override folder) designed to explicitly override every single other xml file in the base game, so that civ 5 will load with the modded game database rather than the original.

• xml files in the "Mods" folder in MPMPM are completely ignored if they don't have a name that matches the hardcoded list from step 1), since MPMPM is treated as DLC
• xml files that DO have the same name as something in the base game will still override, and this is certainly unintentional on the part of the mod creator (because it would not have this effect in the Mods Folder)
• MPMPM actually goes out of its way to check the mods folders for any XML file that matches something in the base game, and appended _RENAMED to the end of the filename so it won't do this. I made this fix somewhere near the start of this thread.

 

[Poizen 87]

And I'm guessing from context, if I see math.random(1,2) that would effectively result in a random choice of 1 or 2? I noticed you didn't list syntax for 2 values separated by a comma.

Also, as an example, I tried your solution for the Inuit but it broke their start bias. Guessing I'm doing something wrong. The scripts variously look something like:

local function ShuffleTiles(ResourceTable) -- This function brought to you by wikipedia
local n = #ResourceTable
while n >= 2 do
local k = math.random(n)
ResourceTable[n], ResourceTable[k] = ResourceTable[k], ResourceTable[n]
n = n - 1
end
n = 2
print("Shuffled")
for k,v in pairs(ResourceTable) do print(k,v) end
return ResourceTable
end

Which I changed to->

local function ShuffleTiles(ResourceTable) -- This function brought to you by wikipedia
local n = #ResourceTable
while n >= 2 do
local k = game.rand(n,"") + 1
ResourceTable[n], ResourceTable[k] = ResourceTable[k], ResourceTable[n]
n = n - 1
end
n = 2
--print("Shuffled")
--for k,v in pairs(ResourceTable) do --print(k,v) end
return ResourceTable
end

The other major statement that required changing was:

local function PlaceInuitStratResource(plot)
local n = #ResS
local k = math.random(n)
if plot:GetPlotType() == PlotTypes.PLOT_OCEAN then
print("This can only be Oil")
local dice = math.random(3)
if dice < 3 then
plot:SetResourceType(GameInfoTypes.RESOURCE_OIL, 2)
else
plot:SetResourceType(GameInfoTypes.RESOURCE_OIL, 4)
end
return true
elseif plot:CanHaveResource(ResS[k], false) then
print("Can have strategic")
local dice = math.random(3)
if dice < 3 then
plot:SetResourceType(ResS[k], 2)
else
plot:SetResourceType(ResS[k], 4)
end
return true
elseif plot:GetTerrainType() == TerrainTypes.TERRAIN_SNOW then
print("It's snow.")
if ResS[k] ~= GameInfoTypes.RESOURCE_HORSE then
local dice = math.random(3)
if dice < 3 then
plot:SetResourceType(ResS[k], 2)
else
plot:SetResourceType(ResS[k], 4)
end
return true
end
else
print("Can't have strategic")
return false
end
end

Which I changed to:

local function PlaceInuitStratResource(plot)
local n = #ResS
local k = game.rand(n,"") + 1
if plot:GetPlotType() == PlotTypes.PLOT_OCEAN then
--print("This can only be Oil")
local dice = game.rand(3,"") + 1
if dice < 3 then
plot:SetResourceType(GameInfoTypes.RESOURCE_OIL, 2)
else
plot:SetResourceType(GameInfoTypes.RESOURCE_OIL, 4)
end
return true
elseif plot:CanHaveResource(ResS[k], false) then
--print("Can have strategic")
local dice = game.rand(3,"") + 1
if dice < 3 then
plot:SetResourceType(ResS[k], 2)
else
plot:SetResourceType(ResS[k], 4)
end
return true
elseif plot:GetTerrainType() == TerrainTypes.TERRAIN_SNOW then
--print("It's snow.")
if ResS[k] ~= GameInfoTypes.RESOURCE_HORSE then
local dice = game.rand(3,"") + 1
if dice < 3 then
plot:SetResourceType(ResS[k], 2)
else
plot:SetResourceType(ResS[k], 4)
end
return true
end
else
--print("Can't have strategic")
return false
end
end

Did I mess up the syntax somewhere?

 

[cicero225]

And I'm guessing from context, if I see math.random(1,2) that would effectively result in a random choice of 1 or 2? I noticed you didn't list syntax for 2 values separated by a comma.

Ah yes. It even mentions that in the documentation but I missed it. math.random(a,b) is only valid for integers, picking one of the integers in the range a, a+1...b-1, b. (Equivalent to math.random(b-a+1)+a-1 )

In the case of 1,2 it would be either 1 or 2, of course. You could do Game.Rand(2,"")+1 instead of math.rand(1,2) in that case.

(The relevant part of the documentation is "Finally, we can call random with two integer arguments, l and u, to get a pseudo-random integer x such that l <= x <= u." It also looks like "" is a valid input to get no message, based on references to Game.Rand I see elsewhere on the site.)

 

[cicero225]

And I'm guessing from context, if I see math.random(1,2) that would effectively result in a random choice of 1 or 2? I noticed you didn't list syntax for 2 values separated by a comma.

Also, as an example, I tried your solution for the Inuit but it broke their start bias. Guessing I'm doing something wrong. The scripts variously look something like:




Which I changed to->



The other major statement that required changing was:



Which I changed to:



Did I mess up the syntax somewhere?

Capitalization. It is, in fact, supposed to be Game.Rand().

 

[Poizen 87]

Silly me. I'm not used to lua or case sensitive coding languages. Thanks for the speedy reply! You've really been a big help.

Edit: oy, capitalization didn't fix the issue

 

[cicero225]

Silly me. I'm not used to lua or case sensitive coding languages. Thanks for the speedy reply! You've really been a big help.

Edit: oy, capitalization didn't fix the issue

Wait, is the start bias still broken, or does it work now but you still have desyncs?

 

[Poizen 87]

Start bias is broken. Haven't been able to test the desync issue yet. Would it have anything to do with the fact mval is a variable? (n)

 

[cicero225]

Start bias is broken. Haven't been able to test the desync issue yet. Would it have anything to do with the fact mval is a variable? (n)

I doubt it.

http://modiki.civfanatics.com/index.php/Game.Rand_(Civ5_API) has many usage examples from the base game but they seem no different than what you have.

Do you have any lua log info? What is going wrong with the start bias?

 

[Poizen 87]

False alarm. I kept spawning in the jungle. Turns out I had the literal rotten luck of drawing 5 random standard sized communitas maps that didn't have any eligible snow or ice starts. It's working fine.

 

[cicero225]

False alarm. I kept spawning in the jungle. Turns out I had the literal rotten luck of drawing 5 random standard sized communitas maps that didn't have any eligible snow or ice starts. It's working fine.

So now the question is desyncs.

 

[Poizen 87]

I'll be running a quick playtest tonight with my girlfriend. I'll play the first 25 turns, throw down an inuksuk and improve a fish resource (which turn into seals for Inuit), and see if it remains stable.

 

[Poizen 87]

Desyncs not fixed sadly, although I am absolutely certain it's related to the Inuksuk now. Desyncs begin happening every turn *immediately* after the first one is finished. At least that's my best guess, so I checked a few hunches:

I thought it might be that tile yields were not updating or applying properly for other players, but I checked my GFs laptop to make sure- tile yields were updated properly.

My other hunch was simply the script being too intensive, due to the plot iterations the inuksuk performs on all surrounding tiles. But this doesn't seem to be the case either, as other Civs that run plot iterator functions don't seem to produce the same issue.

I'm left wondering if there's something I'm missing (Their UA which grants food from tiles outside of workable range area), but these explanations don't gel with the Inuksuk, unless there's some interaction between the Inuksuk and their UA- Perhaps if the Inuksuk boosts food for a tile outside of workable range? Further testing would be required.


The only suspicious thing I could find in the lua.log was a bunch of entries like:

'[179377.718] NotificationPanel: Attempt to remove unknown Notification 28'

But these were occuring prior to the resync so I am not sure what to think of it.

 

[Serp]

kind of offtopic:

Can you explain, why in the start post and everywhere else is written:

"Communitas map
Workaround: After starting the game, IMMEDIATELY SAVE. Quit, and then reload the saved game. Now everyone should have the host's version of the map." ?

I just put the .lua file in my user map directory and I'm able to select and play communitas map in multiplayer without doing the steps above.
Or does in in fact not work and some features of communitas map are missing?

edit:
and while you talking about random: Is Map.Rand() okay, or will there be unsync ?