TerrainBuilder.GetRandomNumber vs Game.GetRandNum and other MP Desync Questions

[Red Key]

For the two random number functions in the title (TerrainBuilder.GetRandomNumber vs Game.GetRandNum) is one better to use than the other? Especially when considering multiplayer desync issues?

I made a mod that makes so unique units, improvements, and buildings are unlocked randomly. Currently I am using TerrainBuilder.GetRandomNumber based on an old thread I saw on here, but when playing with friends one of the players always seem to have a lot of sync issues. A little more detail - did a four player game and player R kept having sync issues, but all other players were fine including me and players B & C. Also tried a three player game without R, but instead player B had all the sync issues. If we do a game without the mod then sync issues go away (well are much more rare).

The random function can be called due to Events.ResearchCompleted or Events.CivicCompleted. Also at the beginning of the game a check is performed on uniques that don't have any prereq tech or civic, and this is called based on GameEvents.OnGameTurnStarted. Could using random numbers during any of these events cause desync issues?

Only other thing I can think of that might cause issues is if for loops iterate in a different order for different players. My for loops iterate over GameInfo like the line below, or use ipairs which from what I understand is better to use than pairs.
for row in GameInfo.Units() do

Here is the full LUA portion of the code. There are some parts I could simplify - at first I thought it would help if I stored some data in tables, but then ended up only using them to retrieve names.

Spoiler :

-- RandomUnique
-- Author: RedKey
-- DateCreated: 10/4/2020 
--------------------------------------------------------------
-- Change these 3 variables to change the chance of uniques unlocking. Represents chance out of 100. e.g. 10 out of 100.
local iUniqueUnitChance : number = 10;
local iUniqueBuildingChance : number = 10;
local iUniqueImprovementChance : number = 10;
--------------------------------------------------------------
local UniqueType = {};
local TechUnlocks = {};
local CivicUnlocks = {};

local UnitData:table = {
    Name = {},
    Tech = {},
    Civic = {},
    Replaces = {},
};
local ImprovementData:table = {
    Name = {},
    Tech = {},
    Civic = {},
};
local BuildingData:table = {
    Name = {},
    Tech = {},
    Civic = {},
    Replaces = {},
};

-- Store a list of all uniques unlocked by a tech
function CachePrereqTech (sTech, sType)
    if (sTech) then
        if (TechUnlocks[sTech] == nil) then
            TechUnlocks[sTech] = {sType};
        else
            table.insert(TechUnlocks[sTech],sType);
        end
    end
end

-- Store a list of all uniques unlocked by a civic
function CachePrereqCivic (sCivic, sType)
    if (sCivic) then
        if (CivicUnlocks[sCivic] == nil) then
            CivicUnlocks[sCivic] = {sType};
        else
            table.insert(CivicUnlocks[sCivic],sType);
        end
    end
end

function GetUnitData()   
    for row in GameInfo.Units() do
        if (row.TraitType == 'RU_TRAIT_CIVILIZATION_NO_PLAYER') then
            local unitType = row.UnitType;
            UnitData.Name[unitType] = Locale.Lookup(row.Name);
            UnitData.Tech[unitType] = row.PrereqTech;       
            UnitData.Civic[unitType] = row.PrereqCivic;
            UniqueType[unitType] = 'UNIT';
            CachePrereqTech(row.PrereqTech, unitType);
            CachePrereqCivic(row.PrereqCivic, unitType);
        end
    end
    for row in GameInfo.UnitReplaces() do
        if (UnitData.Name[row.CivUniqueUnitType]) then
            UnitData.Replaces[row.CivUniqueUnitType] = row.ReplacesUnitType;
        end   
    end
end

function GetImprovementData()
    for row in GameInfo.Improvements() do
        if (row.TraitType == 'RU_TRAIT_CIVILIZATION_NO_PLAYER') then
            local impType = row.ImprovementType;
            ImprovementData.Name[impType] = Locale.Lookup(row.Name);
            ImprovementData.Tech[impType] = row.PrereqTech;       
            ImprovementData.Civic[impType] = row.PrereqCivic;
            UniqueType[impType] = 'IMPROVEMENT';
            CachePrereqTech(row.PrereqTech, impType);
            CachePrereqCivic(row.PrereqCivic, impType);
        end
    end
end

function GetBuildingData()   
    for row in GameInfo.Buildings() do
        if (row.TraitType == 'RU_TRAIT_CIVILIZATION_NO_PLAYER') then
            local buildingType = row.BuildingType;
            BuildingData.Name[buildingType] = Locale.Lookup(row.Name);
            BuildingData.Tech[buildingType] = row.PrereqTech;       
            BuildingData.Civic[buildingType] = row.PrereqCivic;
            UniqueType[buildingType] = 'BUILDING';
            CachePrereqTech(row.PrereqTech, buildingType);
            CachePrereqCivic(row.PrereqCivic, buildingType);
        end
    end
    for row in GameInfo.BuildingReplaces() do
        if (BuildingData.Name[row.CivUniqueBuildingType]) then
            BuildingData.Replaces[row.CivUniqueBuildingType] = row.ReplacesBuildingType;
        end   
    end
end

function DoRandomUnlock (iChance)
    return (TerrainBuilder.GetRandomNumber(100, "Random Unique Unlock") < iChance);
end

function TryUnlockUnit (pPlayer, unitType)
    if (DoRandomUnlock(iUniqueUnitChance)) then
        pPlayer:AttachModifierByID('RU_UNLOCK_' .. unitType);
--        if (UnitData.Replaces[unitType]) then
--            pPlayer:AttachModifierByID('RU_DISABLE_' .. UnitData.Replaces[unitType]);
--        end
        local unitName = UnitData.Name[unitType];
        NotificationManager.SendNotification(pPlayer:GetID(), NotificationTypes.USER_DEFINED_1, unitName .. ' Unlocked', 'Your people have developed a unique unit - the ' .. unitName);
    end
end

function TryUnlockImprovement (pPlayer, improvementType)
    if (DoRandomUnlock(iUniqueImprovementChance)) then
        pPlayer:AttachModifierByID('RU_UNLOCK_' .. improvementType);
        local impName = ImprovementData.Name[improvementType];
        NotificationManager.SendNotification(pPlayer:GetID(), NotificationTypes.USER_DEFINED_2, impName .. ' Unlocked', 'Your people have developed a unique improvement - the ' .. impName);
    end
end

function TryUnlockBuilding (pPlayer, buildingType)
    if (DoRandomUnlock(iUniqueBuildingChance)) then
        pPlayer:AttachModifierByID('RU_UNLOCK_' .. buildingType);
        local buildingName = BuildingData.Name[buildingType];
        NotificationManager.SendNotification(pPlayer:GetID(), NotificationTypes.USER_DEFINED_3, buildingName .. ' Unlocked', 'Your people have developed a unique building - the ' .. buildingName);
    end
end

function UnlockUniques (iTurn)
    if (iTurn == 1) then
        UnlockInitialUniques();
    end
end
GameEvents.OnGameTurnStarted.Add(UnlockUniques);


-- Unlocks uniques with no prereq techs or civics
function UnlockInitialUniques ()
    for _, playerID in ipairs(PlayerManager.GetAliveMajorIDs()) do
        local pPlayer:table = Players[playerID];
        print("Unlocking initial uniques for player" .. playerID);
        for row in GameInfo.Units() do
            if (row.TraitType == "RU_TRAIT_CIVILIZATION_NO_PLAYER") then
                local unitType = row.UnitType;
                if (row.PrereqTech == nil and row.PrereqCivic == nil) then
                    TryUnlockUnit(pPlayer, unitType);
                end
            end
        end
        for row in GameInfo.Improvements() do
            if (row.TraitType == "RU_TRAIT_CIVILIZATION_NO_PLAYER") then
                local impType = row.ImprovementType;
                if (row.PrereqTech == nil and row.PrereqCivic == nil) then
                    TryUnlockImprovement(pPlayer, impType);
                end
            end
        end
        for row in GameInfo.Buildings() do
            if (row.TraitType == "RU_TRAIT_CIVILIZATION_NO_PLAYER") then
                local buildType = row.BuildingType;
                if (row.PrereqTech == nil and row.PrereqCivic == nil) then
                    TryUnlockBuilding(pPlayer, buildType);
                end
            end
        end
    end
end

-- Unlocks uniques with a prereq tech
function UnlockUniqueFromTech (playerID, iTech)
    local pPlayer = Players[playerID]
    local sTechType = GameInfo.Technologies[iTech].TechnologyType
    if (TechUnlocks[sTechType] ~= nil) then
        for i,unlock in ipairs(TechUnlocks[sTechType]) do
            if (UniqueType[unlock] == 'UNIT') then
                TryUnlockUnit(pPlayer, unlock)
            elseif (UniqueType[unlock] == 'IMPROVEMENT') then
                TryUnlockImprovement(pPlayer, unlock)
            elseif (UniqueType[unlock] == 'BUILDING') then
                TryUnlockBuilding(pPlayer, unlock)
            end
        end
    end
end
Events.ResearchCompleted.Add(UnlockUniqueFromTech)

-- Unlocks uniques with a prereq civic
function UnlockUniqueFromCivic (playerID, iCivic, bCancelled)
    if (bCancelled) then return end
    local pPlayer = Players[playerID]
    local sCivicType = GameInfo.Civics[iCivic].CivicType
    if (CivicUnlocks[sCivicType] ~= nil) then
        for i,unlock in ipairs(CivicUnlocks[sCivicType]) do
            if (UniqueType[unlock] == 'UNIT') then
                TryUnlockUnit(pPlayer, unlock)
            elseif (UniqueType[unlock] == 'IMPROVEMENT') then
                TryUnlockImprovement(pPlayer, unlock)
            elseif (UniqueType[unlock] == 'BUILDING') then
                TryUnlockBuilding(pPlayer, unlock)
            end
        end
    end
end
Events.CivicCompleted.Add(UnlockUniqueFromCivic)

function Initialize()
    GetUnitData();
    GetImprovementData();
    GetBuildingData();
end
Initialize();

 

[Red Key]

Mods - meant to post this in the Mod Creation Help subforum if one of you wants to move it. Thanks!

 

[LeeS]

Firaxis uses both methods in their own scripts.

TerrainBuilder.GetRandomNumber only appears to be used now by Firaxis in map gen or UI files whereas Game.GetRandNum is used in scenario gameplay scripts. Try altering to Game.GetRandNum and see if that cures the issue -- there may be a MP specific reason why Game.GetRandNum is better than TerrainBuilder.GetRandomNumber for gameplay scripts (ie, the way the game is coded to get the numerical result based on some internal seed generation method that is different form the TerrainBuilder). The assumption has always been that since there can only be one "map" the TerrainBuilder method ought to be a stable way to generate a random number for all players in an MP match.

The reason we tend to avoid using the lua system's internal math.random (and any similar "math." method) is that it will generate a different number for every computer because lua accesses the computer's internal clock mechanisms to seed its random number generator -- which will be different for every player in a MP match.

Back in the original days of the game I recall TerrainBuilder.GetRandomNumber being used exclusively by Firaxis hence the advices on old threads to use it instead of the Game one -- but both will be far superior to any of the methods that are generic to lua.

 

[LeeS]

Also make sure that all users in the MP match have the same versions of all mods that are being used, and that these mods are loading their alterations into the game database in exactly the same order. (it may be necessary to rework those mods by using stringent LoadOrder values in the modinfo files to ensure that they always load their database content in exactly the same order relative to each other in all cases for all match participants).

 

[Red Key]

Thanks LeeS. Will see if Game.GetRandNum makes a difference next time I can convince my pals to try it.

 

[Gedemon]

also try with the slower computer hosting the game if only one of 4 players start to desync.

for row in GameInfo is fine to use, as ipairs, but yep, never use pairs for gameplay effect. There is a orderedPairs() function in one of Firaxis Lua common file that can be included in your script, but never break out of it or clean __orderedIndex that was added to the table manually.

I never had issues with TerrainBuilder.GetRandomNumber, I wonder if Game.GetRandNum was added later

it's Automation.GetRandomNumber that cause desync IIRC. Note that I think the Lua RNG can be used if you seed it manually.

Events could be in cause, maybe, but I think I've used Events.ResearchCompleted, Events.CivicCompleted and GameEvents.OnGameTurnStarted safely