TableSaverLoader, for persisting Lua table data through game save/load

[DarkScythe]

Wow.

That is worse than I thought!

I had thought that the pattern matching would've included the '.lua' extension, such that ABCTestXYZ.lua would be different from ABCTest.lua (There are 3 whole characters in the middle of that!) but it seems your testing has revealed even that isn't enough.

I wonder if many modders know about this, as being new to the Civ modding scene, certainly everything here is new to me. It's worrisome when it comes to such file conflicts for "common" filenames and utilities, such as TableSaverLoader.

In either case, it seems like to make a filename of a common utility unique, you have to... basically change the entire filename. I did see the comment of adding some sort of parameters to enforce strict pattern matching (using the ^ and $?) but that would most likely only protect your own files, and won't prevent your files from accidentally and inadvertently 'overwriting' someone else's, unless everyone adopts such practices.

What the heck, Firaxis.

 

[whoward69]

That is worse than I thought!

...

What the heck, Firaxis.

You can probably tell from my posts in that thread that I was less than chuffed when I worked it out - I'd wasted many evenings of modding time trying to track down what was happening!

I'm pretty sure that "Xyz.lua" will NOT include "XyzUtils.lua", ie the extension is included as part of the pattern. You can also include paths as part of the pattern (pretty sure it doesn't care if you use \ or / as delimiters, but I seem to have used \), so include("\Xyz.lua") should circumvent most problems, include("\whtools\Xxy.lua") would be even safer

 

[Pazyryk]

I added version as suffix like this: "TableSaverLoader016.lua". If an older mod included older version using:

include("TableSaverLoader.lua")

then everything is OK. If they used

include("TableSaverLoader")

then I think it is random whether their mod uses older version or newer version. But should be OK either way due to backward compatibility. In any case, your mod will use newer version if you use name with version suffix (with or without extension).

I've been using the ".lua" extension in include statements ever since I heard about the issue above. I hadn't thought of using "\" prefix before but I might start doing that now.

 

[DarkScythe]

Oh, sorry, I must have misunderstood.

whoward, for some reason, I thought your test did have the .lua extension in your includes.
Rather, I was not aware you could even set an include on a file without the extension explicitly identified. I'm quite used to these things requiring exact matches.

In other words, the prefix/suffix not being enough then is only if your include() does not explicitly state the .lua extension?

Essentially:
include("Test") will accept Test.lua, ABCTest.lua, TestXYZ.lua, and ABCTestXYZ.lua
include("Test.lua") will accept Test.lua and ABCTest.lua, but not the other two

Is that correct?

Still dumb either way, but what do I know..

That said, I suppose "\Test.lua" would assume that the lua file is in the root of your mod directory structure? I have mine inside a "\Lua" folder, but that might be common enough that it really won't help much.

 

[whoward69]

Is that correct?

correct

Still dumb either way, but what do I know..

... or me

That said, I suppose "\Test.lua" would assume that the lua file is in the root of your mod directory structure?

No. Forget anything you know about directory structure and just treat what's inside the brackets of include() as a pattern and the "directory structure" within the VFS as path like strings, hence "\Test.lua" will match both \Test.lua and \some\path\to\the\file\Test.lua

 

[DarkScythe]

No. Forget anything you know about directory structure and just treat what's inside the brackets of include() as a pattern and the "directory structure" within the VFS as path like strings, hence "\Test.lua" will match both \Test.lua and \some\path\to\the\file\Test.lua

Haha, that is quite annoying, still.
Seems this stuff works completely counter to 95% of how stuff seems to work elsewhere.
I suppose I can attempt to use "\Lua\Test.lua", but considering how common a Lua folder is, I doubt it will do too much over simply making "Test.lua" into "difh9235rbjkfdg.lua" for uniqueness.

 

[bane_]

"difh9235rbjkfdg.lua" for uniqueness.

Get the hell out of here. This one is taken.

 

[JFD]

Hi Pazyryk,

this is a fantastic utility, as, by now, you are aware I've found it to result in better performance than SaveUtils, and certainly it feels more stable. Now just to figure it all out

 

[LeeS]

Pazyryk, am I misunderstanding, but is it impossible to have two different mods each using TableSaverLoader with their own gT hooks ups and "TableName" names ? I've two completely seperate mods both of which I am trying to persist data for, and only when I run Mod A without Mod B also active will the data be persisted for Mod A. I make no change except enabling and disabling Mod B. Mod B's data is always being persisted but Mod A's is not when both mods are running. There are no mod dependancies or mod references between the two.

 

[DarkScythe]

It shouldn't be impossible, since VV and I both use it in our mods, and I routinely play with both of our Civs active.

Are your global tables being defined using the same names in both mods? Maybe you've inadvertently gotten it set up to share the data instead?

 

[LeeS]

hmmm...the only thing that's the same in the table naming is 'gT'. As in TableSave(gT, TableName), but I have TableName = "KnightsLRS" in one mod and TableName = "GGMUGA2" in the other. In setting up my tables to be saved as "part of" gT, in one mod I have this:

Spoiler :

include("TableSaverLoader016.lua")

--------------------------------------------------------------------------------------------------------------------------------------------------
--Variables Definitions not related to TableSaverLoader
--------------------------------------------------------------------------------------------------------------------------------------------------

gPromoMorale = GameInfoTypes.PROMOTION_MORALE
gPromoAlhambra = GameInfoTypes.PROMOTION_DRILL_1
gHeroicEpic = GameInfoTypes.BUILDING_HEROIC_EPIC
gAlhambra = GameInfoTypes.BUILDING_ALHAMBRA
gKnightsTemplar = GameInfoTypes.BUILDING_KNIGHTS_TEMPLAR
gKnightsTemplarTurnIncrement = 3	-- was 10
iFreeUnit = GameInfoTypes.UNIT_CRUSADER_LEGION
iNumFreeUnit = 1
iOBStech = "TECH_RIFLING"
iOBStechID = GameInfo.Technologies[iOBStech].ID

--xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
-------------------------------------------------------------------------------------------------
--TableSaverLoader Table definitions
-------------------------------------------------------------------------------------------------
gT = {}
gTemplarKnights = {}
--gTemplarKnights.gPlayerWhoBuilt = "NONE"
--gTemplarKnights.gCityThatBuilt = "NIL"
--gTemplarKnights.gTurnWhenBuilt = "NIL"
print("Before TableSaverLoader loading table gTemplarKnights was:")
for k,v in pairs(gTemplarKnights) do print(k,v) end

TableName = "KnightsLRS"

gT = { gTemplarKnights }

In the other I have:

Spoiler :

include("TableSaverLoader016.lua")


--------------------------------------------------------------------------------------------------------------------------------------------------
-- ADAPTATION COMMANDS FOR USE IN OTHER MODS
-- TABLE to specifiy the Great People types to be used is "gValidGreatPeopleUnitTypes"
-- VARIABLE to assign table save/load to a specific mod is "tableName"
-- NO OTHER changes in this lua file are required and should not generally be made.

-- add or remove units in the gValidGreatPeopleUnitTypes table to alter which Great People the main function executes for
-- only unit designations for Great People can be included in this table
-- currently UNIT_ARTIST , UNIT_WRITER , and UNIT_MUSICIAN don't work properly because of issues with bringing-along the correct great work
--    with the unique unit name
-- All other great people work with this system: UNIT_GREAT_GENERAL, UNIT_GREAT_ADMIRAL, UNIT_MERCHANT, UNIT_ENGINEER, UNIT_SCIENTIST, and UNIT_PROPHET

-- VARIABLE TO ASSIGN TABLE SAVE/LOAD TO A SPECIFIC MOD IS tableName
-- you need to change from "GGMUGA2" to something unique for your mod
--------------------------------------------------------------------------------------------------------------------------------------------------


gValidGreatPeopleUnitTypes = {"UNIT_GREAT_GENERAL", "UNIT_GREAT_ADMIRAL"}

TableName = "GGMUGA2"


--------------------------------------------------------------------------------------------------------------------------------------------------
--flag variables for DEBUG PRINTING. These can be changed from '0' to force print statements into the lua.log for debugging purposes.
--to shut off all debug printing set both of these variables to '0'.
--------------------------------------------------------------------------------------------------------------------------------------------------

iPrintSeverity = 1	--flag for deciding whether to print LeeS debug printing statements, higher numbers result in more debugging print statements

			--in a line like this: SeverityForPrint(iCivilizationType, iPrintSeverity, 4)
			--	the printing will occur any time "iPrintSeverity" is set to "4" or higher.
			--	the contents of variable "iCivilizationType" will be printed.

			--a line like this: SeverityForPrint(gQActiveCivPlayerNames, iPrintSeverity, 2, "gActiveCivPlayerNames")
			--	will print the full contents of the k,v pairs within table "gQActiveCivPlayerNames". The extra fourth argument will add
			--	 a line at the top of the printing in the form of "Current Data in the Table gQActiveCivPlayerNames are:"


iPrintOverride = 1	--override set to "1" to force always printing the results of the tables gMasterUsedGreatPeopleNames and gJustUsedName at
			--  the end of the function GreatPeopleNames every time it executes. This is set up so that the contents of the two tables
			--  can be verified through debug print statements without burying the lua.log in any other print statements. Only when this
			--  print-out seems incorrect should you begin to use "iPrintSeverity" to force more debug printing to occur.


--------------------------------------------------------------------------------------------------------------------------------------------------
--flag variables for processing control. DO NOT CHANGE. GENERALLY, NO CHANGES SHOULD BE MADE BELOW THIS COMMENT LINE.
--------------------------------------------------------------------------------------------------------------------------------------------------

gLoadFromSave = 0	--do not change from 0

--------------------------------------------------------------------------------------------------------------------------------------------------
--print messages to the lua log when the lua loads properly. DO NOT CHANGE
--------------------------------------------------------------------------------------------------------------------------------------------------

print("The Civilization Appropriate Great People lua loaded properly for " .. TableName .. " data")
print("iPrintSeverity is set to " ..  tostring(iPrintSeverity) .. " , and iPrintOverride is set to " ..  tostring(iPrintOverride))

-----------------------------------------------------------------------------------------------------------------------------------------------------
--create tables for all the 'fall-back' great people names and for active civs in the game. DO NOT CHANGE.
-----------------------------------------------------------------------------------------------------------------------------------------------------
gT = {}
gMasterUsedGreatPeopleNames = {}
gFallbackGreatGeneralNames = {}
gFallbackGreatAdmiralNames = {}
gFallbackGreatMerchantNames = {}
gFallbackGreatEngineerNames = {}
gFallbackGreatScientistNames = {}
gFallbackGreatProphetNames = {}
gFallbackGreatArtistNames = {}
gFallbackGreatWriterNames = {}
gFallbackGreatMusicianNames = {}
gQActiveCivPlayerNames = {}
gAllGreatPeopleNames = {}
gJustUsedName = {}
gTempData = {}
-----------------------------------------------------------------------------------------------------------------------------------------------------
--set up the table saver loader master table tree. DO NOT CHANGE.
-----------------------------------------------------------------------------------------------------------------------------------------------------
gT = { gMasterUsedGreatPeopleNames,
	gQActiveCivPlayerNames,
	gFallbackGreatGeneralNames,
	gFallbackGreatAdmiralNames,
	gFallbackGreatMerchantNames,
	gFallbackGreatEngineerNames,
	gFallbackGreatScientistNames,
	gFallbackGreatProphetNames,
	gFallbackGreatArtistNames,
	gFallbackGreatWriterNames, gFallbackGreatMusicianNames, gAllGreatPeopleNames }

[edit]
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
After some experimentation with changing the global table name from 'gT' in one of the mods to 'gMasterTemplarsTableLRS' I'm not seeing any difference in behavior. I am finding something suspiscious in my lua.log:

 [COLOR="Green"][224232.437] FreeUnitsPerXTurns: TableSave time: 0.05699999999996, inserts: 3, deletes: 0, updates: 0, unchanged: 1, checksum: 154
[224232.484] Great People Names: TableSave time: 0.0039999999999623, inserts: 0, deletes: 0, updates: 0, unchanged: 1542, checksum: 128544[/COLOR]
[224234.046] FreeUnitsPerXTurns: Knights Templar Construction was detected
[224234.046] FreeUnitsPerXTurns: gCityThatBuilt	8192
[224234.046] FreeUnitsPerXTurns: gTurnWhenBuilt	1
[224234.046] FreeUnitsPerXTurns: gPlayerWhoBuilt	0
[224234.046] LocalCityResourcesSpeedWondersBuildings: 412
[224234.046] LocalCityResourcesSpeedWondersBuildings: The city of Rome had 0 gWonderProductionModiferBldg buildings placed in the city
[224234.046] LocalCityResourcesSpeedWondersBuildings: The city of Rome had 0 gBuildingProductionModiferBldg buildings placed in the city
[224234.046] OPCityRules: pPlayer:GetGreatPeopleCreated() gives a value of 0
[224234.046] FreeUnitsPerXTurns: During PlayerDoTurn table gTemplarKnights was:
[224234.046] FreeUnitsPerXTurns: gCityThatBuilt	8192
[224234.046] FreeUnitsPerXTurns: gTurnWhenBuilt	1
[224234.046] FreeUnitsPerXTurns: gPlayerWhoBuilt	0
[224235.343] TurnProcessing: Hiding TurnProcessing
[224239.234] WonderPopup: setting to original size 976 576
[224301.125] [COLOR="Red"]Great People Names: TableSave time: 0.0080000000000382, inserts: 0, deletes: 0, updates: 0, unchanged: 1542, checksum: 128544[/COLOR]

The green "save report" is coming from the auto-turn saving, whereas the red "save report" is coming from using the "SAVE GAME" menu.

DarkScythe: is VV also using the MapData method you are using in Holo ?

 

[DarkScythe]

Are you sure that isn't because the data hasn't changed? I don't remember when the Great General data is supposed to change. Is the green turn autosave coming from the turn 0 autosave, or a midgame one?

As for Holo & Madoka, yes as far as I can tell we both use the MapModData definitions, although I have given mine a different name. I think I did notice before that his Nanoha and Madoka Civs were saving stuff into the same database table, though. It seemed like this was unintentional, but I wasn't sure if it was because he used the same MapModData definition in both of his mods.

The code snippets you've posted also don't seem to show any of that MapModData stuff, nor does it show when you're actually calling TableSave from the save-interception stuff.

 

[LeeS]

The green stuff was from mid-turn auto-save after completion of turn # 0. The data on FreeUnitsPerXTurns: TableSave did in fact change. This bit is actually the verification that the data should have changed, and that the table was changed for the duration of the game session (ie, before SAVE GAME and exit, then reload of saved game):

[224234.046] FreeUnitsPerXTurns: Knights Templar Construction was detected
[224234.046] FreeUnitsPerXTurns: gCityThatBuilt	8192
[224234.046] FreeUnitsPerXTurns: gTurnWhenBuilt	1
[224234.046] FreeUnitsPerXTurns: gPlayerWhoBuilt	0

This is the hook-up for the FreeUnitsPerXTurns as I'm currently running it:

--------------------------------------------------------------------------------------------------------------------------------------------------
--include commands for other lua files that are required
--------------------------------------------------------------------------------------------------------------------------------------------------

include("TableSaverLoader016.lua")

--------------------------------------------------------------------------------------------------------------------------------------------------
--Variables Definitions not related to TableSaverLoader
--------------------------------------------------------------------------------------------------------------------------------------------------

gPromoMorale = GameInfoTypes.PROMOTION_MORALE
gPromoAlhambra = GameInfoTypes.PROMOTION_DRILL_1
gHeroicEpic = GameInfoTypes.BUILDING_HEROIC_EPIC
gAlhambra = GameInfoTypes.BUILDING_ALHAMBRA
gKnightsTemplar = GameInfoTypes.BUILDING_KNIGHTS_TEMPLAR
gKnightsTemplarTurnIncrement = 3	-- was 10
iFreeUnit = GameInfoTypes.UNIT_CRUSADER_LEGION
iNumFreeUnit = 1
iOBStech = "TECH_RIFLING"
iOBStechID = GameInfo.Technologies[iOBStech].ID

--------------------------------------------------------------------------------------------------------------------------------------------------
--flag variables for processing control. DO NOT CHANGE. GENERALLY, NO CHANGES SHOULD BE MADE BELOW THIS COMMENT LINE.
--------------------------------------------------------------------------------------------------------------------------------------------------

gLoadFromSave = 0	--do not change from 0


--xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
-------------------------------------------------------------------------------------------------
--TableSaverLoader Table definitions
-------------------------------------------------------------------------------------------------
gMasterTemplarsTableLRS = {}
gTemplarKnights = {}

print("Before TableSaverLoader loading table gTemplarKnights was:")
for k,v in pairs(gTemplarKnights) do print(k,v) end

TableName = "KnightsLRS"

gMasterTemplarsTableLRS = { gTemplarKnights }

-------------------------------------------------------------------------------------------------
--TableSaverLoader Hookups
-------------------------------------------------------------------------------------------------

function OnModLoaded() --called from end of last mod file to load

	local bNewGame = not TableLoad(gMasterTemplarsTableLRS, TableName)

	if bNewGame then
		print("New Game")
		else print("Loaded from Saved Game")
		gLoadFromSave = 1	--LRS added this line

	end

	TableSave(gMasterTemplarsTableLRS, TableName)
end

function OnEnterGame()   --Runs when Begin or Continue Your Journey pressed
	print("Player entering game ...")
	ContextPtr:LookUpControl("/InGame/GameMenu/SaveGameButton"):RegisterCallback(Mouse.eLClick, SaveGameIntercept)
	ContextPtr:LookUpControl("/InGame/GameMenu/QuickSaveButton"):RegisterCallback(Mouse.eLClick, QuickSaveIntercept)
end
Events.LoadScreenClose.Add(OnEnterGame)

function SaveGameIntercept()	--overrides Civ5 code when player presses Save Game from Game Menu or Cntr-s
	TableSave(gMasterTemplarsTableLRS, TableName)
	UIManager:QueuePopup(ContextPtr:LookUpControl("/InGame/GameMenu/SaveMenu"), PopupPriority.SaveMenu)
end

function QuickSaveIntercept()	--overrides Civ5 code when player presses Quick Save from Game Menu or F11
	TableSave(gMasterTemplarsTableLRS, TableName)
	UI.QuickSave()
end

local autoSaveFreq = OptionsManager.GetTurnsBetweenAutosave_Cached()
function OnGameOptionsChanged()
	autoSaveFreq = OptionsManager.GetTurnsBetweenAutosave_Cached()
end
Events.GameOptionsChanged.Add(OnGameOptionsChanged)

function OnAIProcessingEndedForPlayer(iPlayer)
	if iPlayer == 63 then					--runs on barb turn AFTER barb unit moves (very close to the regular autosave)
		if Game.GetGameTurn() % autoSaveFreq == 0 then	--only need to do on autosave turns
			TableSave(gMasterTemplarsTableLRS, TableName)
		end
	end
end
Events.AIProcessingEndedForPlayer.Add(OnAIProcessingEndedForPlayer)

function InputHandler(uiMsg, wParam, lParam)
	if uiMsg == KeyEvents.KeyDown then
		if wParam == Keys.VK_F11 then
			QuickSaveIntercept()		--F11 Quicksave
        		return true
		elseif wParam == Keys.S and UIManager:GetControl() then
			SaveGameIntercept()			--ctrl-s
			return true
		end
	end
end
ContextPtr:SetInputHandler(InputHandler)

----------------------------------------------------------------------------------------------------------------------------------------------------
--game initialization and table saver/loader loading functions. DO NOT CHANGE
-----------------------------------------------------------------------------------------------------------------------------------------------------

--borrowed and adapted from Darkscythe
function KnightTemplarGameInit()
	KnightTemplarDataLRS()
end
Events.LoadScreenClose.Add(KnightTemplarGameInit)

---xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
-----------------------------------------------------------------------------------------------------
--fills in the table gActiveCivPlayerNames with all the active player civilization names. DO NOT CHANGE
-----------------------------------------------------------------------------------------------------

function KnightTemplarDataLRS()
	if gLoadFromSave == 1 then
		print("Knight Templar Table Was loaded from saved game")
	end
	if gLoadFromSave == 0 then
		print("Knight Templar Table Was created for a new game")
		gTemplarKnights.gPlayerWhoBuilt = -1
		gTemplarKnights.gCityThatBuilt = -1
		gTemplarKnights.gTurnWhenBuilt = -1
		--table.insert(gTemplarKnights, xxxxx)
	end
	print("After TableSaverLoading for new/continued game data in gTemplarKnights table was:")
	for k,v in pairs(gTemplarKnights) do print(k,v) end
end

I copied everything from the Great Generals mod and then only changed the parts as needed so as to avoid a conflict with the Great Generals mod.

At the very end of the lua file I have

-------------------------------------------
-- final TableSaverLoader hook-up command
-------------------------------------------

OnModLoaded()

I changed to this text within the lua file this morning. I thought perhaps I was missing a hook-up, so I copied everything out of the Great Generals file (which is always working) into the Knights Templar file (which is only working when the Great Generals is not enabled). As I type this I cannot be 100% certain this version works without the Great Generals active since in the last couple of days I've run so many tests on it they've all begun to blend together in my memory but I will double check and report back confirmation.

[edit]
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Spoiler :

lua.log from new game loading with the auto-save that occurs right after the player enters the new game:

[231562.796] FreeUnitsPerXTurns: Loading TableSaverLoader.lua...
[231562.796] FreeUnitsPerXTurns: Before TableSaverLoader loading table gTemplarKnights was:
[231562.796] FreeUnitsPerXTurns: New Game
[231562.796] FreeUnitsPerXTurns: Creating SavedGameDB tables for new game: KnightsLRS_Data and KnightsLRS_Info
[231563.000] FreeUnitsPerXTurns: TableSave time: 0.14700000000002, inserts: 1, deletes: 0, updates: 0, unchanged: 0, checksum: 19

lua.log from new game loading with right after the player presses "start Your Journey":

[232131.625] FreeUnitsPerXTurns: Player entering game ...
[232131.625] FreeUnitsPerXTurns: Knight Templar Table Was created for a new game
[232131.625] FreeUnitsPerXTurns: After TableSaverLoading for new/continued game data in gTemplarKnights table was:
[232131.625] FreeUnitsPerXTurns: gCityThatBuilt	-1
[232131.625] FreeUnitsPerXTurns: gTurnWhenBuilt	-1
[232131.625] FreeUnitsPerXTurns: gPlayerWhoBuilt	-1

lua.log when doing "SAVE GAME" on turn 0:

[232221.906] FreeUnitsPerXTurns: TableSave time: 0.048000000000002, inserts: 3, deletes: 0, updates: 0, unchanged: 1, checksum: 154

lua.log info coming from PlayerDoTurn event after the Templars wonder is constructed:

[232540.828] FreeUnitsPerXTurns: During PlayerDoTurn table gTemplarKnights was:
[232540.828] FreeUnitsPerXTurns: gCityThatBuilt	8192
[232540.828] FreeUnitsPerXTurns: gTurnWhenBuilt	1
[232540.828] FreeUnitsPerXTurns: gPlayerWhoBuilt	0

lua.log info coming from "SAVE GAME" after the Templars wonder is constructed on turn # 1:

[232711.921] FreeUnitsPerXTurns: TableSave time: 0.078999999999951, inserts: 0, deletes: 0, updates: 3, unchanged: 1, checksum: 172

Game is then exited and saved game is re-loaded. lua.log before player presses "Continue Journey":

[232953.203] FreeUnitsPerXTurns: Loading TableSaverLoader.lua...
[232953.203] FreeUnitsPerXTurns: Before TableSaverLoader loading table gTemplarKnights was:
[232953.281] FreeUnitsPerXTurns: TableLoad ran without error; time: 0.0010000000000048, checksum: 172
[232953.281] FreeUnitsPerXTurns: Loaded from Saved Game
[232953.328] FreeUnitsPerXTurns: TableSave time: 0, inserts: 0, deletes: 0, updates: 0, unchanged: 4, checksum: 172

lua.log After pressing "Continue Journey":

[232966.906] FreeUnitsPerXTurns: Player entering game ...
[232966.906] FreeUnitsPerXTurns: Knight Templar Table Was loaded from saved game
[232966.906] FreeUnitsPerXTurns: After TableSaverLoading for new/continued game data in gTemplarKnights table was:
[232966.906] FreeUnitsPerXTurns: gCityThatBuilt	8192
[232966.906] FreeUnitsPerXTurns: gPlayerWhoBuilt	0
[232966.906] FreeUnitsPerXTurns: gTurnWhenBuilt	1

Data has been persisted. Only change was to dis-enable the Great Generals mod. 2nd templar unit is spawned on the correct turn (ie, Turn #4, since I decreased the turn-length between unit spawning to make debug and testing a little easier)

My guess is that if I re-build the Knights Templar in ModBuddy, it will be the second mod loaded of the two, and will probably start being the one that always works. I will do so and confirm back sometime later today.

[2nd edit]
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Problem is definitely related to mod load order. With Knight Templar re-built in ModBuddy so that it is the "newest" of the two mods, it now persists data but the Great Generals mod does not. Lua.log from saving the game via the "SAVE GAME" menu, and with both mods active:

[234399.203] FreeUnitsPerXTurns: TableSave time: 0.083000000000027, inserts: 0, deletes: 0, updates: 3, unchanged: 1, checksum: 172

I've added both mods zipped as attachments:

 

[DarkScythe]

I really can't see anything wrong here.. or at least it's not blatantly obvious so far.

It also seems to me as if it's all working properly.

Here are my logs during the first few turns:

Turn 0, on creation:

 ChooseArchaeologyPopup: Creating SavedGameDB tables for new game: NanohaMod_Data and NanohaMod_Info
 ChooseArchaeologyPopup: TableSave time: 0.022000000000048, checksum: 0, inserts: 0, updates: 0, unchanged: 0, deletes: 0

 PMMMRebellionMain: Creating SavedGameDB tables for new game: PMMMConversion_Data and PMMMConversion_Info
 PMMMRebellionMain: TableSave time: 0.045999999999935, inserts: 4, deletes: 0, updates: 0, unchanged: 0, checksum: 76

 Great People Names: Creating SavedGameDB tables for new game: GGMUGA2_Data and GGMUGA2_Info
 Great People Names: TableSave time: 0.048999999999978, inserts: 12, deletes: 0, updates: 0, unchanged: 0, checksum: 228

 FreeUnitsPerXTurns: Creating SavedGameDB tables for new game: KnightsLRS_Data and KnightsLRS_Info
 FreeUnitsPerXTurns: TableSave time: 0.027000000000044, inserts: 1, deletes: 0, updates: 0, unchanged: 0, checksum: 19

 HoloLua: Creating SavedGameDB tables for new game: HoloTable_Data and HoloTable_Info
 HoloLua: TableSave time: 0.043999999999983, inserts: 3, deletes: 0, updates: 0, unchanged: 0, checksum: 57

Next, it looks like both mods appeared to fill in their tables properly:

 Great People Names: Player entering game ...
 Great People Names: Active Civs Table Was created for a new game
 Great People Names: Creating Fallback Tables -- Stand By
 FreeUnitsPerXTurns: Player entering game ...
 FreeUnitsPerXTurns: Knight Templar Table Was created for a new game
 FreeUnitsPerXTurns: After TableSaverLoading for new/continued game data in gTemplarKnights table was:
 FreeUnitsPerXTurns: gCityThatBuilt	-1
 FreeUnitsPerXTurns: gTurnWhenBuilt	-1
 FreeUnitsPerXTurns: gPlayerWhoBuilt	-1

Now, I have my game set to autosave every single turn, so here is the end of Turn 0, right before I regain control on Turn 1:

 ChooseArchaeologyPopup: TableSave time: 0, checksum: 168, inserts: 0, updates: 0, unchanged: 4, deletes: 0
 PMMMRebellionMain: TableSave time: 0, inserts: 0, deletes: 0, updates: 0, unchanged: 4, checksum: 76
 HoloLua: [TableSaverLoader] Preparing table data for an auto save..
 HoloLua: TableSave time: 0, inserts: 0, deletes: 0, updates: 0, unchanged: 3, checksum: 57
 Great People Names: TableSave time: 0.086999999999989, inserts: 1692, deletes: 0, updates: 0, unchanged: 12, checksum: 142220
 FreeUnitsPerXTurns: TableSave time: 0.010999999999967, inserts: 3, deletes: 0, updates: 0, unchanged: 1, checksum: 154

Seems every mod was able to save their tables successfully.
I then spawned two Great Generals for the Dutch, in order to force the Civ-Linked names mod to start recording data. I also gave myself all techs through Theology, but for some reason was unable to build the Knight's Templar wonder, so I can't say anything about that.

 HoloLua: [NewPlayerTurn] Player 0 (Holo) is beginning turn 1
 FreeUnitsPerXTurns: During PlayerDoTurn table gTemplarKnights was:
 FreeUnitsPerXTurns: gCityThatBuilt	-1
 FreeUnitsPerXTurns: gTurnWhenBuilt	-1
 FreeUnitsPerXTurns: gPlayerWhoBuilt	-1

 WorldView: nil
 WorldView: function: 77C3B2C8
 SerialEventUnitCreatedGood: The randomly selected name that would have been added to the MasterUsedGreatPeopleNames is TXT_KEY_GREAT_PERSON_FLORIS_DE_VOOGD
 SerialEventUnitCreatedGood: Current Data in the Table gMasterUsedGreatPeopleNames are:
 SerialEventUnitCreatedGood: Current Data in the Table gJustUsedName are:
 SerialEventUnitCreatedGood: 1	TXT_KEY_GREAT_PERSON_FLORIS_DE_VOOGD
 SerialEventUnitCreatedGood: The original name that should have been added to the MasterUsedGeneralAdmiralNames is TXT_KEY_GREAT_PERSON_FLORIS_DE_VOOGD
 SerialEventUnitCreatedGood: Current Data in the Table gMasterUsedGreatPeopleNames are:
 SerialEventUnitCreatedGood: 1	TXT_KEY_GREAT_PERSON_FLORIS_DE_VOOGD
 SerialEventUnitCreatedGood: Current Data in the Table gJustUsedName are:
 WorldView: nil
 WorldView: function: 77C3B2C8
 SerialEventUnitCreatedGood: The randomly selected name that would have been added to the MasterUsedGreatPeopleNames is TXT_KEY_GREAT_PERSON_SIMON_SPOOR
 SerialEventUnitCreatedGood: Current Data in the Table gMasterUsedGreatPeopleNames are:
 SerialEventUnitCreatedGood: 1	TXT_KEY_GREAT_PERSON_FLORIS_DE_VOOGD
 SerialEventUnitCreatedGood: Current Data in the Table gJustUsedName are:
 SerialEventUnitCreatedGood: 1	TXT_KEY_GREAT_PERSON_SIMON_SPOOR
 SerialEventUnitCreatedGood: The original name that should have been added to the MasterUsedGeneralAdmiralNames is TXT_KEY_GREAT_PERSON_SIMON_SPOOR
 SerialEventUnitCreatedGood: Current Data in the Table gMasterUsedGreatPeopleNames are:
 SerialEventUnitCreatedGood: 1	TXT_KEY_GREAT_PERSON_FLORIS_DE_VOOGD
 SerialEventUnitCreatedGood: 2	TXT_KEY_GREAT_PERSON_SIMON_SPOOR
 SerialEventUnitCreatedGood: Current Data in the Table gJustUsedName are:

And here is the autosave at the end of Turn 1, after spawning those two GG's, and before Turn 2 control is handed to me:

 ChooseArchaeologyPopup: TableSave time: 0.00099999999997635, checksum: 168, inserts: 0, updates: 0, unchanged: 4, deletes: 0
 PMMMRebellionMain: TableSave time: 0, inserts: 0, deletes: 0, updates: 0, unchanged: 4, checksum: 76
 HoloLua: [TableSaverLoader] Preparing table data for an auto save..
 HoloLua: TableSave time: 0.017000000000053, inserts: 1, deletes: 0, updates: 0, unchanged: 3, checksum: 76
 Great People Names: TableSave time: 0.017000000000053, inserts: 2, deletes: 0, updates: 0, unchanged: 1704, checksum: 142388
 FreeUnitsPerXTurns: TableSave time: 0, inserts: 0, deletes: 0, updates: 0, unchanged: 4, checksum: 154

I'm not seeing where the problem is..
I should note that when you actually click on the "SAVE GAME" menu item, the only entry that appears is FreeUnitsPerXTurns.
However, I'm not entirely sure if it means that nothing else was saved successfully. I do recall Pazyryk discussing before that only one particular InputHandler could be active at once, or something to that effect.

I ran another test by removing both your mods, and ran with what I have been playing my last game with, Holo and Madoka, etc. Hitting the "SAVE GAME" menu button with that shows only the TableSave event for Holo, but in comparing with the data in the SaveGame database file, in both scenarios, all data was there.

Edit:
Nevermind that, got the Wonder working. Apparently, it doesn't play too well with being granted via FireTuner. Starting over and giving myself the ability to 'actually' research and 'build' the wonder seems to have done the trick.

 FreeUnitsPerXTurns: Knights Templar Construction was detected
 FreeUnitsPerXTurns: gCityThatBuilt	8192
 FreeUnitsPerXTurns: gTurnWhenBuilt	2
 FreeUnitsPerXTurns: gPlayerWhoBuilt	0
 OniClansLua: Calling Suika's Turn Function
 HoloLua: [NewPlayerTurn] Player 0 (Holo) is beginning turn 2
 FreeUnitsPerXTurns: During PlayerDoTurn table gTemplarKnights was:
 FreeUnitsPerXTurns: gCityThatBuilt	8192
 FreeUnitsPerXTurns: gTurnWhenBuilt	2
 FreeUnitsPerXTurns: gPlayerWhoBuilt	0

And at the end of the turn:

 ChooseArchaeologyPopup: TableSave time: 0, checksum: 168, inserts: 0, updates: 0, unchanged: 4, deletes: 0
 PMMMRebellionMain: TableSave time: 0, inserts: 0, deletes: 0, updates: 0, unchanged: 4, checksum: 76
 HoloLua: [TableSaverLoader] Preparing table data for an auto save..
 HoloLua: TableSave time: 0, inserts: 0, deletes: 0, updates: 0, unchanged: 5, checksum: 89
 Great People Names: TableSave time: 0.0030000000001564, inserts: 0, deletes: 0, updates: 0, unchanged: 1693, checksum: 141296
 FreeUnitsPerXTurns: TableSave time: 0.0070000000000618, inserts: 0, deletes: 0, updates: 3, unchanged: 1, checksum: 173

Seems to me like it's all working. At least for autosaves.

 

[LeeS]

It isn't actually persisting for me. It is directly related to whichever is the "last" mod added to the mods menu. The last one appears to get persisted, the other not so much. The inter-turn auto-saving thing makes it look like it is working, but actually using SAVE GAME menu does not.

I re-built the great generals mod so that it was the last of the two added to the MODS menu, and it persists data for the great generals mod but not for the Knights Templar mod. All I get is the -1, -1 , -1 values, even after the wonder has been created. The second unit never spawns on the turn where it ought to.

Although, it appeared like in one of my tests that everything still worked OK for the great generals mod in one test where I had both mods enabled and the Knights Templar functions were being persisted and working correctly. But I haven't been able to replicate that particular condition again. All I seem to be able to consistently create is the "sorry -- nope" condition.

On a side note, apparently GameEvents.CityConstructed does not fire when you use IGE or LiveTuner to directly "plop" the building into the city. But it works just fine so far as I can see when you let the building or wonder get constructed "naturally" as part of the turn processing.

 

[DarkScythe]

Okay.. I think I've managed to reproduce it.

It seems it may have to do with all of us defining an input handler to the same key strokes? I'd have to defer to Pazyryk here, or someone else who knows about these input systems and their contexts more.

I changed my game's autosaving to every 10 turns, instead of every 1.
I then started the game as before, and waited a few turns until I was certain that the game was not autosaving. I then spawned 3 GG's, built the Knight's Templar, and Holo's Lua also kicked in to flag that Theology had been researched, and associated building upgrades had been handed out.

I then initiated a manual save, and upon checking the savegame database, only the Knight's Templar data made it in. Civ-Linked names, and Holo's data were nowhere to be found. I re-enabled autosaving every 1 turns, and upon the next turn, autosave kicked in, and the database updated with all missing data from the Civ-Linked GG name mod, as well as Holo's flags.

I suppose I never noticed this before since my game always autosaved every turn, so when I initiated a manual save, the data was already there.

I don't know if my hypothesis is correct; potentially only Pazyryk would be able to answer that. However, if it is, I suppose one method would be to serialize all of the TableSave()'s being initiated manually by temporarily storing all of the calls in another table, then iterating through it, calling TableSave() on each one.

Edit:
I guess I should be clearer, and not over-dramatize:
When I said that other data was "nowhere to be found," I meant specifically that the new data was not available. Data that existed from the turn 1 autosave were preserved, but none of the new flags set by Civ-Linked names nor Holo were added until the next autosave.

 

[LeeS]

Okay.. I think I've managed to reproduce it.

It seems it may have to do with all of us defining an input handler to the same key strokes? I'd have to defer to Pazyryk here, or someone else who knows about these input systems and their contexts more.

I changed my game's autosaving to every 10 turns, instead of every 1.
I then started the game as before, and waited a few turns until I was certain that the game was not autosaving. I then spawned 3 GG's, built the Knight's Templar, and Holo's Lua also kicked in to flag that Theology had been researched, and associated building upgrades had been handed out.

I then initiated a manual save, and upon checking the savegame database, only the Knight's Templar data made it in. Civ-Linked names, and Holo's data were nowhere to be found. I re-enabled autosaving every 1 turns, and upon the next turn, autosave kicked in, and the database updated with all missing data from the Civ-Linked GG name mod, as well as Holo's flags.

I suppose I never noticed this before since my game always autosaved every turn, so when I initiated a manual save, the data was already there.

I don't know if my hypothesis is correct; potentially only Pazyryk would be able to answer that. However, if it is, I suppose one method would be to serialize all of the TableSave()'s being initiated manually by temporarily storing all of the calls in another table, then iterating through it, calling TableSave() on each one.

Edit:
I guess I should be clearer, and not over-dramatize:
When I said that other data was "nowhere to be found," I meant specifically that the new data was not available. Data that existed from the turn 1 autosave were preserved, but none of the new flags set by Civ-Linked names nor Holo were added until the next autosave.

Yeah. I think that pretty well conforms to what I am seeing.

I have auto-saves set at 10 turns mostly because I never much use auto-save. So I was never getting anything autosaved to make use of since I was running tests where the turn number never got that high, and the initial auto-save from TableSaverLoader was just saving the -1 values for Knights Templar since nothing had been built yet.

And yeah, I'll also have to wait and hope Pazyryk can take a look at the issue. I'll just hold off on updates to my mods until then and hope it doesn't end up where the answer is like the dll thing wherein "there can be only one".

 

[DarkScythe]

As an update, I have sent Pazyryk a message here informing him about this issue, but we'll see if he can get a chance to take a look. It doesn't seem like he's been around in a little bit.

With that said, I believe I've identified the issue, and have tentatively coded up a workaround for it.

If anyone would be interested in helping me test it out, please let me know. This would probably be an unofficial workaround until Pazyryk returns to update TSL.

 

[bane_]

Why can't I make this work? It seems so simple!!

[32705.500] MasterLuaL5R: Loading TableSaverLoader.lua...
[32705.500] MasterLuaL5R: Creating SavedGameDB tables for new game: BaneL5R_Data and BaneL5R_Info
[32705.516] Runtime Error: *path*\Lua\Lib\TableSaverLoader016.lua:377: bad argument #1 to 'pairs' (table expected, got nil)

Code:

include("TableSaverLoader016.lua")
include("L5RDataStorage.lua")

MapModData.gT = MapModData.gT or {}
gT = MapModData.gT
gT.L5RTable = gT.L5RTable or {}
L5RTable = gT.L5RTable

include("L5RLionLua.lua")
include("L5RSpiderLua.lua")
include("L5RMantisLua.lua")
include("L5RPhoenixLua.lua")

This is the only file in the InGameUIAddin, the rest are VFS'ed and include()'d here.
Please help.

 

[DarkScythe]

What exactly are you trying to do? The line numbers don't match up, and I am not quite sure why you need to nest the table initialization inside such a function; as of v0.16, TableLoad() functions well as a "new game check."

Vice Virtuoso's WFTW (and soon to be Rebellion pack) TSL code has changed substantially, especially as we've revamped his table structure alongside adding in support with my new TSL Serializer. If you are going to utilize TSL for your mod, I would recommend you take a look at including my Serializer as well.

Without seeing the contents of L5RDataStorage.lua, gT.L5RTable would return nil if DataLoad() doesn't run. Where is gT defined?

With regard to your turn-0-restart checking, I use a similar check for Holo, except it's not quite as long:

if Game.GetElapsedGameTurns() == 0 and not pPlayer:IsFoundedFirstCity() then

Usage of Game.GetElapsedGameTurns() lets you condense your two Game.GetGameTurn() checks.