TableSaverLoader, for persisting Lua table data through game save/load

[bane_]

I had this:

gT = {}
gT.iNumFantasticGardensAI = 0

GameEvents.PlayerBuilt.Add(
function(iPlayer, iUnit, iX, iY, iBuild) 
	if iBuild == GameInfoTypes["BUILD_FANTASTIC_GARDENS"] then
		gT.iNumFantasticGardensAI = gT.iNumFantasticGardensAI + 1
	end
end)

...in another file.

 

[DarkScythe]

Not sure how those files are all called together, but if they're in separate files, it's possible you're running into variable scoping issues?

 

[bane_]

I added to the other file and it's working. Sorry about that guys.

Vide below...

 

[bane_]

Ok, I definetively don't know how this works. Now I'm getting this:

[51900.674] Runtime Error: C:\Users\bane_\Documents\My Games\Sid Meier's Civilization 5\MODS\Crane Clan (v 1)\Lua/AIHelper.lua:305: attempt to index global 'gT' (a nil value)

Even though the rest is working perfectly.
The code in question is this:

function(iPlayer, iUnit, iX, iY, iBuild) 
	local pPlayer = Players[iPlayer]
	if iBuild == GameInfoTypes["BUILD_FANTASTIC_GARDENS"] then
[B]		gT.iNumFantasticGardensAI = gT.iNumFantasticGardensAI + 1[/B] --troublesome line 305
		print("Garden built.")
	end
	if not pPlayer:IsHuman() then
		local pUnit = pPlayer:GetUnitByID(iUnit)
		if pUnit:IsHasPromotion(GameInfoTypes["PROMOTION_SELECTED_WORKER"]) then
			pUnit:SetHasPromotion(GameInfoTypes["PROMOTION_SELECTED_WORKER"], false)
		end
	end
end)

...which is in a different Lua file than this:

gT = {}
gT.iNumFantasticGardensAI = 0

function OnModLoaded() [...]

What am I doing wrong this time?

 

[Pazyryk]

Just to expand on the error statement, gT is nil in the scope of the function at the time it runs.

So:

1. is the second file in the same Lua state?
2. does the code in the bottom block run at all?
3. does the code in the bottom block run before the function call?
4. did you at some other place assign nil to gT?

#1 depends on how you added or included these two files. If by using include statement, then same state; if as separate InGameUIAddin, then different state. Different Lua states don't share the same globals (except for some defined by Lua or Firaxis such as "print", "MapModData", etc.). #2 & 3 can be answered with print statements. #4 seems unlikely but I added it for thoroughness. A better question is, did you ever assign to gT again, thus possibly nilling it?

 

[bane_]

1. Both InGameUIAddin. I have no 'include' in the file with the error... looks like this is the issue, correct?
2. Absolutely. The function was called at turn ~80.
3. Checking... No. Only three occurrences, two being commented out on the TableSaverLoader file and the other being the one I posted.

 

[DarkScythe]

I believe his asking about "same Lua state" means if both files are being called into the same "parent" file. If your two files are totally separate, without being "include"d into any other parent file, then you have run into scoping issues -- what is in one file cannot see what is in another file. They need something to actually link the code in the two files together.

This was what I was hinting toward with my previous reply to you.

 

[bane_]

Oh. I see.
Since I thought it was working, I didn't pay attention to your other post, my bad.

I believe I am trying things a step farther than I should right now. I need some basics first, I'll get a good read at the Lua manual (thanks for that, by the way Pazyryk ).

Thanks both for the help!

 

[DarkScythe]

Haha, scoping issues get all of us at one point or another, apparently.

I do wonder if this could be made easier with some tweaks to the implementation instructions, though.
I kind of like how I have it going in my mod, although it's fairly basic, and probably won't work for more complex hookups.

 

[Pazyryk]

Unless you are doing something very advanced, you probably want all your mod's Lua to run in the same state. That means one file added with InGameUIAddin. That file has include statements in it that bring other Lua file code into the same state.

But since your modding you should know that all the UI files are each in their own state. Basically all that means is that they don't share globals that you define (like gT in this case). However, they do share globals that Lua defines (e.g., "print") or that Firaxis defined ("MapModData"). MapModData is a table available from all Lua states (not just UI but even other mods), so it is an easy way to share data among states.

 

[LeeS]

All I know at the moment is that I'm 100% getting TableSaverLoader to persist data for me across game saves and 0% attempting to use SaveUtils or NewSaveUtils. Today I spent ten minutes, maybe, re-reading the new instructions on how to hook up version 16 in an lua-only method, and then adding those hook-ups to an lua I was working on for a friend. No berfs, barfs, or hick-ups -- it just works

[edit]though the 1st mod I used it on I made some dreadful newbie mistakes that DarkScythe's patience helped me resolve, after I had bashed my head on the wall and nearly given up. Plus, well, banging my head on the apostrophes deal

 

[DarkScythe]

So, this is interesting.. I just loaded up a couple mods since I was a few versions out of date, and freshly updated them.

However, FireTuner reported this:

Loading TableSaverLoader.lua...
Runtime Error: Failed to iterate table due to query error.  Check Database.log for more details.

Of course, the Database.log revealed nothing of interest related to tables. (It instead has tons of errors from other people's mods.)

Any ideas what this might be? Going to try saving and reloading, and then restarting a new game with the same Civs to see if I can reproduce this.

EDIT:

After extensive testing, and with Vice Virtuoso's assistance, I have identified the problem.
Turns out his mods were using an old version of TableSaverLoader (v0.14) while mine used the newest one (v0.16.)
This caused some compatibility issues, as his mods loaded after mine did, and it seems that the last TableSaverLoader.lua to be loaded into VFS became the one that would be used by every mod.
In effect, this meant that my mod attempted to use the newly-introduced TableLoad() method of determining a new game with a version of TableSaverLoader that didn't yet support it, I think.

I tested this by removing TSL from my mod, which caused Lua errors (referencing a nil TableLoad()) when I attempted to load it on its own.
However, it loaded fine when the other mods using TSL were also enabled -- it displayed the same error as posted earlier.
Once I swapped out his old version of TSL with the new one, everything worked fine, even though my own mod was still missing the file.

Based on my results, here is my question:
Is it recommended/suggested, or even safe, to have multiple instances of TableSaverLoader going?

My thought is to rename TableSaverLoader.lua on my end to something unique so that I won't run into this issue in the future with other mods, but Vice Virtuoso worried that multiple instances could cause more problems.
Is TSL safe to run multiple instances if each one has a unique filename, or is there a way to enforce loading of the latest version if multiple versions are loaded?

 

[Pazyryk]

This caused some compatibility issues, as his mods loaded after mine did, and it seems that the last TableSaverLoader.lua to be loaded into VFS became the one that would be used by every mod.
In effect, this meant that my mod attempted to use the newly-introduced TableLoad() method of determining a new game with a version of TableSaverLoader that didn't yet support it, I think.

I believe you've identified the problem exactly.

Based on my results, here is my question:
Is it recommended/suggested, or even safe, to have multiple instances of TableSaverLoader going?

(Notwithstanding the issue above...) Yes, it should be fine as long as they are totally separated (i.e., each mod runs in own Lua state using it's own DB tables defined by the unique string you supply). Even if both mods include a file called "TableSaverLoader.lua", it is really two different "closures" on that file. So even the variables declared at the file level have separate existence in each mod.

My thought is to rename TableSaverLoader.lua on my end to something unique so that I won't run into this issue in the future with other mods, but Vice Virtuoso worried that multiple instances could cause more problems.
Is TSL safe to run multiple instances if each one has a unique filename

Renaming file should work. That way both versions exist and your mod would have access only to the newer version. The other solution would be to update 0.14 to 0.16 in the other mod. The problem you had was using new 0.16 functionality with 0.14 loaded. The reverse won't be a problem. But that puts burden on other modder. I'm positive that 0.14 can be updated to 0.16 without any "mod compatibility" problem; however, it will break gamesaves.

, or is there a way to enforce loading of the latest version if multiple versions are loaded?

Good question. It would be nice if there were, since TSL is backward compatible (new versions support anything old versions did). I don't know of any way right now, but I'll think about it.

 

[DarkScythe]

I understand the various instances being in separate states, and the fact that using v0.14 functionality with v0.16 is fine.

The problem I had was that it seems like since both of our mods used the same TableSaverLoader.lua filename, his mod which loaded after mine did seemed to overwrite my mod's TableSaverLoader.lua in the VFS, thus forcing my mod to use his older version, even when my mod had the latest version.

I'm not sure if there's any way to force these mods to play happy when this happens (I've notified Vice Virtuoso and he will be updating it to v0.16 with his next slew of updates) but I'm looking at the possible scenario where a mod has been abandoned by its author for whatever reason, or might be tightly written around the way one particular version of TableSaverLoader works.

That said, it seems the simplest solution of renaming the file so that both versions can load is the best bet, and the easiest to implement.

Thanks for the confirmation.

 

[Pazyryk]

I'm pretty sure that there won't be any way to enforce loading order, which would require re-coding of "include" (which we don't have access to).

Maybe I should make it a general recommendation to rename the file. Or perhaps I should name the file TableSaverLoader016.lua myself so modders always include the right file automatically.

 

[DarkScythe]

If you want to avoid versioning conflicts like this, I believe adding the version number officially to the filename would do the trick very well.

Anyone who uses v0.16 will know it, and it won't conflict with users running v0.17, for example. Yet, everyone who runs those versions won't clutter up the VFS with multiple redundant copies, and hopefully only those that actually do modify the file in some way would save it with a custom file name.

 

[Pazyryk]

OK, done. It's now: "TableSaverLoader016.lua". (No changes in file itself.)

 

[DarkScythe]

Thanks, Pazyryk!

There's still the issue of people modifying the file and/or updating the code without updating the filename, but I suppose those will be the exceptions.

Edit:
I just saw you amended your earlier post with explanations, and a bit about breaking gamesaves.
This is amusing, since I ran into that once I replaced his older version with 0.16 and attempted to load a test game.
The Lua log in FireTuner started throwing up lots of very angry warning messages, and was generally NOT happy.

!!!! TableSaverLoader thinks you are using TableSave before TableLoad on a loaded game; are you sure you want to do that? !!!!

 

[DarkScythe]

So, more interesting tidbits related to the issue of filenames I've discovered today.

I'm not sure why, or if it was intended, or who/what is to blame if it wasn't intended.. but at least through Lua, include() doesn't search for an exact match.

With Vice Virtuoso's assistance, and suggestion, I grabbed a copy of Really Advanced Setup, which he says somehow hijacks our PlotIterators.lua file (using whoward's code.)
On a cursory glance, I wasn't sure why, as it had a different filename (GTAS_PlotIterators.lua) and even the function name was different.

Apparently, since both files ended with 'PlotIterators.lua' our scripts' include() somehow decided to pull the GTAS version from the VFS instead of our explicitly-defined "PlotIterators.lua" which broke our mods with their dependence on PlotAreaSpiralIterator().

I further tested this with TableSaverLoader to confirm.

My Lua is set to call:

include("TableSaverLoader.lua")

I then conducted 3 scenarios:

1. TableSaverLoader.lua - Loads fine, as expected.
2. XYZTableSaverLoader.lua - Loads fine even though logic says it should fail!
3. Loader.lua - Fails; Probably not long enough to match the minimum filename called in the include()

This explains why Really Advanced Setup's GTAS_PlotIterators were hijacking our PlotIterators file.

Again, I don't know why, but I suppose the lesson here is that if you want to make filenames unique (as I was going to do with my mod's version of TableSaverLoader) you need to put your unique identifier at the end of the filename as a suffix, before the extension. Having it as a prefix apparently isn't enough, if the rest of the filename is still the same!

Setting it between the "common" filename and the extension should be enough to defeat anyone else loading TableSaverLoaderXYZ.lua, unless someone else decides to be clever and make an ABCTableSaverLoaderXYZ.lua.

 

[whoward69]

So, more interesting tidbits related to the issue of filenames I've discovered today.

I'm not sure why, or if it was intended, or who/what is to blame if it wasn't intended.. but at least through Lua, include() doesn't search for an exact match.

It's intentional - see post #7 here