SaveUtils.lua -- Itemized Data Storage Across Mods

[Androrc the Orc]

Since the GetScriptData and SetScriptData functions were removed from some instances, SaveUtils may no longer be working.

But Firaxis has added something much more practical, which it uses in it's scenarios, which is a more direct connection to save data. For those that wish to use the new functions, use this code as an include:

gbLogSaveData = false;

-- Connection to the Modding save data.  Keep one global connection, rather than opening/closing, to speed up access
g_SaveData = Modding.OpenSaveData();
g_Properties = nil;
-------------------------------------------------------------- 
-- Access the modding database entries through a locally cached table
function GetPersistentProperty(name)
	if(g_Properties == nil) then
		g_Properties = {};
	end
	
	if(g_Properties[name] == nil) then
		g_Properties[name] = g_SaveData.GetValue(name);
	end
	
	return g_Properties[name];
end
--------------------------------------------------------------
-- Access the modding database entries through a locally cached table
function SetPersistentProperty(name, value)
	if(g_Properties == nil) then
		g_Properties = {};
	end
	
	if (g_Properties[name] ~= value) then
		g_SaveData.SetValue(name, value);
		g_Properties[name] = value;
	end
end

Here is one example of how to use these GetPersistentProperty and SetPersistentProperty functions:

include( "NewSaveUtils" ); -- use the aforementioned code as an include

---------------------------------------------------------

function TheHumanGenomeProjectEffects( iPlayer )
	local player = Players[iPlayer]
	local team = Teams[player:GetTeam()]

	if (team:GetProjectCount(GameInfo.Projects["PROJECT_THE_HUMAN_GENOME_PROJECT"].ID) > 0 and GetPersistentProperty("TheHumanGenomeProjectApplied") ~= 1) then
		player:ChangeExtraHappinessPerCity(1)
		SetPersistentProperty("TheHumanGenomeProjectApplied", 1);
	end
end

GameEvents.PlayerDoTurn.Add( TheHumanGenomeProjectEffects )

 

[Moaf]

Hello,
I've played around a bit with the modding.setvalue, and it seems to be _very_ slow compared to the setscriptdata method (probably because it's a direct db call).
Is there a way to intercept the save events themselves (e.g. quicksave or normal saving), so that it's possible to call the setvalue only when there is actually a persisting of save data needed?

BR,
Moaf

 

[ww2commander]

Apology for the stupid questions up front as I am still trying to learn LUA....

How does one save a global table using the GetPersistentProperty and SetPersistentProperty functions? Would it be as simple the following to save data:

SetPersistentProperty("GlobalValueTable", gGlobalValueTable);

I gather this must be the correct approach (from what I understand) as LUA can save a table within tables!

 

[Androrc the Orc]

Apology for the stupid questions up front as I am still trying to learn LUA....

How does one save a global table using the GetPersistentProperty and SetPersistentProperty functions? Would it be as simple the following to save data:

SetPersistentProperty("GlobalValueTable", gGlobalValueTable);

I gather this must be the correct approach (from what I understand) as LUA can save a table within tables!

I imagine so, but I haven't tested it.

 

[whoward69]

Set/GetPersistentProperty() are wrappers around the underlying Modding database simple name/value methods, so you can only store "simple" values - ie anything that will serialize to and deserialize from a string representation.

If you want to save information in tables you'll need to explore the Query() method on the Modding databases and create/access your own tables.

 

[whoward69]

g_SaveData = Modding.OpenSaveData();
g_SaveData.SetValue("test", {x=255, y=0, z=0, w=255});
[COLOR="Red"]Runtime Error: [string "g_SaveData.SetValue("test", {x=255, y=0, z=..."]:1: bad argument #2 to 'SetValue' (Must be of type boolean, nil, number, or string)
stack traceback:
	[C]: in function 'SetValue'
	[string "g_SaveData.SetValue("test", {x=255, y=0, z=..."]:1: in main chunk[/COLOR]

 

[Gedemon]

Hello,
I've played around a bit with the modding.setvalue, and it seems to be _very_ slow compared to the setscriptdata method (probably because it's a direct db call).
Is there a way to intercept the save events themselves (e.g. quicksave or normal saving), so that it's possible to call the setvalue only when there is actually a persisting of save data needed?

BR,
Moaf

It seemed also very slow to me, that's why I use saveutils for tables and Modding.OpenSaveData() for simple values I don't need often.

As serialization of large table was taking some time even with saveutils, I also used something to catch most of the save events in R.E.D. WWII (rerouting the key shortcut for quicksave and "hijacking" the normal save menu) if you want an example.

 

[ww2commander]

This questions is probably better directed at Gedemon:

Being confused about something I never really understood to begin with (!!!)........Did the patch/G&K break the saveutils used in scenarios like R.E.D WWII or am I confusing two separate ways of saving data?

 

[Gedemon]

SaveUtils mod was able to save on Players, Plots, Units. I use Players and Plots (the latter not in R.E.D.), that's why R.E.D. WWII was not broken even if the units script data has been removed (but if they remove Players and Plots script data, I'm toast)

 

[Androrc the Orc]

SaveUtils mod was able to save on Players, Plots, Units. I use Players and Plots (the latter not in R.E.D.), that's why R.E.D. WWII was not broken even if the units script data has been removed (but if they remove Players and Plots script data, I'm toast)

Oh, excellent that you can confirm that it still works on players!

 

[Gedemon]

Yes it still works on players and plots. I didn't test on units, I've no mods using that one.

 

[ww2commander]

Just to make sure I understand your response correctly, the following snippet from your RED WWII mod should still work fine.

I ask as I use this code in my scenario, but have never understood the black magic that occurs behind it!

function LoadAllTable()
	Dprint("-------------------------------------")
	Dprint("Loading data tables ...")	
	g_UnitData = LoadUnitData()
	g_UnitStatistic = LoadUnitStatistic()
	g_ReinforcementData = LoadReinforcementData()
	g_TrackAllCombats = LoadTrackAllCombats()
end
function SaveAllTable()
	Dprint("-------------------------------------")
	Dprint("Saving data table ...")
	SaveUnitData( g_UnitData )
	SaveUnitStatistic( g_UnitStatistic )
	SaveReinforcementData( g_ReinforcementData )
	SaveTrackAllCombats( g_TrackAllCombats )

 

[Gedemon]

Yes, will work, but I've changed it since to save one big table instead of multiple table (save times), check corresponding code of actual mod.

And see LoadUnitData() in utils, it just call the real magic from SaveUtils.lua, but don't ask me what's going on there, that I do not know, except that I had to kill a chicken before writing the include(), it's very important to make the mod work. I pretty sure it's thanks to the chicken that script data was removed by something else than Players if you ask me.

 

[ww2commander]

but don't ask me what's going on there, that I do not know, except that I had to kill a chicken before writing the include(), it's very important to make the mod work.

Really gives new meaning to the phrase 'the blind leading the blind'!

I guess as long as it keeps working then no further chickens must be sacrificed to the Firaxis gods of change!

 

[Pazyryk]

One DB transaction takes a good fraction of a second, whether you happen to be doing 1 or 10000 inserts in that single transaction. That's why my TableSaverLoader component wraps all of your changes into one (or just a couple) transactions at game save.

 

[DonQuiche]

A nice trick I thought about when writing an article on the wiki on data persistence: an easy way to store tables (for convenience and speed since there is now only one value to store), is to use loadstring on deserialization. Basically we store an lua code string that will create a table when we load it. I proposed a little function that does the job.

Also, regarding the "quicksave and save menu hijack" that Gedemon talked about... As he described it it causes conflicts with other mods (IGE for example if the user uses its quick-save or save-then-reload features, or any mod that wants to override the save menu). However, did anyone try to just replace the UI.SaveXXX method?

I know that objects like units and players are context-bound (the player object you get in context 1 is not the same than you get in context 2, although you can communicate them to other threads and they work fine). However, maybe that static objects like UI are not context-bound? And even if they are, maybe their metatable is not context-bound and can be assigned a new function?

EDIT: Nevermind, I just tried it: UI is contex-bound and anyway it is a read-only table. And its metatable is a... boolean?!

 

[daviiadams]

How do you go about saving more than one table? I'm sure this is going on in some mods, but unsure on what the syntax should look like (whether the key should be inside quotation marks etc) as there appears to be a fair few variations within the help in SaveUtils(?)

 

[DonQuiche]

How do you go about saving more than one table?

You put them all in one big table.

 

[daviiadams]

You put them all in one big table.

Ah ok, that could prove interesting

 

[DonQuiche]

Now there are likely other ways to do it but I am not familiar with SaveUtils and I did not want to bother inspecting its code (especially after those glasses of excellent wine). As a result I opted for a safe and funny answer, probably not the most useful one though. That being said, a single table is likely to offer the best performances if this matters for your mod.