SaveUtils.lua -- Itemized Data Storage Across Mods

I just meant hack as in it doesn't eliminate the need for a tool like SaveUtils. Perhaps it would be more accurate to call it an optimization, as it does offer the potential for improved performance.

I believe Dale is modifying the global scope of TopPanel, which automatically loads from the InGame.xml. Ideally, the operation should be moved to an event after load screen close, but that's easier said than done.

I'm a little torn on how to approach the issue. Sharing the data in global scope rather than load screen close results in an InGame.xml and thus mod merging complications. But I'm guessing people are going to frequently want to share something in the global scope of the built-in lua files. I haven't decided how to make this both flexible and still easy to implement. What I can say is that for anything done after load screen close, SaveUtils will function as expected. For things done before load screen close, and in particular global scope, SaveUtils can appear broken. I don't care for the appearance of broken. :p

At the very least, I'm going to need to add a more explicit explanation in the documentation.
 
Actually, my call to my Lua is direct from TopPanel:UpdateData(). So I'm not using ingame.xml or any other file. The only time my script is touched is during runtime.

I'm not using setCacheState as I believed from the doco not using it defaults it to 1. But I do declare the MY_MOD thing (whatever it's called).
 
What I mean to say is that the TopPanel lua is called according to the InGame.xml file, it's not a InGameUIAddin.

UpdateData is called from the global scope of the TopPanel lua. You can find it near the bottom of the file.

Code: 
-- Update data at initialization
UpdateData();

Anything in global scope is going to be executed before all files have loaded, ie: InGame.xml loads TopPanel, then it loads InGameUIAddins last. All of this occurs before any events, such as load screen close.
 
Well I've pulled my lua out of TopPanel and put into InGameUIAddin. Everything works perfectly now, data is saved and loaded between game saves of the mod.

However, I still get the cache error. Even though everything works. :p

Now I've got the same problem as Alpaca. On the first turn after loading game/save, or when something changes happiness during that turn, it doesn't reflect via my script till the next turn. Time to go hunting. :p
 
Whys said:
That's interesting. I run into a similar problem with my Building Resources mod. Moving a unit will also update the happiness in TopPanel.

As to the cache error, can you show me the script?
Click to expand...

Sure thing:

ICSKiller.lua
Code: 
-- ICSKiller
-- Author: Dale
-- DateCreated: 11/21/2010 10:25:44 PM
--------------------------------------------------------------

include( "SaveUtils" ); MY_MOD_NAME = "DaleKentICSKiller";

-- Globals
----------

Player.bNeedToLoadData = true;
Player.bNeedToSaveData = false;
Player.ICSHappinessFromBuildings = 0;
Player.ICSCheckHandicapHappy = 0;
Player.ICSHappinessFromHandicap = 0;
Player.ICSBureaucracyUnhappiness = 0;

-- Functions
------------

function ICSGetHappiness()
	for iPlayerID = 0, GameDefines.MAX_CIV_PLAYERS-1, 1 do
		local count = 0;
		local pPlayer = Players[iPlayerID];
		if( pPlayer.bNeedToLoadData ) then
			pPlayer.ICSHappinessFromBuildings = load( pPlayer, "ICSHappinessFromBuildings" );
			pPlayer.ICSCheckHandicapHappy = load( pPlayer, "ICSCheckHandicapHappy" );
			pPlayer.ICSHappinessFromHandicap = load( pPlayer, "ICSHappinessFromHandicap" );
			pPlayer.ICSBureaucracyUnhappiness = load( pPlayer, "ICSBureaucracyUnhappiness" );
			pPlayer.bNeedToLoadData = false;
		end
		ICSSetHappinessFromBuildings();
		ICSSetBureaucracyUnhappiness();
		local iPoliciesHappiness = pPlayer:GetHappinessFromPolicies();
		local iResourcesHappiness = pPlayer:GetHappinessFromResources();
		local iExtraLuxuryHappiness = pPlayer:GetExtraHappinessPerLuxury();
		local iBuildingHappiness = pPlayer.ICSHappinessFromBuildings;
		local iGarrisonedUnitsHappiness = pPlayer:GetHappinessFromGarrisonedUnits();
		local iTradeRouteHappiness = pPlayer:GetHappinessFromTradeRoutes();
		local iReligionHappiness = pPlayer:GetHappinessFromReligion();
		local iNaturalWonderHappiness = pPlayer:GetHappinessFromNaturalWonders();
		local iExtraHappinessPerCity = pPlayer:GetExtraHappinessPerCity() * pPlayer:GetNumCities();
		local iMinorCivHappiness = 0;
		local iBureaucracyUnhappiness = pPlayer.ICSBureaucracyUnhappiness;
		-- Loop through all the Minors the active player knows
		for iPlayerLoop = GameDefines.MAX_MAJOR_CIVS, GameDefines.MAX_CIV_PLAYERS-1, 1 do
			iMinorCivHappiness = iMinorCivHappiness + pPlayer:GetHappinessFromMinor(iPlayerLoop);
		end
		if (pPlayer.ICSCheckHandicapHappy <= 0) then
			pPlayer.ICSHappinessFromHandicap = pPlayer:GetHappiness();
			pPlayer.ICSCheckHandicapHappy = 1;
		end
		local iHandicapHappiness = pPlayer.ICSHappinessFromHandicap;
		local iTotalHappiness = iPoliciesHappiness + iResourcesHappiness + iBuildingHappiness + iMinorCivHappiness + iHandicapHappiness + iTradeRouteHappiness + iReligionHappiness + iNaturalWonderHappiness + iExtraHappinessPerCity - iBureaucracyUnhappiness;
		pPlayer:SetHappiness(iTotalHappiness);
		if( pPlayer.bNeedToSaveData ) then
			save( pPlayer, "ICSHappinessFromBuildings", pPlayer.ICSHappinessFromBuildings );
			save( pPlayer, "ICSCheckHandicapHappy", pPlayer.ICSCheckHandicapHappy );
			save( pPlayer, "ICSHappinessFromHandicap", pPlayer.ICSHappinessFromHandicap );
			save( pPlayer, "ICSBureaucracyUnhappiness", pPlayer.ICSBureaucracyUnhappiness );
			pPlayer.bNeedToSaveData = false;
		end
	end
end

function ICSSetHappinessFromBuildings()
	for iPlayerID = 0, GameDefines.MAX_CIV_PLAYERS-1, 1 do
		local count = 0;
		local pPlayer = Players[iPlayerID];
        if( pPlayer:IsAlive() ) then
			if( pPlayer:GetNumCities() > 0 ) then
		        for cityIndex = 0, pPlayer:GetNumCities() - 1, 1 do
    				local pCity = pPlayer:GetCityByID( cityIndex );
    				if( pCity ~= nil ) then
						for building in GameInfo.Buildings() do
							if (pCity:IsHasBuilding(building.ID)) then
								if (building.Happiness > 0) then
									if (building.Type == "BUILDING_COLOSSEUM") then
										count = count + (pCity:GetPopulation() / 4);
									elseif (building.Type == "BUILDING_THEATRE") then
										count = count + (pCity:GetPopulation() / 4);
									elseif (building.Type == "BUILDING_STADIUM") then
										count = count + (pCity:GetPopulation() / 4);
									else
										count = count + building.Happiness;
									end
								end
							end
						end
					end
				end
            end
			if( count ~= pPlayer.ICSHappinessFromBuildings ) then
				pPlayer.ICSHappinessFromBuildings = count;
				pPlayer.bNeedToSaveData = true;
			end
        end
	end
end

function ICSSetBureaucracyUnhappiness()
	for iPlayerID = 0, GameDefines.MAX_CIV_PLAYERS-1, 1 do
		local count = 0;
		local pPlayer = Players[iPlayerID];
        if( pPlayer:IsAlive() ) then
			if( pPlayer:GetNumCities() > 0 ) then
		        for cityIndex = 0, pPlayer:GetNumCities() - 1, 1 do
					count = count + (cityIndex / 3);
				end
			end
			if( count ~= pPlayer.ICSBureaucracyUnhappiness ) then
				pPlayer.ICSBureaucracyUnhappiness = count;
				pPlayer.bNeedToSaveData = true;
			end
		end
	end
end

Events.SerialEventGameDataDirty.Add(ICSGetHappiness);
Events.SerialEventTurnTimerDirty.Add(ICSGetHappiness);
Events.SerialEventCityInfoDirty.Add(ICSGetHappiness);
Events.GameplaySetActivePlayer.Add(ICSGetHappiness);

ICSGetHappiness();

The error:
Code: 
Runtime Error: [string "C:\Users\Thesh\Documents\My Games\Sid Meier..."]:428: attempt to index global 'cacheState' (a nil value)
Runtime Error: Error loading ICSKiller.lua.
Runtime Error: [string "C:\Users\Thesh\Documents\My Games\Sid Meier..."]:428: attempt to index global 'cacheState' (a nil value)
 
Whys, from what I see there's no initialisation for the cacheState variable. So when loading you try to index a nil variable.

Try replacing the declaration (line 175 in SaveUtils) by cacheState = {} or in getCacheState catch for cacheState being nil and return nil in that case
 
Whys said:
I just meant hack as in it doesn't eliminate the need for a tool like SaveUtils. Perhaps it would be more accurate to call it an optimization, as it does offer the potential for improved performance.

I believe Dale is modifying the global scope of TopPanel, which automatically loads from the InGame.xml. Ideally, the operation should be moved to an event after load screen close, but that's easier said than done.

I'm a little torn on how to approach the issue. Sharing the data in global scope rather than load screen close results in an InGame.xml and thus mod merging complications. But I'm guessing people are going to frequently want to share something in the global scope of the built-in lua files. I haven't decided how to make this both flexible and still easy to implement. What I can say is that for anything done after load screen close, SaveUtils will function as expected. For things done before load screen close, and in particular global scope, SaveUtils can appear broken. I don't care for the appearance of broken. :p

At the very least, I'm going to need to add a more explicit explanation in the documentation.
Click to expand...

You are right, of course. For saving, it definitely still needs handling. For sharing, automatic namespace handling is of course useful, too
 
Dale said:
The error:
Code: 
Runtime Error: [string "C:\Users\Thesh\Documents\My Games\Sid Meier..."]:428: attempt to index global 'cacheState' (a nil value)
Runtime Error: Error loading ICSKiller.lua.
Runtime Error: [string "C:\Users\Thesh\Documents\My Games\Sid Meier..."]:428: attempt to index global 'cacheState' (a nil value)
Click to expand...
Tuner fails reporting the correct file with any file that has include statements because it always reports only the main file having errors. Luckily that line number is the actual include-files line number without any lines added to it from the main file. In this case I think it was referring to SaveUtils-file.

It seems that SaveUtils is initialized with shareData-function, but this function is called first time during LoadScreenClose-event. If a mod includes this file and calls immediately (=before LoadScreenClose-event) load or save-functions then getCacheState-function runs into an error because cacheState has not yet been initialized with shareData-function.

I agree that if you wish to share data with other mods then you should use that LoadScreenClose-event as a synchronizing point, but what if you just want to load your own saved data?
 
alpaca said:
Whys, from what I see there's no initialisation for the cacheState variable. So when loading you try to index a nil variable.

Try replacing the declaration (line 175 in SaveUtils) by cacheState = {} or in getCacheState catch for cacheState being nil and return nil in that case
Click to expand...

Actually, it's designed that way for a reason. It's initialized upon load screen close. The code for that is directly below line 175. I could initialize it prior to sharing it, but ideally, it shouldn't be used until after load screen close, so as to avoid merging issues with other mods. Anyway, I'm implementing a flag that will let the modder decide from outside of the file.
 
Whys, just want to thankyou for your mod and help. VERY useful! In fact, in one of my upcoming mod blogs I'll mention it and show how it can make a certain unknown Civ feature AWESOME! :D

Also wanted to mention I will no longer be posting at or visiting CFC. Please find me at WePlayCiv if you need anything.
 
:) Version 7.

Okay, cache and cache-state are now instantiated immediately and you can now call share_SaveUtils() to also share them immediately, otherwise they will still be automatically shared on Events.LoadScreenClose(). However, calls to load(), save(), clear(), and sync() will output a warning to the lua console when calling these functions prior to the data being shared. To supress these warnings, include SaveUtils in the following manner.

Code: 
WARN_NOT_SHARED = false; include( "SaveUtils" ); MY_MOD_NAME = "MyMod";

You can find the full documentation in SaveUtils.lua.
_
 
alpaca said:
You are right, of course. For saving, it definitely still needs handling. For sharing, automatic namespace handling is of course useful, too
Click to expand...

There is some advantage to ShareData when it comes to retrieving data tho. Such as obtaining the entire table of shared globals, or the entire table of a particular context's globals, or the entire table of context tables.

Even if modders could agree on a similar shared data structure attached to a built-in super-global, like say GameInfo, by that point the syntax demands are pretty much the same while at the same time it becomes more difficult to sift custom shared data from preexisting data for that class.

It may not be very often anyone will want to inspect the shared data on whole like that, but it does offer the flexibility to be used that way should someone desire it. For example, ShareData allows context-globals to be added by automatic integer index. So you can just dump globals with no name space whatsoever and those unnamed context-globals can then be easily retrieved as a group by themselves. Again, perhaps useful for someone, maybe not, but it provides the flexibility in a manner predictable to other mods.

Also, while I am not aware of any potential conflicts, there is the matter that built-in super-globals are frequently manipulated by the core engine. While that does not make custom data stored there inherently volatile, it could potentially prove to be a hostile data scope. Whereas ShareData is only being used for one thing, making it predictable and easily manipulated.
 
Whys,

I want to state my appreciation for what you're doing here. I'm working on a long-term (i.e., it'll be done someday) total conversion mod that has many complicated feats/events that have to be remembered. I don't much care about sharing between mods. I see the main use of this (for me) is the ability to save different kinds of objects (especially tables) rather than simply text for plot, player, units. I have a few questions for you below. (I've read much but not all the thread, perhaps understanding 60% of it, so please forgive if these are already answered.)
  1. Does this interfere with my ability to use the base "script" functions to store text for plot, player, units?
  2. If I'm only storing simple, easily parsed information for specific plot, player, units, then how much is your system going to cost me (in processing time) compared to using the existing ability to save text plus some simple parsing?
  3. I don't quite understand where the info is being stored. In the gamesave file? In cache? If I send a gamesave file to my friend (running the same mod on a different computer) will he be able to open it and have it work (i.e., all feats/events and such remembered from gamesave file rather than cache)?
[Edit: Pretend I am almost but not quite a "smart kid" when answering. I've written about a dozen short PERL and python scripts for work and play, but you'll quickly loose me if you start on scopes and threads and such.]
 
Pazyryk said:
  1. Does this interfere with my ability to use the base "script" functions to store text for plot, player, units?
  2. If I'm only storing simple, easily parsed information for specific plot, player, units, then how much is your system going to cost me (in processing time) compared to using the existing ability to save text plus some simple parsing?
  3. I don't quite understand where the info is being stored. In the gamesave file? In cache? If I send a gamesave file to my friend (running the same mod on a different computer) will he be able to open it and have it work (i.e., all feats/events and such remembered from gamesave file rather than cache)?
Click to expand...

1.) If you mean can you still call pPlayer:SetScriptData(), then the answer is no. Realize, SaveUtils is in fact saving a string in the ScriptData, but not just your string, any string any other mod needs to save there as well. It allows all the mods to share the ScriptData slot without interfering with one another. This requires that SaveUtils be allowed to use the ScriptData pretty much exclusively. But as long as all mods use it, then there won't be any conflicts.

2.) The only things that might significantly affect performance would be the saving of large and complex tables of tables etc, as these have to be converted into text and then converted back when retrieved. Obviously, text converts to text very easily, so there shouldn't be much to worry about there. Plus you still gain the advantage of being able to save as many items as you want to the target, not just one. Additionally, if you don't disable the cache, then you only need to deserialize the data once, not every time you ask for it. Big performance improvement there over even just the traditional ScriptData functionality. And tho you say you don't care about sharing, the fact is SaveUtils cache is designed to be automatically shared (via ShareData), so when another mod deserializes the data, your mod doesn't have to, it can just get it from the shared cache. That can also provide significant performance improvement, so sharing might be more important than you think.

3.) It saves it to the game save file at some point, but that's all back end code. SaveUtils simply uses the ScriptData. How script data is treated on the back end, I don't know. What I can say is yes, you can save a file, send that file to a friend, they can load the mod, then load the file, and everything should work. The cache is enabled by default, but the default also serializes the data when ever you call save(). So each time you call save(), that data is immediately stored in the script data slot and should carry over when you reload the game later.
 
Thanks for the answers above! I've read the InGame.xml discussion. Along with confusion and a headache, I got a suspicion (perhaps unfounded) that SaveUtils won't work if I'm trying to retrieve saved information from within an existing but modified UI lua file (since these load from InGame.xml before your SaveUtils, if I understand correctly).

To be a little more specific, my main mod lua script generates some novel unit info and saves using pUnit:save(key, integer). Now I want to get that integer and display it in the unit panel, which means that I need to modify the existing UnitPanel.xml and UnitPanel.lua (writing this from memory so I'm not sure if that name is exactly right) and I need to use pUnit:load(key, integer) from within UnitPanel.lua. So then my questions:

1. Do I need to add
Code: 
include( "SaveUtils" ); MY_MOD_NAME = "myMod";
at the top of both my main mod lua script and the modified UnitPanel.lua?

2. Will the above have a load order problem?
 
You must log in or register to reply here.

Similar threads

Chrisy15
CanSaveUnit is flawed - Introducing CanSaveUnitGood
Replies
2
Views
144
Chrisy15
Chrisy15
G
SaveUtils doesn't work with Unitpanel.
Replies
0
Views
739
GreatBayethe
G
sman1975
"Merlin" Scenario Engine (MSE)
2
Replies
23
Views
2K
sman1975
sman1975
P
How add your custom data into gamecore.
Replies
0
Views
1K
Pusperk
P
SoehnelS
Available Objects in LUA
Replies
5
Views
2K
SoehnelS
SoehnelS
Top Bottom