[Mod Component] TSL Serializer | Add-on for TableSaverLoader

DarkScythe

Hunkalicious Holo
Joined
May 6, 2014
Messages
804
TSL Serializer
Add-on component for use with TableSaverLoader
Available at the CivFanatics Download Database
Latest Version: v3 (Released: March 28, 2015)

What This Is:

This is an add-on component for use with Pazyryk's TableSaverLoader which remedies an issue that may result in potential data loss.

Pazyryk has been notified of this issue, but as he seems to have not been around very much lately, I decided to write up a workaround in the meantime, until he returns to resume work on TableSaverLoader, and can come up with a better solution.

What The Problem Is:

It is suggested that any mod which uses TableSaverLoader for persistence of its data include this component alongside it. As discussed in the thread here, there exists an issue with the fact that, for mods which rely on the Lua-only savegame interception system, we all have to define TableSave() with the details of the table we want saved, and then further define the Input Handler to call these functions.

However, When multiple mods which rely on this system are loaded, a conflict appears -- only one Input Handler can take control of any given input! The savegame interception relies upon capturing Ctrl+S and F11, but only the last input handler loaded will be able to take control of these keys. As well, clicking on the 'Save Game' or 'Quicksave' buttons in the menu is also tied to running each mod's customized savegame interception functions.

What ends up happening, then, is that only the last mod loaded will have its data saved when a manual game save is initiated this way.

Note that autosaves are unaffected! This only affects cases where autosaves run infrequently, and a player manually saves the game and exits or reloads. Any data from other mods since the last autosave will be lost.

What This Does:

This script is a work-around for the game's inherent limitation of the input handler system interfering with TableSaverLoader across multiple mods / contexts. It remedies the input handler limitation by invoking the use of LuaEvents to trigger all associated mods to execute a TableSave() from their own context. At the same time, it re-defines the interception functions to be more generic, so that it can be re-used across contexts.

How It Works:

This component is separated into two files: A "Core" file, and a "Client" file.
The Core file sets up the "TSL Master" for the TSL Serializer for the first mod which loads with this component. This Master handles communication among all of the "Clients" which set up listeners from the Client file.

The Clients request an ID from the Master when they load. This ID is used by the Master to keep track of how many contexts it needs to call. In doing so, Clients will also "register" their tables with the Master, which allows the Master to perform some basic logic to determine whether there are any issues with, conflicts between, or duplication of data among different mods and contexts.

The Client file also hooks up with TableSaverLoader, and adds a listener which waits to conduct a TableSave() when triggered by the Master.

Usage Notes / Requirements:

Both of the files must be included alongside any mod or context which relies on TableSaverLoader. By extension, TableSaverLoader must be present. This component does not include a copy of TableSaverLoader.

Both files must be imported into VFS.

The "Core" file must not be altered. It is already included within the Client file. No further action is necessary with the Core file beyond including it in VFS.

The "Client" file must be renamed. This is important, because the Client file must sit within the same context as the parent mod.

The Client should be included after TableSaverLoader has been loaded and global tables defined, but before the first Events.LoadScreenClose function.

For example:
Code:
	...
	include("TableSaverLoader016.lua")
	gMyGlobalTable = {}
	include("ModTSLSerializer.lua")
	...

Implementation Specifics:

tableRoot and tableName must be defined, either inside the Client file, or as a global before it is included (in which case, delete / comment out tableRoot and tableName inside the Client file.)

tableRoot will be whatever is your mod's main global table. For those that have already implemented TableSaverLoader, this will be the first argument in your TableSave() function. It doesn't matter whether it points directly to the main table, or to a variable which references it, as long as it points to the table with all the data, and all the sub-tables.

For example, if you use the global MapModData table:
Code:
	MapModData.gMyModTable
	MapModData.gT.gMyModData

Simple global Lua tables are also valid:
Code:
	gMyModDataTable = {}

If you have assigned to something to reference that table instead, you may also use it.
Code:
	MapModData.gMyModTable
	gMyModTable = MapModData.gMyModTable
	gTable = gMyModDataTable

For these examples, these are valid tables to define tableRoot as:
Code:
	tableRoot = MapModData.gMyModTable
	tableRoot = MapModData.gT.gMyModData
	tableRoot = gMyModTable
	tableRoot = gTable
	tableRoot = gMyModDataTable

AVOID USING 'MapModData' or 'MapModData.gT'!
These are default tables (the latter being a TSL-specific example) and may contain a whole lot of unrelated data which may cause weird issues if saved all into your table. As the client registers the defined table with the Master, the Master will warn you if you try to do this. It will, however, not fail or prevent you from using these tables.

tableName must be a string and will be the unique name for the table when written into the savegame database. Avoid duplicates! The Master will warn you if a duplicate tableName is submitted.

For those with existing TableSaverLoader hookups, tableName will be the same as the second argument provided in your TableSave() function.

Notes For Modders:

This script introduces a new LuaEvent which effectively replaces the default TableSave():
Code:
	LuaEvents.TSLTableSave()

It sends a notice to the Master to trigger all contexts to conduct a TableSave() immediately.

The biggest usage for this is with TSL's savegame intercept functions, which have already been included.

All manual saves done by your script, as well as autosave need not be modified, although there is no harm in using the above LuaEvents replacement.

Once this component is added and included, it is important that existing TableSaverLoader hook-up code be deleted or commented out, except for OnModLoaded()!

This script already includes all of the standard TableSaverLoader hookup-code (except for OnModLoaded(),) for ease of use, especially the modified savegame interception functions which now call the new LuaEvents instead.

Leaving the original hook-up code in place may end up re-defining these functions back, and render this component useless. OnModLoaded() is not included, because implementations of this particular function vary between mods. In other words, this script only handles saving -- TableLoad() will still need to be called by your own script.

This script has been tested with a couple different mods, but by no means was it an exhaustive test, and so for the time being, consider this a potential Beta. However, I have not seen any issues arise in my tests. This has also only been tested with the latest version of TableSaverLoader as of current writing (v0.16) and may provide unexpected results when used in an environment with mixed versions, since TableSaverLoader v0.16 is incompatible with tables created with older versions of itself.

How To Use:

  1. Import Core & Client files into VFS.
  2. Rename the (non-Core) Client file into something unique.
  3. include() the Client file into your main Lua script.
    • Ensure it's included after TableSaverLoader.
    • Ensure that the global table has already been defined.
  4. Define tableRoot and tableName in, or before, the Client file.
  5. Delete or comment out existing TableSaverLoader hook-up code, EXCEPT for OnModLoaded()
    • At the minimum, SaveGameIntercept() and QuickSaveIntercept() need to be removed, or commented out. Modified versions of those functions are included in this script.
    • Every other TSL hook-up function has also been included, so it may duplicate autosaves if not removed.
    • The only function not duplicated/included is the OnModLoaded() function, as different mods have different ways of defining this.
    • If your mod has a modified InputHandler, ensure it retains the calls defined by TableSaverLoader's stock function, and leave it defined after the Client file include. It will overwrite the included function.

Download:

For the moment, please refer to the CivFanatics download database.
If need arises for an alternate download for some reason, I can provide it then.

Release History:

  • v1 (Jan 13, 2015)
    • Initial private beta testing release
  • v2 (Jan 14, 2015)
    • Separated the code into 'Core' and 'Client' files
  • v3 (Mar 28, 2015)
    • Fixed an issue causing the save game hotkeys to not work properly
    • Added version checking to the Serializer
    • Removed the need to delete tableRoot and tableName if they were already defined
    • Hooked up the debug mode so it actually does something
    • Switched standard save operation method with the previously-experimental debug method for speed
      • Old method is activated by toggling MapModData.TSLMaster.debugMode to true

Mods Utilizing TSL Serializer:

Released:
  • Fate/Stay Night: Archer's Japan [BNW] (by Vice Virtuoso)
  • Madoka Magica: Wish for the World [BNW] (by Vice Virtuoso)
  • Saints Row -- 3rd Street Saints Civilization [BNW/G&K] (by Vice Virtuoso)
  • Lyrical Nanoha -- TSAB Civilization [BNW] (by Vice Virtuoso)
  • Militaires Sans Frontieres Civilization: Subsistence [BNW] (by Vice Virtuoso)
  • Knights Templar (by LeeS)
  • Hulfgar's Modpack Complete Edition (by Hulfgar)
  • CIV-Linked Great Generals (by LeeS)
  • Jojo's Bizarre Adventure - Steel Ball Run: The United States of Valentine [BNW] (by Vice Virtuoso)
  • Hyperdimension Neptunia: Leanbox [BNW] (by Vice Virtuoso)
  • Hulfgar's Modpack Industrial Edition (by Hulfgar)
  • Piety & Prestige (by JFD)
  • Cultural Diversity (by JFD)
  • Hyperdimension Neptunia: Lowee [BNW] (by Vice Virtuoso)

Planned / In Testing:
  • Holo's Wolf and Merchant Empire (Spice and Wolf Civilization) (by DarkScythe)
  • Holo's Spice and Wolf Civilization (BETA TEST) (by DarkScythe)
  • Super Mario Bros. - The Lumas (by Typhlomence)
  • Exploration Continued Expanded (by JFD)
 
Well, this has been out for a month now.

Over 400 views and a dozen downloads by mod authors later, and still not a single comment, complaint, suggestion or feedback whatsoever?

I get that my code-writing may have improved, but it's definitely not flawless, haha. I don't even know who is actually using it in their mods.

Well, regardless, please let me know if anyone has any issues with it. At this point I only see a very minor change if I were to update this to v3, but it'd involve a whole lot of new code-writing to work in some sort of version-checking and Master-hibernation-handoff routines, so unless there are some major issues, or people getting really confused over how to integrate this, I'll hold off on it.

The bunch of mods I use personally at least have all been updated to work with this, and everything seems to be working well for me so far, so I don't see any major issues yet.

I may update the OP at some point to provide more examples and clarify certain steps, but I suppose it's a low priority, since it doesn't look like anyone's actually gotten confused enough to ask about it yet.

Edit:
Also, feel free to let me know if you have included this in your mods, so I can add them to the list in the OP.
 
The whole TSL component was a bit over my head to begin with, so could you provide some examples of implementation in the OP.

I'm not sure what I've done wrong, but whenever loading up a mod with the Serializer I get this error: "TableSaverLoader016.lua:377: bad argument #1 to 'pairs' (table expected, got nil)" and saving returns "TableSaverLoader thinks you are using TableSave before TableLoad on a loaded game; are you sure you want to do that?"

Any global tables that I define do seem to persist, however (though not simple boolean values), so it's all rather confuzzling.
 
Hey there, JFD; Thanks for taking a look at my utility.

Apologies for your confusion; LeeS suggested back in January that I should try to provide exactly that -- more examples of implementation -- but considering how lonely this thread has been, I hadn't felt compelled to do so. I suppose I should get off my lazy butt now and update my OP with more explanations and examples for such a situation going forward.

TSL itself seems quite a complex beast at first, which isn't helped at all by Pazyryk's default instructions for hooking it up as they involve many pieces of code going in several places, seemingly for maximum confusion. It's on this point that I tried to integrate most of that hook-up code into my Serializer, so in theory, it should be much easier to integrate TSL support with my add-on.

I'll assume first that you already understand what TSL is and does -- As complex as it seems, I am glad I chose to use Pazyryk's system, because it really is pretty much "set it, and forget it" to quote those old infomercials. Once it's all set up, no further action is required, and you treat it as any other standard global Lua table, which means it can inherently be expanded without mucking with TSL again. It also avoids having to litter save() and load() all over the place, which I feel is a big plus for code cleanliness.

With that out of the way, in terms of implementation, it is essentially including Pazyryk's TSL component (not included,) throwing my Serializer's files in there, and defining a couple things.

In your case, without seeing more of the code, and how you're including the files, my best guess is that it may be as simple as an incorrect order (since Lua processes items line-by-line) or it could be something wrong with the entire table setup. If you would rather not reveal code publicly, feel free to PM me the relevant section where you set up and call TSL & my Serializer, and I can give more specific advice.

However, generally speaking, you have to pick one particular table to be the "global parent table."
If you have several tables for holding different pieces of information, you can simply stick them into one "main" table, which becomes the global parent table, and is the one that you specify to TSL to save.

The fact that you say global tables still get saved, though, is a bit puzzling.
  1. Are you using MapModData as the basis for your global tables?
  2. Are you loading a game after saving without exiting to Main Menu or Windows first?

In terms of implementation:

These are the necessary files:
  • TableSaverLoader016.lua (Imported into VFS)
  • TSLSerializerCoreV2.lua (Imported into VFS)
  • TSLSerializerV2.lua (Imported into VFS and renamed.)

*** Warning: Long details ahead. ***

Let's assume you are placing everything into one main Lua file, for a Civ we'll creatively name JFD, so we'll be renaming the Serializer Client file to TSLSerializerV2_JFD.lua.

Let's also assume that you want to create tables to hold data on a player's Cities, Units, and Resources.
Code:
[B]tCities = {}
tUnits = {}
tResources = {}[/B]

However, TSL requires a single table to be saved. This can be achieved easily enough, by inserting the aforementioned tables into a parent table:
Code:
[B]tJFDCivTable = {}
tJFDCivTable.Cities = {}
tJFDCivTable.Units = {}
tJFDCivTable.Resources = {}[/B]
tCities = tJFDCivTable.Cities
tUnits = tJFDCivTable.Units
tResources = tJFDCivTable.Resources

The extra defines at the bottom are not strictly necessary, but may be helpful to have a shorter variable name when referencing that particular subtable.

Of course, that's just one way of setting it all up. Lua has several valid methods of setting up a table, so you can choose to use a shortened syntax if you wish, for example:
Code:
tJFDCivTable = {"Cities" = {}, "Units" = {}, "Resources" = {}}
This also only sets up a standard Lua table, which can only be accessed by functions operating within the same context. For cross-context data sharing via a table, you'll want to rely on MapModData -- this is a standard, internal game table, so don't try to use it as-is.

Code:
[B]MapModData.JFDCivData = MapModData.JFDCivData or {}
tJFDCivTable = MapModData.JFDCivData[/B]
tJFDCivTable.Cities = {}
tJFDCivTable.Units = {}
tJFDCivTable.Resources = {}
...

Pazyryk gives an example of MapModData.gT, but since MapModData is structured as a standard table, nothing says you must use that particular subtable, and in fact, things get weird if too many people use that subtable simultaneously. As stated, my Serializer will warn you if you try to use MapModData.gT for this reason but it won't fail.

With all that said, we've defined tJFDCivTable as our parent table, and this is what we want/need to save. This is where we call on the powers of TableSaverLoader. Of course, this means we need to include it first.

Code:
tJFDCivTable = {}
tJFDCivTable.Cities = {}
tJFDCivTable.Units = {}
tJFDCivTable.Resources = {}
tCities = tJFDCivTable.Cities
tUnits = tJFDCivTable.Units
tResources = tJFDCivTable.Resources

[B]include("TableSaverLoader016.lua")[/B]
Note that it doesn't matter where you call the include in relation to those table defines. It can come before, or after that block of defines we have here.

We now have TSL available, but not yet hooked up. This is where it gets confusing with Pazyryk's stock instructions, because you have several pieces of code, but I've incorporated most of it into my Serializer. As such, simply include the Serializer's client file -- the one we renamed earlier.

Code:
tJFDCivTable = {}
tJFDCivTable.Cities = {}
tJFDCivTable.Units = {}
tJFDCivTable.Resources = {}
tCities = tJFDCivTable.Cities
tUnits = tJFDCivTable.Units
tResources = tJFDCivTable.Resources

include("TableSaverLoader016.lua")

[B]include("TSLSerializerV2_JFD.lua")[/B]
My Serializer must be added after boh those tables have been defined and TSL has been included, so order does matter here, but as long as it's after that whole block, it's fine.
The Client internally already calls the Core file, so no action is needed other than VFS'ing it. The two files also include almost every hookup needed to get TSL working -- it will have all of the Lua savegame interception ready to go.

However, I say almost because you still have a few things to do. For one, neither TSL nor my Serializer is aware of what table it is you want to save yet. Pazyryk's standard instructions has you filling in this information all over the place, but I've condensed all that in my script so that you only need to define them once:
tableRoot and tableName.

tableRoot will be the table that we want to save; in other words, our parent table:
Code:
tJFDCivTable = {}
tJFDCivTable.Cities = {}
tJFDCivTable.Units = {}
tJFDCivTable.Resources = {}
tCities = tJFDCivTable.Cities
tUnits = tJFDCivTable.Units
tResources = tJFDCivTable.Resources

include("TableSaverLoader016.lua")

[B]tableRoot = tJFDCivTable[/B]

include("TSLSerializerV2_JFD.lua")
Note that my Serializer requires the presence of tableRoot, either before it's called via include() or defined inside the client file (there is a line in it where you do this.)

Once you have defined tableRoot, TSL will now understand what table you want to save. However, we're still missing one piece of information: tableName. This is essentially the "filename" of the table when it gets written into the savegame database. It can be anything you want, but try not to make it too generic. It must be a string.

Code:
tJFDCivTable = {}
tJFDCivTable.Cities = {}
tJFDCivTable.Units = {}
tJFDCivTable.Resources = {}
tCities = tJFDCivTable.Cities
tUnits = tJFDCivTable.Units
tResources = tJFDCivTable.Resources

include("TableSaverLoader016.lua")

tableRoot = tJFDCivTable
[B]tableName = "JFDCivStuff"[/B]

include("TSLSerializerV2_JFD.lua")

Again, tableName must be defined before or within the Client file.

Now we have TSL able to save our table. But there's one more step: Getting TSL to load said table. This is the only part that of the hookup process that I do not include with my Serializer, because implementations can vary depending on the needs of particular mods (whereas saving is pretty standard -- we all want stuff to be saved when we, uh, hit the save button.)

Pazyryk generally recommends creating an OnModLoaded() function set to fire at the very end of your Lua load chain, although if you can understand what is trying to be done, you can sneak it in elsewhere.

The purpose of firing OnModLoaded() at the end is that such a function is supposed to do one main thing: call TableLoad(). For obvious reasons, this only needs to be done once each time the mod loads, and further, Pazyryk recommends it being at the end of all the Lua loading because in certain mods, as more Lua files load, they may make additions or changes to the Lua table data.

An example would be if you set up a special unit which had to start with 20 Combat Strength, but would vary depending on era.

When your mod first loads, you could spawn such a unit, then set its Combat Strength to 20, and save that information to the table.

However, let's say when you quit the game, you save the data with this unit having 57 Combat Strength. If you attempt to load this data too early, your unit, upon loading the savegame, will momentarily see itself having 57 Combat Strength, but then be overwritten by your Lua's "default" of 20 later on.

In essence, TableLoad() should occur after all "default" changes and additions to your table data have completed, so that it can load the exact data you had before the game save.

Pazyryk, therefore, recommends some variation of this:
Code:
function OnModLoaded() --called from end of last mod file to load

	local bNewGame = not TableLoad(gT, "MyMod")

	-- < mod-specific init code here depending on bNewGame >

	TableSave(gT, "MyMod")	--I talk about this more below
end


OnModLoaded() --add at bottom of whatever file loads last for your mod

However, as long as you're not going to be modifying the table contents at a certain point, you can stick the function in there.

Practical examples:
My Spice and Wolf Civ calls this manually right after I've called my Serializer, and before its main trait script even sets up any trait-related functions:
Code:
-- Attempt to load the table
if [B]TableLoad(tableRoot, tableName)[/B] then
	print("[TableSaverLoader] Initializing to continue a loaded game...")
else
	print("[TableSaverLoader] Initializing for a new game...")
end
This format allows me to print messages depending on the outcome of TSL's ability to load the table, as a way of detecting whether it's a new game or not.

LeeS' Civ-Linked Great General Names mod uses a variation of Pazyryk's method:
Code:
-----------------------------------------------------------------------------------------------------------------------------------------------------
--create tables for all the 'fall-back' great people names and for active civs in the game. DO NOT CHANGE.
-----------------------------------------------------------------------------------------------------------------------------------------------------
gT = {}
gMasterUsedGreatPeopleNames = {}
gFallbackGreatGeneralNames = {}
gFallbackGreatAdmiralNames = {}
gFallbackGreatMerchantNames = {}
gFallbackGreatEngineerNames = {}
gFallbackGreatScientistNames = {}
gFallbackGreatProphetNames = {}
gFallbackGreatArtistNames = {}
gFallbackGreatWriterNames = {}
gFallbackGreatMusicianNames = {}
gQActiveCivPlayerNames = {}
gAllGreatPeopleNames = {}
gJustUsedName = {}
gTempData = {}
-----------------------------------------------------------------------------------------------------------------------------------------------------
--set up the table saver loader master table tree. DO NOT CHANGE.
-----------------------------------------------------------------------------------------------------------------------------------------------------
gT = { gMasterUsedGreatPeopleNames,
	gQActiveCivPlayerNames,
	gFallbackGreatGeneralNames,
	gFallbackGreatAdmiralNames,
	gFallbackGreatMerchantNames,
	gFallbackGreatEngineerNames,
	gFallbackGreatScientistNames,
	gFallbackGreatProphetNames,
	gFallbackGreatArtistNames,
	gFallbackGreatWriterNames, gFallbackGreatMusicianNames, gAllGreatPeopleNames }

-----------------------------------------------------------------------------------------------------------------------------------------------------
--table saver/loader saving functions. DO NOT CHNAGE
-----------------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------
--TableSaverLoader Hookups
-------------------------------------------------------------------------------------------------
include("LRSGPNTSLSerializerV2.lua")
function OnModLoaded() --called from end of last mod file to load

	local bNewGame = not [B]TableLoad(gT, TableName)[/B]

	if bNewGame then
		print("New Game")
		else print("Loaded from Saved Game")
		gLoadFromSave = 1	--LRS added this line

	end

	TableSave(gT, TableName)
end
This is largely a direct variation of Pazyryk's recommendation, which should be self-explanatory, although LeeS added some extra directives in the case of a new game other than simply printing a message.

ViceVirtuoso's most recent Civ eschews all that function nonsense, and calls it directly:
Code:
--Start TSL if any HDN Civs are in the game
if bAnyHDNCivs == true then
	print("At least one of Vice's HDN Civs are present in the game.")

	MapModData.HDNMod.Shares = {}
	MapModData.HDNMod.GreatGameDevProgress = {}
	MapModData.HDNMod.GreatGameDevProgressNeeded = {}
	MapModData.HDNMod.UsedCityHeaders = {}
	MapModData.HDNMod.TransformedThisTurn = {}
	MapModData.HDNMod.ExtraShareSources = {}
	HDNMod = MapModData.HDNMod

	include("TableSaverLoader016.lua");
	include("TSLSerializerV2Neptunia.lua");
	include("GTAS_PlotIterators.lua");

	[B]TableLoad(HDNMod, "HDNMod")[/B]
	TableSave(HDNMod, "HDNMod")
	HDNMod = MapModData.HDNMod
else
	MapModData.HDNMod = nil
end

With all that said, we'll assume that we won't be changing the data after the initial setup, so we'll go with ViceVirtuoso's last example:

Code:
tJFDCivTable = {}
tJFDCivTable.Cities = {}
tJFDCivTable.Units = {}
tJFDCivTable.Resources = {}
tCities = tJFDCivTable.Cities
tUnits = tJFDCivTable.Units
tResources = tJFDCivTable.Resources

include("TableSaverLoader016.lua")

tableRoot = tJFDCivTable
tableName = "JFDCivStuff"

include("TSLSerializerV2_JFD.lua")

[B]TableLoad(tableRoot, tableName)[/B]

...And we're done! TSL is properly hooked up. Note that I've (re-)used tableRoot and tableName because, well, we already defined it. It doesn't quite matter, though, as long as they point to the same table and name that you defined (directly or indirectly) but I personally like having everything use those variables, so that I only ever need to make changes in one place.

You may notice that most of the examples include a TableSave() immediately after the TableLoad() -- this is harmless being there, and technically optional, but it's there to capture any sort of immediate changes prior to the player receiving control of the game.

Apologies it got so long, but that should be a general enough overview to guide most people through.
If you're still stuck, or have specific circumstances, please let me know and I can go through that with you separately.

As an aside, I am currently working on, and about to test, v3 of my Serializer. It fixes a moderate issue that I was just notified of by Typhlomence, and should be available within a few hours, or by tomorrow at the latest.
 
Here is the code as it is within my lua for the latest version of my Knights Templar mod. I have color-coded all the TableSaverLoader hook-up commands in blue, and all the places where I use the saved-and-loaded table information in green:
Spoiler :
Code:
---xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
--include commands for other lua files that are required
---xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

[color="blue"]include("TableSaverLoader016.lua")[/color]

---xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
--Variables Definitions not related to TableSaverLoader
---xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

gValidBuildingPromos = {}
gValidBuildingExperience = {}

gKnightsTemplar = GameInfoTypes.BUILDING_KNIGHTS_TEMPLAR
gKnightsTemplarTurnIncrement = 10
iFreeUnit = GameInfoTypes.UNIT_CRUSADER_LEGION
iNumFreeUnit = 1
iOBStech = "TECH_RIFLING"
iOBStechID = GameInfo.Technologies[iOBStech].ID

---xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
--flag variables for processing control. DO NOT CHANGE. GENERALLY, NO CHANGES SHOULD BE MADE BELOW THIS COMMENT LINE.
---xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

[color="blue"]gLoadFromSave = 0[/color]	--do not change from 0


---xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
--TableSaverLoader Table definitions
---xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

[color="blue"]gMasterTemplarsTableLRS = {}
gTemplarKnights = {}

gMasterTemplarsTableLRS = { gTemplarKnights }

tableRoot = gMasterTemplarsTableLRS
tableName = "KnightsLRS"

include("LRS_KnightsTemplar_TSL_Addon.lua")

---xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
--TableSaverLoader Hookups
---xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

function OnModLoaded() --called from end of last mod file to load

	local bNewGame = not TableLoad(tableRoot, tableName)

	if bNewGame then
		print("New Game")
		else print("Loaded from Saved Game")
		gLoadFromSave = 1	--LRS added this line

	end

	TableSave(tableRoot, tableName)
end[/color]

---xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
--game initialization and table saver/loader loading functions. DO NOT CHANGE
---xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

--borrowed and adapted from Darkscythe
[color="green"]function KnightTemplarGameInit()
	KnightTemplarDataLRS()
end
Events.LoadScreenClose.Add(KnightTemplarGameInit)

---xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
--fills in the table gTemplarKnights with all the info for when/if the wonder has been built. DO NOT CHANGE
---xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

function KnightTemplarDataLRS()
	if gLoadFromSave == 1 then
		print("Knight Templar Table Was loaded from saved game")
	end
	if gLoadFromSave == 0 then
		print("Knight Templar Table Was created for a new game")
		gTemplarKnights.gPlayerWhoBuilt = -1
		gTemplarKnights.gCityThatBuilt = -1
		gTemplarKnights.gTurnWhenBuilt = -1
		--table.insert(gTemplarKnights, xxxxx)
	end
end[/color]

---xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
--during loading build the table with the valid building-granted trained free promos
--	gValidBuildingPromos = {}	--bool Unit:IsPromotionValid(PromotionType promotion)
--doing the lua in this way should give compatibility for all expansion levels and for all mods that add buildings or wonders
---xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

local freeUnitDetails = GameInfo.Units{ID=iFreeUnit}()
sRequiredCombatClass = freeUnitDetails.CombatClass
sRequiredUnitDomain = freeUnitDetails.Domain

for row in GameInfo.Buildings() do
	iBuildingID = row.ID
	sPromoString = tostring(row.TrainedFreePromotion)
	if sPromoString ~= "nil" then
		bWasAPromoMatch = false
		bWasACombatClassMatch = false
		for row in GameInfo.UnitPromotions_UnitCombats() do
			if tostring(row.PromotionType) == sPromoString then
				bWasAPromoMatch = true
				if tostring(row.UnitCombatType) == sRequiredCombatClass then
					bWasACombatClassMatch = true
				end
			end
		end
		if bWasAPromoMatch then
			if bWasACombatClassMatch then
				gValidBuildingPromos[iBuildingID] = GameInfoTypes[sPromoString]
			end
		else gValidBuildingPromos[iBuildingID] = GameInfoTypes[sPromoString]
		end
	end
end

---xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
--during loading build the table with the valid building-granted free experience
--	gValidBuildingExperience = {}
--	sRequiredCombatClass = tostring(GameInfo.Units{ID=iFreeUnit}().CombatClass)
--	sRequiredUnitDomain = tostring(GameInfo.Units{ID=iFreeUnit}().Domain)
--doing the lua in this way should give compatibility for all expansion levels and for all mods that add buildings or wonders
---xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx


print("In looking through buildings to create the XP table")
for row in GameInfo.Buildings() do
	iTotalFreeExperience = 0
	iBuildingID = row.ID
	sBuildingType = tostring(row.Type)
	iExperience = row.Experience
	if iExperience > 0 then
		iTotalFreeExperience = iTotalFreeExperience + iExperience
	end
	for row in GameInfo.Building_DomainFreeExperiences() do
		if tostring(row.BuildingType) == sBuildingType then
			if tostring(row.DomainType) == sRequiredUnitDomain then
				iTotalFreeExperience = iTotalFreeExperience + (row.Experience)
			end
		end
	end
	for row in GameInfo.Building_UnitCombatFreeExperiences() do
		if tostring(row.BuildingType) == sBuildingType then
			if tostring(row.UnitCombatType) == sRequiredCombatClass then
				iTotalFreeExperience = iTotalFreeExperience + (row.Experience)
			end
		end
	end
	---------------------------------------------------------------------------------------------------
	--store the building ID and the total XP given by it
	--do not make additions to the table gValidBuildingExperience anywhere but in the following lines
	---------------------------------------------------------------------------------------------------
	if iTotalFreeExperience > 0 then
		gValidBuildingExperience[iBuildingID] = iTotalFreeExperience
	end
end

---xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
--function to trap info when the wonder is completed
--(Hook) GameEvents.CityConstructed(ownerId, cityId, buildingType, bGold, bFaithOrCulture);
---xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

function KnightsTemplarCompleted(ownerId, cityId, buildingType, bGold, bFaithOrCulture)
	if buildingType ~= gKnightsTemplar then return end
	[color="green"]gTemplarKnights.gPlayerWhoBuilt = ownerId
	gTemplarKnights.gCityThatBuilt = cityId
	gTemplarKnights.gTurnWhenBuilt = Game.GetGameTurn()[/color]
end
GameEvents.CityConstructed.Add(KnightsTemplarCompleted)

---xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
--function that actually spawns the units once the wonder has been completed
---xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

iTurnIncrement = math.ceil(gKnightsTemplarTurnIncrement * GameInfo.GameSpeeds[PreGame.GetGameSpeed()].TrainPercent / 100)
function FreeUnitsPerXTurns(iPlayer)
	local pPlayer = Players[iPlayer]

	if [color="green"]gTemplarKnights.gPlayerWhoBuilt == -1[/color] then return end
	if [color="green"]gTemplarKnights.gPlayerWhoBuilt ~= iPlayer[/color] then return end
	--Define local variables
	local pPlayer = Players[iPlayer]
	pCity = pPlayer:GetCityByID([color="green"]gTemplarKnights.gCityThatBuilt[/color])

	--Actions to be taken to pop out free units

	local iOriginalBuildTurnNumber = [color="green"]gTemplarKnights.gTurnWhenBuilt[/color] 
	local iCurrentTurnNumber = Game.GetGameTurn()
	local iTurnsElapsed = iCurrentTurnNumber - iOriginalBuildTurnNumber
	if (iTurnsElapsed % iTurnIncrement == 0) or (Game.GetGameTurn() == [color="green"]gTemplarKnights.gTurnWhenBuilt[/color]) then

		--------------------------------
		-- local for check for whether wonder is obsolete below
		--------------------------------

		iMyPlayersTeamNumber = pPlayer:GetTeam()
		local iMyTeamHasRifling = Teams[iMyPlayersTeamNumber]:IsHasTech(iOBStechID)

		--------------------------------
		--commands to add XPs below
		--------------------------------

		local iXPS = 0
		for k,v in pairs(gValidBuildingExperience) do
			if pCity:IsHasBuilding(k) then
				iXPS = iXPS + v
			end
		end

		--------------------------------

		--command to spawn units below

		--------------------------------
		if not iMyTeamHasRifling then
			iUnitPlot = pCity:Plot()
			pUnitToSpawn = pPlayer:InitUnit(iFreeUnit, iUnitPlot:GetX(), iUnitPlot:GetY())
			pUnitToSpawn:JumpToNearestValidPlot()
			pUnitToSpawn:SetExperience(iXPS)
			for k,v in pairs(gValidBuildingPromos) do
				if pCity:IsHasBuilding(k) then
					pUnitToSpawn:SetHasPromotion(v, true)
				end
			end
		end
	end
end
GameEvents.PlayerDoTurn.Add(FreeUnitsPerXTurns)


---xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
--misc support functions, some of which are only kept as fallback references in case I f something up
---xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

------------------------------------------------------------
---- whoward69's player cannot train a special unit function
------------------------------------------------------------

local iUnitRanger = iFreeUnit

GameEvents.PlayerCanTrain.Add(function(iPlayer, iUnit) 
  if (iUnit == iUnitRanger) then
	return false;
  end

  return true
end)



--xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
-- final TableSaverLoader hook-up command
--xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

[color="blue"]OnModLoaded()[/color]
Even though the code itself may be a bit OP in what-all it is doing, remarkably very little of it is actually to do with the TableSaverLoader system once using DarkScythe's add-on serializer.

[edit]My Civ-Linked Great General Names mod has an lua that is 830 lines long, and uses and adjusts my persisted tables during each game-session much more heavily than this example does. But the amount of code used to hook-up TableSaverLoader for persisting all that table info and all that in-session manipulation of data is virtually identical between the two mods.
 
Thanks for the addendum, LeeS. The color coding definitely makes it easier to tell apart which sections are doing what.

I definitely tried to streamline the ease of using Pazyryk's TableSaverLoader, though.
Either way, the important thing to note is not the whole wall of code, but rather the placement and order of the different pieces of information (table/variable defines, and file includes.)
 
Thanks for the thorough response, DarkScythe. The issue, for me, was that I wasn't clear on the placement of TableRoot and TableName. It seems to be all working as intended, now, however.

For your reference, this is for Cultural Diversity, but will also be adapted into Piety & Prestige and Exploration Continued Expanded.
 
Ah, thanks for the update. I'm glad it was helpful in clearing things up for you!
If you have any other issues, please feel free to let me know.

I'm also in the process of testing v3 of the Serializer right now, so please check back in a bit to grab the newer version before you push out your updates to those mods. Updating should be as simple as replacing the V2 files with the V3's; nothing should change, code-wise, besides your include() statement to call the V3 client.

I've been avoiding mods which use TSL without this component, mainly due to my other mods relying on TSL, but I'll check out those mods. I had planned to provide support for them, but unfortunately I won't be able to until I finish the additional development of my Civ (mod support will be provided in a separate add-on mod I'll be making once I get the rest of my planned major features implemented in the Civ.) I also have yet to play with them, since I'm still working on balancing my new features for a Vanilla game.

ViceVirtuoso tipped me off last night that you might have been looking to add this into Piety and Prestige, but I wasn't aware that those others also needed it. Either way, I'll update my OP with those mods in the list of 'confirmed compatibility' once I get v3 published.

I'll also probably update my OP with more examples like that giant post, and probably some more emphasis about the ordering of the items in case more people fall into the same type of confusion.
 
The current version of CulDiv and ExCE don't use TSL yet, but the next update to CulDiv will be assigning cultures on a per citizen basis, and this is information that obviously needs to be stored. I will be sure to await the TSL Serializer update before release.

For CulDiv support, the latest version should make it much easier to add new culture groups - which will probably be more useful to your mods than the existing groups. Feel free to let me know on the CulDiv thread or in PM if you need help with that support, when (if) you get the time to add it.
 
Very interesting, thanks for the heads-up, JFD. If you don't mind, I'll place all of those under the "Testing" section of my OP.

Definitely will look into the CulDiv groups and such, but I'm not sure when I can do so. The rewrite of Holo's main Lua is taking far longer than I expected. Still, at some point I will get around to including support for it; I have more plans than I have available time to make happen. :/

In any case,

TSL Serializer v3 has been released!

Please visit the CivFanatics Downloads Database to grab the newest version.

It is strongly advised that every mod author who has made use of v2 update to v3 as soon as possible. v2 is now no longer supported.

v3 fixes a moderately important issue relating to the save game and quicksave hotkeys not working properly. Many thanks to Typhlomence for pointing out the issue which slipped past me and my various beta testers during development.

Changelog is as follows:
v3 (Mar 28, 2015)
  • Fixed an issue causing the save game hotkeys to not work properly
  • Added version checking to the Serializer
  • Removed the need to delete tableRoot and tableName if they were already defined
  • Hooked up the debug mode so it actually does something
  • Switched standard save operation method with the previously-experimental debug method for speed; Old method is activated by toggling MapModData.TSLMaster.debugMode to true

v3 should behave just fine with v2, although depending on load order, v2 may duplicate attempts to save data. This is mostly harmless, since the data is already saved, and TSL knows not to update the database again since nothing has changed. The only potential side effect to the duplication is a tiny reduction in responsiveness when saving the game.

The hotkeys issue, while potentially critical due to the stated goal of this component, is actually not too severe -- so long as a single mod loaded runs v3, the hotkeys will function, although there may be a few errors printed to the logs from those mods which use v2. It is for this reason that updating is recommended.

Finally, the OP will be updated over the next few days to include more information related to this component, and how to set up properly.

For now, the OP will be simply updated with mentions of the new version, and a refreshing of the list of supported mods.

Changes are as follows:
Released:
  • Added Hyperdimension Neptunia: Leanbox [BNW] (by Vice Virtuoso)
  • Added Hulfgar's Modpack Industrial Edition (by Hulfgar)
Planned / In Testing:
  • Removed Madoka Magica Civilization Pack: Rebellion [BNW] (by Vice Virtuoso) -- Mod has been reworked to remove the need for TSL
  • Added Super Mario Bros. - The Lumas (by Typhlomence)
  • Added Piety & Prestige (by JFD)
  • Added Cultural Diversity (by JFD)
  • Added Exploration Continued Expanded (by JFD)
 
Top Bottom