Requesting assistance with Lua and TableSaverLoader

[DarkScythe]

Hello, everyone,

I'm in the middle of making my first ever mod for Civ5, and it is fairly ambitious -- a whole new Civ. Thanks to everyone's help here, I've gotten virtually all of the XML finished extremely quickly and basically working without issue.

I'm at the point where I need to finish off the last couple parts of the UA with Lua, but unfortunately, I have no experience with Lua.

I've been looking at examples, and reading up where I can, though documents that get too technical are beyond my ability. I've also had some help from a few members here for specific issues, and a general guide through various little things I can/should do in Lua.

Right now, though, I am stuck.
I am attempting to come up with code that will iterate through all of a city's plots to determine a few conditions, namely its owner and if it has any resources on it.
That part is done, and it works. However, I need to be able to save this data as I have a need to check these plots each turn, and as I was advised it would not really be possible, I was recommended to use either ShareUtils or TableSaverLoader. I chose the latter, since it appeared to be newer, but otherwise either one would probably do what I need it to do.

I've already gotten some of the code in place, and have properly hooked it up to my Lua scripts, and it is at least hooking into the autosave system correctly.

My problem is that TableSaverLoader did not really come with any documentation. The author, in his thread, states that he provides an ExampleMain.lua, but I cannot seem to find it. Without the documentation, I have been working off of referencing code from other mods (such as the CL Australia civ) and trying to reverse-engineer the code necessary to communicate updates with the database. Unfortunately, as I have little experience with Lua, on top of lacking the documentation to explain how to properly utilize TableSaverLoader, I have not been successful at getting it to work.

A snippet of my code, at the stage where it needs to inject values into a table for storage, is as follows:

print("Plot does not already exist in the table, adding it now..")
		gT.TestStorage[pPlot] = {}
		gT.TestStorage[pPlot][iX] = pPlotX
		gT.TestStorage[pPlot][iY] = pPlotY
		gT.TestStorage[pPlot][sImprovement] = sPlotImprovement

To explain my code:
While I am iterating through a city's plots, selected plots will have their pPlot (pointer?) and its X/Y coordinates, as well as what improvement has been built on it, saved to the database. My desire is that with this data, upon successive "scans" over the city's plots, the code can link it back up to data stored in this database, and compare to see if anything (notably, the improvement on it) has changed or not. There are a few other scenarios I need to handle, but for now I'm limiting myself to this, until I get a working prototype for me to expand on.

I am unsure if my syntax is even correct, which only exacerbates the problem.
Edit: I've also tried wrapping the data being assigned to the table fields in curly braces (for example, {pPlotX}) but it doesn't seem to matter. Either way, the Lua log / Firetuner complains that the table index is nil.

Any help, advice, or direction would be deeply appreciated.

Thank you!

 

[whoward69]

What are the values of pPlot, iX, iY and sImprovement - all are used as Lua variables so they must have values given to them somewhere.

If pPlot is a Lua plot object you can't use it as an index into the gT.TestStorage[] array - as the index must be an integer or a string (or something convertible to a string)

Assuming pPlot is a Lua plot object, you can get its unique map index and use that as the index into the gT.TestStorage[] array. And given that you can always determine X and Y for a plot from its unique map index you don't need to store those. So the only thing you end up storing is sPlotImprovement

gT.TestStorage[pPlot:GetIndex()] = {}
gT.TestStorage[pPlot:GetIndex()]["Improvement"] = sPlotImprovement

or all on one line

gT.TestStorage[pPlot:GetIndex()] = {Improvement = sPlotImprovement}

Given a plot index, you can get the Lua plot object back as

pPlot = Map.GetPlotByIndex(iPlotIndex)

and the co-ords are then just pPlot:GetX() and pPlot:GetY()

 

[DarkScythe]

Thank you for your response, whoward!

To start from the top, value of pPlot is pCity:GetCityIndexPlot(i) as it was defined in the beginning of my loop to iterate through a city's plots. iX and iY are not defined; I had intended to use them as the name for the values of pPlotX and pPlotY which are defined as pPlot:GetX() and pPlot:GetY(), although thinking about it now, if I were to use them as a name, I needed to enclose them in quotes. Apologies there, as I had been iterating on the code all day, and iX and iY were used for a different purpose before I attempted to swap in the table code.

I'm still getting the hang of what values I can reference directly, and which ones I can't. The code has failed a few times when, during my debug statements, trying to print the values of pPlot and pCity would fail, yet other times it could print "successfully" but as a table: XYZ value instead.

I checked the Civ modiki and was unable to find GetIndex() there..
But I assume that this index will not change during the course of the game? I can't rely on the loops iterating through all the plots in the same order each turn, so I needed some way of making sure each plot was tracked back to the same identifier. If pPlot:GetIndex() does so, I might not need X and Y at all, since it would be superfluous and would always be the same.

As well, if I am going to be calling pPlot:GetIndex() over and over again, would it help to make that a local variable ahead of time?

I'm a little confused by the last line though; Can I not use the new ID provided by GetIndex() to alter the plot in question, or must it be converted back to the Lua object/pointer that pPlot was originally in? If so, how do I define iPlotIndex as a value from the table? (That is, how would I query the table for just the one value?)

Sorry if that sounds noobish; I'm still trying to wrap my head around some tutorial code for lua tables and pairs/ipairs, but it seems everyone is fond of using 1-letter variable names, which I find extremely difficult to comprehend.

As well, I want to tack on extra conditions, such as a "CheckedThisTurn" boolean into the table, so that I don't alter the same plot twice. If I read correctly, the method I am using to iterate over plots will simply grab plots 3 tiles out from each target city, regardless of who actually owns it. If this happens, I want the code to know that it's already worked on that plot for the turn. Of course, I haven't quite figured out yet how I would reset those to false at the beginning of the next turn, though. I had found on another thread suggested to use pPlot:GetCityPurchaseID() to try and find out the specific city a plot is attached to, but when I was debugging my code to find failures, I could not find GetCityPurchaseID() on the modiki either, and had removed it.

In either case, to set those values I assume it's along the lines of:

gT.TestStorage[pPlot:GetIndex()] = {}
gT.TestStorage[pPlot:GetIndex()]["Improvement"] = sPlotImprovement
gT.TestStorage[pPlot:GetIndex()]["CheckedThisTurn"] = true

Thank you again for your assistance.

 

[DarkScythe]

Well..

I modified my code a bit with the updated syntax as I understand it, but I'm still running into problems.

Rather than determining pPlot:GetIndex() each time I update the table, I added a local variable declaration, iPlotIndex, immediately after the if condition in my loop that tests that pPlot is not nil.

However, upon attempting to run this in-game, the script fails and halts at that declaration with this error:
attempt to call method 'GetIndex' (a nil value)

Am I not supposed to define it into a variable?

 

[DarkScythe]

Ah!!

I think I've got it, now.
whoward, that was quite the hint I needed -- thank you!
GetIndex() didn't seem to work, but with some Google-fu, I found another thread started by you which mentions that GetIndex() is deprecated, and to use GetPlotIndex() instead.

Unfortunately, I wasn't sure if that was for your modded DLL or not (I want to avoid adding anything more than just XML and Lua for now) and spent some time thinking about how to get it working with the coordinates instead. In the end, I decided it couldn't hurt, and tried replacing GetIndex() in my code with GetPlotIndex() -- at which point, everything started working!

The rest of my conditional checks using this stored data is also working well, so again, thank you!

I am sure I will run into another Lua issue soon, though, but for now I am thankful that this bit is finally working successfully.

 

[Pazyryk]

If you have (or expect) a lot of different needs for persisting data, TableSaverLoader is worth getting up and running. But if you have a very specific need and there is an easy alternative, then I would suggest the alternative.

In this case,

>pPlot:SetScriptData("Was improvement here")
>pPlot:GetScriptData()
Was improvement here

You can use this to persist text strings for any plot. Two things to keep in mind:
1. Get will return "" (not nil) before anything is assigned to it
2. If you are doing some kind of funny serialization (trying to pack data into a small string) make sure none of the bytes are 0.

 

[DarkScythe]

Thanks for the suggestion, Pazyryk!

I am anticipating trying to persist at least a few things, and while I did not know about SetScriptData() before, I think now that I have your TableSaverLoader thing working, I might as well keep it, as it should offer greater flexibility in case I need to do anything fancier later.

I also need to save multiple pieces of information per plot, which I'm unsure if SetScriptData() can do.

My greatest issue was really that I had no idea how to use your utility properly, and even now I'm still confused over whether or not I need to modify the SaveMenu.lua or not.

 

[Pazyryk]

The main problem with using TableSaverLoader is that the base game dll gives you no way to intercept before a gamesave. That's a problem because you want to run TableSave right before the gamesave file is generated. Otherwise, if you change some mod data after TableSave and then save game and exit, that change was not caught by TableSaverLoader.

In the past, I got around this with some very messy hacks involving modding SaveMenu.lua and (even worse) really hacking the heck out of autosave (because it runs exactly in a place where there is are no GameEvents allowing you to intercept there). I solved all these problems more recently by modding dll to generate GameEvents right before saves.

It's all a matter of timing though. You just have to be aware of when your data (that is supposed to be persisted) changes and when TableSave runs.

 

[DarkScythe]

Well, as of right now, by default, I think the code (sans SaveMenu.lua alteration) properly saves on autosaves, quicksaves, and Barbarian turns.

If I need to catch some mid-turn saves, is there any harm in including the TableSave() function at the end of whatever function I run that actually does stuff with it?

To be specific, my function runs in order to scan the plots around a city, and use the stored information (or write information about a newly discovered/acquired plot) in order to do something with it. If I stick TableSave() at the end of that function, would it then technically allow mid-turn saves of the table data?

Outside of that, if the game were to close unexpectedly, you would have to pick up with your last autosave/quicksave/manualsave, and any progress after the last one would've been lost regardless.

Is there a situation where this loss of mid-turn table data does not coincide with loss of regular turn data?

 

[Pazyryk]

If I need to catch some mid-turn saves, is there any harm in including the TableSave() function at the end of whatever function I run that actually does stuff with it?

To be specific, my function runs in order to scan the plots around a city, and use the stored information (or write information about a newly discovered/acquired plot) in order to do something with it. If I stick TableSave() at the end of that function, would it then technically allow mid-turn saves of the table data?

Each time you run TableSave() it will test all of your persisted data against their current values (this is Lua only) and then update DB if something changed (only updating what changed). The change test (Lua only) is probably inconsequential if the number of items is not huge and if you are only running it several times and not 100s of times. The DB updates take up a minimum amount of time that depends on your hard drive (1/6 sec on my old rotating disc HD with Civ5 running). The power of TableSaverLoader is that it can pack many changes (iirc, 600) into one update so you only pay that 1/6 second price once. So, in short, running TableSave() several to 10s of times in a turn is probably not noticable. But running it 100 times will be a real drag even if only one data point is changed each time.

As I said in the other post, DB updates are weird and not the same as C++ writing to a gamesave file. Each DB update requires that the hard drive rotate around twice (assuming you have one that rotates) regardless of how little data is changed. I don't really know how SSDs factor in here (I haven't tested TableSaverLoader on huge data sets since I updated my computer to an SSD).

 

[DarkScythe]

Ah, thanks for the information.

While my functions will loop over many plots and make changes depending on the conditions of those plots, I would stick the manual TableSave at the very end of those functions, after they have completed all their tasks, so as to take advantage of the multiple-changes-written-at-once feature. In this case, I expect there to be 1~3 functions that run on a turn-by-turn basis, as well as 3~4 functions that run based on specific user actions (Player founds a city, Player Captures/Loses a city, etc.) Very well under a dozen, and certainly nowhere near 100 times.

From what you've posted, I think I should be safe enough, from a performance standpoint, to go ahead with this plan.