SaveUtils.lua -- Itemized Data Storage Across Mods

[Ryan F. Mercer]

Yep, just did that. For SaveUtils.lua, Set VSF to true in the file properties in modbuddy. Fixed it for me.

edit: btw, just noticed, print statements now show the full file path preceeding the output, but error statements are still truncated! *sigh*...

 

[alpaca]

Yep, just did that. For SaveUtils.lua, Set VSF to true in the file properties in modbuddy. Fixed it for me.

edit: btw, just noticed, print statements now show the full file path preceeding the output, but error statements are still truncated! *sigh*...

Yeah that's totally annoying because it's so hard to find the file. A solution is to wrap the whole thing in a function and use pcall, which allows you to print the error message but it's still annoying.

 

[lemmy101]

Ow wow awesome cakes. Thanks alpaca!

 

[Thalassicus]

Two questions... I think the second one relates to this tool, so I figure this is as good a place to post as any. First, is there a way to control execution order of scripts / files, and if so, how? The necessary order is:

1. InfoTooltipInclude.lua contains variable buildingStat
2. Anything in mods that alters buildingStat
3. Everything that uses InfoTooltipInclude.lua to create tooltips

The reason is tooltips are built once and cached when the game loads, so if #3 happens before #2 then mods have no effect.

Second, if it is possible to ensure this execution order, are these the steps to take to make buildingStat a variable other mods can alter?

1. Include SaveUtils
2. Add a line after defining buildingStat:
   share("Tooltips.buildingStat", buildingStat)

 

[Ryan F. Mercer]

I think the best thing to do would be to include ShareData.lua as a LuaContext near the top of InGame.xml. If you need to share the cached script data then call share_SaveUtils() explicitly somewhere after the include("SaveUtils); MY_MOD_NAME=... But if you don't need to keep this data across saved games, then just use the share functionality directly with share() and exchange what ever data you want between lua files, including function pointers.

 

[SamBC]

This makes me think it would be a really good idea to define a consensual standard defining a new AddIn type, and a change to a core file to load them, for things that really need to be done early - and agree a file to put this in, and everyone put it in like that so that conflicts don't happen.

Sometimes I just feel like modular modding is so near and yet so far - partial support in Lua, no support in graphics, great support in the DB...

On an entirely conversational note, would it be sensible to have a similar standard for registering functions to a luaevent that gets called once per turn, per player (with relevant arguments), and once per turn, per plot, and so on? I know that such iteration isn't cheap, but it's sometimes the only way to do certain things, so at least we should consolidate such loops... because of the way variables work in context, I believe, it would even be fairly easy to work with things like Building Resources. You'd just want either a standard file per 'loop', or a standard file each for various reasonable combinations of loop. I ask because I'm about to start working on self-upgrading improvements (if my tuner-based testing of the principle works out), and I'd love to do it in a way that's going to be more efficient when combined with anything else that does big loops. I have a horrible image of half-a-dozen mods loaded together, each doing either a per-city or per-plot loop every turn...

 

[Thalassicus]

I think the lua code runs fast enough time isn't really much of an issue, it's usually autosaves that add the most delay between turns for me. What is an issue is the half-done modular support... but as Alpaca pointed out elsewhere, if/when we can modify the c++ portion of the source code and create our own core game dll it'll basically make any mod that does so incompatible with other mods, regardless of how much lua modularity we have.

 

[Ryan F. Mercer]

This makes me think it would be a really good idea to define a consensual standard defining a new AddIn type, and a change to a core file to load them, for things that really need to be done early - and agree a file to put this in, and everyone put it in like that so that conflicts don't happen.

Sometimes I just feel like modular modding is so near and yet so far - partial support in Lua, no support in graphics, great support in the DB...

On an entirely conversational note, would it be sensible to have a similar standard for registering functions to a luaevent that gets called once per turn, per player (with relevant arguments), and once per turn, per plot, and so on? I know that such iteration isn't cheap, but it's sometimes the only way to do certain things, so at least we should consolidate such loops... because of the way variables work in context, I believe, it would even be fairly easy to work with things like Building Resources. You'd just want either a standard file per 'loop', or a standard file each for various reasonable combinations of loop. I ask because I'm about to start working on self-upgrading improvements (if my tuner-based testing of the principle works out), and I'd love to do it in a way that's going to be more efficient when combined with anything else that does big loops. I have a horrible image of half-a-dozen mods loaded together, each doing either a per-city or per-plot loop every turn...

It's worth pointing out that Building Resources really shouldn't be doing everything on the ActivePlayerTurnStart. That's also why its has a minor issue with the TopPanel display. The code should be broken up over a couple different points during events. But that sounds a lot easier than it is. No one else has ever offered a better design.

As for standards, I'm all for them. You just have to remember they are a constant work in progress. So if you have an idea for a standard, build it! Post it! Get others to adopt it! If a better idea or improvement comes along, people can discuss it, collaborate, and push out a new standard. I have yet to see any real competing standards at this point, just a lot of areas without any real attention. That's why I created Mod List. Now people can easily get to the home page of the mods they are running, while they are still in game. Which I think in the long run can do a lot to help draw more people into the modding discussion.

As a programmer and a philosopher, seeking both elegance and perfection, your words of yearning echo thru these primal coding canyons and find sympathy with our small and desperate clan of scavenging survivalists. But there are no fairy tails. These are the trenches. Grab a shovel.

 

[SamBC]

What events should it (Building Resources) be split over, in your view, and what difficulties are found in doing this? Out of curiosity, and to see if I can think of solutions.

 

[Ryan F. Mercer]

What events should it (Building Resources) be split over, in your view, and what difficulties are found in doing this? Out of curiosity, and to see if I can think of solutions.

1.) I don't know.
2.) Refer to 1.

There is a point somewhere in the code that handles when buildings are completed and there is another point somewhere in the code that handles resources. Ideally, BuildingResources would tap directly into those points in the code rather than forcing everything from ActivePlayerTurnStart. Doing so would likely solve the TopPanel issue because everything would be in the proper state when the TopPanel runs rather than changing state after TopPanel runs.

 

[SamBC]

Ah yes, indeed; if there had been an event I could find to hook into for gaining a building (or at least one for building it and one for conquering a city) and losing a building (or at least one for selling a building and one for losing a city), I think I'd've gotten around to buildings producing resources before you did. My mind shied away from a loop every turn, although it doesn't seem to work out too computationally expensive.

 

[Ryan F. Mercer]

Ah yes, indeed; if there had been an event I could find to hook into for gaining a building (or at least one for building it and one for conquering a city) and losing a building (or at least one for selling a building and one for losing a city), I think I'd've gotten around to buildings producing resources before you did. My mind shied away from a loop every turn, although it doesn't seem to work out too computationally expensive.

Actually, I believe the reason I got to BuildingResources first is specifically because I developed SaveUtils. Without it, such a loop isn't possible, because there is no reliable way to keep track of which resources have been applied and which resources haven't, across saved games. I guess you could have got to that before me as well. That would have been helpful, because the deserialize routine required about 30 hours to develop and refine. Tho to be fair, the original deserialize by killmeplease worked well enough, but I guess I did get to the shared cache first. I mean killmeplease had a cache utility but it only worked in a single lua context. It didn't share the cache with every other mod currently running, which made it much less efficient, not to mention confusing to people trying to use the script data slot to share data between lua states. Of course, there were a lot of limitations to sharing data that way anyway, so I developed ShareData, which allowed me to share the cache universally. So I guess it's kind of a doing thing, more than a thinking about and asking others to document stuff that you could easily document yourself.

But yes, I suppose you could have gotten to it before me. But that's the past now and I await what ever improvements you have to offer. In the mean time, I'm currently working on making it easier for people to create their own in game screens without cluttering up the InGame.xml, allowing far greater mod compatibility. What is it you are currently working on? I wouldn't want to get to anything before you.

 

[SamBC]

Actually, I believe the reason I got to BuildingResources first is specifically because I developed SaveUtils. Without it, such a loop isn't possible, because there is no reliable way to keep track of which resources have been applied and which resources haven't, across saved games. I guess you could have got to that before me as well. That would have been helpful, because the deserialize routine required about 30 hours to develop and refine. Tho to be fair, the original deserialize by killmeplease worked well enough, but I guess I did get to the shared cache first. I mean killmeplease had a cache utility but it only worked in a single lua context. It didn't share the cache with every other mod currently running, which made it much less efficient, not to mention confusing to people trying to use the script data slot to share data between lua states. Of course, there were a lot of limitations to sharing data that way anyway, so I developed ShareData, which allowed me to share the cache universally. So I guess it's kind of a doing thing, more than a thinking about and asking others to document stuff that you could easily document yourself.

But yes, I suppose you could have gotten to it before me. But that's the past now and I await what ever improvements you have to offer. In the mean time, I'm currently working on making it easier for people to create their own in game screens without cluttering up the InGame.xml, allowing far greater mod compatibility. What is it you are currently working on? I wouldn't want to get to anything before you.

In between work, study, and freaky illness symptoms, I'm working on improving-as-worked improvements, a la cottages, but it will all be for naught if API functions don't do what they look like they do (I'm not laying odds). ImprovementDuration (a plot property) is just the raw age of a plot, so isn't much use, but I can use SaveUtils to store the number of turns actually worked in ScriptData, checking every turn (don't see much way around that). I've got the design of an extra table worked out to store information on improvement evolution, and the real hinge is the Plot:SetImprovementType method.

On the back burner, as a really-not-interesting-to-do-slogfest that I attack every so often is rewriting chunks of AssignStartingPlots.lua, so as to make it possible to just add resources and have them work as per standard. It's the motivational hump of "I know what I have to do, just have to do it". Harder to choose that sort of thing when free time is limited.

The remaining substantial item is adding back a sort of war-weariness happiness effect, which I think I have a handle on, but would need a lot more time, to learn more UI (or get someone to help with that bit), and the AI wouldn't react to it as it should because we can't change their diplomatic weightings dynamically (as far as I know). However, it would be fun. Whether I get to that before doing substantial bits of AssignStartingPlots is anyone's guess - as is how long I take to do anything!

You are right about things going past save game, I hadn't thought of that in my original plans. I think I was planning to do something that recalculated from scratch after a load... it was something I looked at quite hard for a while after modding tools came out, but I just didn't think of doing a check every turn. I certainly didn't mean to sound competitive over it, I was just reflecting.

 

[Valkrionn]

Wait. Why do you need the API for cottages? The tags exist, do they not?

Quick search turns up:

• ImprovementPillage/ImprovementUpgrade
• UpgradeTime
• RiverSideUpgradeMod
• CoastalLandUpgradeMod
• HillsUpgradeMod
• FreshWaterUpgradeMod

Do they not work the same as in Civ4?

 

[SamBC]

Wait. Why do you need the API for cottages? The tags exist, do they not?

Quick search turns up:

• ImprovementPillage/ImprovementUpgrade
• UpgradeTime
• RiverSideUpgradeMod
• CoastalLandUpgradeMod
• HillsUpgradeMod
• FreshWaterUpgradeMod

Do they not work the same as in Civ4?

I'll admit I've not tested it myself, but I gather they aren't actually implemented. Certainly, if they are, they would appear to use total time built, not turns worked - the former is what's recorded and available.

 

[Valkrionn]

I'll admit I've not tested it myself, but I gather they aren't actually implemented. Certainly, if they are, they would appear to use total time built, not turns worked - the former is what's recorded and available.

They are hooked up; I had developed a test mod that made use of them.

I just can't remember if it was worked turns or not... I know my city was working the improvement the whole time.

Would be worth testing, at least. I'd do it if the mod was still around; I've deleted it.

 

[alpaca]

As for load order: I'm actually wondering what makes the game load all the tooltips before the game starts. I know the lua is loaded in the front-end (UniqueBonuses) but there's nothing telling it to execute once for each entity.

 

[SamBC]

They are hooked up; I had developed a test mod that made use of them.

I just can't remember if it was worked turns or not... I know my city was working the improvement the whole time.

Would be worth testing, at least. I'd do it if the mod was still around; I've deleted it.

I shall try it out later, then

 

[Valkrionn]

I shall try it out later, then

One thing to note: The display for the tags is not hooked up. I wrote a display for them, IIRC, and it might still be in the Economy Mod. It's not a big thing though, quite sure you can write your own.

 

[SamBC]

One thing to note: The display for the tags is not hooked up. I wrote a display for them, IIRC, and it might still be in the Economy Mod. It's not a big thing though, quite sure you can write your own.

The other thing that isn't there is that you can't make it only upgrade after a certain tech... I'll have to think whether that matters. It'll also need testing whether it adjusts for gamespeed. Maybe I'll do that as my main activity for this afternoon, while running tests on making the AI behave sanely when positioning cities.