Natan35
Mayor of St. Natansburg
Well, considering ItR deluxe is released, yeah.
Resource icon
JFD's Rise to Power
- Thread starter JFD
- Start date
- Status
DarkScythe
Hunkalicious Holo
- Joined
- May 6, 2014
- Messages
- 804
Hey JFD,
I took a hiatus from modding for a bit (mostly been distracted with playing a couple other games... cough Neptunia...) but I'm getting back into the swing of things by investigating this issue reported to me regarding your Piety & Prestige mod.
ViceVirtuoso reported to me that Piety and Prestige was causing issues with his Neptunia mods (among others) related to a failure with TableSaverLoader. Specifically, mods that use TableSaverLoader would encounter this error when run alongside Piety and Prestige:
Code:
Failed to iterate table due to query error.It seems like this issue was brought up in late June / early July, though I was only just contacted by ViceVirtuoso a couple days ago about this. You guys should probably ping me earlier on Steam, or something, to get my attention more quickly, haha.
I know that TableSaverLoader was a relatively new addition to your mod, based on the inquiries I fielded for you back when you were attempting to implement TSL alongside my TSL Serializer component, so I went to look at the code to see if there were any issues with implementation. ViceVirtuoso sent me a copy of the mod code he had for me to investigate with.
However, I just pulled a copy of your latest version from your Dropbox link in this thread, and the code looks different from what I was sent. In particular, it looks like you have commented out the entire section related to TableSaverLoader and the TSL Serializer. Before I proceed with an explanation, are you still looking for a way to fix it, or are you planning on removing it entirely?
Let me know what your plans are here.
JFD
Kathigitarkh
I moved the TSL stuff to the Shared/Lua/Utilities/JFD_PietySovereigntyUtils.lua file, so that it could be accessed by both Piety and Sovereignty (as they're technically separate mods). I still definitely need to use TSL, so an explanation on how to resolve the issue (or, on what I'm doing wrong, as the case may be 
) would still be very much wanted.
) would still be very much wanted.
DarkScythe
Hunkalicious Holo
- Joined
- May 6, 2014
- Messages
- 804
Okay, thanks for letting me know, and apologies for the delay. I was too tired when I returned home last night to do anything, and I wanted to test your new version first to see if the problem still existed, haha.
Unfortunately, the problem got worse.
After extensive testing with both versions (and this was made a bit more annoying / difficult with your new version due to your editing TableSaverLoader and my Serializer to remove all of their print statements, hampering my ability to trace what Lua was doing until I realized what you did and reverted the prints) I believe I have figured out what the issue is, as well as how to fix it.
I think the problem may ultimately be that it appears you may not quite understand how the include() function works for including files into Civ's Lua. I did a write-up of include over here. Although it's about a different issue, it should still explain some of its quirks, and you can also check the additional links from that post to responses by whoward and Firaxis.
However, in regards to this specific issue with TableSaverLoader failing, long story short, because of the above issue you've wound up including and reinitializing TableSaverLoader repeatedly, thus causing it to fail.
To start off, let's briefly go over very quickly how include() works out here:
We'll start with a File A which will act as our primary file. With this file, we will execute include() on two other files - File B and File C. For the sake of simplicity, we'll say that every file has a single line of actual code: print() to echo the letter of the file name.
Now, we'll have File B additionally include File D and TSL (File T) while File C includes File B. (These includes are demonstrating a specific scenario I am seeing in your current mod; I'll explain later.)
To Visualize:
So, this would show the links stretching out from each file. The problem, though, is that when include() executes, it goes through the code from the matching file and runs them! I feel like you may have thought (from how I'm seeing the includes being used -- correct me if my assumption is wrong) that including a file simply 'makes it available' for that particular Lua file to use. In actuality, your code is causing Lua to execute every line of code in multiple files.
Here, then, is a visualization of what actually gets loaded in the end by File A:
As a result, following our hypothetical situation above, once File A fully loads, we'll have seen this be printed:
As you can see, we have some redundant code executing here. In particular, notice that we have File T loading twice... This was our stand-in for TSL in this example. Unfortunately, doing so violates one of the rules set by Pazyryk for his TableSaverLoader:
That second point is there more for reference than anything, but the important thing is that you are (potentially inadvertently) setting up multiple TSL calls to save the same table repeatedly.
Looking at specifics, I found this very quickly when I decided to add in some extra debug print() statements into your code to determine when and where TSL was being called and loaded from.
On the bright side, you have manually included both TableSaverLoader and my TSL Serializer in a single file: JFD_PietySovereigntyUtils.lua (Previously JFD_PietyPrestigeUtils.lua.) This is the correct way to go about it ... were it not for the fact that you've included this file all over the place.
For one, this file is included into JFD_PietyBalance_Functions.lua as well as JFD_PietyUtils.lua. But that's not all! The previously-mentioned JFD_PietyBalance_Functions.lua also includes JFD_PietyUtils.lua! This means that JFD_PietyBalance_Functions.lua calls JFD_PietyUtils.lua which calls JFD_PietySovereigntyUtils.lua, only to then call JFD_PietySovereigntyUtils.lua again! This was what my previous simplified example was demonstrating.
Now, to determine how many times this happened, I added this into JFD_PietySovereigntyUtils.lua just prior to the TSL include:
Here's a snippet of the logs:
That was only a snippet.. there were more instances. It was a similar story when I pasted this in the older version:
The old version echoed this print statement a whopping ten times, while your new version now loads it eleven times! That's why I say it sort of got worse. Haha.
I also am not quite sure why you have so many entry points / contexts running, but I believe they're meant to overwrite different parts of the game, so I'll leave those alone for now.
In any case, the fix, then, is fairly simple: You need to place the TSL initialization code in a single place so that it only executes once for your entire mod. The entire issue has come about because of your inadvertent reinitialization of TSL so many times.
As a test fix, here is what I did: I removed the TSL initialization code from JFD_PietySovereigntyUtils.lua (Previously JFD_PietyPrestigeUtils.lua - this fix works for both versions.) I then placed the code in the root file of one of your entry points: JFD_Piety_Functions.lua
I placed it right after your include of JFD_PietyUtils.lua (which naturally meant it would have loaded your JFD_PietySovereigntyUtils.lua as well as your Global Defines file JFD_PietySovereignty_GlobalDefines.lua, which set up your table structure.) While I'm here, I'm not sure why you have Sukritact_SaveUtils running as well, since it seems a bit redundant to be running two different data persistence utilities at once.
As an aside, since you've defined tableRoot and tableName out here, it doesn't need to be redefined inside the Serializer client file (although it doesn't hurt anything for you to have done so.) By default with v3, the client defines tableRoot and tableName to take up the existing value if predefined beforehand.
Of course, I moved your OnModLoaded() function to the bottom of JFD_Piety_Functions.lua as well.
Anyway, this produces this in the logs (I've added a few more debug prints here):
No errors! And TSL only loads once for your mod. Yay!
This serves as a proof of concept both to test the theory that multiple TSL calls were the problem, as well as testing the theory that limiting it to one call would fix said issue. I then attempted to recreate the issue by manually placing a second initialization of TSL into JFD_Piety_ChooseStateReligionPopup.lua, thus causing the game / your Lua to load TSL twice. This immediately caused the same error as before, and for TSL / HDN to fail.
A couple points I should note:
I have attempted to look into the cause of this issue, and as part of doing so, I wanted to eliminate any possibility that it was my Serializer that could have potentially caused this issue. To that end, I repeated these tests after completely removing my Serializer from both Piety & Prestige as well as ViceVirtuoso's HDN/Nepgear Civ. I verified that the Serializer was no longer active by placing all of Pazyryk's default TSL hookup code in the appropriate places as before, and hardcoded the table names into the save interception calls. Triggering a manual save made it apparent that my Serializer was no longer active, as the standard TSL behavior was observed -- only one of the two mods was able to save its data (HDN, in this case.)
Even with these changes, I observed the same conclusions -- a single initialization of TSL caused both mods to operate properly with each other, while adding even a single additional initialization would cause HDN to fail. As a result, I am confident that my Serializer's code did not introduce any additional issues into the mix (and even when I coded in a special version for P&P to detect and ignore duplicate table registrations, it didn't help as TSL itself had already been initialized by that point.) Looking up Pazyryk's documentation linked above I believe that this is an inherent limitation of TSL.
Unfortunately, in doing my investigation, I was only able to narrow it down to an issue occurring at some point during the TableSave() function. Specifically, it probably silently fails during the CREATE TABLE phase, but it definitely fatally errors out during the INSERT INTO phase, as it likely is trying to insert data into a nonexistent table. However, I am unable to pinpoint the issue any further, because I am simply not experienced or knowledgeable enough about SQL and Civ's Lua-to-SQL interaction to go any further than this, although if you would like to help, I would definitely appreciate it. I tried to enter the same exact commands to CREATE TABLE in Firetuner, but I received no messages or notifications, and the database did not update with any new tables when viewed via SQLite Manager. However, pasting the same exact commands into SQLite Manager would create the tables without issue, so I have no idea. Without being able to go further, I'm not sure if I can (or even should) try to come up with a 'patch' for this issue.
Regardless, you need to ensure that TSL is only loaded once throughout your entire mod. I know you have multiple contexts running simultaneously, which is why my 'fix' was not a proposed solution, but rather a proof of concept - I'm not sure how your code truly runs, so I'm not sure that the file I chose would be appropriate, but limiting it to one initialization definitely does fix the primary issue.
Along those lines, you're using MapModData as your global table, which is already shared across contexts, so you do not need to reinitialize it all over the place. You simply initialize it once, and so long as every other context is told to look for your MapModData table, everything should sync up quite nicely. You do, however, need to ensure that your MapModData subtables are cleared on script load, because MapModData is not cleared when you load a save while already in-game, and you can potentially run into 'future data' issues.
You should also, while you're at it, redo and streamline all of your include() statements so that there's less redundancy going on (I only pointed out the one or two examples specific to TSL, but you have far more redundancy issues going on elsewhere because of the nested includes you're doing.) Essentially, you should try and ensure that for each "chain" of includes you have, that in total, each file is only called once. As I noted before, if A includes B which itself includes C, then A will have both B and C executed as if it was written into it. You don't need to explicitly tell A to include C again.
I'm sorry this post grew so large, but I wanted to be clear about what I did, and what the problems I observed were, as well as trying to explain them as clearly as possible. I hope it helped, though!
Feel free to add / poke me on Steam if you need assistance going through this in real-time.
Unfortunately, the problem got worse.
After extensive testing with both versions (and this was made a bit more annoying / difficult with your new version due to your editing TableSaverLoader and my Serializer to remove all of their print statements, hampering my ability to trace what Lua was doing until I realized what you did and reverted the prints) I believe I have figured out what the issue is, as well as how to fix it.
I think the problem may ultimately be that it appears you may not quite understand how the include() function works for including files into Civ's Lua. I did a write-up of include over here. Although it's about a different issue, it should still explain some of its quirks, and you can also check the additional links from that post to responses by whoward and Firaxis.
However, in regards to this specific issue with TableSaverLoader failing, long story short, because of the above issue you've wound up including and reinitializing TableSaverLoader repeatedly, thus causing it to fail.
To start off, let's briefly go over very quickly how include() works out here:
We'll start with a File A which will act as our primary file. With this file, we will execute include() on two other files - File B and File C. For the sake of simplicity, we'll say that every file has a single line of actual code: print() to echo the letter of the file name.
Now, we'll have File B additionally include File D and TSL (File T) while File C includes File B. (These includes are demonstrating a specific scenario I am seeing in your current mod; I'll explain later.)
To Visualize:
Code:
File A
|__ File B
|__ File C
File B
|__ File D
|__ File T
File C
|__ File BSo, this would show the links stretching out from each file. The problem, though, is that when include() executes, it goes through the code from the matching file and runs them! I feel like you may have thought (from how I'm seeing the includes being used -- correct me if my assumption is wrong) that including a file simply 'makes it available' for that particular Lua file to use. In actuality, your code is causing Lua to execute every line of code in multiple files.
Here, then, is a visualization of what actually gets loaded in the end by File A:
Code:
File A
|__ File B
|__ File D
|__ File T
|__ File C
|__ File B
|__ File D
|__ File TAs a result, following our hypothetical situation above, once File A fully loads, we'll have seen this be printed:
Code:
A B D T C [B][COLOR="Red"]B D T[/COLOR][/B]As you can see, we have some redundant code executing here. In particular, notice that we have File T loading twice... This was our stand-in for TSL in this example. Unfortunately, doing so violates one of the rules set by Pazyryk for his TableSaverLoader:
That second point is there more for reference than anything, but the important thing is that you are (potentially inadvertently) setting up multiple TSL calls to save the same table repeatedly.
Looking at specifics, I found this very quickly when I decided to add in some extra debug print() statements into your code to determine when and where TSL was being called and loaded from.
On the bright side, you have manually included both TableSaverLoader and my TSL Serializer in a single file: JFD_PietySovereigntyUtils.lua (Previously JFD_PietyPrestigeUtils.lua.) This is the correct way to go about it ... were it not for the fact that you've included this file all over the place.
For one, this file is included into JFD_PietyBalance_Functions.lua as well as JFD_PietyUtils.lua. But that's not all! The previously-mentioned JFD_PietyBalance_Functions.lua also includes JFD_PietyUtils.lua! This means that JFD_PietyBalance_Functions.lua calls JFD_PietyUtils.lua which calls JFD_PietySovereigntyUtils.lua, only to then call JFD_PietySovereigntyUtils.lua again! This was what my previous simplified example was demonstrating.
Now, to determine how many times this happened, I added this into JFD_PietySovereigntyUtils.lua just prior to the TSL include:
Code:
print("********************")
print("Loading TSL Serializer from JFD_PietySovereigntyUtils")
print("********************")Here's a snippet of the logs:
Code:
JFD_PietyInstitutionMandate_Functions: ********************
JFD_PietyInstitutionMandate_Functions: "Loading TSL Serializer from JFD_PietySovereigntyUtils"
JFD_PietyInstitutionMandate_Functions: ********************
JFD_PietyInstitutionPatriarchy_Functions: ********************
JFD_PietyInstitutionPatriarchy_Functions: "Loading TSL Serializer from JFD_PietySovereigntyUtils"
JFD_PietyInstitutionPatriarchy_Functions: ********************
JFD_Piety_ChooseStateReligionPopup: ********************
JFD_Piety_ChooseStateReligionPopup: "Loading TSL Serializer from JFD_PietySovereigntyUtils"
JFD_Piety_ChooseStateReligionPopup: ********************
JFD_Piety_Functions: ********************
JFD_Piety_Functions: "Loading TSL Serializer from JFD_PietySovereigntyUtils"
JFD_Piety_Functions: ********************
JFD_Piety_InstitutionPopup: ********************
JFD_Piety_InstitutionPopup: "Loading TSL Serializer from JFD_PietySovereigntyUtils"
JFD_Piety_InstitutionPopup: ********************
JFD_TopPanelUpdated: ********************
JFD_TopPanelUpdated: "Loading TSL Serializer from JFD_PietySovereigntyUtils"
JFD_TopPanelUpdated: ********************
HDNMainScript: Hyperdimension Neptunia mod beginning load.
HDNMainScript: At least one of Vice's HDN Civs are present in the game.
[COLOR="Red"] Runtime Error: Failed to iterate table due to query error. Check Database.log for more details.[/COLOR]That was only a snippet.. there were more instances. It was a similar story when I pasted this in the older version:
Code:
print("***")
print("Serializer loading from file [ JFD_PietyPrestigeUtils.lua ]")
print("***")The old version echoed this print statement a whopping ten times, while your new version now loads it eleven times! That's why I say it sort of got worse. Haha.
I also am not quite sure why you have so many entry points / contexts running, but I believe they're meant to overwrite different parts of the game, so I'll leave those alone for now.
In any case, the fix, then, is fairly simple: You need to place the TSL initialization code in a single place so that it only executes once for your entire mod. The entire issue has come about because of your inadvertent reinitialization of TSL so many times.
As a test fix, here is what I did: I removed the TSL initialization code from JFD_PietySovereigntyUtils.lua (Previously JFD_PietyPrestigeUtils.lua - this fix works for both versions.) I then placed the code in the root file of one of your entry points: JFD_Piety_Functions.lua
I placed it right after your include of JFD_PietyUtils.lua (which naturally meant it would have loaded your JFD_PietySovereigntyUtils.lua as well as your Global Defines file JFD_PietySovereignty_GlobalDefines.lua, which set up your table structure.) While I'm here, I'm not sure why you have Sukritact_SaveUtils running as well, since it seems a bit redundant to be running two different data persistence utilities at once.
Code:
-- JFD_Piety_Functions
-- Author: JFD
-- DateCreated: 5/24/2014 12:00:40 AM
--=======================================================================================================================
-- INCLUDES
--=======================================================================================================================
include("EventsAndDecisions_Utilities.lua")
include("FLuaVector.lua")
include("JFD_PietyUtils.lua")
[B][COLOR="Blue"]include("TableSaverLoader016.lua");
tableRoot = JFD_PietySovereignty
tableName = "JFD_PietySovereignty"
print("********************")
print("Loading TSL Serializer from JFD_Piety_Functions")
print("********************")
include("JFD_PietySovereignty_TSLSerializerV3.lua");
print("TableLoad called from JFD_Piety_Functions")
TableLoad(tableRoot, tableName)[/COLOR][/B]As an aside, since you've defined tableRoot and tableName out here, it doesn't need to be redefined inside the Serializer client file (although it doesn't hurt anything for you to have done so.) By default with v3, the client defines tableRoot and tableName to take up the existing value if predefined beforehand.
Of course, I moved your OnModLoaded() function to the bottom of JFD_Piety_Functions.lua as well.
Anyway, this produces this in the logs (I've added a few more debug prints here):
Code:
JFD_Piety_Functions: Loading TableSaverLoader.lua from JFD P&S...
JFD_Piety_Functions: ********************
JFD_Piety_Functions: Loading TSL Serializer from JFD_Piety_Functions
JFD_Piety_Functions: ********************
JFD_Piety_Functions: TableLoad called from JFD_Piety_Functions
JFD_Piety_Functions: TableLoad called from OnModLoaded in JFD_Piety_Functions
JFD_Piety_Functions: TableSave called from OnModLoaded in JFD_Piety_Functions
JFD_Piety_Functions: Creating SavedGameDB tables for new game: JFD_PietySovereignty_Data and JFD_PietySovereignty_Info
JFD_Piety_Functions: TableSave time: 0.035000000000764, inserts: 23, deletes: 0, updates: 0, unchanged: 0, checksum: 437
HDNMainScript: Hyperdimension Neptunia mod beginning load.
HDNMainScript: At least one of Vice's HDN Civs are present in the game.
HDNMainScript: Loading TableSaverLoader.lua from JFD P&S...
HDNMainScript: Creating SavedGameDB tables for new game: HDNMod_Data and HDNMod_Info
HDNMainScript: TableSave time: 0.03400000000056, inserts: 8, deletes: 0, updates: 0, unchanged: 0, checksum: 152
[COLOR="Blue"] HDNMainScript: Nepgear obtained the ''Trait Script Loaded'' Affinity![/COLOR]No errors! And TSL only loads once for your mod. Yay!
This serves as a proof of concept both to test the theory that multiple TSL calls were the problem, as well as testing the theory that limiting it to one call would fix said issue. I then attempted to recreate the issue by manually placing a second initialization of TSL into JFD_Piety_ChooseStateReligionPopup.lua, thus causing the game / your Lua to load TSL twice. This immediately caused the same error as before, and for TSL / HDN to fail.
A couple points I should note:
- This only seems to affect mods loaded after Piety & [Prestige / Sovereignty]
- Mods loaded prior to this mod had no problems communicating with TSL
- For some unknown reason, this issue affects only the mods loaded after it, but these redundant initializations seem to have no impact whatsoever on your own mod
I have attempted to look into the cause of this issue, and as part of doing so, I wanted to eliminate any possibility that it was my Serializer that could have potentially caused this issue. To that end, I repeated these tests after completely removing my Serializer from both Piety & Prestige as well as ViceVirtuoso's HDN/Nepgear Civ. I verified that the Serializer was no longer active by placing all of Pazyryk's default TSL hookup code in the appropriate places as before, and hardcoded the table names into the save interception calls. Triggering a manual save made it apparent that my Serializer was no longer active, as the standard TSL behavior was observed -- only one of the two mods was able to save its data (HDN, in this case.)
Even with these changes, I observed the same conclusions -- a single initialization of TSL caused both mods to operate properly with each other, while adding even a single additional initialization would cause HDN to fail. As a result, I am confident that my Serializer's code did not introduce any additional issues into the mix (and even when I coded in a special version for P&P to detect and ignore duplicate table registrations, it didn't help as TSL itself had already been initialized by that point.) Looking up Pazyryk's documentation linked above I believe that this is an inherent limitation of TSL.
Unfortunately, in doing my investigation, I was only able to narrow it down to an issue occurring at some point during the TableSave() function. Specifically, it probably silently fails during the CREATE TABLE phase, but it definitely fatally errors out during the INSERT INTO phase, as it likely is trying to insert data into a nonexistent table. However, I am unable to pinpoint the issue any further, because I am simply not experienced or knowledgeable enough about SQL and Civ's Lua-to-SQL interaction to go any further than this, although if you would like to help, I would definitely appreciate it. I tried to enter the same exact commands to CREATE TABLE in Firetuner, but I received no messages or notifications, and the database did not update with any new tables when viewed via SQLite Manager. However, pasting the same exact commands into SQLite Manager would create the tables without issue, so I have no idea. Without being able to go further, I'm not sure if I can (or even should) try to come up with a 'patch' for this issue.
Regardless, you need to ensure that TSL is only loaded once throughout your entire mod. I know you have multiple contexts running simultaneously, which is why my 'fix' was not a proposed solution, but rather a proof of concept - I'm not sure how your code truly runs, so I'm not sure that the file I chose would be appropriate, but limiting it to one initialization definitely does fix the primary issue.
Along those lines, you're using MapModData as your global table, which is already shared across contexts, so you do not need to reinitialize it all over the place. You simply initialize it once, and so long as every other context is told to look for your MapModData table, everything should sync up quite nicely. You do, however, need to ensure that your MapModData subtables are cleared on script load, because MapModData is not cleared when you load a save while already in-game, and you can potentially run into 'future data' issues.
You should also, while you're at it, redo and streamline all of your include() statements so that there's less redundancy going on (I only pointed out the one or two examples specific to TSL, but you have far more redundancy issues going on elsewhere because of the nested includes you're doing.) Essentially, you should try and ensure that for each "chain" of includes you have, that in total, each file is only called once. As I noted before, if A includes B which itself includes C, then A will have both B and C executed as if it was written into it. You don't need to explicitly tell A to include C again.
I'm sorry this post grew so large, but I wanted to be clear about what I did, and what the problems I observed were, as well as trying to explain them as clearly as possible. I hope it helped, though!
Feel free to add / poke me on Steam if you need assistance going through this in real-time.
Natan35
Mayor of St. Natansburg
Checked the logs after not getting piety from events... The only log that looked unusual was the localization log. Should I post it?
JFD
Kathigitarkh
Thank you for taking the time to investigate! I should note that I do understand, at least in essence, how include() works - when previously using SaveUtils, I found my previous misunderstanding of its use to be a source of much trouble. However, I had/have no clue how to allow multiple mods and files to otherwise access the data that Piety saves, hence why including the TSL functions in _Utils was a quick (and only) but evidently bad fix. However, I shall carefully peruse what you're written, and see if I can set up TSL properly.
Perhaps the oddest thing is that, IIRC, my Karl XII mod, using also TSL, ran perfectly fine alongside P&S. Almost certainly, ExCE, which similarly uses TSL, does.
No.
Natan35
Mayor of St. Natansburg
Should I post the other mods and bother your modjesty with my problems?
DarkScythe
Hunkalicious Holo
- Joined
- May 6, 2014
- Messages
- 804
I'm glad to help. To your points, yeah, I didn't want to simply assume you didn't understand include(), but at the same time, I was seeing a lot of bizarre redundant file includes going on, which confused me. I still don't quite understand why, how, or if you're still using SaveUtils in your code, but I feel it would be cleaner if you simply moved everything over to TSL. I'm a bit biased though, haha.
In any case, as far as I can tell, you technically have set up TSL properly.. you're just calling it too many times due to the nested includes. TSL, when utilizing MapModData as you currently are, allows for sharing data through that table across your Lua contexts (and other mods.) All that is necessary is for everything to be initialized once, and every other context to query that table for data.
This is why, for sharing data, Pazyryk recommended setting up the table definitions as:
Code:
MapModData.gT = MapModData.gT or {}Utilizing the or operator, each individual context can set up a new table if it doesn't already exist, or simply pick up the existing table if another context has already created it.
As to why you're getting inconsistent results with your mods, I believe I have run into a similar situation (as I am continuing to test right now.) It seems to me that there may be another factor at play here, and that is when the Lua executes its functions.
To be specific, load order does not appear to be the only factor at play for this causing issues. For ViceVirtuoso's Lyrical Nanoha Civ, it has P&P set as a reference, meaning that the game loads Nanoha after P&P. However, during execution (when loading the map, etc. for a new game) Nanoha's files get processed before P&P's files, and it appears to be able to save fine. However, when I forced it to execute after P&P, it failed with the aforementioned table query error (when I restored P&P to the redundant include code.)
To support this, I would like to ask: Do/did Karl XII and ExCE process their files before P&P?
I suppose in light of this, I should also amend my previous conclusion: Redundant re-initializations of TSL may not necessarily cause issues on its own, but rather it may cause unpredictable issues instead.
In the end, I feel like we should 'play it safe' as it were, and adhere to Pazyryk's recommended guidelines of only running TSL once per mod.
Edit:
As well, can you provide a list of your mods using TSL/Serializer? It would help me wade through your, uh, substantial list of offerings in order to be able to test them.
JFD
Kathigitarkh
Piety has gone through much development - some Lua files aren't even executed (the Balance ones you noted, in fact, were from a distant past), and so it's no surprise that there is some redundancy. Originally, the mod used SaveUtils, and this is why some files still include it - I no longer use it, of course, but I suppose there are places I forgot to remove reference to it. Need update I'll do some cleaning
Both Karl and ExCE should load after P&S, as they both reference it. But mod load order has been known to break...
Only Karl, P&S, ExCE, and the Into the Renaissance Deluxe scenario use TSL atm.
Both Karl and ExCE should load after P&S, as they both reference it. But mod load order has been known to break...
Only Karl, P&S, ExCE, and the Into the Renaissance Deluxe scenario use TSL atm.
So I started a game with some mods and EUI.
Now I cant't access the Diplomacy and faith screens via the ui.
Is there a "workaround" ->shortcut?
Here is my list of mods I am playing with:
(I marked the ones I think could cause this problem...)
Barbarians Evolved + Initial Swarm
City-State Leaders for BNW v6
CSD v27
CivIV Diplo v10
Civ Names by Policies v8
CBP v13
CP v66
Contextual Unit Names v3
Culturally Linked Start Locations v4
Ethnic Units v31
Events and Decisions v1
Extra Victory Conditions v1
Global - Fortify Almost Healed
Global - Units Awake in Danger
InfoAddict
JFD's Piety and Prestige v5
More Luxuries v155
UI - Gold Alerts
UI-Great Work Manager
UI - Small Resource Icons
UI-Wonder Planner v13
Unique Cultural Influence v4
Wonder Race v4
Note: whowards Wonder Planer only seemed to work in the beginning, not after loading the save file...
Now I cant't access the Diplomacy and faith screens via the ui.
Is there a "workaround" ->shortcut?
Here is my list of mods I am playing with:
(I marked the ones I think could cause this problem...)
Barbarians Evolved + Initial Swarm
City-State Leaders for BNW v6
CSD v27
CivIV Diplo v10
Civ Names by Policies v8
CBP v13
CP v66
Contextual Unit Names v3
Culturally Linked Start Locations v4
Ethnic Units v31
Events and Decisions v1
Extra Victory Conditions v1
Global - Fortify Almost Healed
Global - Units Awake in Danger
InfoAddict
JFD's Piety and Prestige v5
More Luxuries v155
UI - Gold Alerts
UI-Great Work Manager
UI - Small Resource Icons
UI-Wonder Planner v13
Unique Cultural Influence v4
Wonder Race v4
Note: whowards Wonder Planer only seemed to work in the beginning, not after loading the save file...
Vicevirtuoso
The Modetta Man
Piety has gone through much development - some Lua files aren't even executed (the Balance ones you noted, in fact, were from a distant past), and so it's no surprise that there is some redundancy. Originally, the mod used SaveUtils, and this is why some files still include it - I no longer use it, of course, but I suppose there are places I forgot to remove reference to it. Need update I'll do some cleaning
Both Karl and ExCE should load after P&S, as they both reference it. But mod load order has been known to break...
Any issues with other mods should generally fix themselves if only one instance of TSL is loaded from P&P. As a general rule, I load TSL in the main context of my mod, and if I need to access data from my mod's global table in another context, I explicitly call it from MapModData.
For example, my biggest mod uses MapModData.PMMM as its root table, and within the context of the file which contains the TSL includes, I have it localized to simply PMMM. However, any time I want a UI element to get data from that table, or when I want to edit its data through another mod like Events and Decisions, I explicitly type out MapModData.PMMM. (Localizing MapModData.PMMM in every context I used it in had a tendency for one context to have different data from another, so I dropped that idea.
)This way, only one context is ever handling the task of saving and loading the table via TSL. Could potentially fix the issues we've been experiencing, since Pazyryk did say that multiple TSLs running in the same mod are not a good idea.
Natan35
Mayor of St. Natansburg
It is in the user settings file inside the piety folder.
DarkScythe
Hunkalicious Holo
- Joined
- May 6, 2014
- Messages
- 804
Piety has gone through much development - some Lua files aren't even executed (the Balance ones you noted, in fact, were from a distant past), and so it's no surprise that there is some redundancy. Originally, the mod used SaveUtils, and this is why some files still include it - I no longer use it, of course, but I suppose there are places I forgot to remove reference to it. Need update I'll do some cleaning
Both Karl and ExCE should load after P&S, as they both reference it. But mod load order has been known to break...
Only Karl, P&S, ExCE, and the Into the Renaissance Deluxe scenario use TSL atm.
Thanks, I'll grab those (and I suppose I should update my Serializer thread as well with your new additions) and give it some testing. I'll report back later with my results.
As far as obsoleted files go, I definitely recommend going through and cleaning out all of the older stuff, as it can generally make things confusing since I don't know exactly the path your development took you, and have to make assumptions when I'm leafing through the code trying to troubleshoot it.
As far as referencing, the thing is that all of the HDN mods currently reference P&P as well, but it gave me inconsistent results yesterday when I updated over to your P&S; As far as I can tell, the mod ID hasn't changed, and GetEnabledModsByActivationOrder() reports P&S loading before any of those mods, but I was still getting some of those mods executing ahead of P&S at some point. I couldn't really explain why, and I'm not sure if the "name" parameter needs to be updated to reflect the mod's new name, although I can't see why this would be necessary since it should track references by mod ID's.
Either way, more testing...
Edit:
Haha. Almost immediately, I have results. I grabbed a copy of your Karl XII Civ from your Dropbox (linked from that Steam workshop page.)
I disabled all the HDN Civs and anything else I was testing with previously, so the only things currently loaded are CP, E&D, P&S, Karl XII, Perfectworld3 and Spice and Wolf (the latter two I basically always have loaded, for obvious reasons. Always betting that my Wolf can beat your Karl.
)'lo and behold, the log:
Code:
JFD_SwedenKarlXII_Functions: King Karl XII is in this game
[COLOR="Red"]Runtime Error: Failed to iterate table due to query error.[/COLOR] Check Database.log for more details.If only that second sentence there actually meant they put something in the database log.
In this case, Karl XII most definitely loaded and executed after P&S (this was with the version of P&S that I've edited to re-initialize TSL several times from specific files.) For controlled testing, I tried it with three calls (where my testing ended last night,) then shortened to two calls, and then I also went ahead and removed everything, and restored the original JFD_PietySovereigntyUtils.lua file to its original state, reverting its behavior to the original 'inadvertent' multiple-initialization mode, and every single test produced the same error as above with Karl XII.
I then re-edited the files to return to my previous 'fix,' with just a single call laid inside JFD_Piety_Functions.lua at which point Karl XII began functioning normally with regard to TSL.
For an additional test, I re-enabled the HDN mods Vert/Leanbox and Blanc/Lowee to run alongside Karl XII. All three mods have P&S set as a reference, but curiously during my testing last night, Blanc and Vert's Lua files were being executed before P&S' so I wanted to test my hypothesis here while I have Karl XII executing after it.
With a single call as before, everything worked fine. However, as soon as I re-enabled a second initialization in JFD_Piety_ChooseStateReligionPopup.lua it caused the rror again -- but only for Karl XII! Vert and Blanc once again executed their scripts ahead of P&S, and escaped without issue.
Code:
JFD_SwedenKarlXII_Functions: Processing data for save operation into savegame database.
JFD_SwedenKarlXII_Functions: Table save #1 for '[COLOR="Green"]HDNMod[/COLOR]' initiated from TSL Master.
JFD_SwedenKarlXII_Functions: TableSave time: 0, inserts: 0, deletes: 0, updates: 0, unchanged: 8, checksum: 152
JFD_SwedenKarlXII_Functions: Table save #2 for '[COLOR="Blue"]JFD_PietySovereignty[/COLOR]' initiated from TSL Master.
JFD_SwedenKarlXII_Functions: TableSave time: 0, inserts: 0, deletes: 0, updates: 0, unchanged: 23, checksum: 437
JFD_SwedenKarlXII_Functions: Table save #3 for '[COLOR="Blue"]JFD_PietySovereignty[/COLOR]' initiated from TSL Master.
JFD_SwedenKarlXII_Functions: TableSave time: 0.015000000000327, inserts: 0, deletes: 0, updates: 9, unchanged: 14, checksum: 437
JFD_SwedenKarlXII_Functions: Table save #3 for '[COLOR="Red"]JFD_SwedenKarlXII[/COLOR]' initiated from TSL Master.
[COLOR="Red"]Runtime Error: Failed to iterate table due to query error.[/COLOR] Check Database.log for more details.I think this supports my revised conclusion that it's the script execution order that matters, rather than strict load order. Based on the fact that you said Karl XII worked well with P&S, I have to ask if you can recall (or can reproduce) whether the logs showed Karl's Lua executing before or after P&S.
Bonus:
Just for the sake of completeness, I wanted to figure out exactly what part of the duplicated initialization calls caused these issues. Starting from the "working" setup of just a single call, I slowly un-commented sections of the redundant initialization in the second file, and checked the game for errors at each step.
Results:
- Simply including the TSL file caused no issues.
- Including TSL and my Serializer also caused no issues.
- Including TSL+Serializer, as well as the TableLoad() calls, whether it be standalone or part of OnModLoaded() caused no issues.*
- Issues arose as soon as I un-commented the TableSave() line within OnModLoaded().
*There technically were other errors/warnings that arose when I did this:
Code:
JFD_SwedenKarlXII_Functions: Processing data for save operation into savegame database.
JFD_SwedenKarlXII_Functions: Table save #1 for '[COLOR="Blue"]JFD_PietySovereignty[/COLOR]' initiated from TSL Master.
JFD_SwedenKarlXII_Functions: [COLOR="Red"]!!!! TableSaverLoader thinks you are using TableSave before TableLoad on a loaded game; are you sure you want to do that? !!!![/COLOR]
JFD_SwedenKarlXII_Functions: TableSave time: 0.0019999999999527, inserts: 23, deletes: 0, updates: 0, unchanged: 0, checksum: 437
JFD_SwedenKarlXII_Functions: Table save #2 for '[COLOR="Blue"]JFD_PietySovereignty[/COLOR]' initiated from TSL Master.
JFD_SwedenKarlXII_Functions: TableSave time: 0, inserts: 0, deletes: 0, updates: 0, unchanged: 23, checksum: 437
JFD_SwedenKarlXII_Functions: Table save #2 for '[COLOR="Green"]JFD_SwedenKarlXII[/COLOR]' initiated from TSL Master.
JFD_SwedenKarlXII_Functions: TableSave time: 0, inserts: 0, deletes: 0, updates: 0, unchanged: 1, checksum: 19I suppose, long story short, try not to initialize TSL multiple times. Haha.
Apologies if this has already been answered... I had a little search but 61 pages is a lot! I've used this mod for a while now and have always chose my state religion to be the same as my overall religion. I'm playing a game now though, and I actually didn't manage to get a religion for myself in time (was 3 turns off...) so I decided to adopt a neighbor's religion since I liked their beliefs. Now I have the choice of adopting a state religion and I'm unsure what to choose and how it'll affect me.
I could choose the state religion to be the same as my neighbor's religion which I'm currently spreading to my cities. Or I could also choose a different neighbor's religion as my state religion. What exactly would happen if I chose the other religion? Would I get the beliefs and pantheon of this new religion? Would it override my current religion in my cities? I've never actually had a state religion be different than my chosen or adopted religion, so I'm unsure what happens!
I could choose the state religion to be the same as my neighbor's religion which I'm currently spreading to my cities. Or I could also choose a different neighbor's religion as my state religion. What exactly would happen if I chose the other religion? Would I get the beliefs and pantheon of this new religion? Would it override my current religion in my cities? I've never actually had a state religion be different than my chosen or adopted religion, so I'm unsure what happens!
Natan35
Mayor of St. Natansburg
It will convert any religion- less\ pagan citizens to your state religion. However, it may result in a significant piety loss, and in the loss of your religion, so beware.
zwei833
Emperor
- Joined
- Feb 26, 2011
- Messages
- 1,228
Maybe a little off-topic, but is that E&D, Piety and Cultrual Diversity compatilble with overhaul mods such as GAIA(http://steamcommunity.com/sharedfiles/filedetails/?id=298284903&searchtext=gaia) ?
JFD
Kathigitarkh
Idk, but I would wager that it is, so long as you take some intuition when it comes to the user settings file.
Your cities will slowly convert to the chosen religion, which means they will benefit from the follow beliefs of that religion. Choose to follow the religion that either benefits your cities most, or which benefits your religions most (as you'll get the high piety relations bonus with the founder).
Piety updated (Steam version also updated):
- Piety conversion rate now only applies to the point that a city is following your SR as a majority. This should allow for minorities to persist.
- Reduced the Gold cost and increased the XP gain for Chapter Houses.
- Fixed issues with TSL thanks to DarkScythe.
Again, Secularization, Decisions and costs, etc. are working as intended for me, so any problems that [Natan] is having are mostly likely due to some error on his end.
DarkScythe
Hunkalicious Holo
- Joined
- May 6, 2014
- Messages
- 804
Hurray. I'll grab the new copy and run one more test just to make sure.
Have you verified that your other mods (ExCE and Into the Renaissance Deluxe) work okay, or do you want me to take a look at those as well? I haven't received any reports of conflicts from them so I haven't checked, although on a glance Karl looks fine since it's much simpler in terms of Lua contexts.
- Status
Similar threads
- Sticky