Lua Database Modifications

tvynr

Chieftain
Joined
Mar 12, 2016
Messages
12
Summary of the below: I'm not sure how much the Lua provided by a mod can do. Is it possible for Lua scripts to modify the game database? Or is that only something you can do via SQL and XML?

My wife and I enjoy playing a heavily modded multiplayer Civ V under Linux; to do this, we use MPMPM under Windows to create a modpack and then transfer the modpack to our Linux machines. The process of creating the modpacks is slow (as the Civ 5 menu for mod selection has serious performance problems when you have hundreds of mods in the list) and error-prone (now and then, one of the hundreds of mods won't work correctly and will cause crashes only after it is used in a modpack). MPMPM has been an excellent tool for us, but I'm hoping to be able to experiment with modpacks without so much manual effort.

So I'd like to create a Linux CLI tool that will create modpacks for me without the intervention of Windows or Civ V. The source for Gedemon's MPMPM is easily available (https://github.com/Gedemon/Civ5-MultiPlayer-ModsPack-Maker/blob/master/MPMP_Maker.lua), which helps a great deal.

To do this, I have to load in all of the game's base definitions to create some SQL database out of them; that should be easy. Then, I'd need to do whatever it is that Civ V does when you activate a mod. Finally, I'd take the step that MPMPM does: serialize the database into a collection of XML files that can override every definition in the game and use that as a fake DLC.

The second step above is, of course, the hardest; I have to faithfully execute on the database every operation that Civ V would execute for every mod. I'm hoping that someone tells me that Lua scripts can only read the SQL database and so my program would only have to apply the SQL and XML files from the mod during this phase. I have a creeping feeling, though, that there might be some way in which a Lua file could make database modifications.

It's hard to get any certainty on something like this by reading the existing documentation, since there could always be some tiny detail I missed. So I thought I'd bow at the feet of the community's knowledge: can Civ V mod Lua files alter the Civ V SQL database?

Thank you!
 
Lua CAN modify the database, but CiV has already cached the GAME database by playtime, so it wouldn't make a difference if modders were to do so. You can inspect the Civ5CoreDatabase.db (i.e., vanilla database without G+K or BNW) and the Civ5DebugDatabase.db (i.e., database after expansions, DLC, and mods are applied) with a SQLite browser.
The Windows ones are kept in %userprofile%\Documents\My Games\Sid Meier's Civilization V\.

However, note there is a second database file for the Localization database (which translates all those TXT_KEY entries in the Language_* "tables"--note they're actually Views). This database is NOT cached. On the other hand, very few modders have found a use for that fact.
 
Interesting; thanks for the summary! Based on what you said, it sounds like you don't expect there to be any way for a mod to run database-modifying Lua upon activation; that is, there's no way for a Lua script to run before the game starts. Is that correct?
 
For your purposes, all XML/SQL files execute before Lua files and the Lua can't change the database anyway (at least not without a modded DLL, but if you have a mod that has one of those it's not going to work on Linux anyway!)

What are you going to write the CLI tool in?
 
Yeah, we just ignore mods with DLLs. :)

I've not yet decided what language I want to write the tool in. My best languages (that are suitable here) are Python and Java and there are arguments for either of those. Part of the reason I was looking for this info is 'cause it'd significantly affect my choice of language if I had to be able to build an interface to the DB that the Lua could run against. If I only need to process SQL and XML files, that makes the whole project a lot easier!
 
Back
Top Bottom