Considerations for compatibility with VEM

[Thomdar]

I have just started to play around with modding CIV5. I enjoy what VEM adds to the game but trying to work around it while making a mod can be a little difficult depending on what I'm currently messing around with. While I know LUA is going to be harder to step around I was thinking about SQL/XML database manipulations. I'm trying to make a change to the value of gold provided each turn by the Colossus. VEM has set it to 8 and I wish to set it to 5 in my mod for reasons I don't need to get into right now. The issue is load order it seems. As I do not yet know of a simple measure to defeat this I was thinking of a different approach that VEM can take.

The thing about Vanilla CIV5 is that modding it works but with VEM since it is a mod can butt heads with other mods over who gets to make changes to the database. I'm sketching SQL statements and tables that VEM and other mods that want to work well with VEM can include.

The concept is that a table of protected changes can be made. The mod that wants to make first creates the table (if not made already) than adds to the table information for the table and row that wants to protect. VEM than would need to be changed to have the updates or inserts it does into the vanilla tables check the protected table as a requirement. If the protected table says that particular row in the table is blocked than it doesn't make the change.

The basic logic is if VEM loads first there is no protected table therefore it makes its changes than the mod comes after and overwrites them. If instead the mod loads first it creates the protected table and makes its change then when VEM loads it checks the protected table and ignores those changes.

Below is some rough and untested SQL statements I have cobbled together as a beginning towards this concept. First thing I'm looking for is some feedback and editing of any of my many mistakes in my SQL I'm sure exists (only started SQL yesterday).

Spoiler :

-- Code for mod to update a table with compatibility with VEM

CREATE TABLE IF NOT EXISTS Compatibility_Check (
ModName text, -- Considerations for possibly cross compatibility between mods
CompatibilityType text, -- Future thought to have different ways to do compatibility
-- but more work for VEM to consider all possibilities
-- might be more useful for cross compatibility
TableName text, -- Name of the table you are modifying and wish to protect
ColumnName01 text, -- ColumnNameXX and ColumnValueXX lists the Column/Value pairs
ColumnValue01 text, -- that your mod is modifying
ColumnName02 text, -- The actual value you are changing does not need to be mentioned
ColumnValue02 text,
ColumnName03 text, -- Future thoughts for table to include acceptable ranges for
ColumnValue03 text, -- other mods to change or maybe entire tables you wish to ignore
ColumnName04 text, -- VEM or other mod changes to
ColumnValue04 text,
ColumnName05 text,
ColumnValue05 text,
ColumnName06 text,
ColumnValue06 text,
ColumnName07 text,
ColumnValue07 text,
ColumnName08 text,
ColumnValue08 text,
ColumnName09 text,
ColumnValue09 text,
ColumnName10 text,
ColumnValue10 text
);

-- General insertion into table for protected change
INSERT INTO Compatibility_Check (
ModName, CompabilityType, TableName, ColumnName01, ColumnValue01, ColumnName02, ColumnValue02)
SELECT 'Thomdar Mondo Marathon Madness', 'OVERWRITE_VEM', 'Building_YieldChanges',
'BuildingType', 'BUILDING_COLOSSUS',
'YieldType', 'YIELD_GOLD';

-- General insertion into table of actual change
UPDATE Building_YieldChanges (BuildingType, YieldType, Yield)
SET Yield = 5
WHERE BuildingType = 'BUILDING_COLOSSUS' AND YieldType = 'YIELD_GOLD';

--------------------------------------------------------

-- Code for VEM to update table with compatibility considerations for mods
-- Yet to test but thinking VEM would need something like this with each change to tables
-- It would check to see if table entry is protected
-- The logic is that if VEM loads first it makes the changes and mod overwrites
-- If mod loads first it creates the protection table and VEM will not make its change
-- Either way the mod gets its change and VEM is overwriten
UPDATE Building_YieldChanges (BuildingType, YieldType, Yield)
SET Yield = 8
WHERE BuildingType = 'BUILDING_COLOSSUS' AND YieldType = 'YIELD_GOLD'
AND NOT EXISTS (SELECT * FROM Compatibility_Check
WHERE TableName = 'Building_YieldChanges' AND CompatibilityType = 'OVERWRITE_VEM'
AND ColumnName01 = 'BuildingType' AND ColumnValue01 = 'BUILDING_COLOSSUS'
AND ColumnName02 = 'YieldType' AND ColumnName02 = 'YIELD_GOLD');

 

[Thalassicus]

Load order is a particularly tricky issue for any mods that change a similar concept. It's why we really need that "Associations" tab on the modbuddy project properties! Load order is determined by the order mods are installed in. So to get VEM first, then a mod second, you do this:

1. Move all mods out of the /mods folder.
2. Move in VEM and enable it in the mod browser.
3. Move in mod #2 and enable it.

A comparability table is a great idea and would like to add something like what you're doing to the mod. The only downside of sql is it can slow down development, since it's often fastest to copy-paste lines from vanilla files to mods, and translating it from xml to sql takes time. It's more reliable than instructing people to install mods in a particular order, but does take more time from the modder.

 

[Thomdar]

After cleaning up my code and testing I can confirm it does work. For the modders only a small amount has to be done in SQL which is to create the compatibility table.

Spoiler :

CREATE TABLE IF NOT EXISTS Compatibility_Check (
ModName text,
CompatibilityType text,
TableName text,
ColumnName01 text,
ColumnValue01 text,
ColumnName02 text,
ColumnValue02 text
.
.
ColumnNameXX text,
ColumnValueXX text -- Not sure what would be the best number to end it at
);

That can be a premade file that all the modders can add (which also helps ensure a standard). Everything else can be done in XML. They can make their changes in XML to the tables. The additional work comes from adding to the compatibility table but that can also be done in XML and can be easily templated for quickness. All the additional SQL work comes from VEM changing over to it for database updates/additions. I'm probably going to poor over the files personally. I'm mostly just doing this as a learning experience. The final SQL code that works which would be for VEM additions:

Spoiler :

UPDATE Building_YieldChanges
SET Yield = 8
WHERE BuildingType = 'BUILDING_COLOSSUS' AND YieldType = 'YIELD_GOLD'
AND NOT EXISTS (SELECT * FROM Compatibility_Check
WHERE TableName = 'Building_YieldChanges' AND CompatibilityType = 'OVERWRITE_VEM'
AND ColumnName01 = 'BuildingType' AND ColumnValue01 = 'BUILDING_COLOSSUS'
AND ColumnName02 = 'YieldType' AND ColumnValue02 = 'YIELD_GOLD');

 

[Thalassicus]

I've been thinking this over a while... I'm not sure it's practical. VEM has over 4,000 row, set, and delete statements. Converting these to SQL, creating entries in the compatibility table for them, and keeping it up to date would be a very time-consuming process.

A simpler solution is available: we can load modmods after VEM, so modmods override changes in VEM. This takes only a few seconds each time VEM or modmods are updated.