[ModBuddy Extension] ModMerger (WIP - input required)

I'm working on my third ModBuddy extension, this time targeted at modpack makers.

The extension is supposed to detect and help solve conflicts between mods.

I already have some code, but before I start writing the core functionality, I have to make sure what the extension is actually supposed to do (I have never made a modpack).

How do I imagine the merging process:

1. Create/open the project the target mod will be merged INTO in the ModBuddy.
2. Click the "Add Merged Mod" in the "Tools" menu.
3. The extension asks to locate the target *.modinfo file.
4. List of mod files is loaded from the modinfo file. The extension compares this list with files already present in the mod.
5. Another dialog box similar to TortoiseSVN's UPDATE window pops up. The window contains a list of all the files present in the target mod. Each file has one of these statuses:
   • Added - the file is not present in currently opened mod yet, the file will be added to the mod as it is in the target mod.
   • Merged - the file is present in both mods, but the their changes relative to the vanilla file version do not conflict with each other and can be automatically merged.
   • Conflicted - the file is present in both mods and their changes relative to the vanilla file version are not compatible. Clicking on this file will open some (most likely external, possibly TortoiseMerge) merging tool, which will assist in resolving the conflict manually.
   • Renamed - the file with the same name is present in both mods, but every file contains something different (irrelevant to the other file), so the newer file has to be renamed and all references to it updated.
   The user also check which files will be applied into the current mod (or uncheck those that won't).
   No changes are written until the next step.
6. Click the "Merge" button and all the changes are written into the opened project.

So what input do I need? Of course any (smart and useful) suggestions are more than welcome.

But more than that I need advice about what kinds of conflicts between mods can happen and how to solve them. I know about these:

• File replacement conflict - solving these is the primary focus of this extension. This kind of conflict can be solved by simply text-wise merging the two files. But there is one thing I'm not sure about: do the replaced files have to be in location (folder) precisely identical to the original one in Assets folder or do they have to have the same name only?
• File name conflict - this happens when both mods contain "FunnyPicture.dds", but with different contents. One of these has to be renamed and all references to it updated.
• Database conflict - this happens when both mods change value of one row in the database. I don't see how I could detect these without actually executing all the queries on the game database.
• DLL conflict (in future) - solving these will be impossible without access to the modded DLL source code. The extension can merely inform the user, that there is and DLL conflict.
  
  
  

CommanderZ said:

But more than that I need advice about what kinds of conflicts between mods can happen and how to solve them. I know about these:

• File replacement conflict - solving these is the primary focus of this extension. This kind of conflict can be solved by simply text-wise merging the two files. But there is one thing I'm not sure about: do the replaced files have to be in location (folder) precisely identical to the original one in Assets folder or do they have to have the same name only?

Click to expand...

Great idea! It should be tremendously useful to those who make mod packs.

With regard to your question above, they only need the same name as far as I'm aware. I know that Kael's guide doesn't reconstruct the entire directory structure for modifying core UI files. I personally tend to recreate the structure to help track which files have been changed/overridden.

Thank you for confirmation, I thought so.

You wrote " what kinds of conflicts between mods can happen ".

I'm not a modder myself, but I've been playing Civ since CivII, and I follow what modders do.
I think that you may find that there are also "logical conflicts", meaning when you make one tech a prerequisite for another, and another mod reverses them. Or a certain tech reveals something on the map or opens a further tech path, and another mod has a different mechanism for doing the same thing. I don't know whether these instances would fall under "Database conflicts".
If so, please forgive my ignorance.
All the best, and thanks for all your work.

Not to discourage you, but I guess to do this properly is more complex.

First, file names and file location are completely random, at least for anything but lua (I assume that for lua the filenames (not location) generally stays the same, so you know when two mods change the same file and therefore are conflicting, at least for standard files, not for any additional custom lua files).

As the filename therefore does not tell you anything, you will need to look at the content to find potential conflicts.

Second, thanks to Civ 5 allowing SQL like updates (where a whole lot of entries are affected because of the where condition), you will have a hard time finding conflicts for those kinds of changes. I guess I would not even try to do so and only try to identify them in 'regular' xml.

Finally, if you get this right, you still 'only' find changes that change the same object. This does not mean they are conflicting however or need to be merged into one file, as the same object (but different attributes of it) can be changed in different files without any problems.
Logical conflicts which are not on a object basis probably cannot be figured out (at the very least only with a lot of additional effort).

Thanks to the new xml logic, there are pretty much no more xml conflicts, other than logical ones. They were a lot more likely in Civ 4. So to me the only / main source of conflicts will be lua - and there a tool like WinMerge will already tell you what the differences between the files are.

If you create a tool which is smart enough to merge lua files, more power to you, for xml it probably is not really needed any more from a strictly technical perspective.
The only conflicts you could identify (relatively easily) in xml are conflicting changes to the same attribute of the same object in multiple files. You could also offer to merge all changes to one object into one file, whether this is actually desirable is up to the user (I would definitely keep it an optional step) as by keeping the changes from different sub-mods in different files, updating one of the sub-mods is easier than in a merged file.

In this regard Civ 5 clearly is a step forward over Civ 4 where the xml was far less flexible / powerful, making conflicts far more frequent.

First, file names and file location are completely random, at least for anything but lua (I assume that for lua the filenames (not location) generally stays the same, so you know when two mods change the same file and therefore are conflicting, at least for standard files, not for any additional custom lua files).

Click to expand...

Merger would focus primarily on these LUA file conflicts, once these are sorted out, I can think about how to fix other conflicts. I'm afraid I won't be able to do much about those though.

CommanderZ said:

Merger would focus primarily on these LUA file conflicts, once these are sorted out, I can think about how to fix other conflicts. I'm afraid I won't be able to do much about those though.

Click to expand...

So you'd take 2 or more Lua files, diff them with the vanilla assett, and try to merge the diffs?

SamBC said:

So you'd take 2 or more Lua files, diff them with the vanilla assett, and try to merge the diffs?

Click to expand...

Exactly.

That doesn't seem practical for anything but the simplest of mods.

The idea is to automatically merge mods, which edit the same files, but in different areas. Or if the mods modify the same area of one file, but only add stuff, the merge should spit out working code.

One of the reasons why I made this thread so soon is to ask people, who actually do some mod merging, what king of programmatic help would they find useful.

If I find out, that it wouldn't be possible to make a very useful tool in reasonable time, I would scrap this extension and do something useful instead (or actually play the game ). If you have any ideas on what else you would find more useful, tell me.

What happens when there are common function names, or common variable names, etc?