Mod switcher

[tdb]

I've updated the elements.txt in the above URL to work with Immigration mod.

Impaler: You had made some mistakes with your updates.

ReligionInfo Civ4ReligionInfo toplevel
[b]ReligionInfos ReligionInfo set[/b]
ReligionInfo GlobalReligionCommerces list
ReligionInfo HolyCityCommerces list
ReligionInfo StateReligionCommerces list

This one has the wrong toplevel element (a typo I suppose). The ReligionInfos element should be container and ReligionInfo an object. However, the typo prevented this line from working and the hardcoded rules for Info stuff took over. Set and list elements can only contain value elements!

LSystemInfos LNode container
LSystemInfos LProduction container
LSystemInfos ArtRef list
LSystemInfos To list

This just won't work. Containers and objects rely on each sub-element being uniquely identifiable from the element name or key. See also a few posts above.

UnitFormations EventMaskList list
UnitFormations UnitMaskList list
UnitFormations PromotionMaskList list
UnitFormations Position list
UnitFormations UnitEntry list

Position should be container. UnitEntry is basically a container too, but can potentially cause problems since it can occur multiple times. I'll have to see if I can figure something out - there might be an easy-ish fix for it. The masks should be sets, since each value is unique and order doesn't matter (I suppose).

These are fixed in the above elements.txt as well.

 

[tdb]

Yup, the multiple UnitEntry problem was easy enough to fix. You'll see the result in the next version. Unfortunately this doesn't help too much with the LSystem files, since it relies on the number and order of entries with the same name staying constant, and adding resources requires adding stuff to those files.

 

[tdb]

I managed to edit EFA and MSDA slightly to make them combine successfully, but it depends on one code change too, so I'll give you the edited versions at the same time as the next version of the program.

 

[tdb]

I tried CivMore, which is 180MB packed and 300MB unpacked (and happens to contain some things I want). As I suspected, when it got to the final saving phase, the program was using 705MB of RAM. SevoMod is even larger, so no wonder if it crashes with only 512MB of RAM.

 

[tdb]

New idea for future: "bridge" mods.

This would allow features that are only present when certain two (or more) are selected, but would allow each of those mods to be used separately as well.

Example: Mod A adds a new civilization. Mod B adds a new leader trait and. Mod A would like to use this trait, but would also like to maintain independency. Now the author of mod A could create a bridge mod that automatically gets enabled when both A and B are enabled, and changes the traits of mod A's leaders.

What do you think?

 

[Impaler[WrG]]

I'm not familiar with the concept of Bridging, its sounds like a kind of "soft" dependency "use if its their but not if it aint" which sounds good in practice but I worry how easy this will be to configure. Mod makers are lazy and tend to make one of two types of mods, mod components which are intended to be used with other things and big modpackages. I see ModSwitcher being used by the makers of the components as a vehicle for easier merger and greater use of their component. The package makers will use it to save themselves time, but as these people already spend vast sums of time hand merging I would expect them to still be willing and able to do some "stitching" nessary to combine components. So for example the new Trait comes as one component in a diff file and the Leader as another file. The package maker merges them and then edits the leader to use the new trait. If a bridge mod is more work then stitching they will probably skip it as their only concerned with their imediate modpackage.

I'm not shure what the best method may be for achiving the goal but it should reflect the typical user/moder and how the creator->consumer flow actualy works in most situations.

 

[tdb]

I just invented the concept, more or less. The idea is to "smooth the edges" when combining mods. Of course, my vision might be different from how people actually use the program, so the idea might or might not be useful. In my vision, mod makers would ultimately create relatively small mods and release them in the Civ4Mod format, ready to be plugged in to Civ4Modder. Players could then choose exactly what they want to have in the game.

 

[Impaler[WrG]]

Could people areadly do something similar by the following steps

Diff 2 seperate mods from the Firaxis Original Assets

Merge the mods but use an empty directory as the Base upon which to merge (am I correct in thinking that would give you all files back with just the added elements in the file).

Create a copy of the merged mods and edit the copy with linkages between the various elements (such as give a new leader from one mod the new traits from the other mod).

Diff the edited files from the un-edited files to record the changes as yet another Diff file, this one being a "Bridge" and the first 2 (or more) diffs being its dependencies.

Now some other guy downloads these 3 files and opens up the UI which lists the Bridge as an option requiring the various dependent files. The merging process is now 2 step (hopefully this can be automated as its rather straitforward) firsts all non-bridge mods are merged with the normal Firaxis original code (from properties) as the base, then all Bridges are merged using the results of the first merge as the base. This resulting final merge overwrites the first and the end user gets a rather seamless end result out of it all. Is this what you had in mind?

 

[tdb]

Something like that. It isn't really necessary to use an empty base though - applying a mod and then reading the result back in will produce the same mod. In fact, it would only cause problems when creating the combined mod, since the program would see all files as "new".

As for the merging process, mods are always applied one by one, i.e. there will be as many "phases" as there are enabled mods. Bridge mods would automatically override all of their dependencies without having to ask the user about conflicts.

 

[tdb]

Version 0.5 released. Be sure to also check http://civ4.tdb.fi/?page=mods.

 

[Michelangelo]

Downloaded it, and tried it..........................

It works now for the EFA and MSDA (downloaded from your site). The ones I create myself don't. I'll take a look at it what you changed but I have the idea that it has to do with screwed up formatting in regard to the original, while not actually changing anything substantial.

The problem with the invisible modnames (probably due to the incorrect reading of the version number) still exists.

I'll try to combine some other mods now myself.

 

[tdb]

Of course they work, since I made them work It was pretty simple really, if you know how diffs work. Then again, if you don't... I can give a more in-depth explanation later, if you want.

Unfortunately, I wasn't able to reproduce the invisibility problem on my computer (tried both Windows and Linux). The program uses exactly the same code for updating the mod list both on startup (or refresh) and after changing its properties. This would point to something being wrong with the mod file, but that evidently isn't the case as your files work here. Thus, I'm pretty baffled here...

Does the mod information show up in the properties dialog? Are you using any non-ASCII characters? Do you have other computers you could try it on? Perhaps with different versions of Windows?

I guess I'll have to create a special version of the program that shows some debugging information...

 

[Michelangelo]

Does the mod information show up in the properties dialog? Are you using any non-ASCII characters? Do you have other computers you could try it on? Perhaps with different versions of Windows?

Everything shows exept the version number.

This is what I get when I open the properties of a mod.

If I change the version to anything i've tried, normal characters including numbers, f.i. "1" "v95" "0.95".and I click on save, close the window, I can see the mod. But when I go back to the properties screen or press refresh, the version number is gone again.

I would like to know a bit more about the diff's. I'm hesitant on making changes as I see a location indicator of the change (@@ .....@@). I don't know what happens if I remove / add something from a diff. No hurry though as for now I can just ask you to make it work for me

 

[tdb]

That's wery strange indeed. I'll create the special debug version in the near future so we can get to the bottom of this.

Each line beginning with @@ starts a new diff block. The first pair of numbers is the starting line and line count in the original file and the second pair has the same information for the modified file. Civ4Modder ignores everything except the original starting line (even that can be wrong and it'll still work).

The block data consists of lines, prefixed with a character telling the type of the line. ' ' means a "context" line, i.e. it is the same in both files and is used to find the correct location. '-' is a removed line and '+' is an added line. When applying a diff, all context and removed lines must match to the original, otherwise an error is thrown.

Moving on to the problem with MSDA and EFA... If we take a diff of the CvScreensInterface.py file from both, we get the following (I've cut some unnecessary stuff):

$ diff -u Assets/Python/EntryPoints/CvScreensInterface.py Mods/ModifiedSpecialDomesticAdvisor/Assets/python/entrypoints/CvScreensInterface.py 
@@ -1,7 +1,14 @@
 ## Sid Meier's Civilization 4
 ## Copyright Firaxis Games 2005
 import CvMainInterface
-import CvDomesticAdvisor
+
+## 12monkeys - Modified Special Domestic Advisor - begin
+##--------------------------------------
+## import CvDomesticAdvisor
+import CvModSpecialDomesticAdvisor
+##--------------------------------------
+## 12monkeys - Modified Special Domestic Advisor - end
+
 import CvTechChooser
 import CvForeignAdvisor
 import CvMilitaryAdvisor

$ diff -u Assets/Python/EntryPoints/CvScreensInterface.py Mods/ExoticForeignAdvisor\ v0.96/Assets/python/entrypoints/CvScreensInterface.py 
@@ -3,7 +3,10 @@
 import CvMainInterface
 import CvDomesticAdvisor
 import CvTechChooser
-import CvForeignAdvisor
+# Exotic Foreign Advisor Change 1/2
+#--------------------------------------
+import CvExoticForeignAdvisor
+#--------------------------------------
 import CvMilitaryAdvisor
 import CvFinanceAdvisor
 import CvReligionScreen

After applying the first one, it looks the same as the file in MSDA:

## Sid Meier's Civilization 4
## Copyright Firaxis Games 2005
import CvMainInterface

## 12monkeys - Modified Special Domestic Advisor - begin
##--------------------------------------
## import CvDomesticAdvisor
import CvModSpecialDomesticAdvisor
##--------------------------------------
## 12monkeys - Modified Special Domestic Advisor - end

import CvTechChooser
import CvForeignAdvisor
import CvMilitaryAdvisor

Now, when the program tries to apply the second diff (from EFA), it first tries to find a line that says "import CvMainInterface". That's found at line 3, since MSDA didn't change it. However, according to the diff, the next line should be "import CvDomesticAdvisor". But since MSDA changed this, the program doesn't find what it expects and decides "nah, this block didn't fit here after all" and goes looking for more possible positions to put it in. There aren't any in the file, so it eventually throws an error.

And what's the solution? Remove the surrounding context from the diffs, leaving only the removed and added lines. Now the program will only look for that one line and replace it with another. Since the mods change different lines, they won't interfere with each other.

 

[Michelangelo]

Thanx for the explanations. I'll probably need it when incorparation more mods.

Please contact me if you need some testing done for resolving the bug, or other bugs/features etc. I'll be happy the help.

 

[Michelangelo]

Bug Update,

The version is only deleted when I restart or press refresh. If I check the preferences when the mod is visible in the modlist, then the version number is displayed.

 

[Impaler[WrG]]

Any new developments?

I was thinking that their should be a collection of pre-diffed mods made so people can easily download and merge them. I think it would be prudent to have a thread under ModComponents page with a number of the popular mod Components for download as well as a post onthe completed mods page with diffs of the popular compilation mods. People can then do their own add-ons much more easily.

 

[tdb]

I've been pretty busy actually playing the game, sorry

I'll be adding the quick merge feature next and maybe fixing some of the existing problems listed in the readme.

There's one thing I'd like some feedback on: dynamic context size for python scripts. Way back at version 0.2 or so there was some talk about this. However, I'm a little sceptical about whether mod makers will use the feature, and it's about as easy to edit the diff by hand with the built-in editor. Basically, it would work like this:

import Module1
[b]#!context 0
import MyNewImport
#!context default[/b]
import SomeOtherModule

The program would not generate context for the bolded lines, and thus would ignore any possible changes to the other imports when merging. Given the fact that mod makers probably won't use this, do you think this is worth implementing (1) before version 1.0 or (2) at all?

As for the pre-diffed mods... Did you notice I already put some up at http://civ4.tdb.fi/?page=mods? Feel free to submit additions.

 

[Impaler[WrG]]

Yes I know what its like to get distracted actualy playing the game

by the way the link is broken, and I agree that feature is very unlikly to be used by mod makers so theirs little point in including it.

You mentioned that diffs are applied one by one. I had always though their was some means of doing them all at once. My thoughs were something along the lines of...

The first hunk of each diff is read and its line recorded, the line numbers are compared and the lowest line number/first diff pointer and linenumber are recorded in another location. The original files are copied untill the first hunk line is reached, then the hunk is writen in. The diff that it came from then advances to the next hunk and lowest-line-hunk is checked again and updated. When the lowest-line checking finds that 2 or more hunks modify the same line then it throws a conflict and we go into conflict resolution mode. This might make it easier to resolve sticky situations as all the relevent diffs are on-hand, it also allows for a 3+ way conflict to be resolved better as the user can be presented with all the conflicting data at once rather then just 2 at a time.

 

[tdb]

Seems like the forum decided to interpret the end-of-sentence question mark as part of the url. The correct one is:

http://civ4.tdb.fi/?page=mods

It would basically be possible to apply all diffs at once as you suggested. The result would be the same however, and it wouldn't make automatic conflict resolution any easier. It is also possible to collect all mods that conflict on a single file and ask the user to order them, but this will create complications if we also want to report all of the conflicting files for a single mod pair. I'll have to think about ways to include both of these.