Modular XML loading

[Impaler[WrG]]

I have an idea that could be of Huge assistence to mod makers who need to combine work made by multiple people. This always requires that people paste new XML elements on to the existing ones. But their is one area ware they dont, the Text files. Its possible to add an unlimited number of xml files in the text folder and they all load. I though, what if the same could be done for Units, Buildings and Techs the 3 most commonly moded objects. After examing the source that loads the Text files and the other files I think its possible.

This is how the Text files are loaded

// load all files in the xml text directory
//
std::vector<CvString> files;
gDLL->enumerateFiles(files, "xml\\text\\*.xml");
int i;
for(i=0;i<(int)files.size();i++)
{
bLoaded = LoadCivXml(m_pFXml, files); // Load the XML
if (!bLoaded)
{
char szMessage[1024];
sprintf( szMessage, "LoadXML call failed for %s. \n Current XML file is: %s", files.c_str(), GC.getCurrentXMLFile().GetCString());
gDLL->MessageBox(szMessage, "XML Load Error");
}
if (bLoaded)
{
// if the xml is successfully validated
if (Validate())
{
SetGameText("Civ4GameText", "Civ4GameText/TEXT");
}
}
}

DestroyFXml();

enumerateFiles is key it grabs everyfile in the specified directory, it could be used to grap new files out of a new directory we create within the Units/Tech/Buildings folders which would hold many small xml files each holding a single element.


I think I can use some of the standard C++ functions to enumerate directories, If so this will be even better as we can have a directory of directories and each game object could be packaged under a single folder containing all the objects XML such as the ArtDefine, UnitClass and Artfiles (.nif and .dds).

Adding a new object would be achived by dragging and dropping the folder!!
The only downside is it would likly require that the cach be ignored for these 3 particular files which would slow down loading by about a second or 2, but as most people are constant switching mods and not loading from the catch much as it is they wont mind.

 

[The Great Apple]

Very good idea! I certainly would use it!

It has more uses that then combining mods BTW. Imagine a fantasy scenario where all the Civs have a different unit list - you could keep them in different files, which would be alot easier to organise.

 

[Kael]

Very good idea! I certainly would use it!

It has more uses that then combining mods BTW. Imagine a fantasy scenario where all the Civs have a different unit list - you could keep them in different files, which would be alot easier to organise.

Nice for scenerio makers too if they could build on top of an existing mod and throw in a handful of new units and buildings. You may want to consider loading the new files from a seperate location so people can create their own scenerios/expansions for mods they enjoy.

Maybe a checklist on the main menu that would let you select what extra files you wanted to include or exclude. That way modmakers could include stuff that some players might not like in seperatly named files and those players can choose to play without them. It might be nice for any new units/building/etc that you guys choose to roll into the project if you do.

The dependancies are going to be hard though. Developers are going to have to be careful that the only place the techs (for example) are referenced are in the matching unit and building files. The unitclass and buildingclass files are going to need to be added. And then the trickier stuff like the CIV4CityLSystem.xml will need to be merged if you want the buildigns to be displayed in the city graphic (i dont envy the person to try to code a merge of that file).

 

[Impaler[WrG]]

I think a 2 tier system could work, it would go something like this

Someware (CustomAssets most likly) I create a "CustomModuals" folder, which can conttain any number of modual folders having any name. You use the Modual folders for containing content which has cross-referencing dependency such as an extensive game mod. Inside the modual you can have up to one folder for each of the various object types (Units/Buildings/Tech ect ect...) these will all be strictly named to identify the object type. Inside each of these an unlimited number of Object files is allowed and their names are loose so you can have them identify the object your adding "Dragon v1.5" "Alchemy Tech" and such. Inside the Object folder are several tiny xml files which each contain a single <UnitInfo>, <ArtDefine>, <BuildingInfo> and other such tags as are required to fully describe and implement the object. All these files are stricktly named to match the original assets files they come from, essentialy they are trunkated versions of thouse files.

A visual guide, Bold text is Rigid naming, Italic is flexible

CustomModuals
.........Moduals (unlimited)
...............ObjectTypes
.........................Objects (unlimited)
.............................Infos.xml

 

[member66170]

That would be good, but what about mods where 2 different things are changed for, lets say, 1 unit? It is a pain in the @ss to have to copy and paste a tag from 1 file to another.
Would it be possible to have the game combine both of the xml? It might be possible if you do something like this.

1) Load XML from MOD123

2) Look at the first tag of MOD456
2.b) If it is different to the corresponding tag in the vanilla XML, copy the tag and have it over-ride the corresponding tag in MOD123

3) Repeat for second tag...


Infact, you would only have to do this once, if you save the final XML file.

Does this make any sence?

 

[SimCutie]

I think a 2 tier system could work, it would go something like this

Someware (CustomAssets most likly) I create a "CustomModuals" folder, which can conttain any number of modual folders having any name. You use the Modual folders for containing content which has cross-referencing dependency such as an extensive game mod. Inside the modual you can have up to one folder for each of the various object types (Units/Buildings/Tech ect ect...) these will all be strictly named to identify the object type. Inside each of these an unlimited number of Object files is allowed and their names are loose so you can have them identify the object your adding "Dragon v1.5" "Alchemy Tech" and such. Inside the Object folder are several tiny xml files which each contain a single <UnitInfo>, <ArtDefine>, <BuildingInfo> and other such tags as are required to fully describe and implement the object. All these files are stricktly named to match the original assets files they come from, essentialy they are trunkated versions of thouse files.

A visual guide, Bold text is Rigid naming, Italic is flexible

CustomModuals
.........Moduals (unlimited)
...............ObjectTypes
.........................Objects (unlimited)
.............................Infos.xml

I think that some folder below XML fodler is better than "CustomAssets/CustomModuals"
e.g.) CustomAssets/XML/Custom and MODS/Assets/XML/Custom/.
In this folder there may be several XML files like "ModXXX.xml", "ModYYY.xml"

You scheme would be great if all resource of a mod ( Python file, XML, image DDS, etc) is put there in below single folder. But currently it isn't so. And it is bit too complicated.
Particular mod can have multiple XML for each object as you suggested, but it would be nice if single XML for a mod ( for simple mod ) is supported, too.

 

[Impaler[WrG]]

How changes to the original files can be loaded is not clear at this point, I cant NOT load the regular files as this would screw everything up. I was thinking that it might be possible to completly chop up and re-organize the original content into Moduals (this sounds like a Herculean task so I think its best avoided)

It might only be possible for New content to be loaded in this manor, and it will likley not be possible to resolve conflicting information between changes. It might be possible for the CustomModuals to over-ride and replace the original game content but this is unlikly.

After looking at the XML loading function I see that its automaticaly pre-pending "Assets" to it,
After looking at the manor inwhich the BasicInfo.xml is loaded I think I can allow mixing of elements which do not normaly occur on the same document such as a <UnitType>, <UnitClass> and <ArtDefine> allowing moders to further consolidate an objects info onto one XML document. A new Schema for each type of mixed document would be probably be required. Its not clear how picky Civ4 is about the placement of Schema's I see they are stored in a cache so hopefully I dont need to place the Schema right next to the document it is referenced by.

 

[Impaler[WrG]]

With the return of tbd and new versions of Civ4Modder program which can merge all changes to the Assets direcory the need for modular loading is greatly reduced, I think I will focus on assisting him with debugging and improving his program (he released the source on his website) as it is in my opinion an even better solution to the problem of Mod merging being quicker and easier. Consider this mod/idea on indefinate suspention...