[SDKMOD] Modular XML Loading 2.0

[Morfydd]

I would LIke to See A new Forum Created for A Plug And Play Compatible Library

*hint hint THUNDERFALL

I have to agree with purplexus on this one modules will need there own download database <im thinkign there will be many modules <civilizations, units, buildings, hmmn thats a lot of files then theres terrain etc >

 

[purplexus]

Note as of now there is no way to include buildings into a modular plug and play system

Every building will need a cut and paste section to include into the LPlotSystem.xml as well as a quick how to add that section. Perhaps someone will write this and put it at the top of a read me which is included with each and every building for a plug and play system or civilization with a building already included in the civ.

 

[purplexus]

Impaler I have petitioned Thunderfall for a plug and play forum. It will require a sticky for your CCCP add on as well as a how to include the CCCP add-on (don't forget to let every one know the options that the CCCP can do).
categories for your forum may I suggest a separate one for a HOW TO:
and each of the following. CIVILIZATIONS... BUILDINGS... TECHS... UNITS... WONDERS... SOUNDPACKS... MISC...
May I also suggest a Library record be kept in a separate sticky if someone wants to take on this challenge?

 

[Impaler[WrG]]

The only good think about the LSystem is that it dose not blow up with a dangling reference (Translation: The LSystem can contain references to Buildings that arnt loaded without harmfull affects). We can thus build up a single LSystem with all know buildings on it and keep it in a toped thread for people to use.

Currently the modular loading the CCCP is behind that of the 2.0 release that I made earlier, it dosn't load as many of the smaller more obscure file types. The next release will be Modular loading 2.1 and CCCP 2.07 which will each contain the most advanced loading.

Also I've come across some issues with Schema's it seems that the game will use schemas found in moduals to validate stuff out side that modual, sometimes the base data and sometimes another modual. As long as all the Schemas are the same their were no issues but trying to use any kind of variety screws things up badly so I'm going to include instructions for re-naming schemas as well. Its very trivial, pre-pend the modual name to the Schema and update the schema line at the top of the file.

I'll also try to fix some of the bad error messages that have been poping up when something dosn't load right and try to add a means of skipping Base File content by use of Global Defines.

The modual Forum sounds interesting, it will be tricky though to set them up in such a way that they are not passed over and neglected. What we realy need are more modual makers to create a more extensive library.

 

[Dale]

Impaler, this is AWESOME work!

Some questions:
1. If I completely throw out the default units in my mod, does my module units file overwrite the original units? EG: default - Unit[0] = SETTLER, my mod - Unit[0] = RIFLEMAN (same with buildings, improvs, etc)
2. If I add a new tag to units, do I need to add the new tag to both the default schema and the module's schema, or just the module's schema?
3. If I have less items in the module file (EG: 21 units) do the default items over that number still exist (EG: units 22+)?

I'm sure I'll think of some others. But that's enough for now.

Dale

 

[Impaler[WrG]]

1) Original (Base) units are over-writen if and only if they they have the same name so <UNIT_SETTLER> in a Modual would replace the original settler without any modification to the Base file, single unit modifications are even easier to do then a full new element as you only need to include the files your modifying (typicaly the Big Info file). To get ride of a unit entirely you would have to use the traditional method of modifing and including a Big Civ4UnitInfos.xml file that has had that unit removed. An alternative is to modify just the UnitClass for that Unit and make the Default unit of the Class None, the unit would then be unbuildable but would still show up in the Pedia.

2) Include the Modified Schema in the modual but rename it "DCM_Artilery_UnitSchema.xml" or some such thing, and have the other XML files point at it, version numbers might also be a good idea if you plan to keep modifying that Schem in the future. It seems the game creates a cach of all Schema file it finds and if the schemas in the moduals have the same names but are different it can mis-apply them. Specific linking each modual with its own schema should allow to be as secure against incorrect validations and any future modifications by yourself or others.

3) The quantity of items in a modual is irrelivent because the data is sorted by Name, if its present in the Base file (which is loaded first) then it over-writes. If not it appends it and increments the total item count. Thus most moduals contain only a single child element on each of their xml files, the bare-bones to add say a new unit or Leader to the game. If multiple elements are present (for example a single Civ4UnitInfos.xml that holds the the Catapult, Treb, Artillery, Battleship witch have been modified to have artilery range) each child element will be processed on its own just as it would be if it was a seperate file so clumping together several elements is a handy way to reduce the amount of cutting and pasting nvolved especialy for modifications to large numbers of existing units. (New stuff is best served ala-cart so people can pick and choose)

P.S. Most of the core functionality I've just explained is realy coming from CA's code, specificaly the AddGlobalClassInfo function and some variations on it for specific hard to handle files. I created a nice wrapper function ProcessClassInfo around it to extend the loading to many many more file types, most importantly Text and Global Defines. The wrapper provides consistent regular expression file enumeration. The Schema stuff was the last major hurdle in getting full plug and play moduals and I figured that out mostly by trial and error. I'd be interesting in what you think of the code and how it could be improved.

 

[purplexus]

I have had a though on this for a bit.... but what if someone were to totally strip the warlords of all civs and also make those modular too? then you have a base in which the game would work with and you just pick and chose which civ's you want in the game. This would make moding for those like FFH where everything is different easier to make.

Just an idea though?

Am I to understand that there are issues with diplomacy xml files?

 

[Gaurav]

I have had a though on this for a bit.... but what if someone were to totally strip the warlords of all civs and also make those modular too? then you have a base in which the game would work with and you just pick and chose which civ's you want in the game. This would make moding for those like FFH where everything is different easier to make.

Just an idea though?

Am I to understand that there are issues with diplomacy xml files?

Actually, just provide such a "blank template" CIV4CivilizationInfos.xml file and instruct module makers to include it if and only if they are planning to remove all (or most of) the original civs. The modual would then still use FFH_CIV4CivilizationInfos.xml or whatever, and both files can be in the same folder.

I think it will be very rare for anyone to try to merge two total conversion mods.

 

[Impaler[WrG]]

Yep that would be the easiest way to knock out all the existing Civs, the game would then load the modual/s and they would be the only data. The Blank file though would need to be in the traditional pathway and file name for examle xml/Civilizations/Civ4CivilizationInfos.xml

 

[purplexus]

Actually, just provide such a "blank template" CIV4CivilizationInfos.xml file and instruct module makers to include it if and only if they are planning to remove all (or most of) the original civs. The modual would then still use FFH_CIV4CivilizationInfos.xml or whatever, and both files can be in the same folder.

I think it will be very rare for anyone to try to merge two total conversion mods.

I was thinking more in a sense such as this that if I wanted to make say for instance a North american map and have the north american tribes only fight it out. I would like to include the Americans the native american tribes and canada with mexico. So in this sense to have the american civ still in the game but as a modular only would be easier for such a scenerio to make. Know what I mean? know what I mean? Nudge nudge... wink wink???

 

[Gaurav]

I was thinking more in a sense such as this that if I wanted to make say for instance a North american map and have the north american tribes only fight it out. I would like to include the Americans the native american tribes and canada with mexico. So in this sense to have the american civ still in the game but as a modular only would be easier for such a scenerio to make. Know what I mean? know what I mean? Nudge nudge... wink wink???

No big deal. Now three files in your module:

1. CIV4CivilizationInfos.xml (blank template in traditional path)
2. America_Civ4CivilizationInfos.xml (copied America from original version.)
3. NorthAmercianCivs_Civ4CivilizationInfos.xml

 

[purplexus]

No big deal. Now three files in your module:

1. CIV4CivilizationInfos.xml (blank template in traditional path)
2. America_Civ4CivilizationInfos.xml (copied America from original version.)
3. NorthAmercianCivs_Civ4CivilizationInfos.xml

This would work for sure! But I think what we are trying to accomplish correct me if I am wrong... But take the blank template and then just drag and drop all the civs you want to use in the scenario. it should be that easy although there is still and i think there always will be an issue with the Lplotsystem file.

Wouldn't it be cool if for now on someone creates a civ and if you want it in your game you just drag it into your custom assets folder? As it stands right now I would have to cut and paste every single xml and include them into the original .xml file

I think this is the most powerful tool yet to be built.. IMHO

What I would like to do with my original civ 4 files is set it up as a blank template period. and then just throw in the civ's that I like. The reason this is so valuable to me is this.... Having worked on the Civ Gold team a little bit... I noticed how much friggin work it was to add yet just one more civ. Now it should be just alter one file and VIOLA! done.

 

[Gaurav]

Wouldn't it be cool if for now on someone creates a civ and if you want it in your game you just drag it into your custom assets folder? As it stands right now I would have to cut and paste every single xml and include them into the original .xml file

I think this is the most powerful tool yet to be built.. IMHO

Yes, I think it is great too. As long as Impaler can follow the 80/20 rule (do only the work that solves 80&#37; of the problems) and not try to solve every freak edge case of XML mod merging...

 

[primem0ver]

Very interesting impaler and chinese american! I will have to check this out when we are ready to merge other mods.

 

[Impaler[WrG]]

I'm doing several Wonder Moduals from art work made by a number of different artist, I've got most of the wonders from my earlier wonder mod pack converted to moduals along with some new ones too.

In the screen shoot below theirs

Leonardos Workshop, The Brandenburg Gate, The World Trade Center, The Gateway Arch, The CN Tower, BigBen, The Harbor Opera House, The White House, The Apollo Program and the Sphinx

On my to do list, The Porcalin Tower, The Arc de Triomphe, The Mausoleum, The Statue of Zeus, The Code of Hamurabi, The Terracotta Army, Petra, The Leaning Tower of Pisa and Neuschwanstein Castle

All these moduals will soon be uploaded to the CCCP in the Library Branch of the project as freely downloadable and plugable moduals, they will require the use of a modified LSystem file which will also be their.

 

[MrUnderhill]

I'm having a bit of trouble merging this with the FFH SDK.

// Start Addition Extended GlobalDefines by Impaler[WrG]
    std::vector<CvString> files;
    gDLL->enumerateFiles(files, "xml\\*_GlobalDefines.xml");
    int i;
    bool failure = false;
    for(i=0;i<(int)files.size();i++)
    {
        if (!ReadGlobalDefines(files[i], cache))
        {
            failure = true;
        }
    }
    if (failure)
    {
        return false;
    }
    // End Addition by Impaler[WrG]

	gDLL->destroyCache(cache);
////////////////////////////////////////////////////////////////////////

It says here:

if (!ReadGlobalDefines(files[i], cache))

That:

CvXMLLoadUtilitySet.cpp:206: error C2664: 'CvXMLLoadUtility::ReadGlobalDefines' : cannot convert parameter 1 from 'std::allocator<_Ty>::value_type' to 'char *'
CvXMLLoadUtilitySet.cpp:5297: warning C4129: '*' : unrecognized character escape sequence
:: === Build finished: 1 errors, 1 warnings ===

I think it's having trouble converting a CvString (or whatever that is) to a C string, but I'm not sure. Is there a way to fix this?

 

[Impaler[WrG]]

The first error is due to not including a change to the functions protype on the .h which I aparently forgot to label (my bad) change the function protype on the .h too

DllExport bool ReadGlobalDefines(const TCHAR* szXMLFileName, CvCacheObject* cache);


and change the function definition on the .cpp too

bool CvXMLLoadUtility::ReadGlobalDefines(const TCHAR* szXMLFileName, CvCacheObject* cache)


The second unrecognized character thing is a harmless warning message caused by the * in the regular expession, I dont know a way to get rid of it but it wont cause any trouble.

 

[MrUnderhill]

Yep, that did it. It seems to work for the most part (see my sig), but a few things aren't quite working.

New promotions don't seem to accept other promotions as prerequisites (non-modular promotions still work fine), which means they can be gained in any order and they don't stack together (ex. Promotion I and Promotion II appear separately, rather than together). Is this fixable?

 

[Impaler[WrG]]

Its due to the way the game hashes all the names of thouse Promotions, youll experence similar behavior with Techs. Fortunatly my 3.0 code will remove this limitation by Hashing all names first before loading data, this will also allow universal referencing of any object by any other object, currently your restricted to only referencing something thats already loaded and the files load in a very precice order to make shure all these references are valid.

 

[Morfydd]

now that sounds usefull <ive been biding my tiem waiting on completion of the 3.0 version and next step in CEP and CIV Gold relearning XML coding and teaching myself python syntax <this old dog isnt going to be attempting SDK alteration and c++ while built on the programming languages i learned is well going to take me a goodly while to catch up on ..ie it will prolly be replaced by somethign else by the time i catch up >