[SDKMOD] Modular XML Loading 2.0

[Impaler[WrG]]

3.0 version released get it on the first page!

[Morfydd]

Originally Posted by Impaler[WrG]

3.0 version released get it on the first page!

Thanks a heap

[Gaurav]

Okay, so if I created the file <MODS>/<ModName>/Assets/XML/Units/My_CIV4UnitInfos.xml would it be in the "right" directory? Meaning, would it take advantage of cache loading or not? Would it require a schema in the same directory or not?

Do you think Firaxis is going to include this thing?

[purplexus]

Are we now to break it down to the smallest parts such in a civilization for instance there are what Soundpacks...Units... Flags...Unique Units... Cities...Leaderheads???....Etc.???? or Am I reading that wrong???

[purplexus]

Originally Posted by Morfydd

now that sounds usefull <ive been biding my tiem waiting on completion of the 3.0 version and next step in CEP and CIV Gold relearning XML coding and teaching myself python syntax <this old dog isnt going to be attempting SDK alteration and c++ while built on the programming languages i learned is well going to take me a goodly while to catch up on ..ie it will prolly be replaced by somethign else by the time i catch up >

Bah Fortran, Cobal, and Pascal aren't that far off C++

they are all still compilers

[Morfydd]

Originally Posted by purplexus

Bah Fortran, Cobal, and Pascal aren't that far off C++

they are all still compilers

haha okay but it has been 10 years since ive written in any of those languages its the newer syntax that im having a bit of difficulty with ...its new terms to utilize ..as to fortran its not a compilier or at least fortran 77 wasnt ..nor was cobal in the incarnation i learned..pascal was a compiler and yea im relearnign as i said its the syntax thats getting me <shortened versions of words instead fo the entire woord ie bool to denote boolean ...as an example not a big deal but takes an old dog like me time to relearn>

[Morfydd]

Originally Posted by purplexus

Are we now to break it down to the smallest parts such in a civilization for instance there are what Soundpacks...Units... Flags...Unique Units... Cities...Leaderheads???....Etc.???? or Am I reading that wrong???

check the exampel libraries impaler has made and yea its the smallest parts ..ie you have all all the files for a single unit in a small directory ..you can have many of the mini mod directories and they should all run together iot eliminates having to have a huge line by line item file that you have to change to add a unit ..you just add one unit and its done..<note i used units in the example but its just about any item thats doable like this if you dont like leader head a .just add leaderhead b can replace a or use them together ..no sweat>

[Impaler[WrG]]

Okay, so if I created the file <MODS>/<ModName>/Assets/XML/Units/My_CIV4UnitInfos.xml would it be in the "right" directory? Meaning, would it take advantage of cache loading or not? Would it require a schema in the same directory or not?

Do you think Firaxis is going to include this thing?

The DLL would load such a file and write the info to cache during loading but the better pathway would be ...

<MODS>/<ModName>/Assets/XML/Custom Units/My/My_CIV4UnitInfos.xml

As for inclusion we can only pray they pillage the code as it would solve the chicken and egg problem (Cant use a Module if you don't have the moded DLL and why both if theirs no content for it, why both to make your content modular if no one has this special DLL). Early adopters like yourselves will push the envelope by creating the modules that get the ball roling. If it starts to take off and becomes a de-facto standard it has a much better chance of being picked up officially.

Are we now to break it down to the smallest parts such in a civilization for instance there are what Soundpacks...Units... Flags...Unique Units... Cities...Leaderheads???....Etc.???? or Am I reading that wrong???

check the exampel libraries impaler has made and yea its the smallest parts ..ie you have all all the files for a single unit in a small directory ..you can have many of the mini mod directories and they should all run together iot eliminates having to have a huge line by line item file that you have to change to add a unit ..you just add one unit and its done..<note i used units in the example but its just about any item thats doable like this if you dont like leader head a .just add leaderhead b can replace a or use them together ..no sweat>

Morfydd has the right Idea, organize at the level of usable game elements, units, buildings, civics etc etc. Generally when I do a unit its got a single folder with all the file types in it and no sub-directories, I dont both with 'Art' or 'xml' because in total their are only about 8 files. Their might be situations ware it makes more sense to combine two dependent objects. For example I'm working on some Resource Modules and one is going to be for Sevo's Natural gas. Sevo made a new improvement for the gas "Pipelines" and modified the Offshore platform to access the resource as well. I would combine the modified Platform, the new Pipeline and the Gas Bosus all into one folder because none work without the others. That what I mean by "stand alone" you can load just that one module and it will work . When it comes to Civs I recommend that the UUnits, UBuildings and Leaderheads be nested inside the Civ because even though these thing could be made as separate modules and would load fine they cant be accessed/played-with without a Civ that calls on them. Nesting them inside a Civ help make the plug and play process easy and you makes it easier to swap the contents out later, say for example to switch around a Leaderhead or extract a UU and turn it into a standalone unit.

[Gaurav]

Originally Posted by Impaler[WrG]

The DLL would load such a file and write the info to cache during loading but the better pathway would be ...

<MODS>/<ModName>/Assets/XML/Custom Units/My/My_CIV4UnitInfos.xml

Right, but when I read your new post, it sounded like this "better pathway" would not be cached and would require the schema file.

[Edgecrusher]

Im just wondering if this is happening for anyone else.

So I build the directory as described. There isnt anything in the folders. I create a unit to test it out, it loads, everything is fine, yet for some reason The Sphinx, Leonardos Workshop, the Aboringal Civ and the Medic (i.e. all the examples you had) load as well.

This confuses me because the files for them aren't in any of the XML directories, I dont even have the files on my computer.

Anyway I download the examples, i Put them into the correct folders and load again, for some reason it doesnt load, just hangs. Anyway thats whats going on here.

I love the work, it makes things a hell of alot easier for me.

[Impaler[WrG]]

That sounds like your trying to load from cache after changing the Assets, always load un-cached after changing or switching out modules. The other examples were preserved in your cache between loads. Also remember it loads everything in Custom Assets and the Mod folder.

[purplexus]

Just curious... how is the project going on the Civ Gold Modularization?
well I hope.!

[Impaler[WrG]]

Slowly at the present time, Wyz is concentrating on getting the new Civs made for CivGold4, making all the separate 1-civ modules is time consuming so I'm now thinking once CivGold4 is ready I will make a single "Super-Module" out of it in which which the files arn't divided into individual civs. This would lack the cherry-picking ability you would have to use all or none but it would be about a 100 times faster to make.

[purplexus]

Originally Posted by Impaler[WrG]

Slowly at the present time, Wyz is concentrating on getting the new Civs made for CivGold4, making all the separate 1-civ modules is time consuming so I'm now thinking once CivGold4 is ready I will make a single "Super-Module" out of it in which which the files arn't divided into individual civs. This would lack the cherry-picking ability you would have to use all or none but it would be about a 100 times faster to make.

Time consuming!!!... TELL ME ABOUT IT!

as for making a super-module will it be as easy as dragging the files into a certain civs folder?

OR!!!

will that be even necessary?

[Impaler[WrG]]

I've been thinking of some improvements I could make to Modular Loading code. I've got two big ideas for next version

The foremost is to allow art file pathways to be determined dynamically at runtime. Currently your forced to use names like >xml/TYPE_OF_MODULE/MODULE_NAME/File.DDS< and the person using the module has to put it in that exact file pathway or the art file won't be found. The xml files in the module will load from any location in the xml folder but the art is so restrictive is strangling the inherent flexibility of the xml loading. Dynamic runtime artpathways would simply combine the file pathway of the xml file with then art sting so you would just have >File.DDS< on the ArtDef and so long as you put the art in the same folder as that xml file everything would work fine. Their would of course be a new boolean tag for each pathway that activates this feature so older modules remain backwards compatible and you can continue to mix the games original assets (like animations) with new ones (like NIF and Skins). Modules making use of this feature would have total flexibility in their pathway and the user could create their own directory structure that best fits their mod.

The other idea I'm a bit shakier on, I'm considering adding a new folder dirrectly under Assets called "Modules" and making it the only valid location for modular loading. This is mostly for conceptual clarity, modules are typically art+xml so they don't really fit into any existing directory well. Kicking modular loading out of the xml directory would force some updating of existing modules but it would be rather easy to do and I'm hoping that the run-time pathways help to smooth that transition. The principle advantage would be that mod makers can easily drag out or drag in this new directory to turn their modular content on or off rather then having to root around in the xml directory. I'm considering grandfathering in the xml directory too just to give people the option and maximize back compatibility which so far has never been broken. Any suggestions?

[Gaurav]

If you are looking for new ideas...

It would also be useful I think to have the possibility of deciding whether a file should be loaded from python. This could allow "meta mods" like Ruff's Cobbled SG Modpack to for example disable an XML mod like the included GP type mod.

There was someone once working on a project to fully expose arbitrary XML to python as well, so as to allow for new XML attributes without SDK modding.

[purplexus]

Impaler what would the Load times be on modifying the CCCP DLL to go around searching for art be like. I imagine it would really increase this time as it would have many XML files and a lot of them have Art directions for them. If you take out the Art directions in these files YIKES!!!

I do like the Modules Folder idea a lot actually

I would also suggest to anyone who does use a module that they also have a modules storage folder too.

[Impaler[WrG]]

The run-time pathways aren't going to add anything (noticeable) to the load time, its just combining the string of the XML files directory (minus the xml file itself) with the string in the art tag. This feature will only work if the art is in the same folder as the xml file that references it which is already how all existing modules work anyways (at least the ones I've made), if for any reason you need to separate them just don't use this feature. Their wont be any searching beyond the search thats currently done for XML files via the regular expression search.

[Impaler[WrG]]

It would also be useful I think to have the possibility of deciding whether a file should be loaded from python. This could allow "meta mods" like Ruff's Cobbled SG Modpack to for example disable an XML mod like the included GP type mod.

There was someone once working on a project to fully expose arbitrary XML to python as well, so as to allow for new XML attributes without SDK modding

If were talking Python event triggered mods the Dr Elmer Jiggles Event Manager would be the thing to use, Ruff's mod uses an interesting option control panel to control a lot of python options but its all hard coded. I've been doing some brainstorming on extending or adapting this into some kind of registration based Option Manager much like the Event manager. Many INI file based python mods could relativity easily be switched to work with it in substitution. The Option manager itself would generate a UI for everything thats registered with it and could load the "default" configuration options from any source desired such as the INI parser or XML global Defines.

As for the global exposure of XML values to Python, I've heard this idea before but I'm just now beginning to think of a way this could be done, by a series of Hash maps on CvInfoBase. This could be loaded up with every tag or at least the ones that aren't nested and exposed to Python through some kind of call similar to the GC.getIndexforString call. I think this is doable but would probably result in a rather hefty load slowdown and all calls to the data will be slowed by the string interpretation.

[Gaurav]

Originally Posted by Impaler[WrG]

If were talking Python event triggered mods the Dr Elmer Jiggles Event Manager would be the thing to use, Ruff's mod uses an interesting option control panel to control a lot of python options but its all hard coded. I've been doing some brainstorming on extending or adapting this into some kind of registration based Option Manager much like the Event manager. Many INI file based python mods could relativity easily be switched to work with it in substitution. The Option manager itself would generate a UI for everything thats registered with it and could load the "default" configuration options from any source desired such as the INI parser or XML global Defines.

Most everything else can be already be done through python, I just need some exposed method call to set an XML file to load or not. I don't expect it to be fully dynamic given the way XML loads, just some way to tell the XML loader not to load a file before the load occurs. Then I just call it from python script before the XML loading takes place. Anyway it is just an idea, you can think about it.