MXMLL 3.0 by Impaler[WrG] & Chinese American DOWNLOAD GET MODULAR XML LOADING 3.0 DLL, Source Code, Templates and Instructions HERE INTRODUCTION Modular XML Loading (MXMLL) is a mod to the Civ4 Warlords DLL which allows the game to load and read multiple XML files of many types. It uses a standardized naming and organizing convention which has the promise of delivering full drag and drop modularity to the games content (almost). Properly configured modules will be able to alter the games default data or add new content such as a Tech, Wonder, building, Unit, Leader or whole Civs with such elements nested inside it. The end of tedious cutting and pasting to assemble Mod content will herald a new age in Civ4 Moding. NEW IN THIS VERSION 3.0 is a deep re-write of MXMLL 2.0 itself an update and extension of modular loading 1.2 by Chinese American which can be found at http://forums.civfanatics.com/showthread.php?t=184907. Despite this all modules made under 2.0 remain compatible with this 3.0 version which removes the restriction in 2.0 which prevented a base element from referencing a module element. In addition moders can safely reference any XML element on any supported file or within a file rather then just those that were earlier in the loading order. Lastly the ability to read and write modular data to the games cache allows for faster loading both when using modules or not. Future version will seek to address the remaining shortcomings particularly in the area of Audio and Diplomacy text. HOW IT WORKS The mod operates entirely during the xml loading portion of the game. First the games "Base" content is added to a list of targeted files. Base contents are the games original XML files as they would otherwise be loaded meaning Custom Assets at the highest precedence, then the Mod if any and lastly Assets from the Civ4 directory. Next regular expression searching is done of the whole XML folder Mod folder and custom assets are both found and adding to the file list. All Tag names on these files are then hashed for future reference and the data indexes assigned. Then a second round of loading occurs and the remaining data is loaded into the predetermined indexes (as opposed to post-determined as was originally done). Modules load after Base content and can step-on aka replace Base data that has a matching Type Tag name, modules will also step on each other with the last one to load being the one seen in game. Module loading order is not affected by Custom Assets vs Mod, either could step on the other so simply avoid ever having two conflicting modules. WHAT DOSN'T WORK Unfortunately do the the nature of the LSystem.xml files for Buildings and Improvements its not possible to modularize these files because they don't pass through the SDK. Sound and Audio data has not been addressed yet either and several file types are linked with Enums in the SDK thus any extensions of these files would have no effect without a corresponding modification in the SDK. All other data both XML and art can be made into a module but without LPlot data the building or improvement will be invisible in the game, though it will have every other game play affect and display in the Pedia. A copy of a modified LSystem or instructions on how to modify it should be included in a Building or Improvement module and some notes made on the readme.txt. Similarly custom Diplomacy Text common in new Civs isn't supported due the the complex nature of this file. Both of these files will require traditional cut and paste. SUPPORTED FILE TYPES The following types of files are supported, in all instances the game will search and find files matching xml/*_TYPE.xml ware TYPE is one of the following strings. CIV4ArtDefines_Unit CIV4ArtDefines_Building CIV4ArtDefines_Improvement CIV4ArtDefines_Civilization CIV4ArtDefines_Leaderhead CIV4ArtDefines_Bonus CIV4GameSpeedInfo CIV4TurnTimerInfo CIV4WorldInfo CIV4ClimateInfo CIV4SeaLevelInfo CIV4AdvisorInfos CIV4TerrainInfos CIV4EraInfos CIV4UnitClassInfos CIV4SpecialistInfos CIV4TechInfos CIV4FeatureInfos CIV4TraitInfos CIV4GoodyInfo CIV4HandicapInfo CIV4CivicOptionInfos CIV4UpkeepInfo CIV4HurryInfo CIV4SpecialBuildingInfos CIV4ReligionInfo CIV4CultureLevelInfo CIV4BonusClassInfos CIV4VictoryInfo CIV4BonusInfos CIV4BuildingClassInfos CIV4BuildingInfos CIV4SpecialUnitInfos CIV4ImprovementInfos CIV4CivicInfos CIV4LeaderHeadInfos CIV4ColorVals CIV4PlayerColorInfos CIV4EffectInfos CIV4BuildInfos CIV4UnitInfos CIV4CivilizationInfos CIV4GameText GlobalDefines NAMING RULES The file naming system that must be used to get a modular xml file to load is strict but simple and universal. Because * is a wild card you can place a descriptive name at the front such as Ninja_CIV4UnitInfos.xml also their can be any number of intervening directories under the xml directory so data can be nested to your hearts content. This is in contrast to the games default loading which was and still is very rigid in its structure. Failure to have a file match the regular expression is the most likely reason for content to silently (no load warning) not show up in the game. SCHEMAS The main drawback of this directory flexibility is that every XML file still needs to be validated before loading and the games validation code is off limits to me. It demands the Schema be in the same directory as the file its validating, if not the loading fails and a warning is thrown which you can OK past to continue loading but their will then likely be subsequent "ITEM NOT FOUND" warnings as the other files associated with the module are left with dangling references. Their have been odd cross validation errors I cant explain arising from modules using schemas of the same name, thus I recommend that all module schemas be renamed with a simple "module name_" at the front to match the module the module their in. Remember update the header of the files that use that Schema. This is especially true for Modules that are going to use a new XML tag that that interacts with a modified DLL, the local Schema will allow your module to load under a unmoded DLL and will isolate the modified Schema from invalidating other XML files. ORGANIZATION Though not required I would strongly recommend content creators and moders use my organization proposals to keep everyone on the same page and make modules easy to use. Within the xml their will be two folders for each of the major data types, one for Modifications and one for new material called Custom. Things like a Civ which have dependent sub elements should have their sub elements nested within their packages so the single package contains everything necessary to add or remove the Civ from the game. In general content should be in the smallest pieces able to standalone such as a single Unit, Tech or Building. Only if multiple elements are inter-dependent should they be packaged together. TEMPLATES To help make the creation of modules easier I have included as set of templates for several of the major data types in the core download. These folders contain an empty copy of each fill that needs to be included to get a new element of that type into the game. A modified element only requires a copy of the modified file so for example to change the Pedia of a Unit and Its art Def you would need only a Text and ArtDef file. A typical template looks like this. Code: <?xml version="1.0"?> <!-- edited with XMLSPY v2004 rel. 2 U ([url]http://www.xmlspy.com[/url]) by Soren Johnson (Firaxis Games) --> <!-- Sid Meier's Civilization 4 --> <!-- Copyright Firaxis Games 2005 --> <!-- --> <!-- Tech Infos --> <Civ4TechInfos xmlns="x-schema:CIV4TechnologiesSchema.xml"> <TechInfos> </TechInfos> </Civ4TechInfos> EXAMPLES As a separate download (at the CCCP Module Library Source Forge repository at http://sourceforge.net/projects/civ4ccp/) you can find a growing collection of modules of all types. These modules are free to download, use, modify and distribute. Moders are invited to submit new modules at the Home thread (see top) or by E-mail to me at firstname.lastname@example.org MAKING MODULES Because all previous mods have been in the "Monolithic" format, aka one file of each type, most new Modules will be made by extraction of existing content out of these mods or efforts to break them down completely into Modules. Another popular method will be from scratch construction especially with new Art assets that have just been released. Once you've chosen the type of module your making and have the contents at hand begin by select the appropriate Template folder, make a copy and rename it, use a good descriptive name that is unlikely to be used twice. Dump all the art files directly into the folder .nif's .dds's .kfm's .etc .etc. Then if your doing an extraction simply copy and paste the appropriate elements on to the template xml's. Rename each file by adding the Module name before the '_' also add it for the schemas. The ArtDef Info will need to be edited, change what ever file patway was being used before to "xml/TYPEOFMODULE/MODULENAME/FILENAME". TypeOfModule would be things like "Custom units" or "Modified Buildings", you might need a different organization thats more appropriate for your purposes but try to keep it organized. Some file types have Button pathways on them directly (Units) be sure to edit these as well. Pink/Magenta buttons result if the pathway is wrong, wrong .nif and .kfm files will appear blank or generate a python exception if Debugging is on. If your creating from scratch copy and paste from an original object that is a close match for what your making and edit it into what you want. Lastly discard any files you you didn't use, for example if making a Unique Unit you don't need to edit the UnitClass.xml so just discard it. Once your ready load up and look in the Civilopedia or start a game (don't try to run a save made without the module it will be incomputable). You may have to go through a few cycles of tweaking Art scales fixing broken pathways, balancing stats etc etc. When building up a large multi layered Module like a whole Civ its recommended that you start at the top level the Civ get it working and then add Leaders, Units and Buildings as sub-folders, this helps organize them and makes it easier to add and remove them later, be sure to take the parent directory into account when doing the art pathways. If any loading errors occur track them down by activating logging and examine the xml.log generated at load time, it will list each file loaded and what elements were found on them, errors are usually the result of copy paste errors or failure failure to properly reference the Schema. Finally fill out the readme so others who use your module can credit you and contact you, include any special configuration options or dependencies that will be needed. CACHE LOADING The 3.0 release includes the ability to load from the Cache like a normal unmodified DLL (this ability had been sacrificed in the 2.0 version). When cache loading is active the regular expression searching will be skipped for each file that can be loaded in this way (the 13 largest files comprising the bulk of the content), other file types will still search and load modules. When the game detects changes in Custom Assets it will conduct the full search and write the contents of all modules it finds into the Cache files, these will load on subsequent games at near the speed of the unmodified DLL's cache loading. It appears from testing that the game can in some case be tricked into falsely trying to cache load when Modules are added or removed so its recommended that you explicitly DE-activate cache loading after adding or removing modules to be on the safe side. If you get an error during Cache loading just reload and it should be fine. SDK MERGING This mod is highly mergable with other SDK mods. Not only dose it operate at a narrow time and place unlikely to interfere with any other code the files themselves are easy to combine because of the four files modified, three (CvXMLLoadUtility.h , CvXMLLoadUtility.cpp CvXMLLoadUtilitySet.cpp) are almost never touched in other mods and the third (CvInfos.cpp) has only a few modest additions in the XML readings. You are free to merge this code with your own so long as I am credited and the end user is given links to the SF Module library. Of particular interest is the Civilizeditor by Lopez which has great synergy with with mod. CHANGELOG Spoiler : 1.0 - By Chinese American, Sep 4th, 2006 Reads partial xml files for *_CIV4ArtDefines_Unit.xml, *_CIV4UnitClassInfos.xml, *_CIV4UnitInfos.xml. 1.1 - By Chinese American, Sep, 2006 Will replace old types with new ones. Reads partial xml files for *_CIV4ArtDefines_Building.xml, *_CIV4BuildingClassInfos.xml, *_CIV4BuildingInfos.xml. 1.2 - By Chinese American, Sep 9th, 2006 Workaround for bug where buildings get obscene happiness from new buildings. Added reading techs, improvements, builds, traits, leaderheads, civilizations. Reads partial xml files for *_Arts.xml, *_Classes.xml, and *_Infos.xml in xml\units\ and xml\buildings\. Reads partial xml files for *_UnitArts.xml, *_UnitClasses.xml, *_UnitInfos.xml, and *_Units.xml in xml\units\. Reads partial xml files for *_BuildingArts.xml, *_BuildingClasses.xml, *_BuildingInfos.xml, and *_Buildings.xml in xml\buildings\. Reads partial xml files for *_CIV4TechInfos.xml, *_TechInfos.xml, *_Techs.xml, and *_Infos.xml in xml\technologies. Reads partial xm files for *_CIV4ImprovementInfos.xml, *_ImprovementInfos.xml and *_Improvements.xml in xml\terrain, and xml\art\*_CIV4ArtDefines_Improvement.xml and xml\terrain\*_ImprovementArts xml. Reads partial xml files for *_CIV4BuildInfos.xml, *_BuildInfos.xml and *_Builds.xml in xml\units. Reads partial xml files for *_CIV4CivilizationInfos.xml, *_CivilizationInfos.xml and *_Civilizations.xml in xml\civilizations\, and xml\art\*_CIV4ArtDefines_Civilization.xml and xml\civilizations\*_CivilizationArts.xml. Reads partial xml files for *_CIV4LeaderheadInfos.xml, *_LeaderheadInfos.xml and *_Leaderheads.xml in xml\civilizations\, and xml\art\*_CIV4ArtDefines_Leaderhead.xml and xml\civilizations\*_LeaderheadArts.xml. 2.0 Initial Re-Release by Impaler[WrG] on Dec 20th, 2006 standardizes all file expressions to "xml/*_File.xml" Adds support for following files CIV4GameSpeedInfo, CIV4TurnTimerInfo, CIV4WorldInfo, CIV4ClimateInfo, CIV4SeaLevelInfo, CIV4AdvisorInfos, CIV4TerrainInfos, CIV4EraInfos, CIV4UnitClassInfos, CIV4SpecialistInfos, CIV4FeatureInfos, CIV4GoodyInfo, CIV4HandicapInfo, CIV4CivicOptionInfos, CIV4UpkeepInfo, CIV4HurryInfo, CIV4SpecialBuildingInfos, CIV4ReligionInfo, CIV4CultureLevelInfo, CIV4BonusClassInfos, CIV4VictoryInfo, CIV4BonusInfos, CIV4SpecialUnitInfos, CIV4CivicInfos, CIV4ColorVals, CIV4PlayerColorInfos, CIV4EffectInfos, CIV4GameText, GlobalDefines 3.0 Update by Impaler[WrG] Feb 1st, 2007 deep re-write adds caching of module data, universal cross file referencing, fixed incorrect error messages durring loading, many earlier functions by CA/Impaler[WrG] and Firaxis obsoleted and removed.