Fun with xml extraction

Rwn · Apr 27, 2014

Part 2 + whole table in .txt attached

Spoiler :

Building | Bonus resource | :gold:

|

|

|

Building | Bonus resource | :gold:

|

Rwn · Apr 27, 2014

Some oddities I've encountered during my analysis (things that likely give the correct effect in-game, but through confusing syntax):

- BUILDING_SHAMAN_CATHEDRAL and BUILDING_SHAMAN_CATHEDRAL2 have two "PrereqTech" (not in Techtypes like the others) with the second being "NONE"

- BUILDING_SLAVERY_BAD_I has two iHappiness entries

- BUILDING_3DPRINTING_MILL, BUILDING_NANOPRINTING_MILL, BUILDING_PARTS_PLANT and BUILDING_WEMBLEY have only one <CommerceChanges><iCommerce> entries (instead of 4 for most buildings)

- The following have 3 <CommerceChanges><iCommerce> entries (instead of 4 for most buildings)

Spoiler :

- BUILDING_HIGHWAY_HIDEOUT is the only building which has only one <BonusCommerceModifiers><BonusCommerceModifier><CommerceModifiers><iCommerce> for each resource (instead of 4 usually, one for :gold:

,

and

):

Code:

			<BonusCommerceModifiers>
				<BonusCommerceModifier>
					<BonusType>BONUS_COPPER</BonusType>
					<CommerceModifiers>
						<iCommerce>5</iCommerce>
					</CommerceModifiers>
				</BonusCommerceModifier>
				<BonusCommerceModifier>
					<BonusType>BONUS_SILVER</BonusType>
					<CommerceModifiers>
						<iCommerce>5</iCommerce>
					</CommerceModifiers>
				</BonusCommerceModifier>
				<BonusCommerceModifier>
					<BonusType>BONUS_GOLD</BonusType>
					<CommerceModifiers>
						<iCommerce>5</iCommerce>
					</CommerceModifiers>
				</BonusCommerceModifier>
			</BonusCommerceModifiers>

Fun facts:
- no building has an :espionage:

bonus with resources (all 4th <BonusCommerceModifiers><BonusCommerceModifier><CommerceModifiers><iCommerce> are equal to 0).
- Hypermarket is the building that has the most resources giving bonus (57, giving each 1% :gold:

!)

Rwn · Apr 27, 2014

For the whole list of 179 buildings with resources giving bonuses sorted by number of different resources:

Spoiler :

Rwn · Apr 27, 2014

Faustmouse said:
No this has nothing to do with buildings. Quotes are a Tech-Thing. So for this you would need to have a look at the technology file, which should be rather easy because I think there is only one (probably more with alt time-lines.)

Attached is the list I've built based on all text files in C2C (either containing Gametext or located in xml/text).
There's something odd, though. I'm sure that some missing entries are in fact not missing (for example Writing, whose Civilopedia appears correctly in-game), I just can't find them anywhere - looking for "TXT_KEY_TECH_WRITING_PEDIA" in the whole C2C folder returns only one entry (the TechInfos file), the associated text is nowhere to be found. Is it possible that the C2C is taking the vanilla's entry instead?

Anyway, when I found an entry, I put the 40 first characters (that should be enough to identify duplicates); for the <SoundMP> entry, I just looked whether if was filled or not.

Here's an extract of the table to show what it looks like (the full list is too long for the forum):

Spoiler :

Regarding building duplicates, I've understood why they were there

The only "full duplicate" I've found is the Bambooworm one

Rwn · Apr 27, 2014

A few other oddities in the tech field:

- TECH_SPECIAL_PROMOTION has "Special Promotion" (instead of something like "TXT_KEY_TECH_SPECIAL_PROMOTION") in <Description> tag and nothing in the <Strategy> tag.

- Several techs have <Strategy> entries that do not look like they match the same technology. For example TXT_KEY_TECH__SIMULATED_REALITY_STRATEGY has a typo (one too many underscore), TECH_CRIMINOLOGY has TXT_KEY_TECH_MARXISM_STRATEGY entry, etc.

Here's the list:

Spoiler :

TECH_3D_PRINTING | TXT_KEY_TECH_3D_MODELLING_STRATEGY
TECH_ACYBERIMM | TXT_KEY_TECH_CYBERIMM_STRATEGY
TECH_ADHESIVES | TXT_KEY_TECH_WEAVING_STRATEGY
TECH_ANTIBIOTICS | TXT_KEY_TECH_MEDICINE_STRATEGY
TECH_ASTEROID_MINING | TXT_KEY_TECH_ASTEROID_MINING2_STRATEGY
TECH_ASTRO_ENVIRONMENTAL_SYSTEMS | TXT_KEY_TECH_SPACE_FLIGHT_STRATEGY
TECH_ASTROLABE | TXT_KEY_TECH_ASTRONOMY_STRATEGY
TECH_ATLATL_MAKING | TXT_KEY_TECH_SPEARFISHING_STRATEGY
TECH_ATOMPUNK | TXT_KEY_TECH_ROCKETRY_STRATEGY
TECH_BARBED_WIRE | TXT_KEY_TECH_STEEL_STRATEGY
TECH_BARK_WORKING | TXT_KEY_TECH_WOOD_WORKING_STRATEGY
TECH_BIOPUNK | TXT_KEY_TECH_CLONING_STRATEGY
TECH_CALCULUS | TXT_KEY_TECH_ALGEBRA_STRATEGY
TECH_CALLIGRAPHY | TXT_KEY_TECH_WRITING_STRATEGY
TECH_CHRISTIANITY | TXT_KEY_TECH_CHRISTIANITY_STRATGEY
TECH_CLOCKPUNK | TXT_KEY_TECH_CLOCKWORKS_STRATEGY
TECH_COMPULSORY_EDUCATION | TXT_KEY_TECH_CIVIL_ENGINEERING_STRATEGY
TECH_COSMOLOGY | TXT_KEY_TECH_THEORY_OF_RELATIVITY_STRATEGY
TECH_CRIMINOLOGY | TXT_KEY_TECH_MARXISM_STRATEGY
TECH_CRITICAL_THOUGHT | TXT_KEY_TECH_DIVINE_RIGHT_STRATEGY
TECH_CYBERPUNK | TXT_KEY_TECH_CYBERNETICS_STRATEGY
TECH_DIESELPUNK | TXT_KEY_TECH_COMBUSTION_STRATEGY
TECH_DIVINATION | TXT_KEY_TECH_PRIESTHOOD_STRATEGY
TECH_EARTH_OVEN | TXT_KEY_TECH_COOKED_FOOD_STRATEGY
TECH_EMANCIPATION | TXT_KEY_TECH_REPRESENTATIVE_DEMOCRACY_STRATEGY
TECH_ENLIGHTENMENT | TXT_KEY_TECH_SOCIAL_CONTRACT_STRATEGY
TECH_FUNGICULTURE | TXT_KEY_TECH_AGRICULTURE_STRATEGY
TECH_GROUND_STONES | TXT_KEY_TECH_NATURAL_PIGMENTS_STRATEGY
TECH_HARPOON_MAKING | TXT_KEY_TECH_SPEARFISHING_STRATEGY
TECH_HUNTING_TACTICS | TXT_KEY_TECH_WARFARE_STRATEGY
TECH_HYDROELECTRICITY | TXT_KEY_TECH_ELECTRICITY_STRATEGY
TECH_IDEOGRAMS | TXT_KEY_TECH_WRITING_STRATEGY
TECH_IMPERIALISM | TXT_KEY_TECH_GRAND_WAR_STRATEGY
TECH_KNOWLEDGE_MERCANTILISM | TXT_KEY_KNOWLEDGE_MERCANTILISM_STRATEGY
TECH_LUNAR_MEGASTRUCTURES | TXT_KEY_TECH_LUNAR_TOURISM_STRATEGY
TECH_MEGACORPORATIONS | TXT_KEY_TECH_CONGLOMERATES_STRATEGY
TECH_MELECTRONICS | TXT_KEY_TECH_MDESIGN_STRATEGY
TECH_MERITOCRACY | TXT_KEY_TECH_LITERATURE_STRATEGY
TECH_MICROBIOLOGY | TXT_KEY_TECH_BIOLOGY_STRATEGY
TECH_MINORITY_RIGHTS | TXT_KEY_TECH_REPRESENTATIVE_DEMOCRACY_STRATEGY
TECH_MOLECULAR_BIOLOGY | TXT_KEY_TECH_BIOLOGICAL_WARFARE_STRATEGY
TECH_NITROGLYCERIN | TXT_KEY_TECH_EXPLOSIVES_STRATEGY
TECH_ORBITAL_MEGASTRUCTURES | TXT_KEY_TECH_LUNAR_TOURISM_STRATEGY
TECH_PATENT_RIGHTS | TXT_KEY_TECH_CORPORATION_STRATEGY
TECH_PICTOGRAPHS | TXT_KEY_TECH_WRITING_STRATEGY
TECH_PLANETARY_DEFENSES | TXT_KEY_TECH_LUNAR_TOURISM_STRATEGY
TECH_QUADRUPLEHELIX | TXT_KEY_QUADRUPLEHELIX_STRATEGY
TECH_RAFT_BUILDING | TXT_KEY_TECH_SAILING_STRATEGY
TECH_RELIGION | TXT_KEY_TECH_MYSTICISM_STRATEGY
TECH_RUDDER | TXT_KEY_TECH_SHIP_BUILDING_STRATEGY
TECH_SCAVENGING | TXT_KEY_TECH_HUNTING_STRATEGY
TECH_SEXTANT | TXT_KEY_TECH_NAVIGATION_STRATEGY
TECH_SIMULATED_REALITY | TXT_KEY_TECH__SIMULATED_REALITY_STRATEGY
TECH_SOLAR_POWER | TXT_KEY_TECH_ELECTRONICS_STRATEGY
TECH_SPECIAL_PROMOTION |
TECH_STEAMPUNK | TXT_KEY_TECH_STEAM_POWER_STRATEGY
TECH_SUPERFLUOUS_MANUFACTURING | TXT_KEY_TECH_PLANET_MANUFACTURING_STRATEGY
TECH_TELEVISION | TXT_KEY_TECH_MASS_MEDIA_STRATEGY
TECH_TENGRIISM | TXT_KEY_RELIGION_TENGRIISM_STRATEGY
TECH_WOMENS_SUFFRAGE | TXT_KEY_TECH_LABOR_UNION_STRATEGY

- TECH_COOKED_FOOD is the only one to have a <Help> entry (TXT_KEY_TECH_COOKED_FOOD_HELP)

Dancing Hoskuld · Apr 27, 2014

Rwn said:
Buildings with absent/empty/NONE <Civilopedia> tag
...

Buildings with absent/empty/NONE <Advisor> tag (duplicates not included)
BUILDING_CANNIBALISM
BUILDING_CANNIBALISM_BAD_I
BUILDING_CANNIBALISM_BAD_II
BUILDING_CANNIBALISM_BAD_III
BUILDING_CHAMPAGNE_CLUB
BUILDING_SLAVERY
BUILDING_SLAVERY_BAD_I
BUILDING_SLAVERY_BAD_ZORO_I
BUILDING_SLAVERY_BAD_ZORO_II
BUILDING_WHITE_WINE_CLUB

I have added adviser tags to those missing them but the ones ending with a Roman number are auto build so is the adviser even used?

I looked at the first two you say are missing pedia entries only to find that they do have them and they show up in the Pedia. They are missing strategy text.

edit

Rwn said:
Attached is the list I've built based on all text files in C2C (either containing Gametext or located in xml/text).
There's something odd, though. I'm sure that some missing entries are in fact not missing (for example Writing, whose Civilopedia appears correctly in-game), I just can't find them anywhere - looking for "TXT_KEY_TECH_WRITING_PEDIA" in the whole C2C folder returns only one entry (the TechInfos file), the associated text is nowhere to be found. Is it possible that the C2C is taking the vanilla's entry instead?

Yes, or BtS or Warlord.

Rwn said:
- The following have 3 <CommerceChanges><iCommerce> entries (instead of 4 for most buildings)

This is because they were made before the forth commerce was added to the XML

It is optional to have them so you only need to put in as many as you need to get the result you want. The missing ones get a value of zero. It apparently speeds up load time not to have the extras in.

Rwn · Apr 27, 2014

Dancing Hoskuld said:
I have added adviser tags to those missing them but the ones ending with a Roman number are auto build so is the adviser even used?

I have no idea, I'm just reporting what I extracted from the files

I looked at the first two you say are missing pedia entries only to find that they do have them and they show up in the Pedia. They are missing strategy text.

Sorry, I inverted <Strategy> and <Civilopedia> in my message. It should read:

Buildings with absent/empty/NONE <Strategy> tag (duplicates not included)
BUILDING_ADRIANOS_WALL
BUILDING_ARCHIMEDES_WORKSHOP
BUILDING_BURJ_DUBAI
BUILDING_CANNIBALISM_BAD_I
BUILDING_CANNIBALISM_BAD_II
BUILDING_CANNIBALISM_BAD_III
BUILDING_CORPORATION_MOONBEAN_SHOP
BUILDING_EMPCLOTH_STORE
BUILDING_FASHION_STORE
BUILDING_HUMAN_RIGHTS_DECLARATION
BUILDING_INDIAN
BUILDING_NASA
BUILDING_SLAVERY_BAD_I
BUILDING_SLAVERY_BAD_ZORO_I
BUILDING_SLAVERY_BAD_ZORO_II
BUILDING_TOMB
BUILDING_TRIUMPH_COLUMN

Buildings with absent/empty/NONE <Civilopedia> tag (duplicates not included)
BUILDING_CORPORATION_EMPCLOTH_HQ
BUILDING_CORPORATION_FASHION_HQ
BUILDING_CORPORATION_MAPSTER_HQ

This is because they were made before the forth commerce was added to the XML It is optional to have them so you only need to put in as many as you need to get the result you want. The missing ones get a value of zero. It apparently speeds up load time not to have the extras in.

Wouldn't it be useful to remove the extra <BonusCommerceModifiers><BonusCommerceModifier><Co mmerceModifiers><iCommerce> then? See table above, most buildings only have a resource :gold:

bonus (1st item), and the 4th item

espionage

is never used...

Iceman68 · Apr 28, 2014

Can I ask how you have managed to extract them? I am struggling to come up with a list of Movie Defines, which should help me.

Rwn · Apr 28, 2014

Iceman68 said:
Can I ask how you have managed to extract them? I am struggling to come up with a list of Movie Defines, which should help me.

I've come a long way since my first attempts, but here's how I do it now. Still, don't expect an easy road...

First, unless it's all in only one file, you have to get all xml files together in one file. Usually they are either located in a given C2C folder or have a similar pattern (such as *BuildingInfo*.xml for buildings); you may use your OS' built-in search function, or specific software like grepwin if you're looking for info inside files. I then open them all with notepad++ and merge them with a short python script I found somewhere on the internet, but there are surely other ways to do that.

Then you have to "clean" the file: usually each file starts and ends with a given tag (fore example <BuildingInfos>...</BuildingInfos>): you have to delete all the intermediary such </starttag><startag> to only have one at the start and one at the end. Notepad++ has a replace function which allows regexp that is useful for this (for example replace all </starttag>.*?<starttag> instances by nothing). If you don't do that, the importation will at best stop at the first file. Also, depending on the file, you may find later on when importing the xml that you have more cleaning up to do...

Then the key is Excel's built-in xml import function (I'm very familiar with Excel from work; I have no idea if opensource spreadsheet softwares can also do that). When you open an xml file, you get an option to build a table based on identified <tags>, see this video for example. I don't know how large is your file, but for large files (lots of entries and different tags) it's useful to select only the tags you're interested in, especially if your computer is short on memory (with 4GB I struggled a lot with the - admittedly quite large - building file).

Now, things would be too easy if it ended there. Excel tries to build its own list of headers based on the tags it finds, which causes several issues. First, it sometimes fails to build the proper list: for example if there are both <tag1>value1</tag1> and <tag1><subtag>subvalue</subtag></tag1> for the same <tag1>, you'll lose all the value1 entries (I got this one in the gametext due to the use of plural and gender tags). When that happens, you have to go back to notepad++ and regexp to try to fix things...
Once that's done, Excel will still keep only the last tag (giving it a number if there are multiple versions), you have to manually write the name of each parent tag if you want to differentiate final tags with the same name.
And finally, the format of the table imported is usually very messed up: each value is alone on its own line (you may quickly end up with files in the 10k+ or even 100k+ lines), empty tags are not displayed (thus you get different lengths for each entry), you have an issue of having multiple value for a given tag (for example a building may have several prerequisite technologies) which are also displayed each on its own line, but on the same column (as they correspond to the same tag)... I don't know where that comes from, some xml files are better than others, though (HandicapInfos was quite easy to import for example and looked like the one in the youtube video above; maybe you'll get lucky with your file

).

Building a nice table with what you want from this is the most time-consuming task. I use a lot of formulas (especially vlookup/hlookup, best excel formula ever!) on a different sheets to get exactly what I want. Usually it starts by finding the unique ID tag, the one that identifies each object (such as <Type> for a building), and copying it on all empty lines below it; then removing all lines with empty cells in the column whose value you want to extract. Of course, it's becoming much more complicated if you are looking for several (or all) tags, especially if that's one of those tags that has multiples value for each entry... It's hard to be more specific at this point, it really depends on what I'm looking for and what the raw imported table looks like, but it's mostly a matter of using vlookup formulas to get the data from the raw table, filtering excess lines and deleting them.

As you can see, the whole process is still quite experimental, unfortunately there's no "generic pre-built" spreadsheet where you just put in the imported xml to get the nice table... Hope this helps nonetheless!

Dancing Hoskuld · Apr 28, 2014

Iceman68 said:
Can I ask how you have managed to extract them? I am struggling to come up with a list of Movie Defines, which should help me.

All movie defines have to be in Assets/Modules/zMovieFix or they wont play.

Rwn · Apr 28, 2014

Just to have all info in that topic for future search convenience:

Link to the message with the current unit database.

Iceman68 · Apr 29, 2014

Dancing Hoskuld said:
All movie defines have to be in Assets/Modules/zMovieFix or they wont play.

True, but they also need to be in the buildinginfos files otherwise they don't get called from the zMovieFix file.

Rwn · Apr 29, 2014

Full unit database with (unprocessed) duplicates + same with buildings

Thunderbrd · Apr 29, 2014

@THE WHOLE TEAM:

If we are to really make good use of Rwn's documents we will need to update them with EVERY change we make just as faithfully as we do the xml itself. Is this possible for us all to agree to maintain if we do so on say, an Excell spreadsheet? How can we 'share' this spreadsheet so it's very easy for us to keep up on? Should it be included in the SVN (this is the only way I can see it working)? Can we ALL open and manipulate an Excel spreadsheet and if we can, at what file type version can we all do so?

Now - this could be VERY helpful for evaluating the game structures we have and balancing and aligning things. A hugely useful tool for development. But it WILL enforce us to be very disciplined and to consider it as important to keep updated as all direct modifications.

If we can do this with buildings, we should be able to do it with almost all game object classes - and imo it's something we SHOULD do.

But we'd need to have a 100% consensus and full cooperation from all of us to make it worthwhile or someone's going to end up getting very frustrated when it falls out of whack with our game files!

The other option is to keep asking Rwn to update these files at various intervals... something I don't think we can reasonably do. Otherwise, if we don't commit to the above noted effort to keep these spreadsheets in synch with the xml we'll have these files as useful for a very short term duration and then be wanting for them again all too soon.

Dancing Hoskuld · Apr 29, 2014

1) I don't have Excel but I do have LibreOffice. I am not sure how compatible they are.

2) I have a bunch of terrain and improvement spreadsheets I use that someone posted here.

3) Keeping both XML and spreadsheet updated is very unlikely to work as someone will forget and tings will get out of sync. Also the values for some things probably depend on which options and modmods you are playing with.

Having a program that will create the spreadsheet at need would be better but would loose the comments we can make on the longer term spreadsheet, unless we can come up with a means of keeping that information.

Rwn · Apr 29, 2014

A few things:
- I built those files to help you all with C2C modding: less time spent by competent C2C modders on boring xml-reading tasks is more time for them to put real thought in improving the mod

If maintaining this file requires more time than occasionnally reading xml, that's not worth it.
- From my perspective, even if those files become soon unused due to obsolescence, well, they'd have served their purpose. I certainly don't want anybody to feel that it must be maintained to make it worth the time to build them initially.
- These two files are already (a bit) obsolete, since they are based on the SVN from a few weeks ago... so there's already a step to climb to make it updated.
- Honestly, I'm not sure I'll want to do again the work to build those files anytime soon (though it was an interesting exercise to find a way to do it initially)... Still, in case anyone else would want to do it, one thing I'd point out is that one of the most time consuming part is debugging the files - duplicates, units missing a unit class, etc. It could be worth at some point to "streamline" how files are built and organized.

Faustmouse · Apr 29, 2014

I can open excel and I'm willing to update my Spreadsheet (Buildings) with new entries. Needs some time to get used to it I think.

Thunderbrd · Apr 29, 2014

Dancing Hoskuld said:
1) I don't have Excel but I do have LibreOffice. I am not sure how compatible they are.

If I uploaded an excel file can you check if it can open for u?

Dancing Hoskuld said:
2) I have a bunch of terrain and improvement spreadsheets I use that someone posted here.

And thus you may recognize the great value in keeping them clean and updated right?

Dancing Hoskuld said:
3) Keeping both XML and spreadsheet updated is very unlikely to work as someone will forget and tings will get out of sync. Also the values for some things probably depend on which options and modmods you are playing with.

I agree it will be difficult but if we can all agree to TRY it won't be nearly so frustrating and I think if it becomes a habitual part of the process (as it is for me now to update the Combat Class document) it would so often pay off handsomely for the effort! The options and modmods all have xml entries themselves and in these documents we're allowing them to show up as independent entries so those would be maintained just exactly as they are in the xml right?

Having a program that will create the spreadsheet at need would be better but would loose the comments we can make on the longer term spreadsheet, unless we can come up with a means of keeping that information.

The comments are less important but the format of the spreadsheet is so it can easily be used with excel or even googledocs so we can really make the most of it by filtering for what elements we're looking for and extracting those definitions at will to align into more specific analysis documents whenever we need to. But I do get what you're saying - Nimek was the closest to coming up with that for us - nearly had it I think. May, in some ways, HAVE gotten it down but I'm still unsure about how to USE the tool properly. Koshling's tool is great but stumbles at some issues like multiple files for example.

Rwn said:
A few things:
- I built those files to help you all with C2C modding: less time spent by competent C2C modders on boring xml-reading tasks is more time for them to put real thought in improving the mod If maintaining this file requires more time than occasionnally reading xml, that's not worth it.

The thing is, it doesn't take long to update an adjustment you made - it's just hard to remember to do so and a real problem if it's forgotten because then the adjustments are hard to find again. So time is not the issue - discipline is. BUT IF done correctly it would severely reduce the amount of time in planning and real strong analysis efforts. It also means we don't have to over-rely on wingrep and opening the game to get the information we need.

Rwn said:
- From my perspective, even if those files become soon unused due to obsolescence, well, they'd have served their purpose. I certainly don't want anybody to feel that it must be maintained to make it worth the time to build them initially.

I'LL use them and strive to keep up on them - particularly the units! It will be VERY useful for me!

Rwn said:
- These two files are already (a bit) obsolete, since they are based on the SVN from a few weeks ago... so there's already a step to climb to make it updated.

This is a little unfortunate since I just made a huge number of unit adjustments... I'll have to try to update the document at some point.

Rwn said:
- Honestly, I'm not sure I'll want to do again the work to build those files anytime soon (though it was an interesting exercise to find a way to do it initially)... Still, in case anyone else would want to do it, one thing I'd point out is that one of the most time consuming part is debugging the files - duplicates, units missing a unit class, etc. It could be worth at some point to "streamline" how files are built and organized.

I certainly don't blame you - I've tried this sort of thing before and seeing what you had to do to get around the issues I couldn't figure out how to get around I can see what an effort it is.

Sounds like the sum total of what you were up against in your 'debugging' issues was a matter of modular edits and the way those work. Such duplicates should NOT have all tags defined, including UnitClass which, just like buildings, is NOT the primary key. The primary key is actually <Type> tag but even then there's a possibility of the primary key having a 'last name' as it were in the replacement tag which would make it a truly different entry even if the type was the same. Don't think we have any unit replacement entries but if you were doing this for traits you'd see a full duplicate optional replacement set.

Faustmouse said:
I can open excel and I'm willing to update my Spreadsheet (Buildings) with new entries. Needs some time to get used to it I think.

That's the spirit!

Hydromancerx · Apr 30, 2014

@TB

I do not think I could keep up such a list. However I have not modded nor probably will be modding buildings for awhile so my statement may be moot.

Rwn · Apr 30, 2014

Thunderbrd said:
Sounds like the sum total of what you were up against in your 'debugging' issues was a matter of modular edits and the way those work. Such duplicates should NOT have all tags defined, including UnitClass which, just like buildings, is NOT the primary key. The primary key is actually <Type> tag but even then there's a possibility of the primary key having a 'last name' as it were in the replacement tag which would make it a truly different entry even if the type was the same. Don't think we have any unit replacement entries but if you were doing this for traits you'd see a full duplicate optional replacement set.

Yes, I understood the painful that Type was really the primary key, not Class

It's the fact that Class is before Type in each entry that is causing a lot trouble due to the way I extracted the xml (now i think about it it could probably be solved with regex replace in notepad++). I'm not sure I understand what you mean with the 'last name' as a primary key, tough?

Anyway, regarding file maintenance, I'm under the impression that many files in the \Module folder are in fact nor really part of a modular/optional part of the game and could be merged. For example, I think in one of the modules there are tens of heroes with each its own file; are there really instances where only some of these heroes are used?
It'd be much easier to extract/work with a limited number of files whose location is well known; ideally I'd see something like this for units for example:
- Three files in XML/units (animals, heroes, rest)
- One file in Module/Alt_timeline and other "triggerable" modules (are there?)
For Buildings we could have something like a file for ordinary buildings, another for wonders and one for special + one for modular parts, etc.
Amount of work aside (I'd be ready to help for that), are there downsides I don't see to that approach?

Fun with xml extraction

King

Attachments

King

King

King

Attachments

King

Deity

King

Chieftain

King

Deity

King

Chieftain

King

Attachments

C2C War Dog

Deity

King

Deity

C2C War Dog

C2C Modder

King

Similar threads