Frustrated modder question

[Gavagay]

Hi, everyone. A few days ago I decided to try myself in modding. I took another mode for a base, a K-mod, and tried to make some XML changes, which were mostly related to civics. At first everything went on smoothly: I made desired changes and ran quite a few simulations to see, how AI would use rebalanced civics. After a while I felt myself satisfied and even started a game with my new mode. Than I decided to make some more rebalancing and changed CIV4CivicInfos.xml yet again. After that I tried to run a simulation and had a CTD in industrial era. I unmade my recent changes and tried to run a simulation again. To my astonishment, I got a CTD. Only after I restored original CIV4CivicInfos.xml CTDs stopped. Than I decided to make changes to it step-by-step, running simulations after every one of them - a VERY time-consuming process, btw. From that I discovered, that now almost everything I do in CIV4CivicInfos.xml leads to CTD.
At this point I felt myself completely at a loss. How is it even possible for a variable in an .xml file to lead to CTD? And even if it is possible - how didn't I ran into it in my earlier experiments? It is plausable to assume, that I'm getting CTDs because of some other reason, but they do stop after I restore original CIV4CivicInfos.xml.
I would really appreciate, if someone can help me to solve this bizzare problem.

[deanej]

It's really hard to tell you what's wrong if we don't know what you're changing. It's probably some type of typo; not every XML field actually undergoes error checking before being read in.

[Gavagay]

I thought about "typo" theory, but it is highly unprobable: I several times started from the original file. Basicly, here are the minimum alterations in CIV4CivicInfos.xml after which CTDs start to accure:

In representation:
changed <SpecialistExtraCommerces> value to zero, changed <CommerceModifiers> to 25 in the first line (gold)

In police state:
changed upkeep to medium

In Universal Suffrage:
changed <YieldModifiers> to 10 in the first line (food), changed <iImprovementUpgradeRateModifier> to 100, changed <iCivicPercentAnger> to 400, changed <iWarWearinessModifier> to 100, removed all previous bonuses (ihurry and +1 hammer from town), changed upkeep to high.

It looks like that changes to representation and police state don't cause CTDs, but every change to Universal Suffrage taken separately (except possibly food bonus) does cause it. And I also do know that CTDs happen even if I leave Universal Suffrage untouched and instead change one of economic civics.
What else can I add? I made changes to other XML files: added one unit (bombard) and altered effect of pyramids. But it looks like these changes are OK as bu themselves they don't cause CTD. CTD always happens closer to endgame (which may be connected with the fact that I changed only late game civics). If I test different versions of CIV4CivicInfos.xml on one save, CTDs happen at different time, but they happen invariably within 25 turns from the last autosave.

[deanej]

Could you post one of the versions of Universal Suffrage that causes a crash? Or maybe all of them?

[Gavagay]

This is my basic version of Universal Suffrage which caused crash. I tried to reset different parts of it to default values, but crashes seemed to stop only after I reset to zeros everything but line about food. Though now I suspect that may be I just didn't wait long enough in that test.

Spoiler:

<CivicInfo>
<CivicOptionType>CIVICOPTION_GOVERNMENT</CivicOptionType>
<Type>CIVIC_UNIVERSAL_SUFFRAGE</Type>
<Description>TXT_KEY_CIVIC_UNIVERSAL_SUFFRAGE</Description>
<Civilopedia>TXT_KEY_CIVIC_UNIVERSAL_SUFFRAGE_PEDI A</Civilopedia>
<Strategy>TXT_KEY_CIVIC_UNIVERSAL_SUFFRAGE_STRATEG Y</Strategy>
<Button>,Art/Interface/Buttons/Civics/Universal_Sufferage.dds,Art/Interface/Buttons/Civics_Civilizations_Religions_Atlas.dds,8,3</Button>
<TechPrereq>TECH_DEMOCRACY</TechPrereq>
<iAnarchyLength>1</iAnarchyLength>
<Upkeep>UPKEEP_HIGH</Upkeep>
<iAIWeight>0</iAIWeight>
<iGreatPeopleRateModifier>0</iGreatPeopleRateModifier>
<iGreatGeneralRateModifier>0</iGreatGeneralRateModifier>
<iDomesticGreatGeneralRateModifier>0</iDomesticGreatGeneralRateModifier>
<iStateReligionGreatPeopleRateModifier>0</iStateReligionGreatPeopleRateModifier>
<iDistanceMaintenanceModifier>0</iDistanceMaintenanceModifier>
<iNumCitiesMaintenanceModifier>0</iNumCitiesMaintenanceModifier>
<iCorporationMaintenanceModifier>0</iCorporationMaintenanceModifier>
<iExtraHealth>0</iExtraHealth>
<iExtraHappiness>0</iExtraHappiness>
<iFreeExperience>0</iFreeExperience>
<iWorkerSpeedModifier>0</iWorkerSpeedModifier>
<iImprovementUpgradeRateModifier>100</iImprovementUpgradeRateModifier>
<iMilitaryProductionModifier>0</iMilitaryProductionModifier>
<iBaseFreeUnits>0</iBaseFreeUnits>
<iBaseFreeMilitaryUnits>0</iBaseFreeMilitaryUnits>
<iFreeUnitsPopulationPercent>0</iFreeUnitsPopulationPercent>
<iFreeMilitaryUnitsPopulationPercent>0</iFreeMilitaryUnitsPopulationPercent>
<iGoldPerUnit>0</iGoldPerUnit>
<iGoldPerMilitaryUnit>0</iGoldPerMilitaryUnit>
<iHappyPerMilitaryUnit>0</iHappyPerMilitaryUnit>
<bMilitaryFoodProduction>0</bMilitaryFoodProduction>
<iMaxConscript>0</iMaxConscript>
<iUnhealthyPopulationModifier>0</iUnhealthyPopulationModifier>
<iExpInBorderModifier>0</iExpInBorderModifier>
<bBuildingOnlyHealthy>0</bBuildingOnlyHealthy>
<iLargestCityHappiness>0</iLargestCityHappiness>
<iWarWearinessModifier>100</iWarWearinessModifier>
<iFreeSpecialist>0</iFreeSpecialist>
<iTradeRoutes>0</iTradeRoutes>
<bNoForeignTrade>0</bNoForeignTrade>
<bNoCorporations>0</bNoCorporations>
<bNoForeignCorporations>0</bNoForeignCorporations>
<iCivicPercentAnger>400</iCivicPercentAnger>
<bStateReligion>0</bStateReligion>
<bNoNonStateReligionSpread>0</bNoNonStateReligionSpread>
<iStateReligionHappiness>0</iStateReligionHappiness>
<iNonStateReligionHappiness>0</iNonStateReligionHappiness>
<iStateReligionUnitProductionModifier>0</iStateReligionUnitProductionModifier>
<iStateReligionBuildingProductionModifier>0</iStateReligionBuildingProductionModifier>
<iStateReligionFreeExperience>0</iStateReligionFreeExperience>
<YieldModifiers>
<iYield>10</iYield>
<iYield>0</iYield>
<iYield>0</iYield>
</YieldModifiers>
<CapitalYieldModifiers/>
<TradeYieldModifiers/>
<CommerceModifiers/>
<CapitalCommerceModifiers/>
<SpecialistExtraCommerces/>
<Hurrys/>
<SpecialBuildingNotRequireds/>
<SpecialistValids/>
<BuildingHappinessChanges/>
<BuildingHealthChanges/>
<FeatureHappinessChanges/>
<ImprovementYieldChanges/>
<WeLoveTheKing>TXT_KEY_CIVIC_WE_LOVE_PRESIDENT</WeLoveTheKing>
</CivicInfo>

Actually I now remembered that I also changed Civ4LeaderHeadsInfo in regards to favourite civics of leaders. Completely forgot about that, sorry. Now will try to to run a sim with this file reset to default.
Update. Hmmm, now, after I copied that, I see strange gaps in words "strategy" and "pedia". I use notepad to edit files, and these spaces aren't seen there. Nevertheless, I never touched these lines, so this gaps should have been already present in original mod. And it works perfectly.

[deanej]

Nothing looks bad... looks like you need to get a debug version of the DLL to figure out what's going on. The standard way to do that is to compile the SDK with the debug option (tutorial on the modiki); I don't know if anyone has posted one or not.

[Gavagay]

Thank you for the answer. I'll try to do it. I will also try to ask this question to the author of the original mod. May be this signifies a bug in his mod which he would want to fix.

[deanej]

Entirely possible. If that mod has its own DLL you'll need to use its source.

[isenchine]

Those gaps in words are not normal. Since you said that you used Notepad, you might want to read this recent post from God-Emperor, copied below.

Originally Posted by God-Emperor

Those symbols are the UTF BOM (Byte Order Mark). That is the wrong format: it should be saved in a format without the BOM (which does exactly nothing in the UTF-8 format anyway). Notepad is notorious for inserting the useless BOM into the beginning of UTF-8 files. While it is missing some useful features of other programs as well, this is possibly the main problem with trying to use Notepad to mod Civ. Apparently, if Notepad thinks the file is UTF-8 instead of 7-bit ASCII or 8-bit ISO 8859-1/"Latin-1" (or Windows-1252, which is slightly different in ways that are frequently irrelevant for text) then it will insert the BOM.

In fact, as far as I know all of the Civ4 XML files, at least those in the version sold in North America (and probably South America too) and western Europe, are actually 7-bit ASCII format, regardless of what the encoding property in the xml tag says they are. All 8 bit characters are encoded using the HTML numeric encoding method to make them 7-bit ASCII (like, for example, "ö" is put into the file as "& #246 ;" (without the 2 spaces)) and the only Unicode symbols out of the 8 bit range are encoded using a "[THING_FOO]" type encoding (like the production hammer icon is "[ICON_PRODUCTION]").

The usual advice is to use Notepad++ instead. Not sure if it would solve your problem of course but anyway it is a better (free) tool to edit the XML.

Edit: personally, I have edited this file in similar ways and I never had a crash with any change in the values.

[The_J]

Originally Posted by isenchine

Those gaps in words are not normal.

That's a problem with the forum settings, not with his XML file. The software breaks up too long words to prevent that somebody tries shenanigans with the forums software.

[isenchine]

Ah, OK!

[Gavagay]

Thanks, everyone, it looks like, I figured out what caused crashes. It was a mistake in a completely different XML file. For some reason it didn't manifest itself in my first sims, that is why I blamed civics for it.