I did a preliminary investigation, using the BIQC functionality in the editor. The notes here are more of a scratchpad than an answer; I'm saving them to build on later.
While it showed that there are some differences in areas such as BLDG and GOVT compared to a "vanilla" BIQ, before the unit changes, it does look like the TERR area is the area of concern. The diff results for that section begin:
Code:
--> Terr <--Standard.biq | Dummy Mod2.biq
name: Desert
DataLength: 233 | 237
NumPossibleResources: 26 | 16
FoodBonus: 1 | 2
AllowAirfields: 1 | 0
AllowOutposts: 1 | 0
AllowRadarTowers: 1 | 0
LandmarkEnabled: 1 | 0
LandmarkFoodBonus: 1 | 2
The key difference that I suspect is resulting in the crazy numbers in Firaxis's editor is the DataLength difference. My editor does its best to ignore that field (other than updating it as necessary, for sections where it is modified); Firaxis's editor follows it religiously. Consequently, any case where it becomes an improper length can result in data corruption, and the sort of strange issue you have run into.
As evidence that this is the problem, you can note that the Firaxis editor (I have version 1.03, which is somewhat newer than Civinator has) displays the Desert terrain at least mostly correct (aside from the Landmark Civilopedia entry), but as soon as you switch to Plains, the second terrain, you have values that are nonsensical. This is consistent with a data length problem.
The question then becomes, how did the data length become 237 instead of 233? I'm going to have to take a look at that when it isn't super late. Looking through the code, it
looks like the only places my editor updates that are when it reads a BIQ file. Although, the NumPossibleResources field should at some point have an impact on DataLength, which makes me wonder if I am missing something. Still, the overall number of resources is the same between Standard.biq and Dummy Mod2.biq, and the number possible for desert is
lower in Dummy Mod2.biq, so nothing jumps out as explaining why DataLength would be higher.
Comparing the non-working BIQ and the one Civinator fixed in a hex editor, I see
both have a terrain data length of 237, but Civinator's has more data on the "possible resources (binary)" area - the non-working one has two bytes for 16 resources, and Civinator's has 8 bytes for 59 resources. The bytes per resources amount passes a non-rigorous validation in my mind, but those additional six bytes
should affect the data length. The non-working one has a manually measured length of 0xEB per resource, or 235, whereas Civinator's has a manually measured length of 0xF1, or 241. Neither of those is 237, but data length doesn't necessarily include the length necessary for the data length variable, and Civinator's is a version 12.06 BIQ, instead of 12.08, which may also be a factor. Still, the fact that Civinator's fix produced a BIQ where the data length variable remained the same, but the actual data length increased by six, points to an inconsistency.
Meanwhile, the Standard.biq file says it has a DataLength of 233, and actually has one of 237. This is consistent with the Civ practice of the real data length not including the 4 bytes that report what the data length is.
Conclusions:
- An investigation into how to reproduce this should likely focus on adding, and perhaps removing, resources. I suspect that this arose due to a changing number of resources at some point, and am mildly suspicious that I don't see code in my editor to increase the dataLength as the number of resources increases (although something increased that number).
- Even among reported good BIQs, we see some inconsistencies; why does Standard.biq has a length 4 less than reality, but Civinator's fix match reality? Data length has always been one of the more confusing parts of the BIQ, but this still manages to be a top 5 case for "confusing data length problems".
I will likely revisit this when less tired, with a focus on trying to establish how to reproduce this problem, and going from there to how to prevent it in the future.