civchecker - avoid crashes in your mod

Well, I ran afterworld on my machine, and it found one missing art file which is really missing:

MissingArt: Art/Units/Cannon Ancient Chinese /Cannon_Ancient_Chinese.nif
Used: after art/civ4artdefines_unit.xml at line 4316

There is an extra space before the slash. I did not bring up the mod and try to show that I get a red circle or purple box or whatever from the missing art; but it seems to work correctly on my setup. I am using win-xp, sp3. Another poster was running on vista, had some trouble understanding how to use the program, but apparently succeeded on vista with no change in the program.

There is a curious thing about the particular message you pasted, which is about ExtractionFacility. That is not inside the Afterworld mod at all; you can find the file in [BTS]/Assets, not inside [BTS]/Mods/Afterworld/Assets. So the file search is failing to find files in your installation. Perhaps in addition, it is failing to find files inside your mod; but it is very curious that it is not finding files in the basic installation.

When you activated the "FileRead" print statements, did it show it was reading any files from the BTS directory? Maybe it is only reading from vanilla, warlords, and your mod. That would explain the Afterworld message but maybe not your silk message.
 
davidlallen said:
When you are modding XML files, the game should check for all kinds of missing definitions when it loads your XML. It checks for some, but not all. If you have used a symbol but the definition is missing, the game may randomly crash, either at startup or during play. This checker tries to catch all kinds of missing definitions so your mod doesn't crash.

See the user documentation for full details.

I have submitted it to the forum download page, but it is in the "moderation queue".

You can download the latest version of the program from this link.
Click to expand...

Sounds useful - downloaded.:goodjob:
 
Thanks davidlallen. I tried running your program and it found 30 or so items that need changed in my mod.
 
How did you make it look into packed art for the vanilla check? I use packed art in my mod and it makes civchecker useless for finding missing art because it reportes over 800 files as missing that are in the files. This makes it impossible to tell if I can ignore a missing art entry or if I have to deal with it, so I have no choice but to ignore them all and hope for the best.
 
deanej said:
How did you make it look into packed art for the vanilla check? I use packed art in my mod and it makes civchecker useless for finding missing art because it reportes over 800 files as missing that are in the files. This makes it impossible to tell if I can ignore a missing art entry or if I have to deal with it, so I have no choice but to ignore them all and hope for the best.
Click to expand...

Maybe run CivChecker on an unpacked version of your mod to find errors.
 
Yes, exactly. You can have any number of directories in your search path. I usually keep an unpacked copy of my art around somewhere, so I just add that directory as another path.
 
Here is the setup I use for Dune Wars:

add_directory dune "$install_dir/beyond the sword/mods/dune wars/assets"
add_directory unpak "c:/documents and settings/dave/my documents/civ-local/dune-17-pak"

It may be that if you use a single backslash, or some other spelling error, the command is silently ignored. The subdirectory "art" should be directly under the path you specify.
 
I'm a total convert to civchecker now. It saves so much time. It would help for new users if you document the fact it must use forward slashes in the path; alternatively you could code it to accept back slashes. I think 90% of the people in here thought it didn't work because of the whole forward slash thing.

Also I have two requests that would really increase the functionality of this:

First, could you add a check for the LSystem files? This is one of the most difficult to read and mod files, yet there is absolutely no in game help for it, even a debug dll will just crash, and not tell you that the art is missing; which is really annoying.

Secondly, could you add a checking mechanism to find unused art? Because of the way civ4 modding works, it often happens where mods that have ethnic packs end up with alot of unused unit art that just takes up space; it would be nice to get pointed to these (I'd just have it point to folders in the Assets/Art directory that are not referenced by anything, instead of individual files, which would get messy) so that we could know what to cut.

:deal:

Also good job on this, again it saves alot of time.
 
Thank you for the feedback. The program is written in another language, "tcl". It uses front slashes, or double backslashes. Even in python, if you want to put a filename into a string, you must use double backslashes; single backslashes will not work. The configuration file you have to edit says this right at the top, and all the examples right in the file say this. It is possible that 90% of the people miss all of these clues; but I doubt it.

I can add a better error message when a directory is not found; but I don't see any reason to special-case single backslashes.

I have a local version with some improvements. It does check for undefined art, for exactly the reason you suggest. In Dune Wars, and other mods I have looked at, there are many MB of unused art. But, usually when this happens, the art is actually referenced by xml/art/civ4artdefines_something. The way to detect it is one level further back; there is no reference to the object ART_DEF_UNIT_AXEMAN or whatever.

So the feature I added will print all unused objects with a certain prefix; you can find unused art by printing unused objects like ART_DEF_anything. Then you should edit the related art defines file, remove that block, and remove the related art directory.

I've been using it for a while but I never updated the documentation, so I never released it. I should put some time into that, thanks for the push.

Regarding the L-system file, the syntax of this file is a little scary to me. So I never tried to mod it, and also I never tried to have civchecker do anything to it. The problem is that it overloads a lot of information into strings that need to be subdivided, such as
Code: 
<ArtRef Name="generic:caveset.nif::1x1_01">
<Attribute Class="Era">ERA_MEDIEVAL,ERA_RENAISSANCE,ERA_INDUSTRIAL,ERA_MODERN,ERA_FUTURE</Attribute>
<Attribute Class="Scalar">szNIF:Art/Structures/Buildings/Harbor/HarborModern.nif</Attribute>
Are there specific things you find often go wrong in this file? Adding checks for one or two things may not be too much work, but I've never been able to understand the full syntax.
 
The main problem that occurs is that the art doesn't exist. Unlike unit art, or building art defined in ArtDefinesBuilding, missing art in the LSystem causes a crash to desktop. LoR dealt with all of this long ago, but I know that this is a major problem that effects most of the big mods here, as they all seem to use the ethnic city modcomp by Geomodder. Geomodder isn't a God, so considering he's added 1000s of lines of code, it's no surprise that there are typos in some of the file paths he has and what not; but it would be nice to get civchecker to recognize this, even if for nothing else then to get rid of the critical bugs in ethnic city styles.

Also having some sort of art check on the LSystem files is essential for figuring out the unused art, otherwise all the LSystem art would be returned as unused.


Also the user most definitely needs a prompt telling them when stuff isn't found because they used backslashes instead of forward slashes. Virtually everyone I've recommended use this neat little utility made this mistake, and couldn't figure out what went wrong. Actually everyone thinks their mod is fine, because when this happen civchecker just reports no errors. Applications should prompt the user when things go wrong.
 
Please help me to understand more about the lsystem file. I see file references like:
Code: 
<ArtRef Name="generic:ren_eastern.nif::2x1_03">
<Attribute Class="Scalar">szNIF:Art/Structures/Buildings/Harbor/HarborModern.nif</Attribute>
<Attribute Class="Scalar">NIF:Art/Structures/Improvements/Farm/crops_w.nif::2x1</Attribute>
Are there any other places where filenames can be given? For files like "ren_eastern.nif" how should I know what directory to look in?

Also, there are other similar-looking things such as:
Code: 
<Attribute Class="Scalar">szSpecialLayout:Coast</Attribute>
Which are not files. In order for this check to be useful, I have to know how to tell which things are expected to be files, and which aren't. Is there any documentation of this format?

The check for unused art is still useful without including this file, since all the used improvement art is still listed by improvementinfos -> artdefines_improvements. If something is not defined in improvementinfos, but it is defined in one of the lsystem files, I can't see how that would matter; the art can never get into the game, so the lsystem file can't cause any problem.
 
I don't know the answer to your first few questions; I can guess, and I have tweaked the LSystem files some (mainly to fix bugs in it), but I don't want to answer since I can't be certain about things. For this reason I've posted in the Ethnic City Styles thread, and asked Geomodder to come over here and help out. You may want to check out that thread and ask him directly, I'm not sure he understands how useful civchecker would be to him if you teach it to search and validate the LSystem files.
davidlallen said:
The check for unused art is still useful without including this file, since all the used improvement art is still listed by improvementinfos -> artdefines_improvements. If something is not defined in improvementinfos, but it is defined in one of the lsystem files, I can't see how that would matter; the art can never get into the game, so the lsystem file can't cause any problem.
Click to expand...
No, this is not how the LSystem works, the LSystem overrirdes the ArtDefines buildings and improvements Infos files. The art in the LSystem files is not defined anywhere else.
 
davidlallen said:
Please help me to understand more about the lsystem file. I see file references like:
Code: 
<ArtRef Name="generic:ren_eastern.nif::2x1_03">
<Attribute Class="Scalar">szNIF:Art/Structures/Buildings/Harbor/HarborModern.nif</Attribute>
<Attribute Class="Scalar">NIF:Art/Structures/Improvements/Farm/crops_w.nif::2x1</Attribute>
Are there any other places where filenames can be given? For files like "ren_eastern.nif" how should I know what directory to look in?
Click to expand...

Hi David.
In Civ4 LSystem, stuff with a "generic" reference can only be called from the art/structures/cities folder. At least, as long as in CIV4ArtDefines_Misc the generic building directory this path is given. In vanilla civ, warlords, and BtS, it is used for cottages in PlotLSystem and cityset art in CityLSystem. But it is possible to add generic art to other improvements (see farms in the Planetfall mod).
Normal building -and improvement art can be called up from whatever folder. No hardcoded pathway for those, except perhaps the watermill as far as I know.

davidlallen said:
Also, there are other similar-looking things such as:
Code: 
<Attribute Class="Scalar">szSpecialLayout:Coast</Attribute>
Which are not files. In order for this check to be useful, I have to know how to tell which things are expected to be files, and which aren't. Is there any documentation of this format?

The check for unused art is still useful without including this file, since all the used improvement art is still listed by improvementinfos -> artdefines_improvements. If something is not defined in improvementinfos, but it is defined in one of the lsystem files, I can't see how that would matter; the art can never get into the game, so the lsystem file can't cause any problem.
Click to expand...

No need to bother with the szSpecialLayout stuff. As far as I know it's a sort of pre-coded routine that determines how to lay stuf on the map like the aqueduct, city walls, lighthouse (water) and dike/watermill (river).
Hope this answers your questions.
 
I have added some checking for the two L-system files.

There are a number of symbols like IMPROVEMENT_FARM in these files, so I added checking for these symbols. However, in the vanilla files, I see that BONUS_TOBACCO and BONUS_OLIVES are used. These are never defined anywhere. So, is it worthwhile to print these as errors? I guess the system works fine without these being defined anywhere.

I am checking for two types of file references:

<ArtRef Name="generic:med_egypt.nif::2x2x2_01">
<Attribute Class="Scalar">NIF:Art/Structures/Improvements/Farm/crops_w.nif::2x1</Attribute>

Are there any other file references you can think of?

For the first type, I add art/structures/cities to the filename before checking. For the second type, I ran it on Dune Wars, and it found dozens of missing files. There is no problem in running Dune Wars despite the missing files. So this ""error"" must be OK:
Code: 
		<ArtRef Name="goal:IMPROVEMENT_PLANTATION">
			<Attribute Class="Improvement">IMPROVEMENT_PLANTATION</Attribute>
			<Attribute Class="Bonus">[...]
			<Attribute Class="Scalar">NIF:Art/Structures/Improvements/soilenr/massdriver.nif</Attribute>
			<Attribute Class="Scalar"></Attribute>
			<Scale>0.9</Scale>
		</ArtRef>
The soilenr directory doesn't exist. But on the other hand, in Dune Wars, IMPROVEMENT_PLANTATION is not defined either. So since the improvement doesn't exist, the missing file doesn't matter. Removing this type of "false violation" is an extra complication. If I don't remove them, then users may complain that there are all these errors reported which don't really matter. What do you think?
 
Thank you for the sugggestion. My point was that testing to find out if the IMPROVEMENT_whatever is defined or not, is not convenient. It is obviously possible, but at the point when this file is read in civchecker, the improvements file may or may not have been read. I can put a few lines into the documentation, which says these may or may not be real errors.

But the "right" solution is to reorganize the program so that the Lsystem files are guaranteed to be read after the terrain/*.xml files, and then enhance the Lsystem parser so that it keeps the name of the related improvement and checks to see if it's used. This is a little complex, and it isn't something I will be able to do right away.
 
phungus420 said:
I'm a total convert to civchecker now. It saves so much time. It would help for new users if you document the fact it must use forward slashes in the path; alternatively you could code it to accept back slashes. I think 90% of the people in here thought it didn't work because of the whole forward slash thing.

Also I have two requests that would really increase the functionality of this:

First, could you add a check for the LSystem files? This is one of the most difficult to read and mod files, yet there is absolutely no in game help for it, even a debug dll will just crash, and not tell you that the art is missing; which is really annoying.

Secondly, could you add a checking mechanism to find unused art? Because of the way civ4 modding works, it often happens where mods that have ethnic packs end up with alot of unused unit art that just takes up space; it would be nice to get pointed to these (I'd just have it point to folders in the Assets/Art directory that are not referenced by anything, instead of individual files, which would get messy) so that we could know what to cut.

:deal:

Also good job on this, again it saves alot of time.
Click to expand...

So I did as Phungus mentions, tried it, thought it didn't work then figured out the backslash thing. It would have helped me to have that in the readme as I was about to give up on this.
 
You must log in or register to reply here.

Similar threads

FlammieJerynZ
[BNW] How to diagnose Random Crashes because .lua format in a Custom Mod or Modpack?
Replies
2
Views
249
ZheBerg
Z
C
Civ Crashes Related to Mod
Replies
0
Views
336
CivMachine
C
WildWeazel
[REFERENCE] "Smoke testing" a mod: how to detect most crashes before you start playing
Replies
1
Views
797
Takhisis
Takhisis
Knightfall
[C3C] No Tech Trading Scenario
Replies
5
Views
888
Knightfall
Knightfall
The_J
  • Sticky
Old World update #121
Replies
3
Views
494
Solver
Solver
Top Bottom