checkArtFiles.pl - checks for missing art files and missing calls to ART_DEF

primordial stew

Emperor
Joined
Sep 1, 2005
Messages
1,219
Location
a puddle
Unfortunately any missing art file will CTD civ*, so here is a script do some basic checking:

http://forums.civfanatics.com/downloads.php?do=file&id=9361

This program creates a hash of all the dds/kfm/nif files (in pre-digested files), and then goes through the files in the assets/xml/art folder and reports any missing files.

It also scans various files for ART_DEF* statements, and then reports if there is no match found in the files from assets/xml/art/*xml.

Here is how I run it:
1. using the unpack utility Firaxis has provided, unpack all the (warlords for example) .fpk files into some dir.
2. in a cgywin shell, do what this psuedo-code .csh script would:
Code:
foreach d ( "dirs containing art files")
   foreach t ( dds nif kfm )
      cd $d
      find . -type f -name "*$t" > art_${t}.txt
   end
end

"dirs containing art files" are root dirs of where those .fpks were unpacked to, and, and your mod's art dir. The script is setup for RFRE.

eg) in ....\mods\RFRE\Assets\Art\art_kfm.txt
./Art/Terrain/resources/amber/gems.kfm
./Art/Terrain/resources/Barley/tobacco.kfm
./Art/Terrain/resources/Barley/wheat.kfm
etc...

Never mind the forward vs backward slash directory separator, it doesn't matter.

3. in cgywin, run: checkArtFiles.pl

4. examine: $mod/assets/art_debug.log. Mine is 900k, but mostly this is debug info. Check for lines with "-E-" in them (ie near the end of the file).

eg)
-I- opened source file: xml/Civilizations/CIV4LeaderHeadInfos.xml
-E- xml/Civilizations/CIV4LeaderHeadInfos.xml: art_def_leader_alexander NOT found
-E- xml/Civilizations/CIV4LeaderHeadInfos.xml: art_def_leader_genghis_khan NOT found
-I- opened source file: xml/Civilizations/CIV4TraitInfos.xml

So the game will CTD if the reference to either art_def_leader_alexander or art_def_leader_genghis_khan get called (eg meet these leaders during diplomacy).


I'm pretty sure this doesn't check for all files, but it's not hard to expand it. Let me know if there are changes to make.
 
since the uploaded files were wiped out, here is the code:

Code:
#!perl
#
# purpose:  verify art files
#
# method: foreach art xml file, check for missing files.  reading art_*.txt, which is created by cd'ing to the dir, then:
# find . -type f -name "*.dds" > art_dds.txt  (kfm + nif too)
# MUST RUN FROM cgywin!!
#
# $Header$
#########################################################

$| = 1;

use File::Copy;

## either give a warlords or vanilla xml path.  modDir files will be converted to this schema
# $civ4XmlDir = "d:/civ4/Assets/XML";
$civ4XmlDir = "d:/civ4/Warlords/Assets/XML";
# $civ4XmlDir = "f:/program files/firaxis games/sid meier's civilization 4/Warlords/Assets/XML";

# pointer to your mod to be converted
# $modDir = "f:/program files/firaxis games/sid meier's civilization 4/warlords/mods/rfre.r1/Assets/XML.orig";
$warDir =  "d:/civ4/Warlords";
$civDir =  "d:/civ4";
$modDir =  "$warDir/Mods/RFRE/Assets";
# $modDir =  "$warDir/Mods/RFRE.r1.ror/Assets";
# $convertedDir = "d:/civ4/warlords/mods/rfre.r1/Assets/XML.converted";

### dirs where file contain either ART_DEF references, or the actual definitions (ie xml/art)
@xmlDirsDef = ("xml/art");
@xmlDirsRef = ("xml/Civilizations","xml/Units","xml/Technologies");

### don't know how to read .fpk files, so user must unpack and index in some dir
@artSrcs = ("$warDir/Assets/Art","$civDir/Assets/Art","$modDir/Art","c:/tmp/unpack/art");

# debug support
$ENV{DEBUG} = 1;
chdir($modDir);
open(LOG,">$modDir/art_debug.log") || warn "$! trying to create debug.log\n";
##############################################

# create hash of graphics files
foreach $d ( @artSrcs ){ 
   opendir(DIR,"$d") || warn "$! trying to read dir $d\n";
   foreach $each ( readdir(DIR) ){
	print LOG "reading $d/$each\n" if $ENV{DEBUG};

	open(SCHEMA,"$d/$each") || die "$! trying to read $d/$each\n";
        while(<SCHEMA>){
	        if ( $each =~/art_\w+\.txt/i ){
		   chomp;
		   $_ =~s/^\.\///;
		   $_ =~s/(\.\w\w\w).*/$1/;  # finish the chomp..
	           $_ =~tr/A-Z/a-z/;
		   $gameArtHash{"$_"} = 1;
		   print LOG "$d: $_\n";
		}
	}
	close(SCHEMA);
   }
   closedir(DIR);
}

foreach $d ( @xmlDirsDef ){ 
   opendir(DIR,"$modDir/$d") || warn "$! trying to read dir $modDir/$d\n";
   foreach $each ( readdir(DIR) ){
	next unless ( $each =~/\.xml$/i );
	print LOG "found $modDir/$d/$each\n" if $ENV{DEBUG};
	push(@modXmlDef,"$d/$each");
   }
   closedir(DIR);
}


foreach $f ( @modXmlDef ){
# 	next unless ( $f =~/CIV4UnitInfos/ );

   if ( $f =~/LSystem/ ){ 
	   next;
   }
   open(FILE,"$f") || warn "$! trying to read dir $f\n";
   print LOG "-I- opened source file: $f\n";

   while(<FILE>){
	 if ( />(.*)</ ){
	     $data = $1;
	     $data =~s/::.*//;
	     $data =~tr/A-Z/a-z/;
	     if ( $data =~/,/ ){   # this is a single file containing many icons, there is no way to check if the element exists, so just check for the file.
		 ($element,$array) = split(/,/,$data);
		 $data = $array;
	     }

	     if ( $data =~/dds/ || $data =~/nif/ || $data =~/kfm/ ){
		     if ( $gameArtHash{"$data"} ){
			     # print LOG "-I- $f: $data found\n";
		     } else {
			 print LOG "-E- $f: $data NOT found\n";
		     }
	     }

	     if ( $data =~/(art_def\w+)/ ){
		     $artDefHash{"$1"} = 1;
	     }
         }
   }	
   close(FILE);
}

##### ok, that was checking for actual graphics files, now check for references:
foreach $d ( @xmlDirsRef ){ 
   opendir(DIR,"$modDir/$d") || warn "$! trying to read dir $modDir/$d\n";
   foreach $each ( readdir(DIR) ){
	next unless ( $each =~/\.xml$/i );
	print LOG "found $modDir/$d/$each\n" if $ENV{DEBUG};
	push(@modXmlRef,"$d/$each");
   }
   closedir(DIR);
}


foreach $f ( @modXmlRef ){
# 	next unless ( $f =~/CIV4UnitInfos/ );

   if ( $f =~/LSystem/ ){ 
	   next;
   }
   open(FILE,"$f") || warn "$! trying to read dir $f\n";
   print LOG "-I- opened source file: $f\n";

   while(<FILE>){
	 if ( />(ART_DEF\w+)</i ){
	     $data = $1;
	     $data =~tr/A-Z/a-z/;

	     if ( $artDefHash{"$data"} ){
		     # print LOG "-I- $f: $data found\n" if $ENV{DEBUG};
	     } else {
		 print LOG "-E- $f: $data NOT found\n";
	     }
         }
   }	
   close(FILE);
}
 
I have written a checker program which does all of the work in this script automatically. It comes with a "cache" of the 7000-odd files in the BTS assetN.pak files. It also checks for symbols which are used without being defined. Take a look at it in this thread. It's in the download forum under "civchecker".
 
Back
Top Bottom