Stop specific files/directories being indexed.

[billw2015]

Is there some mechanism to do this? Currently every single file in the mod folder is indexed before the mod dll is even loaded, and if there is loads of files it takes ages. I want to be able to have files in there that are NOT indexed (i.e. source files etc.) Any ideas? Hiding the files/directories via "attrib +H" works, but I can't submit hidden attribute into source control so can't distribute this solution to other devs.

 

[Nightinggale]

How about making your own folder next to assets and place your files in there? Next will be a path issue and while it could take a while to go through all files, if you can get it to work with one file, you should be able to get it to work with all files. Something like adding a ..\ prefix to the paths. CvXMLLoadUtilitySet.cpp contains the paths to the xml files, in case you want to move those too. The path to text xml files is also somewhere in the DLL, though I can't remember where offhand.

I did some experimentation at some point and it seems to spend way longer on dds and nif files than say xml files. Moving those two file types would likely make a huge difference and then you can skip the rest.

I have no idea why it has to spend ages indexing because apparently if you pack the files, then indexing is instant or not done. Packing the files has memory sideeffects though and there are examples of computers crashing from reading packed files while they managed unpacked files just fine.

 

[billw2015]

How about making your own folder next to assets and place your files in there?

The problem is that every file in the entire mod directory is indexed. This would include any folder next to assets, and currently includes all our source files etc. It is a pain! I need a way to stop it indexing a directory.

I have no idea why it has to spend ages indexing because apparently if you pack the files, then indexing is instant or not done.

We have 30k files to index when they aren't packed. Even on an SSD that is still going to be slow. Packing files makes indexing fast because indexing is literally just looking at which physical files exist on disk from what I can tell. Once all physical files are known it will open FPKs and any files not found physically on disk are loaded from the FPKs themselves.

 

[Nightinggale]

Even on an SSD that is still going to be slow.

What goes on with indexing is a mystery. Disk I/O is nearly idle, the CPU is idle and memory usage isn't super active either. From what it looks like, it seems the scan code calls something, which isn't responding and then it's waiting for a timeout before moving on to the next file.

 

[billw2015]

CPU is idle

Not for me, it is 100% on one core while I can see the file handles being iterated over in process explorer.
I tried reorganizing my mod directory so the actual mod was one folder deeper, but of course the civ menu mod loading can't understand that, even though it will work when specified in the ini file or command line.