Caveman 2 Cosmos (ideas/discussions thread)

[raxo2222]

I'm afraid it's because of network error during that commit.
I will try to fix it after I get all tree conflicts resolved on my side.
(Commit pushed to server, yet on my side it says failed.)

Yeah, game goes to main menu but you can't start new game or load save due to that error - game will crash then.

With PPIO I can cause this crash if I try open building entry.

 

[olduser]

I'd prefer to ban this practice in C2C space - it would be preferable to have the core be updated to new tags even if those tags are only used for a module. Just so that we have one 'bible' and not many.

One of the reasons I've asked for Alberts2 to chime in is that he was doing a restructuring of schema for a while there and stopped for some reason and IIRC it was going to establish a central schema library, so there may be both a method he can expose and the reason he stopped the project, which may or may not be due to it being a bad idea I'm not aware of. I think it was that we wouldn't stop updating and screwing up his efforts and he got frustrated. Can't recall. He would also know something about the DLL on this that I definitely do not.

I have a big set of changes waiting to be finshed and commited to the svn.

This includes a complete move to xsd
schema validation. It took so long because insted of just creating new schemas for everything i ended up constantly fixing all kinds of stupid errors. That was just frustrating and i stopped working on C2C a few times.

@Anq
I have created a xsd schema for everything already don't waste your time in this.

 

[Anq]

Yeah, game goes to main menu but you can't start new game or load save due to that error - game will crash then.

With PPIO I can cause this crash if I try open building entry.

You are right - I didn't test run the game, only opened the pedia from main menu. I think LSystemSchema should be placed in XML/Buildings and not moved. I made a big mistake.

 

[raxo2222]

I have a big set of changes waiting to be finshed and commited to the svn.

This includes a complete move to xsd
schema validation. It took so long because insted of just creating new schemas for everything i ended up constantly fixing all kinds of stupid errors. That was just frustrating and i stopped working on C2C a few times.

@Anq
I have created a xsd schema for everything already don't waste your time in this.

Well @Anq already committed his schema changes in 10675, though it failed at some point.

 

[olduser]

I'm afraid it's because of network error during that commit.
I will try to fix it after I get all tree conflicts resolved on my side.
(Commit pushed to server, yet on my side it says failed.)

The Audio and LSystem xml is beeing parsed by the exe the dll does not parse those.

Did you test if your xml schema changes work i remember not beeing able to use relative paths for those xdr Schemas.

 

[Anq]

The Audio and LSystem xml is beeing parsed by the exe the dll does not parse those.

Did you test if your xml schema changes work i remember not beeing able to use relative paths for those xdr Schemas.

Those paths ("URI"s) cannot contain backward slashes, only forward slashes. This is what the xml plugin in Notepad++ told me. Both the validator and the game pedia run fine.

 

[Anq]

The LSystem Schema was added in r6430 as an exact clone of the same file under vanilla Civ4's Assets folder. I guess it only exists to support C2CXmlValidator's function. It can't be modified because the exe is hard-coded to read LSystem definitions in only one way. This schema reflects how the exe reads the LSystem, and similarly, AudioDefinesSchema.xml and AudioScriptSchema.xml are essentially readonly files.

@alberts2 mentioned that relative paths could not be used for XDR schemas like this one.
I think this is true for msxml3, the decoding and validating library used by the exe.
Therefore the game panicked when I changed the schema reference for the two LSystem files: CIV4CityLSystem and CIV4PlotLSystem

xmlns="x-schema:CIV4LSystemSchema.xml"

through a relative path:

xmlns="x-schema:../Schema/CIV4LSystemSchema.xml"

This trick only works with the Xerces validator, which is newer than msxml3 and has compatibility for the XDR schemas.

Surprise with the Xerces validator, isn't it?

 

[olduser]

This trick only works with the Xerces library, which is newer than msxml3 and has compatibility for the XDR schemas.

Surprise with the Xerces validator, isn't it

Xerces does not support XDR validation at all
thats the reason the game starts with your changes.
We had to move away from msxml3 for the FinalRelease dll because the schema validation was too slow, xml loading took a few minutes with msxml3. The Debug dll was using msxml3 until some changes in C2C broke it and we changed the xml parser to xerces.
This made XDR Schemas useless until i created the C2CXmlValidator.exe which validates bouth XSD and XDR to make possible for modders to check their xml.

 

[Toffer90]

I'd prefer to ban this practice in C2C space - it would be preferable to have the core be updated to new tags even if those tags are only used for a module. Just so that we have one 'bible' and not many.

I concur with this executive decision. ^^

 

[JosEPh_II]

A Big Thank You to the Coding Team! It's fantastic now that T-brd has some real help thru Anq, billw, and Toffer. And I'm very Thankful that this help has brought alberts2 back to the Mod as well. Thank you All for what you do to keep this Mod running! .

EDIT: And Thank you raxo for your diligence and enthusiasm. Your skills seem to be growing by leaps and bounds as well.

 

[TaylorItaly]

I am 100% in the opinion of Joseph !!!

 

[KaTiON_PT]

Don't forget Jman as well!

 

[strategyonly]

A Big Thank You to the Coding Team! It's fantastic now that T-brd has some real help thru Anq, billw, and Toffer. And I'm very Thankful that this help has brought alberts2 back to the Mod as well. Thank you All for what you do to keep this Mod running! .

EDIT: And Thank you raxo for your diligence and enthusiasm. Your skills seem to be growing by leaps and bounds as well.

speaking of which, could u or anyone else really, go to the "credits" file and UPDATE it. please, thx, hasnt been updated in a few years. . . .thx... SO

 

[raxo2222]

speaking of which, could u or anyone else really, go to the "credits" file and UPDATE it. please, thx, hasnt been updated in a few years. . . .thx... SO

This file is here: Caveman2Cosmos\Assets\Python\BUG\Tabs\BugCreditsOptionsTab.py, so @Thunderbrd can find it and update it.
Last time it was updated in 2014 by you.

 

[Thunderbrd]

Folders "unique" and "units" still are here in art folder and they contain stuff.
Also why education book xcf file isn't packaged?

EDIT: Repackaged those files too - some units were missing some files.
As for that XCF file it wasn't needed for anything at all.

There are numerous file types that cannot be compressed into an FPK and work in the game. Some of those I knew couldn't be and there were some I did not find in the FPKs I searched for those types in so I didn't include because I wasn't sure they could be safely packed. Where those types were located they were left in the file paths they were in.

 

[Thunderbrd]

You are right - I didn't test run the game, only opened the pedia from main menu. I think LSystemSchema should be placed in XML/Buildings and not moved. I made a big mistake.

I would have preferred this wait till after the release. Please all... nothing further that has any chance of creating instability. I'm glad to see enthusiasm but we don't want a release with new and unexpected bugs!

 

[raxo2222]

There are numerous file types that cannot be compressed into an FPK and work in the game. Some of those I knew couldn't be and there were some I did not find in the FPKs I searched for those types in so I didn't include because I wasn't sure they could be safely packed. Where those types were located they were left in the file paths they were in.

Then why all graphics except movies are stored in FPK?
All folders except two could be removed in Unique folder, since you added all of them except those two to FPK but you didn't remove them in art folder..
I bet tons of nonmodular units and other stuff had noncompressable art files too.

Also this could explain why atomic bomber and biological warfare missile doesn't have nuking animations, and black hole feature never worked.
Probably same thing with some combat animations.
That is someone already let compress files, that can't be compressed.

All FPK files take 896 MB of space.
After extracting them all files take 887 (957 on hard drive) MB of space potentially indicating duplicates.

After packing them all to single file with unselected "Compress All" and Compression Level of 0 they take 889 MB of space.
I bet duplicates are at fault here.

 

[Thunderbrd]

Then why all graphics except movies are stored in FPK?
All folders except two could be removed in Unique folder, since you added all of them except those two to FPK but you didn't remove them in art folder..
I bet tons of nonmodular units had noncompressable art files too.

Also this could explain why atomic bomber and biological warfare missile doesn't have nuking animations, and black hole feature never worked.
Probably same thing with some combat animations.
That is someone already let compress files, that can't be compressed.

All FPK files take 896 MB of space.
After extracting them all files take 887 (957 on hard drive) MB of space potentially indicating duplicates.

Possible but its also that some bombing animation calls are disabled in code due to some animations being bad somewhere and that may be why so well. Nobody could figure out why so they were disabled for now. We can turn them back on after the release of we want to do more to figure it out. I left some empty file structure because it may remain useful to have the suggested shell there in future efforts.

There are places for non packable files outside of art... I am just unsure where they are supposed to go so just left them.

 

[raxo2222]

Possible but its also that some bombing animation calls are disabled in code due to some animations being bad somewhere and that may be why so well. Nobody could figure out why so they were disabled for now. We can turn them back on after the release of we want to do more to figure it out. I left some empty file structure because it may remain useful to have the suggested shell there in future efforts.

There are places for non packable files outside of art... I am just unsure where they are supposed to go so just left them.

So something else was problematic

By the way you missed my comment, but single uncompressed FPK file (compression level of 0, "compress all" unselected) takes slightly less space than fpk files currently - 896 MB now VS 889 MB after unpacking and repacking to single file.

This means there are duplicates in those FPK files.
Replacing FPK packages like that is tempting now.

In experiment I removed all DDS and NIF files, that were in unpackaged art. They take only 20 MB
of space.
If I don't let files replace each other while extracting them then they take 894 MB of space.

Terrain and Terrain1 share 11 duplicate files.
Units1 and Units2 share 6 duplicate files.
There is 8 more duplicated files.

So 25 files meant to replace other files were packaged wrongly.

 

[Toffer90]

Some art files are duplicated on purpose though because it is the same model but linked with different textures and/or different animation files.
Some instances are just two different files with the same name too, where a rename should have been done but weren't.

Just mentioning it.
One needs to look at every duplication pairs on a case by case basis, to figure out if they are really duplicates or if they just appear to be duplicates.
Art that is unused (never referenced by XML or NIF or kf files) can be removed.
Identical art in two locations where two xml entries reference their own version can be optimized by making both reference the same and delete the now un-referenced duplicate.
etc.

When art from two different FPK's have the same folder path and name then one of them is surely redundant. If they are different then one would need to inspect the difference and decide wich one is the better one before removing the other.