Improving load times of Religion and Revolution.

Hello folks.

On my computer, the mod took minutes to load.

By packaging the Art folder using Pakbuild the load time dropped to seconds, comparable to Vanilla Civ4Col.

This pakbuild utility is for Civilization 4, but as both game uses the same graphic engine, it works for us too.

Happy gaming.

At this time, I am zipping and uploading the resulting pak.file ( built from RaR 2.3 Art folder ) I will bump and give simplified instructions when it is done.

The downside of packbuild though is, that using fpks slows down performance ingame a bit.
(We did several tests and had many discussions with other modders about fpks.)

That was the main reason why we purposely never used it.
(We had this discussion many times already over the years.)

But of course every modmodder is free to do whatever works best for him.

If you have a link towards these discussions, I'd be interested.

I understand the reasons that led you to leave the Art folder unpackaged. This, and of course the fact that individual files are easier to work with.

The performance argument may have lost some of its weight over time, as our PCs are renewed and go towards recommended configuration and beyond for this old game.

At any rate, this has worked for the way I want to play the game. If it helps anyone, then it's for the better

Most of those discussions were private messages.
Some in forums as well, but I don't have the links at hand.

If people like to use them, they shall do so.
Do whatever works best for you.

But sorry, I simply had discussions about fpks too often already and thus won't again participate in such discussions myself.

Tried it on my quad core i7 930, 6GBRam + 5500RPM HD machine and I couldn't notice any significant graphics lag. Nice find.

EDIT - Other than the occasional animation lag.

Yeah, I also always use the mods' art files packaged into fpks with pakbuild.
Generally it improves performance, AFAIK
For loading times it's significantly better to use this way
But also shouldn't be slower ingame, or if it is, the difference must be minimal
Ray, what did your tests show?
I thought Civ IV loads the files on starting up the game/mod, and uses most of them from the memory

Ok, I will try to summarize what I (believe to) know AGAIN.
(I am not pissed. It is just a bit frustrating for me to discuss the same issues again and again.)

As I said, we - several experienced Civ4Col modders and a few experienced Civ4 modders - had been exchanging on this topic several times over the years:
(Mostly outside of normal public forums, since most players would not have been interested in such technical discussions.)

1. Using FPKs significantly improves loading times.
That is easy to verify.

2. Using FPKs is a bit of a hassle, when still working a lot on graphics.
But that is an aspect of modding and does not affect the players.

3. Loading times extremely depend on your hardware and OS.
Things are quite different for 64Bit environments with SSDs and very strong graphic cards for example.
(Loading RaR with cleaned cache takes about 12s on my machine.)

4. It is possible to mix FPKs with normal folder structure.
Data in normal folders will override data in FPKs.

5. When using FPKs there are some "general advices" gathered over the years:
(Maybe those are not correct anymore today though because hardware and OS have changed so much.)

- It is better to have 5 FPKs of 100MB than 1 FPK of 500MB
(Try to do your packaging wisely.)

- Some graphics (e.g. some animated screens or some movies) do not work when packed in FPK

- Some glitches with normal graphics (units / buildings / improvments) could be solved by moving the graphics out of FPK to normal folders again.
(It is not know exactly though if those issues were really related to FPKs. Similar things were reported without FPKs as well.
Could have been some temporary problem of the graphic card.)

6. The way graphical caching is done by Civ4-Engine is generally unknown to us modders.
There are many speculations about that, but nobody I ever talked to really knew for sure.

7. The concept of FPKs is quite old (10years).
However, I don't know any other game than Civ that uses them.

-------

Here it already ends with "facts".

The rest is experience, opinions and basic tests several of us had done.
(We did not use advanced methods of testing to verify.)

Those tests had been done quite some time ago though.
(Last discussions about FPKs I remember were more than a year ago.)

Most of us agreed that FPKs caused minimally worse performance ingame.
We are talking here about 5% to 10% longer Autoplays for example.

I ran 4 Autoplays with FPKs and 4 Autoplays with normal folder structure and then compared the times.
(The average time for Autoplays was better with normal folder structure. Don't know the exact numbers anymore though.)

Nobody ever said that FPKs improved performance ingame though.
(I also could not think of any reason, why it should.)

But all of this is more "feeling" than really "scientific testing".
Since so many experienced modders generally agreed on this, I do trust in what I heard.

-------

As I said, everybody shall do whatever works best for him.
(Using normal folder structure without FPKs works best for me though.)

Thanks for the answer!
And of course also for the effort the write all of it down

raystuttgart said:

(Loading RaR with cleaned cache takes about 12s on my machine.)

Click to expand...

I get that kind of speed with a FPK, but more than a minute without it. In fact I can make a complete recompile of the DLL faster than I can start RaR without FPK file(s).

raystuttgart said:

Most of us agreed that FPKs caused minimally worse performance ingame.
We are talking here about 5% to 10% longer Autoplays for example.

Click to expand...

I have the impression that graphics and "math" (essentially all in the DLL) are in two different threads. This mean that while this statement might have been true at one point, it might not affect CPUs with a core for each task.

I think I will run some performance tests on my quad core at some point, but it certainly isn't top priority.


It would also be worth noticing that I have encountered FPK problems on 32 bit systems where 64 bit systems ran the same files just fine. Somebody else confirmed that he experienced the very same result.

Using normal folder structure without FPKs works best for me though

Click to expand...

For me as well - so let's say it again:

We will NEVER use FPK's in our mod!

Every mod-modder can do what he likes.

Nightinggale said:

I get that kind of speed with a FPK, but more than a minute without it. In fact I can make a complete recompile of the DLL faster than I can start RaR without FPK file(s).

Click to expand...

I can only advise to move your Colonization installation to a SSD. Loading RaR from scratch takes 15 sec. on my rig.

Commander Bello said:

I can only advise to move your Colonization installation to a SSD. Loading RaR from scratch takes 15 sec. on my rig.

Click to expand...

I also installed Civ4Col on a SSD drive.
As I said, it takes only about 12s to load RAR with empty caches for me. (Which is similar to your 15s)