Memory-fix by Harkonnen is out!

I've also checked out the patch.

I have a Clevo laptop with 2Ghz Pentium-M, 1GB RAM, 256MB X800XT Mobility video

With the 'safe' setting of vid200 it spared me 126MB RAM and the insane mode 115MB more.

But both settings slowed my in-game scrolling speed down (i'm using 1680×1050 res, 2×AA, max detail)

The insane mode slowed the map scrolling very much but even with insane disabled it got slower.

So for now i've disabled your patch but i'm awaiting new developments to be able to play a huge/18civs game without crashing (i think it's able even now with the patch just i have a good large/9civs/epic game on the way with i'm on the lead :-)).

A lot of gratulations for your efforts and success at reducing memory needs for this game!!!
Checking further developments...
 
Hi Harkonnen
I installed your patch and must say it works well for me. I couldn't load a huge map before, and with your patch I started playing on a huge map. Now I just see how far I'll get. But on a large map it should work perfectly for me. It will be a huge change for me after playing on standard map size.
Thanks man.
 
Harkonnen said:
Fallblau
Page file size doesn't show exact virtual memory usage. Virtual memory tab in task manager shows that per-process (this column must be explicitly added, by default it's not shown).

I think you should try following settings:
insane_mode = 0, vid_mem_usage = 80

Also note that game might swap for the first time you see some land or some country, but optimizations made by my fix allow more data to stay in memory than without my fix, so system swaps less.

For example, 1Gb systems should experience no swapping at all.

Also please read post #1 again (I made update about zlib1.dll there). You *MUST* download it not because I made some improvements to zlib1.dll (I didn't), but because it loads PatchByHarkonnen.dll into civilization4.exe process. Same stands for those who used python24.dll earlier.
Click to expand...

Hi Harkonnen...

Yeah, I don't have the VM tab enabled, but with the same background processes running in the with/without scenarios, the general pagefile size is still relevant...

Will try insane_mode = 0 and vid_mem_usage = 80 and see how it does...

I was indeed using your zlib1 when your patch was installed... Verified by seeing the task button included the "Patched" text...

I'll give it another go and see how it does this morning...
 
First off, :eek:! I'm amazed at the whole idea and that the fan community can do this! I'm new to Civ4, but with a community like this I may be around a while :D. Good work!

Now onto the results...

I installed the patch and have had some mixed, but better :) results.

Here's my sys specs
Athlon 64 3000
MSI K8N Neo2 Platinum mobo
1024 MB memory
nVidia GF4 Ti4200 w/128 MB memory.
AGP Aperture is 128 MB.

Civ4 stuff
BlueMarble mod installed
High textures, no AA, 1024x768.
I didn't change the D3D9 and DynamicAnimPaging settings because it worked the first time, so no need to change those, if I understand your advice correctly.
EDIT - this is full-screen mode, BTW

Harkonnen.ini
vid_mem_usage = 136

I tried loading my crazy save game that always CTD'd. Memory usage prior to the patch was 1195172. With the patch on insane mode, 1047592 is used for a net gain of 147580. As expected, Alt-TAB does not work (you get no textures or city buildings, though you can see animations from tile improvements and units!). Not really much of a loss since Alt-TAB had a tendency to CTD under the old one anyway. Before switching out to test memory usage I zoomed all the way out and back in (I figured that'd be good since it swapped like crazy before). Zoom's faster now, less chunking.

But then I tried the patch with insane mode off and got a bit of a surprise. Memory usage is 1192804, for a net gain of...get ready...2368. I guess that's something, right? :). But there's more...Alt-TAB actually works now without any CTD. Without the patch it was unstable. Zooming out and in is about half as slow as without the patch. It doesn't really make sense, does it, given what I've been reading in this thread and how there's some problem with globe view you want to fix, but on my box it actually helped. The game still swaps like a poker cheat, but it's faster and more stable. Go figure.

Any suggestions for how I could improve swapping? Should I try uninstalling BlueMarble? (seems like those nice textures would have some kind of a price tag, right?)
 
And now for your local P.I.T.A report... :blush:

OK, this morning's results:

Set for insane_mode = 0, vid_mem_usage = 80, turned the VM column on in Task Manager as you suggested.

Fullscreen, 4xAA, All High

WITHOUT PATCH
534,984K Mem Usage
729,596K VM size
--------------
1,218,328 Total

WITH PATCH
645,804 Mem Usage
642,764 VM size
-------------
1,288,568 Total

An INCREASE of 70,240K

Windowed, 0xAA, All High

WITHOUT PATCH
345,376 Mem Usage
733,084 VM Size
------------
1,078,460 Total

WITH PATCH
628,776 Mem Usage
625,756 VM Size
-------------
1,254,532 Total

An INCREASE of 176,072K

So, on (and I'd like to stress this) my machine, it is actually detrimental, and visibly so (scroll is actually slower, etc...). But that's MY machine, and many others are seeing improvement, so don't think I'm trying to take anything away from your accomplishment.

Now my machine always ran the game "OK", if not really well, so I was not quite as in need as others... Consider this your "really odd testing case outside of the statistical norm"... :)

By the way, I'm running Blue Marble as well, so perhaps there's something with it's texture set (which definitely "runs" better than the standard, seeded textures) that is throwing your patch off...

Again, you've done some righteous work here, regardless of whether my box likes it or not, and you've made the game playable for many who could not play it before... So "hats off"! :goodjob:
 
I can run my game very well but once every 10 minutes or so I get a random slowdown on huge maps late into the game. It's a noticeable slowdown but it's not permanent and only lasts a few seconds.

Will this fix it?
 
FallBlau, that's odd...

Your physical memory usage is increasing by leaps and bounds but your VM/swap file is decreasing. How does the game feel, speed-wise and disk-activity-wise? Seems like less VM should mean better performance.

I always wonder why, when I look at memory usage, Civ IV puts soo much into the swap file when there's plenty of physical available that it hasn't used...
 
kettyo said:
I've also checked out the patch.

I have a Clevo laptop with 2Ghz Pentium-M, 1GB RAM, 256MB X800XT Mobility video

With the 'safe' setting of vid200 it spared me 126MB RAM and the insane mode 115MB more.

But both settings slowed my in-game scrolling speed down (i'm using 1680×1050 res, 2×AA, max detail)

The insane mode slowed the map scrolling very much but even with insane disabled it got slower.

So for now i've disabled your patch but i'm awaiting new developments to be able to play a huge/18civs game without crashing (i think it's able even now with the patch just i have a good large/9civs/epic game on the way with i'm on the lead :-)).

A lot of gratulations for your efforts and success at reducing memory needs for this game!!!
Checking further developments...
Click to expand...
I have to agree your patch doesn't help me much either. I used your saved game to test my performance. I have AMD3800+,2gb of ram, 7800gt card. It seems if you have 2gb of ram then your patch doesn't help for the more memory the game uses the better it runs. The movies did run better with your patch though. My game still sometimes crashed when i try to reenter even with setting vid_mem_usage= 200 and insane mode=0

BUT..... then I tried to use atlas map generator to build a impossible size map ( this is superhuge 256 X 160) and notice a huge improvement. This is probably because these superhuge maps starts using over 2gb of ram from the very beginning. When maps are bigger than my ram I notice it take a lot longer to load the map and also can lock up for a few minutes if you scroll around the map (as if there are traffic jams in my pc). This time with your patch it started out only using 1.4 GB when I look around using worldbuilder with no traffic jams. that's over 600mb of ram difference. (Still I doubt this size is playable when it gets loaded toward late in the game)

So in my case your patch doesn't help much until the map need more ram than I have. Because of your patch I can now play even bigger maps than before. Still if Firaxis does included your fix in the next patch I hope we still have an option to turn it off since I won't be playing on these superhuge map often.

P.S. I believe you are right it's the cities that slows down the globel view. Even with a superhuge map with no cities I'm getting over 60fps in GV.
 
HERE GO A LOT OF REPLIES. IF I FORGOT SOMEONES, PLEASE ASK AGAIN.

ALL
One general thing I forgot to mention. If you ever turned AGP off (smartgart for example with ATi), you should turn it on for this patch to bring most of its benefits.

kettyo
I don't understand what d3dquery and dynamicanimpaging is at all, sorry
Well, I don't have exact answer too... just these things might affect how my patch works,
so I tried to repeat my config in order to make fixes/suggestions easier when someone faces problems.

As for savegame... I can't attach it because it's size is above 500Kb, even zipped. Please mail me at 'headden@karelia.ru', so I will mail you back.

jegui
That's strange... Please try this 'id_mem_usage=32, insane_mode=0'. This should be stable with every system. Was it crashing when you zoom out before the patch? If not, please mail me 'headden@karelia.ru' with your savegame.

Stilgar08
Yes, default settings should do. I am not sure my fix will solve movie stuttering problem. It's a bonus I didn't expect, and some people over here still had intro movie problems both before and after the patch...

There are two ways to uninstall it:

1) Uninstall in terms of functionality. Just set 'patch_on = 0' in Harkonnen.ini

2) Uninstall so that game can't detect anything later when it gets patched from Firaxis: download and replace original 'zlib1.dll' (see bottom of the post #1).

Amazigh
Well, this might be GPU overheating (for eaxmple dusty cooler) or it can be problem with power supply as well. Modern video cards state 300W as minimum. I could live with 250W for some time, but I have P3, and even putting 2nd HDD sometimes caused HDDs to restart on-the-fly, it was very scary sound with all that info... so if your power supply is below 300W, I suggest upgrading it to 300, or better 350-400W.

mr. Y
Same problem, same recommendation :)

Amazigh, mr. Y
This game has more geometry than it deserves. And from my experience, at least ATi cards eat much more power within geometry-processing unit rather than with rasterizer. So, any overheating or power problems will surely strike you via civ4 if you have a chance to have them.

Halk
2Gb systems sometimesw catch CTD, this patch solved some of those problems for 2Gb systems. I mean that with 2Gb systems speed won't be improved, but stability will.

JackTheNarrator
Yes, probably :) Thanks! I still don't know if there is memory leak or not. If there is, my tech just reduces its effects. I really faced invalid memory allocation by XML part and another oldschool ASM programmer faced invalid geometry deallocation, but I didn't investigate origins of these problems.

Amazigh
Just yesterday I had a problem of my HDD going off and on. I was damn freaking scared... all sources were there. Then my ATi after reboot told that its power plug is not in place. I opened PC box, plugged it in better, and the problem seems to have been solved.

Defectiv
Try 'insane_mode=0'. It's fast as long as video memory is fast... with 1Gb it's not so important.

Michelangelo
I think yes. Some intel graphics card got improved above in this thread. Though I don't remember if it was TnL card or not. Just give it a try. A patch is easy to be turned off, just replace downloaded 'zlib1.dll' with original 'zlib1.dll' (see post #1) for the link. I'd recommend 'vid_mem_usage = 32', 'insane_mode = 1' for starters. In case of bugs, set 'insane_mode = 0', it should work fine.

JorenCombs
1. You should check VM_Size in task manager instead to get improvement measurements.

2. My patch works with geometry data and it was always able to free at least 100Mb with 'insane_mode = 0'. Does blue marble affect geometry data or just textures?

3. As I saw above 'Ti4200' cards impose performance penalties with my patch which are comparable to performance increase with reduced memory usage, so overally it usually works around same level.

Fallblau
Hmm... another small decrease with Blue Marble (was expecting ~200Mb). I'm going to their thread to ask some questions about geometry. Try reducing 'vid_mem_usage' down to 40, probably it'll help some.

Bast
Probably yes, it depends on the origin of these slowdowns. If they come from memory usage, fix will fix that. Sorry for tautology :)

JorenCombs
I think Windows is just reserving space in swap file for later swapping out of physical memory pages. Changing file size is quick operation, so it increases swap file size, but doesn't actually write something to it. Just a guess... Also VM_size is entire game allocations size. Mem_usage is how much of this is devoted to physical memory plus how much of physical memory is devoted to cache files.

Smidlee
Yes, 2gb people either received no gain from this patch or the gain was increased stability. If you had no CTD yet, then probably you don't need it.
 
Vidmem=80 setting wihout insane option seems to work very well for my laptop. No slowdown indeed scrolling seems faster now.
Without patch system RAM usage with Civ4 and a fully packed large map was around 990MB (out of 1024 :-))
Now it's about 830MB with the same map.
And faster too.
So it's a HUGE improvement now.

It's strange though that i have 256 meg vidcard and use high res graphics so i should use vidmem=200 but it causes slowdown while vidmem=80 does the opposite.
 
Hakonen!!
Thank you very much for your help, I seriosly find you 100x more usefull than firaxis useless tech support that treats me like a totaly n00b.
I know it can't be a heat problem because i have added a Zalman CU700 cooler to my graphic card and I have before played heavy graphic demanding games with a heavily over clocked graphic card without any problems whats so ever. My power suply is a 500W one and with more than 15 amps on the 12V rail, which should be more than enough for my system. I really think it is a civ4 problem but i don't know how.
 
kettyo
You may try some value in-between like 150Mb...

Kalim
This is for you too... Probably my assumption of 180-200Mb for 256Mb video cards was wrong.
 
kettyo
Can you please give me list of DLL files it has in its main directory? If there is no open-sourced DLL, I will write a loder.exe to inject my patch from outside, but that's not preferable due to game restarts and savegame double-clicks (it would restart/load upnpatched with loader)
 
Amazigh
Well, I dunno... very few games use actually 100% of CPU or video memory, I mean all of its capabilities. As for CPU, guess which game suffered the most from overclocking?.... It wasn't Unreal1 in software mode... It was Starcraft... Probably civ4 has something like that with its geometry load or my patch has something like that with CPU. I optimized code for speed in every aspect including powerful optimization by MS compiler. Well, I don't know for sure...

If your card is ATi, try running windowed mode to see if VPU recover dialog appears (and provided it is allowed to appear in conrol panel display settings). I sometimes caught it when I had invalid allocations.
 
Harkonnen,
I experimented a bit with your fix, to see if it can help a poor Intel graphics notebook user (onboard graphics with 64 MB video ram, no TnL); 512 MB RAM, Centrino, etc. (Set the configurable parameter in your ini to 40)

I did not notice any improvements. I did not experience any crashes before (nor after). The game is just as slow as always (but playable).

Does it make sense to experiment some more or the fix ain't gonna help me?
 
crunch
You might try 'insane_mode = 1' if it does not cause problems. Also I'd suggest to set graphics settings to 'low', especially textures.

This fix is primarily for cases when it becomes uplayable due to vast memory usage. I think this is not the case with your current gamestate, it's just slow video... Probably later in game you'll want this fix.

You may try my 1961 savegame (see post #1) to see if it makes any improvement with your system in overused memory situation.
 
Harkonnen said:
JorenCombs
1. You should check VM_Size in task manager instead to get improvement measurements.
Click to expand...

I am checking VM_size, that's where I got the total memory figures. I just added the memory column plus the virtual memory column listed for Civilization4.exe.

Blue Marble...I have no idea what it affects, to be honest.

Harkonnen said:
JorenCombs
I think Windows is just reserving space in swap file for later swapping out of physical memory pages. Changing file size is quick operation, so it increases swap file size, but doesn't actually write something to it. Just a guess... Also VM_size is entire game allocations size. Mem_usage is how much of this is devoted to physical memory plus how much of physical memory is devoted to cache files.
Click to expand...

Um, not to talk back to the ASM whiz, but I don't think that's it... I'm looking at a list of processes right now, some of which has a VM Size smaller than its Mem Usage, and some of which is the other way around. How can VM Size be total phys mem + total swap file usage? VM Size should always be larger if that's true, right? I've always operated under the impression that VM size means how much of it has been paged to the swap file and Mem Usage means how much of it is in physical memory.
 
There are 2 files in Pirates main dir.
binkw32.dll and mss32.dll

BTW i don't think savegame loadup is that much important.
'Load a game' is just 1 click away.
 
You must log in or register to reply here.

Similar threads

Lazy sweeper
Minimum specs... Civ 6 crash heritage... out of memory???
Replies
0
Views
613
Lazy sweeper
Lazy sweeper
ElliotS
[BNW] More memory patches like Skyrim?
Replies
5
Views
549
Tekamthi
Tekamthi
C
Memory Allocation Crashes
2 3 4
Replies
65
Views
6K
Walter Hawkwood
Walter Hawkwood
kitfox
All menus on right side of screen are missing/disabled
Replies
16
Views
579
Necamijat
Necamijat
bssth
[UTILITY] Civ 4 Studio (Map properties editor)
Replies
16
Views
2K
bssth
bssth
Back
Top Bottom