Memory Leak?

It's strange that most people with 1Gb couldn't make that (afaik most used 4.12 or so).

Anyway, that's not a solution for me since I'm bounded to 512Mb by Pentium3 chipset :( Well, at least I never faced CTD yet :)
 
hi reading this forum. cool work y'are doing harkonnen wish i could help.:goodjob:
i've got the reboots with 1 gig memory, a visionmaster pro x800 xt platinum and my cpu is an amd athlon mobile 3000. drivers version 8.18.0.0
tried the old drivers option but gave me funny new problems. like i didn't see the resolution options anymore starting civ4. again it crashed and rebooted.
i think maybe the problems are related with the mobile cpu and or the "less than normal" use in the system manager.

btw i'm experiencing massive defragmentation on my hd
 
BTW, I have discovered in my driver folder that I probably was on 5.0 drivers :P I think drivers not supporting post-9800XT cards (X800 and such) should all work fine. Well... except for the owners of those cards...
 
Emerentius
Thanks... I'll need your and others help when it'll be time to test it. I have almost finished, just gone to the point where nothing more will come into my head when I write the last effort which considers everything (threads, heaps, sizes, paging, life-span of objects, origin of allocations, etc... etc... etc...).
 
I have 1 gb of ram and would love to help. I already have a clean install imaged and ready to go with 1 year old drivers for video and sound. I am working on creating a few more images with different driver combos. Clean installs of coarse. I have several machines that I have tested on and find the same ctds later in the game.
more then willing to stand on the firing line and have to reinstall again. Just let me know.
 
blue3c said:
I have 1 gb of ram and would love to help. I already have a clean install imaged and ready to go with 1 year old drivers for video and sound. I am working on creating a few more images with different driver combos. Clean installs of coarse. I have several machines that I have tested on and find the same ctds later in the game.
more then willing to stand on the firing line and have to reinstall again. Just let me know.
Click to expand...


It seems to me you have to be an expert in computing to play this game, is this correct?
 
chris.giddins said:
It seems to me you have to be an expert in computing to play this game, is this correct?
Click to expand...


Not really, this is just many people trying to find a way, any way to get the game to work for them. The game either works right or it doesn't, depending on some unknown factors.
 
Emerentius said:
hi reading this forum. cool work y'are doing harkonnen wish i could help.:goodjob:
i've got the reboots with 1 gig memory, a visionmaster pro x800 xt platinum and my cpu is an amd athlon mobile 3000. drivers version 8.18.0.0
tried the old drivers option but gave me funny new problems. like i didn't see the resolution options anymore starting civ4. again it crashed and rebooted.
i think maybe the problems are related with the mobile cpu and or the "less than normal" use in the system manager.

btw i'm experiencing massive defragmentation on my hd
Click to expand...


Same reboot problem, latest Catalyst drivers, 9600 Pro, A64 3200+
 
Sevenhertz said:
It is simply UNACCEPTABLE by industry standards to release a game so unstable.
Click to expand...
Not all users have such immense problems with CTDs. I only have crashes every hour or so. Sometimes there are three CTDs in 30 minutes, then the game runs for three hours without one single CTD.

I've seen games which were far more unstable than CIV4.

heppcatt
 
Not being a programmer, I found this little treatise I found from a google search to be of great value explaining some things about how Python and memory allocation works. Fairly easy to read and understand.

I don't quite understand it all yet, especially the recommended fixes, and I am not sure how current it is; i.e. if it applies to the latest version of Python - there is no date on the site - but it does explain the way Python uses memory in terms someone like myself can understand.


http://evanjones.ca/memoryallocator/
 
oldStatesman said:
Not being a programmer, I found this little treatise I found from a google search to be of great value explaining some things about how Python and memory allocation works. Fairly easy to read and understand.

I don't quite understand it all yet, especially the recommended fixes, and I am not sure how current it is; i.e. if it applies to the latest version of Python - there is no date on the site - but it does explain the way Python uses memory in terms someone like myself can understand.


http://evanjones.ca/memoryallocator/
Click to expand...

oldStatesman, that is an excellent article you found. That article is from a guy at my old alma mater, the University of Waterloo, which has one of the very top computing schools in the world.

As for how current it is, he comments that pymalloc became imbedded in 2.3.
The most current version of python available at python.org is 2.4.2, and was released Spet 28th this year.

The first release of 2.3 was in late July, 2003.

I had a quick scan of the release notes of 2.3.1 through 2.3.5 (2.3.5 was released Feb 2005), and I saw nothing about modifying pymalloc in the notes.
It was a quick scan though, so it may be there be something there that a better coder than I would recognize as a change.

This Evan Jones is a grad student at U of W, as per his personal web page, and is still active in Python. According to his web page, he released a Python
parser less than 2 weeks ago.

I am emailed him to see if he knows if this allocation problem was solved in later versions of Python.

This may be a waste of time, but I could see two pieces of info that need to be figured out:

1. I am sure Harkonnen is already all over this, but does Civ IV fit the model that this Evan Jones decribes as a candidate for better allocation of memory.
2. If this pymalloc problem was indeed fixed in later versions of Python, and I am not sure this is even considered a "problem", were the Firaxis coder using this "fixed" release, or an old one?
 
I_batman said:
oldStatesman, that is an excellent article you found. That article is from a guy at my old alma mater, the University of Waterloo, which has one of the very top computing schools in the world.
Click to expand...
Thanks!

If you contact him, please tell him that his writing style is very accessable..a non-programmer such as myself (I am system engineer/sdmin type) was able to comprehend it easily. I found it very helpful - now I really know what Harkonnen and you and others are talking about when they discuss mallocs and allocs and such...big thumbs up!

I have to also say I have pretty much resigned myself to a spectator role now...it seems that the solution will lie in coding, not hardware/systems issues, and I am not qualified to explore there. However, if you all do need any assistance testing, or with more hardware or sytem work, please let me know. I will continue to follow this thread with great interest as long as it is relevent, and will be here ready to be a guinea pig if needed.
 
As I said in a parallel thread, python results in about 22Mb when huge map loads. That's is, I account every page of RAM with at least one byte occupied as whole 4096 bytes. And it results in 22Mb for all allocations from python24.dll (I've redirected their pymalloc to simply malloc/realloc/free which are hooked in my DLL and targeted into my manager, module-separated, thread-separated, etc...). Every piece of junk operates in own private sub-heap to reduce fragmentation. It's 92% (used_page_size / commited_pages_total) - almost ideal, and yet somethings eats more and more.

BTW, I have spotted the devourer. It is d3d9.dll. But it does not seem like a cache of textures or something, too small objects... and too strange Win3.1 legacy function to allocate them is being used.... And sometimes they fail to free blocks, so heap becomes corrupted. This looks to me like a soft memory leak.
 
Harkonnen said:
As I said in a parallel thread, python results in about 22Mb when huge map loads. That's is, I account every page of RAM with at least one byte occupied as whole 4096 bytes. And it results in 22Mb for all allocations from python24.dll (I've redirected their pymalloc to simply malloc/realloc/free which are hooked in my DLL and targeted into my manager, module-separated, thread-separated, etc...). Every piece of junk operates in own private sub-heap to reduce fragmentation. It's 92% (used_page_size / commited_pages_total) - almost ideal, and yet somethings eats more and more.

BTW, I have spotted the devourer. It is d3d9.dll. But it does not seem like a cache of textures or something, too small objects... and too strange Win3.1 legacy function to allocate them is being used.... And sometimes they fail to free blocks, so heap becomes corrupted. This looks to me like a soft memory leak.
Click to expand...
Interseting. Check this article from MSDN website on DSDSPY, a tool in the DirectX developers Kit.

http://msdn.microsoft.com/library/d...c_Summer_03/directx/graphics/tools/d3dspy.asp

This tool is used to basically debug dirextX apps..and it substitutes it's d3d9.dll for the release version of d3d9.dll. you don't think that the Civ4 DirectX routine is making calls for the debug dll instead of the production dll do you? Would this cause memory problems if the Civ4 program thinks the production dll is the debug dll? Just speculation...thinking out loud...

On edit: From some more quick google research this dll seems to contain shader info for direct3d (ref this site for SwiftShader... http://www.transgaming.com/swiftshader_technology.php )...if it is part of the problem, this ties in with a lot of the graphics issues out there...black terrain, cheshire cat bug, weird terrain colors with cards with older versins of T&L and shader support. Hmm....just thinking out loud again...
 
oldStatesman
Thanks, but I know this stuff :) Just simply monitioring would not be enough as I need to change calls - that's what I am trying to do right now...
 
Harkonnen said:
BTW, I have spotted the devourer. It is d3d9.dll.
Click to expand...

Bingo.
That is the file that CIV4 would NOT run without on my system - even though I had DX9.0C it still asked for it.

I had to install the DX version off the game disk.
Cooincidence? It seems not, now.
Question is, is the file supplied on the game disk that is packaged with DX9 faulty/corrupt? Is this possible that it is not an updated version but a corrupt one? Or something else?

BTW - you guys are doing some great work in solving this mystery. You should get a medal.:goodjob:
 
The news are:
my memory manager works, but it almost does not improve the situtation. Fragmentation is not the issue as I suspected when started writing it. There are ~300Mb allocations coming from civ4.exe, and another ~300Mb from d3d9.dll. With the rest stuff whis results in overall 700Mb.

I can't know the origin of civ4.exe allocations since I have no code for it, and I can guess that d3d9.dll eats mem due to D3DPOOL_MANAGED or D3DPOOL_SYSTEMMEM for resources (textures, vertex buffers, etc...).

I am not sure that they ship corrupt D3D9, but I am sure they ship debug version of at least D3DX library (that nasty d3dx9_26.dll file, normally it's contents are in .lib, i.e. linked into executable during making a build, and no such DLL is necessary).

So, that week spent with memory manager is almost in vain since it wasn't the reason (except I discovered a lot of interesting things, but that's useful just for me).

Now I am hooking into D3D to find what and where it allocates. Currently I can't track textures creation. Well, I can, but so far I find only ~256Kb allocated textures, so I wonder where others come from. I don't believe they lock/unlock/render everything through a single small texture, but performance is too low even without swapping, so it might be the case.

Now I'm downloading recent D3D9 SDK to match vtables, perhaps I'm just hooking the wrong function.

My earlier assumption was memory fragmentation. Not the issue though. My current assumption is that a lot of SAME resources are kept in-mem by both civ4.exe and DirectX core. I can't affect civ4.exe calls, but interface of D3D is known, thus hackable. So I can play with it.
 
You must log in or register to reply here.

Similar threads

Lazy sweeper
Minimum specs... Civ 6 crash heritage... out of memory???
Replies
0
Views
606
Lazy sweeper
Lazy sweeper
Larsenex
Civ 4 and 64 bit.
Replies
1
Views
735
a pen-dragon
A
C
Memory Allocation Crashes
2 3 4
Replies
65
Views
5K
Walter Hawkwood
Walter Hawkwood
E
UI: zooming out more
Replies
4
Views
799
axatin
A
kitfox
All menus on right side of screen are missing/disabled
Replies
16
Views
547
Necamijat
Necamijat
Back
Top Bottom