Memory leaks and Inter-turn lag

Abegweit

Anarchist trader
Joined
Aug 6, 2003
Messages
4,879
Location
One step ahead of the authorities
Has anyone else noticed that memory usage grows over time? I started a new game and the software was using 2.6 GB of memory. I re-started civ and did the same thing. Only 850 MB, one third as much.

In a similar vein, the inter-turn lag seems to grow the longer you stay in the game. I noticed a huge difference after re-starting. Not sure if it's related to the lost memory.

Oh. And the multi-processing claims are bullcrap. It is rather obvious from watching the Windows task manager and playing with processor affinity that this application is fundamentally single-threaded. It really is a b!tch to see my quad-core hyper-threading computer with 15% CPU usage.
 
Has anyone else noticed that memory usage grows over time? I started a new game and the software was using 2.6 GB of memory. I re-started civ and did the same thing. Only 850 MB, one third as much.

In a similar vein, the inter-turn lag seems to grow the longer you stay in the game. I noticed a huge difference after re-starting. Not sure if it's related to the lost memory.

Oh. And the multi-processing claims are bullcrap. It is rather obvious from watching the Windows task manager and playing with processor affinity that this application is fundamentally single-threaded. It really is a b!tch to see my quad-core hyper-threading computer with 15% CPU usage.
the "memory leak" issue
Civ IV does this even after final patch. So if you really expect them to fix this your dreaming. Seeing as they began dev using the CIV iv engine im amused that this is still there.
 
I have seen no issues with memory leaks or increasing time lags in CIV since Vanilla.

As a developer myself, I find these kinds of problems to be inexcusable. They are easy to test for and, with proper coding practices, easy enough to guard against. At least the big leaks that throw away gigabytes at any rate.
 
This game doesn't use hyperthreading that well, if at all. On my quad core the CPUs are being used evenly, but nothing really with hyperthreading. I think the did a good job. This isn't prime 95, and it's not going to stress your pc at 100%.

Try going into your bios, disable hyperthreading and all but one core, and tell me if the game plays the same.
 
I have seen no issues with memory leaks or increasing time lags in CIV since Vanilla.

As a developer myself, I find these kinds of problems to be inexcusable. They are easy to test for and, with proper coding practices, easy enough to guard against. At least the big leaks that throw away gigabytes at any rate.

Ya its a small leak.. after an hour or so your looking at 500mb or so. Which is my guess why they never took the time to fix it in civ IV. Doesnt become an issue until late game as the threshold for memory allocation errors grows smaller as the game grows. As to the Civ V problem i haven't looked at it in depth yet but i imagine its the same issue and is a hold over from "borrowing" some of Civ IVs code.
 
I guess that it grows with the time playing because the map is getting busy with the turns, more cities, units, improvements, terrain discovered, all civs at sight on the map, etc.
 
I guess that it grows with the time playing because the map is getting busy with the turns, more cities, units, improvements, terrain discovered, all civs at sight on the map, etc.

Yes its expected to grow as the game itself grows. The issue is when it claims some memory then doesn't release it after it done causing a slow steady stream of wasted memory. In long enough sessions this will lead to eventual memory allocation errors. Depending on your OS and mem limits per program it may be a very long session (64 bit OS with lots of mem) before you see this issue or a shorter one( 32 bit OS). Good news is simply restarting civ and/or reloading the latest save game solves this for the most part. Which again is why they will probably never worry about fixing it as they ignored it in Civ IV.
 
Ya its a small leak.. after an hour or so your looking at 500mb or so. Which is my guess why they never took the time to fix it in civ IV. Doesnt become an issue until late game as the threshold for memory allocation errors grows smaller as the game grows. As to the Civ V problem i haven't looked at it in depth yet but i imagine its the same issue and is a hold over from "borrowing" some of Civ IVs code.

Quite honestly, it obvious that you have no clue what you are talking. 500 MB/hr is NOT a small leak. Even the first version of CIV IV didn't leak like that. And, if the current version leaks at all, it can't be more than a few K.
 
Quite honestly, it obvious that you have no clue what you are talking. 500 MB/hr is NOT a small leak. Even the first version of CIV IV didn't leak like that. And, if the current version leaks at all, it can't be more than a few K.

Lol I'm speaking of late games.. and yes it does on huge maps 18 civs late games it'll lead to memory allocation errors every hour or so. I've seen the issue repeated on multiple systems so don't give me this BS that i don't know what im talking about cause i've seen it. 1.9gb usage save game restart civ load up save game POOF 1.3 gb used. The issue grows exponentially with the amount of processing. Late games huge maps leaking occurs way faster. 500mb/hr was just an estimation it could be significantly lower or higher but the issue remains. Huge maps and more civs adds to it/ its an old issue i doubt they will fix.
 
This game doesn't use hyperthreading that well, if at all. On my quad core the CPUs are being used evenly, but nothing really with hyperthreading. I think the did a good job. This isn't prime 95, and it's not going to stress your pc at 100%.

Try going into your bios, disable hyperthreading and all but one core, and tell me if the game plays the same.

I have tried. Have you? You don't need to go into the BIOS. Windows supports it through the task manager. Result: 100% of usage on a single core. And a single hyper-thread of that as well and small amounts elsewhere. It takes the same amount of time.

As for the even usage of your cores, the answer is simple: Windows spreads a single thread across all cores, using each one in turn (I think this is done for heat reasons).

On my computer I get 15-17% CPU usage. Given that a hyper-thread is not as good as a core and that there is some other work that needs to be done, this shows that the game basically doesn't multi-thread at all.
 
Lol I'm speaking of late games.. and yes it does on huge maps 18 civs late games it'll lead to memory allocation errors every hour or so. I've seen the issue repeated on multiple systems so don't give me this BS that i don't know what im talking about cause i've seen it. 1.9gb usage save game restart civ load up save game POOF 1.3 gb used. The issue grows exponentially with the amount of processing. Late games huge maps leaking occurs way faster. 500mb/hr was just an estimation it could be significantly lower or higher but the issue remains. Huge maps and more civs adds to it/ its an old issue i doubt they will fix.
I always played standard maps so it's possible that I never noticed this. If indeed you are right, it is certain that they will never fix it as there will never be another CIV IV patch. I remain skeptical though. I can't believe that all of the code-divers, modders, and the rest of the civ 4 community never noticed it.
 
Probably just fragmentation, not a genuine leak. RAM is cheap, though, so buy more or restart occasionally and you'll be fine.
 
I always played standard maps so it's possible that I never noticed this. If indeed you are right, it is certain that they will never fix it as there will never be another CIV IV patch. I remain skeptical though. I can't believe that all of the code-divers, modders, and the rest of the civ 4 community never noticed it.

http://www.civfanatics.com/search/g...cac8f8&form_id=google_cse_searchbox_form#1092

Lol just a quick search of JUST civfanatics. It was reported widely but Firaxis pretty much ignored it as they didn't really care about what ppl did on huge maps.And again there was always the simple fix of saving your game and reloading it every hour or so which was much cheaper than development time spent towards us "niche" players who liked huge maps. My experience with firaxis support forums was them telling me my system that had 2gb of ram at the time just wasnt enough.. i beleived them til i moved to my current system which has 4gb of ram which simply delays the issue.
 
Illuminate me. What is memory fragmentation?

It's when you allocate memory, free it, and repeat with different sizes. Eventually the memory becomes fragmented such that you cannot allocate larger chunks of memory even though you have far more total memory free than the chunk you want to allocate.

There are techniques to mitigate this, so it's not an excuse, but a shortcoming. I don't know if Civ suffers from this, but I have seen 32-bit apps crash for lack of memory with only 1.3 GB out of 2 GB allocated.
 
I always played standard maps so it's possible that I never noticed this. If indeed you are right, it is certain that they will never fix it as there will never be another CIV IV patch. I remain skeptical though. I can't believe that all of the code-divers, modders, and the rest of the civ 4 community never noticed it.

Fully patched Civ4BTS definitely had a stutter problem the longer you play that you could have worked around by simply restarting the game's exe.
 
On my computer I get 15-17% CPU usage. Given that a hyper-thread is not as good as a core and that there is some other work that needs to be done, this shows that the game basically doesn't multi-thread at all.

Windows will schedule all physical cores before Hyper-threaded cores. For example, when I run task manager on my i7 it shows four of the eight cores as having all most all of the activity when under low load. 50% would mean you're maxing your physical cores and 12.5% means maxing one physical core. So 15-17% does imply that it isn't scaling much or at all, but Hyper-threading isn't involved yet and won't be unless the game scales to use 100% of four cores or more.
 
On my computer I get 15-17% CPU usage. Given that a hyper-thread is not as good as a core and that there is some other work that needs to be done, this shows that the game basically doesn't multi-thread at all.

Well, if you have a fast enough video card, at least the rendering engine will utilize more than one core, but than it might be the just video driver. And they never claimed explicitly that the AI was multithreaded, it's just that folks ASSUMED it because of all that marketing blurb about "scalability" :D

But fast turn times lategame on large maps won't earn you good review scores and the attention of the casual crowd, fancy graphics will :mischief:
 
Well, if you have a fast enough video card, at least the rendering engine will utilize more than one core, but than it might be the just video driver. And they never claimed explicitly that the AI was multithreaded, it's just that folks ASSUMED it because of all that marketing blurb about "scalability" :D

But fast turn times lategame on large maps won't earn you good review scores and the attention of the casual crowd, fancy graphics will :mischief:

Oh. My video card is great and I have no problems with display update at all. Perhaps the rendering engine is indeed using multiple cores.Dunno. But my issue is with the amount of time taken when I push the <enter> button.
 
It cannot be that bad as I only have about 1.6GB of available memory when I play and it works fine. It takes the same at the start and the end (and I have had a few 4 or 6 hour sessions).
 
Top Bottom