MAF/CTD now each turn

[Toffer90]

As far as I know the new Graphical Paging just means you should not need to use Viewports as much. They do work together and using both should use less memory.

I can confirm that they work well together, after using them together for many hours, and (even though graphical paging reduced the need for viewports) that using viewports still has a large impact on memory usage.

Impact in numbers from a large viewport (X:100 Y:60) taken from one of my gigantic map games before all the ethnic unit skins came along to eat all the RAM:

2.5 GB - Graphical paging without viewport
1.4 GB - Graphical paging and viewport
1.8 GB - Only viewport

This game was in the industrial era with 33 AI; and the only MAF's I got were from memory leaks that build up after playing for a long time.

PS: Don't play a gigantic map without viewports, it will never work. Be careful about number of civs as well; the game from my example will probably crash and burn sooner or later due to there being too many civs in it.

 

[draz]

This game was in the industrial era with 33 AI; and the only MAF's I got were from memory leaks that build up after playing for a long time.

What version was this? The current SVN is unplayable for me since even with only 13 civs on a gigantic map, I'm getting CTD's now after almost every turn and it's only classical era. It's not my rig either since it's more than adequate, so I guess I'm reverting.

 

[Sparth]

Ok ill stop adding new skins for now and Ill start testing C2C.
Lets see where is problem.

 

[Toffer90]

What version was this? The current SVN is unplayable for me since even with only 13 civs on a gigantic map, I'm getting CTD's now after almost every turn and it's only classical era. It's not my rig either since it's more than adequate, so I guess I'm reverting.

I did the test roughly 6-7 months ago, sorry that I can't be of more help.

Edit: If you can find the version where graphical paging was introduced to the mod, then you are close to the version from my test.

 

[JosEPh_II]

What version was this? The current SVN is unplayable for me since even with only 13 civs on a gigantic map, I'm getting CTD's now after almost every turn and it's only classical era. It's not my rig either since it's more than adequate, so I guess I'm reverting.

@Draz,

Try 13 Civs on a huge or large map with out REV and Barb Civ options being on and see if you can make it thru Classical Era.

And as Toffer90 posted try using Both Viewports and Graphic Paging to reduce the ram foot print.

I started a new game with SVN 7187 with 10 AI on a Huge Map, Epic, w/o REV and Barb Civ and have been having no problems yet. I've reached 1700BC late Classical with 20 cities and have not had a hiccup yet.

JosEPh

 

[draz]

I do have both viewports and paging on, tracking RAM usage though it seems that the game only goes up to ~1.8 GB before dying, which just seems odd to me. C2C is the only mod I have installed on this Civ 4 installation, and the cache (both of them, since on win 7 at least there seems to be two separate folders in diff. locations) purged so I don't think it's due to a faulty/corrupted installation, either. But I mean, it should go higher than that before crashing, right? I'll need to do more testing.

But huge or large... those are just too small for me to bother with, I'd rather not just play at all if they're the only real option. Paradox games have spoiled me in that regard.


Does Rev and Barb civs cause additional memory stress even when there aren't any? I usually turn barb civs off, but Rev i've been normally playing with.

 

[JosEPh_II]

@draz,
Realize that Rev generates more Civs thru revolts, thereby increasing the # of Civs and all their graphics to the game as you play.

I MAF'd on this new version for the 1st time in 4 years. SVN version 7106 caused Civ IV exe in the task manager to hit 2.8GB for me. I have 8GB DDR3 ram, 2GB DDR5 vid ram and 8124MB (8GB) of Paging file set. I should not have had a MAF, Except! Civ IV BtS is a 32 bit engine with an 8 year old set of graphic loading programs in the exe which Modders can't touch.

JOsEPh

 

[draz]

@draz,
Realize that Rev generates more Civs thru revolts, thereby increasing the # of Civs and all their graphics to the game as you play.

Yeah I know, but it was so early in my game that that hadn't happened yet. I've a similar system (just more vram) and I full well know the limits of the game. But something has recently happened with the mod that caused all of this, I don't think people had this many problems with MAFs before. What you wrote here (great post btw.) makes complete sense to me, and I think a slimming down / making things more modular like has been suggested already is definitely worth a shot.
I, personally, would prefer to play with just a handful of civs and a larger map, but I know others are the opposite. If, thanks to engine limitations, we can't have both, it'd be great if we could at least choose which one we prefer.

 

[God-Emperor]

There are multiple memory columns you can enable in the task manager that show different things. I don't remember which one is the default, but if it isn't the default you might want to enable the "Memory - Commit Size" column (well, it is called that in Windows 7, but they love to change the names of things so it may be different if you are on some other version) via the "Select Columns..." item on the View menu. That one shows you how much virtual memory is reserved for the process and is, out of all the memory items, probably the most relevant one (although the others could be interesting too). It is the total virtual memory address space that is limited to 4GB (for this 32-bit process) on a 64-bit version of windows, or 3GB or less on a 32-bit version, depending on settings.

Note that you can get a MAF even if you have some virtual address space remaining - even if you have quite a bit left. This is because it can fail not just from not having enough but also from not having a single piece that is big enough to meet an allocation attempt. If it wants to allocate a 10 megabyte chunk all at once and you have 200 MB of space it seems like it should just work, but what if that is divided into 100+ chunks (with already allocated memory ranges between the free parts) none of which are bigger than 5 MB? The allocation fails, that's what. This can happen when you have things of different sizes and different lifespans grabbing space wily-nilly. Some of those things release it quickly, others last the rest of the game, and many are between those two time spans. So you end up with free spaces of various sizes sprinkled in amongst the allocated memory ranges. This sort of thing is known as "virtual memory fragmentation".

Something that might this situation even worse on Windows is that the smallest piece of virtual memory you can allocate is 4 KB. In many cases this is not an issue since the various libraries allocate a chunk and then let you use it in smaller pieces (and it, itself, not why I bring this up). This can have an interaction with the another thing it does: it apparently divides the virtual memory addresses up into 64 KB chunks and you can only ever have one allocation per 64 KB chunk so if you ask for less than 64 KB the rest of the address space in that chunk is wasted - asking for 4 KB gets you 4 KB and "hides" 60 KB so you can't access it until those 4 are released (likewise, asking for 68 KB gets you 68 KB but makes another 60 KB inaccessible since it allocates two contiguous blocks of 64 Kb each). Once again, this is usually not an issue since the libraries and such used to allocate things in high level languages are, in fact, normally allocating things in multiples of 64 KB and then letting you use the allocated memory in smaller pieces. Nevertheless, even though high level languages do things to reduce the problem, it does not eliminate it (if just one thing persists after everything else has gone away it can block the memory management from releasing an entire allocated range, for example).

There are also other complexities in memory management. For example the Direct 3D stuff needs to allocate a bunch of memory and is using memory on the graphics card but I have no idea how it is accounted for. Is the memory in use on the graphics card part of your (or anybody's, or the OS's) virtual address space or just some I/O buffers for passing the data to and from it? Beats me. If it is in your virtual address space then it ought to show up in the column mentioned in the first paragraph.

Anyhow, the "tl;dr" version is that being out of memory is not the only cause of memory allocation failures.

There is a tool called VMMap that it might be interesting to use on the game. (I have never actually used this tool, but it looks interesting.)

 

[ZackBlastoff]

hi

I'm having the same problem since I installed the latest version of C2C.

The game runs fine for over an hour but then it starts to crash every 2 - 4 turns. Sometimes it reports MAF but most times its a quick CTD.

This is on top of a complete re-install of Civ4 and BTS, including a total wipe of everything related in MyGames. I have no other mods installed.

The only changes I have made to the BTS ini file is to enable error logging and autosave every turn.

Here are all the log files I can find and the last AutoSave.

I hope this helps someone.. somewhere...

(I don't know where to obtain this fabled "SVN" number though. The included ReadMe pdf does say "SVN" at the top but there's a blank space after that...)

 

[mskero]

hello i have recently crash about memory allocation failure

http://forums.civfanatics.com/attachment.php?attachmentid=373281&stc=1&d=1397054685

how i can fix it ? anybody knows ?

thanks in advance

 

[Dancing Hoskuld]

In the bug thread first post there are the details about what you need to post so that we can figure out what went wrong in situations like this. With out that info we are only guessing.

If your game session was over 5 hours then memory leaks in the exe may cause the error. Exiting Civ, restarting and loading a save should fix the problem.

If it doesn't then restart Civ and Caveman2Cosmos with a new game turn Viewports on and then load a save game.

 

[Athmos]

Well, I think that does it for me. The property bug (now arise much sooner than before) and the constant crashes have finally convinced me to abandon this mod. The addition of a flower beetle sanctuary doesn't actually makes for the frustration of having 100+ hours games die down on me halfway through. The game was actually more stable in v33 I think.

But I thought I should update to solve a few bug, and then the number of buildings double, the units almost did too, there are now hundreds more of stuff I can't ever use because the game actually crashes a long time before I get there...
Half the settings that are in actually don't work (larger maps, minor civ, barb civ, etc) and will destroy your game once you invest enough time in it.

The worst is that there are so many nice features in there, but in the actual experience you can't actually use them. The mod could more appropriately be called caveman to medieval, since going past renaissance with more than a few civs on a smallish map is impossible.

 

[Harrier]

The mod could more appropriately be called caveman to medieval, since going past renaissance with more than a few civs on a smallish map is impossible.

Not entirely true. I am playing on a standard map size. There are 13 rival civs still alive and I am in the mid part of the Industrial era.

Some MAFs are not just due to the game.

There is reference to a post by 'Maska zgz' somewhere on this forum. (can not remember where)

In it are given some good suggestions for improving your PCs gaming performance.

 

[KingArthur666]

Playing in windowed mode has stopped all the MAFs I was getting.

 

[Shomy035]

Sorry for hijacking this thread but i dont feel like it is necessary to open a new one since my issues are similar to the op.
Right now i am playing a large map, v34, lowest graphics settings, 22 civs + rev barbarians on an i5 laptop and the game tends to become unplayable in late medieval or early renessance. I do not have repeatable ctd's (unless i enter worldbuilder 4-5 times in the same load) but the game tends to hang or go unresponsive similar to the infinite loop bug of AND if i recall correctly. Right now i am waiting for around 30 minutes for ai to finish its turn (or i can close civ and reload the last autosave) but this issue keeps repeating every few turns. I checked the task manager, i have enough ram and the cpu usage of civ4beyondsword.exe is never more than 25% with system idle process 70-75% (the rest goes to chrome, xbmc or some other app). I feel bad for criticizing what is a great effort of your free time and skills but i grew tired of playing prehistoric era over and over.
Again, i really love your work but instead of focusing on adding so many new features, units, buildings, graphics etc etc i feel maybe you should iron out the issues that apparently others have too. I lack the programming skills but the least i could do is test through playing.

 

[Sparth]

Again, i really love your work but instead of focusing on adding so many new features, units, buildings, graphics etc etc i feel maybe you should iron out the issues that apparently others have too. I lack the programming skills but the least i could do is test through playing.

I can only say about my field of work:
In last month or so I removed almost 700mb data from mod and now mod is even smaller before I start adding new graphics (skins mostly). Also many of the textures was resized and alpha channel removed. In the end old units take less data in memory.

Nevertheless if someone want to use only standard set of skins it easy to do. Just XML work (not need programing skills . I can show how to made it.

 

[Shomy035]

I tried downloading the latest svn and loading my game from v34 but the same error remains, artinfo was not found. That error was the reason i reverted from svn 72something to 34, idk what to play anymore.

 

[Sparth]

I tried downloading the latest svn and loading my game from v34 but the same error remains, artinfo was not found. That error was the reason i reverted from svn 72something to 34, idk what to play anymore.

Any more info which art info was not found?

 

[Athmos]

Not entirely true. I am playing on a standard map size. There are 13 rival civs still alive and I am in the mid part of the Industrial era.

Some MAFs are not just due to the game.

There is reference to a post by 'Maska zgz' somewhere on this forum. (can not remember where)

In it are given some good suggestions for improving your PCs gaming performance.

standard map is a small size. Specially in a mod that propose even larger maps than usual. I don't think I ever played on anything else than the largest size in any other mods; but the gigantic I used here proved too much, and the civs multiply faster than they get eradicated with the options I'm using. The issue is compounded by the fact that in my previous game, where I didn't have barb civ on, the gigantic map was handled fine. So I thought it could work...

But since that version (the previous one, v33), the number of building doubled or so, I'm not sure about what causes the rest of the issues, but frankly I'd rather have less "stuff" but be able to complete a game. If I have to play on a standard map size, without barb civs, I'm not really interested.
The appeal of C2C -for me- is the whole size of the experience, but in actuality it doesn't deliver what's written on the box, because it is simply impossible to actually complete a game.

I don't know if my current game would have been further playable if I didn't use barb civs and minor civs, other settings being the same. It is quite possible. I just don't want to invest 100 hours to find out if it is, and how much further down the timeline I could go.

There are lots of good stuff in there, and I can stand a certain amount of bugs given the qualities of some of the features, but the game being cut off at one third to one half of what it should be by constant crashes and MAFs is just too frustrating for me.

I don't know where the mod is going, but it is worrying that while some annoying bugs get fixed with updates, the experience actually gets worse with time. For example, I updated from v33 to v34 mainly to get the air promotion bug fixed. I suppose this should work now, but I'll never find out if I don't play an advanced start since the game will crash a looooooong time before the first air units become available.

I think the mod could really be great, but I burned myself on it one too many time now. I'll check in a few months/years to see where it's at, best of luck to the team, but I'm afraid it will soon crumble under its own weight if a serious culling doesn't happen quite fast.