1. We have added a Gift Upgrades feature that allows you to gift an account upgrade to another member, just in time for the holiday season. You can see the gift option when going to the Account Upgrades screen, or on any user profile screen.
    Dismiss Notice

MAF/CTD now each turn

Discussion in 'Bugs and Crashes' started by Tuvok99, Mar 14, 2014.

  1. Toffer90

    Toffer90 C2C Modder

    Joined:
    Oct 16, 2011
    Messages:
    6,085
    Location:
    Norway
    I can confirm that they work well together, after using them together for many hours, and (even though graphical paging reduced the need for viewports) that using viewports still has a large impact on memory usage.

    Impact in numbers from a large viewport (X:100 Y:60) taken from one of my gigantic map games before all the ethnic unit skins came along to eat all the RAM:

    2.5 GB - Graphical paging without viewport
    1.4 GB - Graphical paging and viewport
    1.8 GB - Only viewport

    This game was in the industrial era with 33 AI; and the only MAF's I got were from memory leaks that build up after playing for a long time.

    PS: Don't play a gigantic map without viewports, it will never work. Be careful about number of civs as well; the game from my example will probably crash and burn sooner or later due to there being too many civs in it.
     
  2. draz

    draz Chieftain

    Joined:
    Oct 25, 2009
    Messages:
    285
    Location:
    Finland
    What version was this? The current SVN is unplayable for me since even with only 13 civs on a gigantic map, I'm getting CTD's now after almost every turn and it's only classical era. It's not my rig either since it's more than adequate, so I guess I'm reverting.
     
  3. Sparth

    Sparth C2C Team Member

    Joined:
    Oct 6, 2013
    Messages:
    2,311
    Ok ill stop adding new skins for now and Ill start testing C2C.
    Lets see where is problem.
     
  4. Toffer90

    Toffer90 C2C Modder

    Joined:
    Oct 16, 2011
    Messages:
    6,085
    Location:
    Norway
    I did the test roughly 6-7 months ago, sorry that I can't be of more help.

    Edit: If you can find the version where graphical paging was introduced to the mod, then you are close to the version from my test.
     
  5. JosEPh_II

    JosEPh_II TBS WarLord

    Joined:
    Jun 20, 2007
    Messages:
    15,673
    Gender:
    Male
    Location:
    Western IL. cornfields
    @Draz,

    Try 13 Civs on a huge or large map with out REV and Barb Civ options being on and see if you can make it thru Classical Era.

    And as Toffer90 posted try using Both Viewports and Graphic Paging to reduce the ram foot print.

    I started a new game with SVN 7187 with 10 AI on a Huge Map, Epic, w/o REV and Barb Civ and have been having no problems yet. I've reached 1700BC late Classical with 20 cities and have not had a hiccup yet.

    JosEPh
     
  6. draz

    draz Chieftain

    Joined:
    Oct 25, 2009
    Messages:
    285
    Location:
    Finland
    I do have both viewports and paging on, tracking RAM usage though it seems that the game only goes up to ~1.8 GB before dying, which just seems odd to me. C2C is the only mod I have installed on this Civ 4 installation, and the cache (both of them, since on win 7 at least there seems to be two separate folders in diff. locations) purged so I don't think it's due to a faulty/corrupted installation, either. But I mean, it should go higher than that before crashing, right? I'll need to do more testing.

    But huge or large... those are just too small for me to bother with, I'd rather not just play at all if they're the only real option. Paradox games have spoiled me in that regard. :p


    Does Rev and Barb civs cause additional memory stress even when there aren't any? I usually turn barb civs off, but Rev i've been normally playing with.
     
  7. JosEPh_II

    JosEPh_II TBS WarLord

    Joined:
    Jun 20, 2007
    Messages:
    15,673
    Gender:
    Male
    Location:
    Western IL. cornfields
    @draz,
    Realize that Rev generates more Civs thru revolts, thereby increasing the # of Civs and all their graphics to the game as you play.

    I MAF'd on this new version for the 1st time in 4 years. SVN version 7106 caused Civ IV exe in the task manager to hit 2.8GB for me. I have 8GB DDR3 ram, 2GB DDR5 vid ram and 8124MB (8GB) of Paging file set. I should not have had a MAF, Except! Civ IV BtS is a 32 bit engine with an 8 year old set of graphic loading programs in the exe which Modders can't touch.

    JOsEPh
     
  8. draz

    draz Chieftain

    Joined:
    Oct 25, 2009
    Messages:
    285
    Location:
    Finland
    Yeah I know, but it was so early in my game that that hadn't happened yet. I've a similar system (just more vram) and I full well know the limits of the game. But something has recently happened with the mod that caused all of this, I don't think people had this many problems with MAFs before. What you wrote here (great post btw.) makes complete sense to me, and I think a slimming down / making things more modular like has been suggested already is definitely worth a shot.
    I, personally, would prefer to play with just a handful of civs and a larger map, but I know others are the opposite. If, thanks to engine limitations, we can't have both, it'd be great if we could at least choose which one we prefer.
     
  9. God-Emperor

    God-Emperor Chieftain

    Joined:
    Jul 18, 2009
    Messages:
    3,551
    Location:
    Texas
    There are multiple memory columns you can enable in the task manager that show different things. I don't remember which one is the default, but if it isn't the default you might want to enable the "Memory - Commit Size" column (well, it is called that in Windows 7, but they love to change the names of things so it may be different if you are on some other version) via the "Select Columns..." item on the View menu. That one shows you how much virtual memory is reserved for the process and is, out of all the memory items, probably the most relevant one (although the others could be interesting too). It is the total virtual memory address space that is limited to 4GB (for this 32-bit process) on a 64-bit version of windows, or 3GB or less on a 32-bit version, depending on settings.

    Note that you can get a MAF even if you have some virtual address space remaining - even if you have quite a bit left. This is because it can fail not just from not having enough but also from not having a single piece that is big enough to meet an allocation attempt. If it wants to allocate a 10 megabyte chunk all at once and you have 200 MB of space it seems like it should just work, but what if that is divided into 100+ chunks (with already allocated memory ranges between the free parts) none of which are bigger than 5 MB? The allocation fails, that's what. This can happen when you have things of different sizes and different lifespans grabbing space wily-nilly. Some of those things release it quickly, others last the rest of the game, and many are between those two time spans. So you end up with free spaces of various sizes sprinkled in amongst the allocated memory ranges. This sort of thing is known as "virtual memory fragmentation".

    Something that might this situation even worse on Windows is that the smallest piece of virtual memory you can allocate is 4 KB. In many cases this is not an issue since the various libraries allocate a chunk and then let you use it in smaller pieces (and it, itself, not why I bring this up). This can have an interaction with the another thing it does: it apparently divides the virtual memory addresses up into 64 KB chunks and you can only ever have one allocation per 64 KB chunk so if you ask for less than 64 KB the rest of the address space in that chunk is wasted - asking for 4 KB gets you 4 KB and "hides" 60 KB so you can't access it until those 4 are released (likewise, asking for 68 KB gets you 68 KB but makes another 60 KB inaccessible since it allocates two contiguous blocks of 64 Kb each). Once again, this is usually not an issue since the libraries and such used to allocate things in high level languages are, in fact, normally allocating things in multiples of 64 KB and then letting you use the allocated memory in smaller pieces. Nevertheless, even though high level languages do things to reduce the problem, it does not eliminate it (if just one thing persists after everything else has gone away it can block the memory management from releasing an entire allocated range, for example).

    There are also other complexities in memory management. For example the Direct 3D stuff needs to allocate a bunch of memory and is using memory on the graphics card but I have no idea how it is accounted for. Is the memory in use on the graphics card part of your (or anybody's, or the OS's) virtual address space or just some I/O buffers for passing the data to and from it? Beats me. If it is in your virtual address space then it ought to show up in the column mentioned in the first paragraph.

    Anyhow, the "tl;dr" version is that being out of memory is not the only cause of memory allocation failures.

    There is a tool called VMMap that it might be interesting to use on the game. (I have never actually used this tool, but it looks interesting.)
     
  10. ZackBlastoff

    ZackBlastoff Chieftain

    Joined:
    Mar 26, 2014
    Messages:
    3
    hi

    I'm having the same problem since I installed the latest version of C2C.

    The game runs fine for over an hour but then it starts to crash every 2 - 4 turns. Sometimes it reports MAF but most times its a quick CTD.

    This is on top of a complete re-install of Civ4 and BTS, including a total wipe of everything related in MyGames. I have no other mods installed.

    The only changes I have made to the BTS ini file is to enable error logging and autosave every turn.

    Here are all the log files I can find and the last AutoSave.

    I hope this helps someone.. somewhere...

    (I don't know where to obtain this fabled "SVN" number though. The included ReadMe pdf does say "SVN" at the top but there's a blank space after that...)
     

    Attached Files:

  11. mskero

    mskero Chieftain

    Joined:
    Nov 29, 2013
    Messages:
    2

    Attached Files:

  12. Dancing Hoskuld

    Dancing Hoskuld Warlord

    Joined:
    Jul 5, 2004
    Messages:
    23,289
    Gender:
    Male
    Location:
    Canberra, Australia
    In the bug thread first post there are the details about what you need to post so that we can figure out what went wrong in situations like this. With out that info we are only guessing.

    If your game session was over 5 hours then memory leaks in the exe may cause the error. Exiting Civ, restarting and loading a save should fix the problem.

    If it doesn't then restart Civ and Caveman2Cosmos with a new game turn Viewports on and then load a save game.
     
  13. Athmos

    Athmos Chieftain

    Joined:
    Jan 2, 2006
    Messages:
    236
    Well, I think that does it for me. The property bug (now arise much sooner than before) and the constant crashes have finally convinced me to abandon this mod. The addition of a flower beetle sanctuary doesn't actually makes for the frustration of having 100+ hours games die down on me halfway through. The game was actually more stable in v33 I think.

    But I thought I should update to solve a few bug, and then the number of buildings double, the units almost did too, there are now hundreds more of stuff I can't ever use because the game actually crashes a long time before I get there...
    Half the settings that are in actually don't work (larger maps, minor civ, barb civ, etc) and will destroy your game once you invest enough time in it.

    The worst is that there are so many nice features in there, but in the actual experience you can't actually use them. The mod could more appropriately be called caveman to medieval, since going past renaissance with more than a few civs on a smallish map is impossible.
     
  14. Harrier

    Harrier Chieftain

    Joined:
    Feb 19, 2002
    Messages:
    2,386
    Location:
    UK
    Not entirely true. I am playing on a standard map size. There are 13 rival civs still alive and I am in the mid part of the Industrial era.

    Some MAFs are not just due to the game.

    There is reference to a post by 'Maska zgz' somewhere on this forum. (can not remember where)

    In it are given some good suggestions for improving your PCs gaming performance.
     
  15. KingArthur666

    KingArthur666 Chieftain

    Joined:
    Sep 17, 2006
    Messages:
    277
    Location:
    Montana
    Playing in windowed mode has stopped all the MAFs I was getting.
     
  16. Shomy035

    Shomy035 Chieftain

    Joined:
    Jan 22, 2013
    Messages:
    17
    Location:
    Serbia
    Sorry for hijacking this thread but i dont feel like it is necessary to open a new one since my issues are similar to the op.
    Right now i am playing a large map, v34, lowest graphics settings, 22 civs + rev barbarians on an i5 laptop and the game tends to become unplayable in late medieval or early renessance. I do not have repeatable ctd's (unless i enter worldbuilder 4-5 times in the same load) but the game tends to hang or go unresponsive similar to the infinite loop bug of AND if i recall correctly. Right now i am waiting for around 30 minutes for ai to finish its turn (or i can close civ and reload the last autosave) but this issue keeps repeating every few turns. I checked the task manager, i have enough ram and the cpu usage of civ4beyondsword.exe is never more than 25% with system idle process 70-75% (the rest goes to chrome, xbmc or some other app). I feel bad for criticizing what is a great effort of your free time and skills but i grew tired of playing prehistoric era over and over. :D
    Again, i really love your work but instead of focusing on adding so many new features, units, buildings, graphics etc etc i feel maybe you should iron out the issues that apparently others have too. I lack the programming skills but the least i could do is test through playing. :(
     
  17. Sparth

    Sparth C2C Team Member

    Joined:
    Oct 6, 2013
    Messages:
    2,311
    I can only say about my field of work:
    In last month or so I removed almost 700mb data from mod and now mod is even smaller before I start adding new graphics (skins mostly). Also many of the textures was resized and alpha channel removed. In the end old units take less data in memory.

    Nevertheless if someone want to use only standard set of skins it easy to do. Just XML work (not need programing skills :). I can show how to made it.
     
  18. Shomy035

    Shomy035 Chieftain

    Joined:
    Jan 22, 2013
    Messages:
    17
    Location:
    Serbia
    I tried downloading the latest svn and loading my game from v34 but the same error remains, artinfo was not found. That error was the reason i reverted from svn 72something to 34, idk what to play anymore. :(
     
  19. Sparth

    Sparth C2C Team Member

    Joined:
    Oct 6, 2013
    Messages:
    2,311
    Any more info which art info was not found?
     
  20. Athmos

    Athmos Chieftain

    Joined:
    Jan 2, 2006
    Messages:
    236
    standard map is a small size. Specially in a mod that propose even larger maps than usual. I don't think I ever played on anything else than the largest size in any other mods; but the gigantic I used here proved too much, and the civs multiply faster than they get eradicated with the options I'm using. The issue is compounded by the fact that in my previous game, where I didn't have barb civ on, the gigantic map was handled fine. So I thought it could work...

    But since that version (the previous one, v33), the number of building doubled or so, I'm not sure about what causes the rest of the issues, but frankly I'd rather have less "stuff" but be able to complete a game. If I have to play on a standard map size, without barb civs, I'm not really interested.
    The appeal of C2C -for me- is the whole size of the experience, but in actuality it doesn't deliver what's written on the box, because it is simply impossible to actually complete a game.

    I don't know if my current game would have been further playable if I didn't use barb civs and minor civs, other settings being the same. It is quite possible. I just don't want to invest 100 hours to find out if it is, and how much further down the timeline I could go.

    There are lots of good stuff in there, and I can stand a certain amount of bugs given the qualities of some of the features, but the game being cut off at one third to one half of what it should be by constant crashes and MAFs is just too frustrating for me.

    I don't know where the mod is going, but it is worrying that while some annoying bugs get fixed with updates, the experience actually gets worse with time. For example, I updated from v33 to v34 mainly to get the air promotion bug fixed. I suppose this should work now, but I'll never find out if I don't play an advanced start since the game will crash a looooooong time before the first air units become available.

    I think the mod could really be great, but I burned myself on it one too many time now. I'll check in a few months/years to see where it's at, best of luck to the team, but I'm afraid it will soon crumble under its own weight if a serious culling doesn't happen quite fast.
     

Share This Page