Random(?) Crashes

Huh. I posted it the same way I did all the other photos. Let me try again.

upload_2020-6-13_11-47-0.png


Also, from playing some with the assert dll, I found that the bug still occurs sometimes on first load. The proton log is from one of those times. I would have mentioned just after I posted the log but my internet went out.
 
Code:
bool CvAnimationPathInfo::read(CvXMLLoadUtility* pXML)
{
    if (!CvInfoBase::read(pXML))
    {
        return false;
    }

    pXML->GetChildXmlValByName( &m_bMissionPath, L"bMissionPath" );
    if (!pXML->TryMoveToXmlFirstChild())
    {
        FErrorMsg("MissionPath invalid");
        return false;
    }

    const wchar_t* tmp = pXML->GetXmlFirstText();
    TCHAR animPathName[1024];
    if (tmp)
    {
        mbstate_t mbs;
        mbrlen(NULL, 0, &mbs);
        wcsrtombs(animPathName, &tmp, 1023, &mbs);
        FAssertMsg(tmp == 0, "Failed to convert animation path name");
    }
So on your linux/wine sutup the FAssertMsg(tmp == 0, "Failed to convert animation path name") triggers on game launch. Meaning wcsrtombs(. . . &tmp . . .) doesn't change tmp as expected.
wcsrtombs is not a C2C specific function so I'm guessing it's from a system dll or similar library.

The issue is related to file paths, which typically are handled differently by different OS's.
Maybe the above code can be rewritten in a way that is more compatible with unique setups like yours, but likely the issue won't be solvable by us. Modders with a lot of technical coding insight like @billw2015, @AIAndy or @alberts2 would be better suited to comment further in that regard.

It is related to animations... @fluffy465 could you try to see if using the "animation frozen" option stops the repeatable end turn crash in your save from happening?

Edit: issues with animations have been known to cause CTD's in the past, we once had a long period where all combat animations in C2C were disabled through hardcoding in the dll because of CTD's.
 
Last edited:
Ends with "No Buildable Unit for selected AI!!" three times. That sounds helpful...

That happens every game over and over. It's one of the many things that should have been fixed when it started happening.
This is normal and expected if you don't yet have a unit available for training for seeing through any of the invisible type categories. It happens with each type it doesn't have a valid unit for yet. So for example, if it can't find a unit that has any abilities in seeing through an invisibility veil, which it won't for much of the game, when it checks to see if it does and finds it doesn't, said message will flash. This message was setup for other checked AI type needs as well and used to mean a more buggy problem than it often does now, such as if it can't find a valid city defender AI type unit. So the unfortunate part is that this is creating a bit of a false alarm. In a future AI overhaul, I'll probably try to find a way to divide out the triggers for this message into more specific messages.

I might also try to improve the check itself so that it knows if one should be available before it even bothers so as to save time on it constantly doing the routine to look for one each time.
 
It is related to animations... @fluffy465 could you try to see if using the "animation frozen" option stops the repeatable end turn crash in your save from happening?

Edit: issues with animations have been known to cause CTD's in the past, we once had a long period where all combat animations in C2C were disabled through hardcoding in the dll because of CTD's.

Trying to turn on frozen animations didn't seem to change the outcome. I remember seeing a few weeks back there was one unit that just A-posed during movement on rare occasions, so I'm inclined to believe it uses a fallback animation when it can't find the one it needs.

As for the "No Buildable Unit for selected AI!!\n", I do personally find that suspicious. Every proton log that I can generate ends with 3 of those. There is also 2 "\nnew m_accruedCostRatioTimes10000 : 10000 "s, and a few hundred lines of healing stuff with every log. It might be a common log output, but I think if it always crashes with that at the end it must be related in some way to the issue, even if it is tangential (like if the time of the crash just coincides with it or something).
 
As for the "No Buildable Unit for selected AI!!\n", I do personally find that suspicious. Every proton log that I can generate ends with 3 of those. There is also 2 "\nnew m_accruedCostRatioTimes10000 : 10000 "s, and a few hundred lines of healing stuff with every log. It might be a common log output, but I think if it always crashes with that at the end it must be related in some way to the issue, even if it is tangential (like if the time of the crash just coincides with it or something).
See my last post as I attempted to explain this phenomenon.
 
Trying to turn on frozen animations didn't seem to change the outcome. I remember seeing a few weeks back there was one unit that just A-posed during movement on rare occasions, so I'm inclined to believe it uses a fallback animation when it can't find the one it needs.
Yeah, it was a long shot anyway.
As for the "No Buildable Unit for selected AI!!\n", I do personally find that suspicious. Every proton log that I can generate ends with 3 of those. There is also 2 "\nnew m_accruedCostRatioTimes10000 : 10000 "s, and a few hundred lines of healing stuff with every log. It might be a common log output, but I think if it always crashes with that at the end it must be related in some way to the issue, even if it is tangential (like if the time of the crash just coincides with it or something).
It's hard to say how many hundreds of codelines are between the last log output and the crashpoint. There are in many cases a lot of codelines between each log entry line in the log you posted.
All C2C games trigger that assert a lot between turns without it crashing the game, so we can't even say for certain that that huge chunk of looped code where the AI looks for what to build in a city, where that one debug output is from, is actually close to the actual crashpoint, especially if the crashpoint is not even in our dll which is often the case in a operating specific CTD.
 
Last edited:
Another long shot:
We know there are CTD's on windows 7 that are not on other windows versions, and found that those go away when setting the game to windows vista compatibility mode, is it possible for you to run the game in compatibility modes in your wine setup.
Would you mind trying out some of them and see if the crash still happen?
 
I tried changing the compatibility mode in winecfg to XP, Vista, and 10 and all of them still crash. The proton log looks the same for all of them, as well as the failed assert (not that I was expecting it to be different).
 
So on your linux/wine sutup the FAssertMsg(tmp == 0, "Failed to convert animation path name") triggers on game launch. Meaning wcsrtombs(. . . &tmp . . .) doesn't change tmp as expected.
wcsrtombs is not a C2C specific function so I'm guessing it's from a system dll or similar library.
wcsrtombs is a function from the standard C library:
https://en.cppreference.com/w/cpp/string/multibyte/wcsrtombs
That could be C locale related. Maybe you have a file name or path that contains non-ASCII characters.
You can find out what wide character it is by checking what tmp points to after that function call.
 
I would look at what tmp is if I knew how to read it. The problem is the window doesn't tell me anything more than tmp == 0 is false, which doesn't really help. If I click the debug button it just crashes.

And this is the directory I have the mod in:
/home/kartoffelsaft/Hard_Disk/Steam Library/steamapps/common/Sid Meier's Civilization IV Beyond the Sword/Beyond the Sword/Mods
or as the game sees it:
Z:/home/kartoffelsaft/Hard_Disk/Steam Library/steamapps/common/Sid Meier's Civilization IV Beyond the Sword/Beyond the Sword/Mods

So if the issue is with non-ASCII characters then it has to be something generated by wine.

But this does give me the idea to see if installing some of the other vc++ runtimes might help with the issue. Do you guys use any versions of vc++ other than 2003? (or 71. I don't know how the version things)
 
I'm surprised I wasn't pinged on Discord or so since I seem to be the resident linux user of this mod. ^.^

But still, let my copy/paste a summary of what I put on Discord.

In short, the code posted above is:
Code:
    if (tmp)
    {
        mbstate_t mbs;
        mbrlen(NULL, 0, &mbs);
        wcsrtombs(animPathName, &tmp, 1023, &mbs);
        FAssertMsg(tmp == 0, "Failed to convert animation path name");
    }

A couple of problems with this code. First let's look at the `wcsrtombs` function, it's return value isn't being checked. First of all it returns the length processed or it returns -1 on failure, both of those need to be checked for reasons I will get to shortly.

Second, the `tmp` is only altered by walking the pointer forward, so in the `FAssertMsg` the `tmp == 0` should only be false if it didn't successfully walk it the entire length, which can happen for two main reasons:

1. It can happen with the output destination, `animPathName` in this case, is too short to hold the entire path.
2. Invalid characters in the wide encoding.

I honestly doubt that #2 happened because wine/proton is very good about 'fixing up' the path, so that leaves #1.

So now why would #1 happen, well that would be because the full path to read that exceeds the 1023 buffer of `animPathName`, which is only 1024 length (1023 chars + null terminator).

So that then begs the question of how this would be read correctly, well first of all the `wcsrtombs` function is being misused here. First it should be used with a null initial argument and 0 length, this will return the size of the buffer needed (without the terminating null, so add 1 to it) or it will return `-1` if their are invalid multibyte characters. Then another `wcsrtombs` should be called with an appropriately sized buffer.

Because `wcsrtombs` is being misused according to the spec itself by using a hardcoded buffer length when the size of the converted string is unknown then it is entirely expected for this to fail, and you would also hit this assert on windows as well if Civ4 were installed into a long enough path.

So at the very least this code is wrong. `wcsrtombs` needs to be called once with a null destination and 0 length valure, there should be `FAssertMsg` on the return value that it is not `-1` (assuming you don't have an appropriate fallback, make sure to handle the failure case), then allocate a long enough buffer (a reusable `std::vector<char>` would probably be best so you can keep reusing that cache but it can grow as needed) to hold the data and call `wcsrtombs` again with the correct buffer length (+1 for null terminator).

The person that hits this assert should be able to work around it by making their path shorter and maybe making sure only ascii in the path just as a double-check.

I have Civ4 installed on a rather short path so that would explain why I haven't hit this issue.
 
I'm wondering how you manage to get the directory to be a shorter length. I listed the directory I have the mod in and the majority of it is from the steam library onward, which is the part of the directory I have the least control over. Would it be possible to install the mod closer to root and give civ 4 the absolute path to the mod? Every modding tutorial I've seen just uses a relative directory so I don't know if it's possible.
 
I'm wondering how you manage to get the directory to be a shorter length. I listed the directory I have the mod in and the majority of it is from the steam library onward, which is the part of the directory I have the least control over. Would it be possible to install the mod closer to root and give civ 4 the absolute path to the mod? Every modding tutorial I've seen just uses a relative directory so I don't know if it's possible.

I don't have steam install games into my home partition, I have it installed into `/mnt/storage0/steam`, which is on my RAID, and it makes for a very nicely short directory for some occasional broken games that hardcode pathlengths. :)

As for Civ4, if you have another partition then link that partition to a nice short path and have steam use it perhaps? If not, what is your full and complete steam path to your Civ4 install?

Regardless though, that function is being misused and it is a bug in the code.
 
I'm wondering how you manage to get the directory to be a shorter length. I listed the directory I have the mod in and the majority of it is from the steam library onward, which is the part of the directory I have the least control over. Would it be possible to install the mod closer to root and give civ 4 the absolute path to the mod? Every modding tutorial I've seen just uses a relative directory so I don't know if it's possible.
You can define custom game library locations with steam, like I have these steam games installed:
B:\Steam\steamapps\common\Crusader Kings II
C:\Steam\steamapps\common\Hearts of Iron IV​
While my steam exe is here:
D:\Steam\steam.exe
with games installed there too lol
D:\Steam\steamapps\common\Dishonored​
 
Trying to shorten the path to civ 4 turns out to be really hard. I ran into a few problems:

1. steam doesn't list symlinks when adding a library.
2. if you force it to use a symlink by adding a folder and replacing the folder with the symlink, it uses the destination of the symlink as a folder
3. you can't remove a library with a game in it, so you have to unmount the drive for steam to forget about it without moving/deleting the games

This basically means my only option is to unmount the partition, then tell it to remount somewhere else and tell steam where it went if I want to shorten the directory.

You can define custom game library locations with steam, like I have these steam games installed:

I already had it in a separate drive. I don't expect you to know the linux filesystem, so no big deal

If not, what is your full and complete steam path to your Civ4 install?

I mentioned what it was in the post just before yours:

/home/kartoffelsaft/Hard_Disk/Steam Library/steamapps/common/Sid Meier's Civilization IV Beyond the Sword/Beyond the Sword/Mods

Though now I'm moving what was /home/kartoffelsaft/Hard_Disk/Steam Library to /mnt/hd/stmlb to see if that helps
 
Though now I'm moving what was /home/kartoffelsaft/Hard_Disk/Steam Library to /mnt/hd/stmlb to see if that helps
Change "Sid Meier's Civilization IV Beyond the Sword" to "Civilization IV" too.
 
It didn't seem to help. I still get the failed asserts and it still crashes on the testing save.

Change "Sid Meier's Civilization IV Beyond the Sword" to "Civilization IV" too.

If I try that steam complains about a missing executable.
 
Looking at the actual length of the paths in the mod (using pwd | wc -c while in the directory), it doesn't actually come out anywhere close to 1023. most of them are in the 120 - 180 range in terms of characters for me. So if the issue is directory length, then the animation files have to be named with 800+ character long strings, which I doubt is the case.

@OvermindDL1 Have you tried using the assert dll given in this post? Because you might be having the issue with the asserts too, and if you are having them but not the crashing then we could rule out the failed asserts as causing the crash.
 
@OvermindDL1 Have you tried using the assert dll given in this post? Because you might be having the issue with the asserts too, and if you are having them but not the crashing then we could rule out the failed asserts as causing the crash.
He would also need the save with the repeatable crash ready to be easily repeated, which is attached in this post.

Edit: Oh, and he would also need to be on the exact same version as you got the crash in with the finalRelease dll provided in SVN or manually built if in the git setup.
SVN rev 11179 or git version v40.1.2401
 
Second, the `tmp` is only altered by walking the pointer forward, so in the `FAssertMsg` the `tmp == 0` should only be false if it didn't successfully walk it the entire length, which can happen for two main reasons:

1. It can happen with the output destination, `animPathName` in this case, is too short to hold the entire path.
2. Invalid characters in the wide encoding.

I honestly doubt that #2 happened because wine/proton is very good about 'fixing up' the path, so that leaves #1.
This is about reading a path in the XML which is usually relative and not absolute.
But now that I look at it more closely this is from reading an XML file that I don't think is even present in C2C so the default XML from the BTS installation is used.
And finally looking even more closely this code makes no sense. It reads something that it claims is a MissionPath but it enters the first child which is Type in all Civ XML files including that one. It reads it, converts it and then only uses it in an error message.
To me that looks like remnant code from a bug hunt that was not removed.
This should probably be removed but is not a likely candidate for a later crash (if someone removes it, don't forget to adjust the TryMoveToXmlNextSibling line later in the same function).

If this triggers an assert in this case, the base Civ XML data might be overwritten or overridden though.
 
Top Bottom