[GS] CTD - is there a really good debugger?

[ReichAdler]

3 weeks of my life trying to track down a random CTD. I have pulled apart & rebuilt everything. Pasted all the pages into Mod Buddy to check code. Pulled my hair out in frustration. Cut sections out to try & isolate what area may be causing it.

I am out of Ideas & ready to walk away from a mod that has been going since day 1 & has almost 100,000 users across its different faces.

Is there a tool I can run that actually tells me why its crashing... as the error logs & Firetuner report nothing. It happens about 30 mins to an hour in, nothing in particular or constant about it... as in doing the same thing... or action or in the AIs turn... nothing I can fathom.

High powered help needed now. I am a broken man.

 

[Zugaikotsu]

How deep into the rabbit hole are you comfortable going?

We are probably running into a C++ memory allocation/access problem, which tend to be the worst things to track down even with the code base in front of you. But, let's try...
First, let's look through the Window's Event Manager. If the application crashed, it is fairly likely the fault will appear there. This will likely be fairly unhelpful, but it may provide 1 of 2 things we can explore deeper with. 1st, we may be able to track down the fault error code, which will correspond to something (a quick Google search should reveal the underlying error). 2nd, we may be able to find some system logs created at the time of the crash. These generally aren't always helpful, but can sometimes provide useful insight.

The next part of this comes much easier if you can isolate a game right before a crash, with a save file, as it gets more...hands on. Visual Studios can attach a debugger process to applications that are running outside of its environment. Now, as with above, without the code, it will prove much more complex to actually get anything too useful without the code, but there have been times where it will provides grains of information to the root of the problem. Next, create a simple mod that iterates through all events to print to FireTuner. If the debug functions of lua are enabled, this gets much easier. If lua debug functions aren't enabled, you'll need to get creative on finding out how to distinguish between events. By doing this, it will tell us the absolute last thing that happened before the CTD.

Pseudo/Untested-Code:

iCounter = 0

function PrintCalls(...)
    print("Callback: " .. iCounter);
    iCounter = iCounter +1;
    print(debug.traceback());

    print("\n");
    print("Arguments: ")
     for i,v in ipairs(arg) do
        printResult = printResult .. tostring(v) .. "\t"
      end
     printResult = printResult .. "\n\n"

     print(printResult);
end

for iIndex, pEvent in pairs(Events) do
    pEvent.Add(PrintCalls);
end

for iIndex, pEvent in pairs(GameEvents) do
    pEvents.Add(PrintCalls);
end

Depending on the results, the path somewhat diverges, so I'll stop there to give you a bit of room to play with it. Good luck.

Disclaimer: I'm not the most familiar with the Civilization backend or debugging from previous series. I am familiar with debugging in a more general approach, though.

 

[ReichAdler]

I am ready to try anything, I hate to admit defeat!

I am grateful for your insight; but lack the knowledge of where to put this code in Firetuner or what I meant to do with it? I have only used it rarely... mainly just to spawn in units to test graphics.

Again, thanks in advance.

 

[Zugaikotsu]

View FireTuner as your main source of debugging pretty much everything in Civilization. You can modify the contents of any control (or make your own) to produce any lua to on-the-fly ingame commands. It is extremely powerful and useful. Not only that, but it will provide live-action console output for any "print" lua functions as well as faults in lua logic to the main console.
But first, I still recommend to look at the https://support.microsoft.com/en-us...oblems-with-event-viewer-in-microsoft-windows to understand what the error is first. The Lua will only provide the last point before the crash, but the Event Viewer should determine what the actual reason for the crash was, on a program level.

Edit: If you post the savegame, with the mods you are having trouble with, I will see if I can debug sometime this week or next.

 

[isau]

What kind of code are you running?

I can tell you one thing that will crash the game FOR SURE in database code is if you are using an ATTACH modifier (like MODIFIER_PLAYER_CITIES_ATTACH_MODIFIER) and the Modifier you link to isn't valid. This can also happen if you type the ModifierArgument as 'ModifierID' instead of 'ModifierId". It still compiles and runs, only to crash when the Modifier is supposed to get applied.

 

[Infixo]

@ReichAdler Which of your many mods is causing CTD?

 

[ReichAdler]

Almost all of them... as I use the same base generally & then just change the needed customisations. I have taken a break from this for a week... back at it now. It actually seems to be an issue between my civ mods & my Wonder Tweaks... but I am still testing things. I am rebuilding the icons at the moment as oddly I suspect it may be related to that.

I just want to get it a bit more sold before I ask you guys to test it with the knowledge you have. I have tried the events search & it is not pulling up anything useful that I can fathom.

I am still at it. Hope to be ready for testing version 427 soon

 

[ReichAdler]

@Zugaikotsu

I thought I had cured it with redoing all the icons, but today it crashed again, & is crashing for others. I would be most grateful to take you up on your offer to debug (as would many others). I have tried your method with the event viewer but am not getting anything for the time period.

It seems to happen with my two mods combined. Normally 20-30 mins in. I would be grateful for your insight.

https://steamcommunity.com/sharedfiles/filedetails/?id=1159810605
https://steamcommunity.com/sharedfiles/filedetails/?id=871073157

The submission.pck has this, it is about the only clues I can get.

00009EA325161c9eb49e5edb347f23f2d253ccf000006AB5{"Catalog":{"Dump":{"File":"","MD5":""},"Log":{"File":"","MD5":""},"Attachments":[{"File":"Renderer.log","MD5":"b36363e9d7a285d262b7c4a42f53dab6"},{"File":"Modding.log","MD5":"de24cb06046392117f28142a4d735044"},{"File":"Engine.log","MD5":"87a762aa809d28749c78f3a228b21495"}]},"Exception":{"Message":"Unhandled Exception\r\nCode:  EXCEPTION_ACCESS_VIOLATION\r\nError reading address 0x67FBBE070\r\n","StackTrace":"Unhandled Exception\r\nCode:  EXCEPTION_ACCESS_VIOLATION\r\nError reading address 0x67FBBE070\r\n"},"Session":{"Hash":7225658133770718269,"Time":1558203192},"Version":{"Major":1,"Minor":0,"Rev":0,"Build":317},"Host":"5D38169D","Time":"2019-05-19 07:07:22","Application":"Sid Meier's Civilization VI (Win64 DX12 FinalRelease)","Source":"EXTERNAL","Project":"","SubmissionType":"CRASH","Line":0,"File":"","Key":"","Callstack":[5382270487,5382273474,140720173442684,140720231121355,140720231021910,140720231106431,140720230452207,140720231101678,6435711798,6435711026,6435721035,6435720667,6435717482,6428940231,6435751797,6428940231,6435788767,6436885033,6430934068,6430903733,6428319933,6428326019,6428243106,6429881785,6428493060,6428499275,6428496440,5371184641,5382286674,140720185440628,140720230867569],"ModuleInfo":[{"file":"D:\\Steam\\steamapps\\common\\Sid Meier's Civilization

 

[Zugaikotsu]

"Unhandled Exception Code:  EXCEPTION_ACCESS_VIOLATION Error reading address 0x67FBBE070"

This is really what we were hoping not to see. Do you happen to have a save file? I went around to turn 120 turns in an hour (Online Speed, Pangaea, Small, 2 CPUs, settler difficulty, no barbarians, no city-states). The only thing that stood out was my memory was slightly higher than normal (hitting about 5GB over the 2-3 GB I normally run), which may have eventually caused issues, but it didn't seem like it rose much after that point. One other thing I noticed, and it likely isn't correlated to anything, but the Relics would pop up twice after discovering a new Natural Wonder.

 

[ReichAdler]

The saves rarely cause a crash again. I had tried compressing graphics to lessen the memory footprint, but that caused other issues, they displayed as translucent. I am just going to accept it as unfixable & give up modding Civ6. I only seem to get this crash on non-Civ6 nations, as in when I add the all same files to an existing Civ, it doesn't do it. So I am converting those that I can into existing Civs.

That relic thing is annoying, I must look into that again.

Thanks for your efforts.