Rebuilding Dune Wars with RevDCM 2.83

I have rebuilt Dune Wars using RevDCM 2.83 as a base. The prime motivation in doing this is to potentially get Dune Wars working in multiplayer (RevDCM 2.83 is reportedly stable for multiplayer).

There are other benefits to getting a RevDCM 2.83 version up and running, namely:
i) All the latest BUG features are included (such as tech tree colour coded by Era):

ii) Further AI improvements over those in Better BTS AI 1.01 by people like Fuyu and Afforess.

Unfortunately, right now the game crashes on start up the majority of the time. This seems to happen after map generation but before the first turn.

On the occassions when the game starts successfully the game seems stable - I have run 300+ turn autoplays - and Dune Wars features such as the Homeworld work fine.

I have attached the rebuilt mod together with all C++ source code. Also, included is a working savegame - this loads up but then crashes as soon as you found a city.

I'm running Windows 7 so the issue may not occur in XP which we have established is more forgiving of small variable/memory issues.

When I've tried to debug the break point is something to do with the Hotseat code which hasn't changed - I've pretty much come up against a brick wall other than that.

I ran civchecker across the mod and nothing significant came up that I could see.

Given the stablity when the game does initialise successfully, it feels like I'm not far off having a stable version of this - but since I haven't been able to debug this successfully myself I'm uploading to see if anyone else can help out.

The file is just a 7-Zip file to be unzipped to your Mods folder. Any help anyone can give would be appreciated.

Download Dune Wars - RevDCM 2.83 here

.

hey!

buddy do you want me to upload my semi merged code of dune with rev?

Thanks for the offer, but I've already merged everything myself - I just need some help fixing this crash.

Here is what I've got from attempting to debug the crash myself. On founding a city with the included savegame - I get this:


The break in the debug dll is the line highlighted in red:


This code hasn't changed at all so I think there is some memory/initialisation problem somewhere and this is just how it is manifesting itself.

Further info: I think the issue could be that RevDCM doesn't like something in the merged XML.

I have made three mods:

Build 1: Vanilla RevDCM 2.83 DLL, Dune Wars terrain XML only. Vanilla leaders, civs, units, buildings, etc. No Dune Wars Python.

Build 2: Vanilla RevDCM 2.83 DLL, Dune Wars terrain, units, buildings, etc. Vanilla leaders, civs and traits only. No Dune Wars Python.

Build 3: Vanilla RevDCM 2.83 DLL, all Dune Wars XML entities. Dune Wars Python included.

In terms of stability:

Build 1, I have never seen crash seems completely stable.

Build 2, occasionally crashes on start up like my full merge (seems like a different crash as shows as a runtime error and pretty rare.)

Build 3, almost always crashes on start up like my full merge.

The main difference between Build 2 and Build 3 is bringing in the Dune Wars civs, leaders and traits so it could be something there. I have been careful to merge the XML schemas and can't see any issues. I can upload these mods if anyone would find them useful.

ill take a wild guess and say its something in the python perhaps?

maybe some f the components there - try to add part after part maybe.

ill tell you, long time ago, i tried merging rev with dune, the compile ws ok, but when istarted a game, one turn after the game i had a crash.

god i wish this to work!

Yeah I feel for you trying to get the build stable, in what can be really complicated scenarios. Yeah like Kel it looks like adding the python is the issue. I'm not as experienced as Afforess because I've only ever worked in the tiny world of RevDCM, but the way to go would be to seperate out whether it is an XML issue or python. The builds above seem to be a bit unclear in that area. In build 2 you add both python and some new XML. The question in my simple head is, can you lay down the entire XML infrastructure without any new python infrastructure and see if that CTD's? My suspicion is that it will be in the python but often I have been surprised to find it elsewhere. If you are like me, then you do not have a proper python debugging environment and must deduce python problems in primitive ways.

It is so much work I understand! I suggest taking a break, and then when your energy returns, absolutely cleanly separate out the possibilities. The DUNE DLL with just the XML infrastructure. Test that XML build, then add the python layer. Is it possible to do that?

Just talking out loud and I realise it may not be that simple. Enjoy Christmas, Kel you too. The good news is that real attention on modding Civ5 is not yet in full swing, so one last effort on Civ4 modding is still worth it and not wasted.

Cheers

Come to think of it, I personally haven't had CTD issues with Python. Always XML issues or DLL issues. Python always either reports an error or just freezes the game, but does not CTD to the desktop. If the game freezes, is the core processor still maxed out trying to compute something or has it given up? If it straight CTD's to the desktop, always for me it has been either a DLL fault, an XML fault or a compiler problem.
Cheers

hey glider,

glad to see you here.

even though i usually blame python, your right about alwats getting an error, and nit necessarily a ctd.

long time ago i did merged the two, but i had issues with it.
might i add, the first dune build where based on revdcm, i only wish deliverator here will be able to bring it back.

Eh Kel
Yeah would be a pity not to have Dune up and running. Is there any unusual XML or graphics infrastructure in Dune that could cause the BTS engine to hang? The function Deliverator talks of CvPlot::updateRouteSymbol() is outside of my familiarity but wouldn't want to get distracted by the break point there because often the actual problem is somewhere else as Deliverator realises.
Cheers

Deliverator said:

This code hasn't changed at all so I think there is some memory/initialisation problem somewhere and this is just how it is manifesting itself.

Click to expand...

The code setLayoutDirty(true); doesn't look right. From what I can tell, this code should be accessed by the indirect operator (->) since a pointer is being used to point to the data (if I am correct). The comment "//pAdjacentPlot->setLayoutDirty(true);" is what makes me think something is missing. This would explain the memory access violation. VS 2010 (w/ Vista/Win 7) will always crash when encountering these kind of errors. VS 2005 (and XP) do not flag them (thus the unpredictable behavior). Basically, the code is saying (an undefined pointer which is NULL or garbage)->setLayoutDirty(true); Also, the indirect operator normally accesses class data or structures (e.g. RECT struc) (the main benefit being you can access class data without get/set functions {but you do have to retreive the pointer though}). I hope this helps narrow down the problem. I would help more, but I am coding my own game engine, so I already have plenty of bugs to trouble shoot.

Building on glider1's comment, it may be worthwhile to try a build 2b, which has all the xml and none of the python. If that crashes, which seems likely to me also, then maybe peeling back half the xml, then another half, etc would reach a stable point and at least highlight which type of xml is causing the problem. Putting in any python without the xml probably wouldn't work because so much of the python uses object numbers defined in the xml.

I made a mistake, Build 2 doesn't have any Dune Wars Python. So the issue does seem to be XML related. This makes sense since as glider1 says I haven't seen many CTDs caused directly by Python.

If that crashes, which seems likely to me also, then maybe peeling back half the xml, then another half, etc would reach a stable point and at least highlight which type of xml is causing the problem.

Click to expand...

This is the best way forward, but it is pretty tricky add in entities bit by bit with a total conversion mod like this. Almost everything is interdependent. It definitely seems to be an XML issue though - I'm using the RevDCM DLL which works fine with different XML.

Do you have the game option "Lock Modified Assets" checked, or are you trying to start up a multiplayer game (which automatically locks modified assets)? If so, for some reason things screw up when there are files in the custom assets folder, which we can't figure out why, since RevDCM's ini file specifically says not to load custom assets... but it happens.

That function you that is crashing is unaltered from Base BTS, I suppose you need to figure out where it is getting called from maybe, since it, itself is obviously not the problem.

When you use the debug DLL, and the visual studio breaks on the crash, you can also see the 'call stack', which is basically the chain of function calls until the point of the crash.
If it's not visible, try the 'Debug' menu under 'Windows->Callstack'.

This can you give a hint on what's the problem.

When in break mode, you can also double click on other places in this stack to go to the correct place in code. You can then check the values of variables either by hovering over them or by looking at the 'locals' or 'autos' windows.

Try posting the callstack. It might give more hints.

Jester Fool said:

The code setLayoutDirty(true); doesn't look right. From what I can tell, this code should be accessed by the indirect operator (->) since a pointer is being used to point to the data (if I am correct). The comment "//pAdjacentPlot->setLayoutDirty(true);" is what makes me think something is missing.

Click to expand...

The '//pAdjacentPlot->setLayoutDirty(true)' you're referring to is in a loop going over adjacent plots and calling setLayoutDirty() for each of them. Here it's calling it for this plot, so there's no need for the indirection.

Nice one Asaf. The call stacks are great thanks for reminding me. If Deliverator sets the break point at the call to this setLayoutDirty() in the CvPlot object and that is the last that is ever heard of proper execution, and if we intuit that it is not a python problem, and we realise that the final break point is not necessarily the source of the problem but a possible indicator of the incompatible use of the class somehow, but we also realise that the setDirty() mechanism used recursively as it is here, could possibly be suggesting some corruption in the memory allocation within plot structures, you could take a guess and say that it is something to do with Dune's alternative XML definitions/resources for the plots themselves. You could then forget about all the other types of XML definitions and just focus on plots, focusing on anything in the XML that links plot resources together into groups of plots. Is there such an arrangement?

Of course ideas, like this probably just lead up the garden path to know where, and a methodical approach, systematically singling out the problematic XML, might end up with a quicker positive result.

Best wishes for Dune. Intermittent issues are a total @#$@
Cheers

I decided to change tack and have successfully rebuilt Dune Wars using Fuyu's Better BUG AI as a base. I'll post the new version tomorrow. Better BUG AI isn't fixed for multiplayer, but starting with a base that has only the stuff needed for Dune Wars seemed a better way to go. I'll post more about it tomorrow.