Infinite turn / grouping loop (Tester.py)

Asserts report while running the mod (unfortunately, none will solve the big problem):

1) In GlobalDefines.xml, NUM_BUILDING_PREREQ_OR_BONUSES is set at 4. You have two buildings that require more than 4 bonuses. The fifth bonus is then ignored.

2) Two buildings have FLAVOR_ECONOMY which doesn't exist in GlobalTypes.xml.

3) In BuildingClassInfos, for the BUILDINGCLASS_PUBLIC_SCHOOL, the Default building is spelled wrongly with the last L not capitalized. EDIT: no text, nor art available but an ART_DEF entry => pink button.

4) In CivilizationInfos, the derivative Civ for Germany is Holy Rome which does not exist in your mod.

Assert 5) I leave to Nightinggale :p because I don't get it.

It's in CvXMLLoadUtilitySet.cpp, line 1605: ... bSuccess...OWN TYPE - dependency not found... file NONE.

- - - - - - - - -- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Tell me if the new version of Tester.py works for you, in another game.

For your 2 save games, they now crash on me to Desktop when I hit the End of Turn button (this even with your original dll). I still don't understand how I could continue the two games yesterday. As you hinted, the problem might not be an infinite loop, or something else is also bugging the game. I guess I'll leave it to Nightinggale for now.

- - - - - - - - -- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Edit: Assert 6) a bit trivial but the map used for your savegames has on plot x=68,y=36 a RiverWEDirection=0. It is incorrect as 0 is North. It should be either 1 (East) or 3 (West).

Assert 7) [CvPlayer.cpp line 14844] About Improvement Yield Change and Yield Types. Strange: removing both the Improvements and Yields files from your mod (to revert to BtS) still triggers the assert (I was using a WBS save out of your savegames).
 
It's in CvXMLLoadUtilitySet.cpp, line 1605: ... bSuccess...OWN TYPE - dependency not found... file NONE.
It appears to be modded code to handle modular loading. Looks like you didn't copy paste the text correctly though as it should say "in file", not just file. It would appear that it fails to find type "" in file NONE :dubious:. What that mean is... well your guess is as good as mine. Hopefully the debugger can reveal more info.

The code isn't perfect btw. It sets the string in one line and then it is used for assert in the next. Placing both in the same line would make the compiler skip creating the string when asserts are disabled. Also the assert is checked twice, meaning it will create two identical asserts for each condition to trigger this assert. The code should still work though. It's just not what I would classify as well written.

Also setting up the project files for makefile 2.5 seems to be a good idea. Playing with an assert dll would have told about those asserts long ago.

For your 2 save games, they now crash on me to Desktop when I hit the End of Turn button (this even with your original dll). I still don't understand how I could continue the two games yesterday. As you hinted, the problem might not be an infinite loop, or something else is also bugging the game. I guess I'll leave it to Nightinggale for now.
Luckily it makes no big difference if it is a crash or infinite loop. Both events are something well defined the tools can detect and reveal the precise lines where it goes wrong. Why it goes wrong is a different story and might not be equally easy to figure out.
 
Well, let me thank you (plural) a lot for all the effort you put into this - it's very much appreciated, especially as this all must take a lot of time!

I highly suspect there is one, or a set of, issues underlying these crashes; they pop up in every single game I launch and, as a result, all games become practically unplayable at around the exact same time.

I wonder what will happen if I start a game in the modern age... Let me test that, and see how it goes. Perhaps the fact that movement speed of later naval units is larger is relevant?

Regarding isenchine:

1) I put that tag, and similar ones, at 10 now. I wonder if there's a reason for not putting them that high - why not have them at 100, for example?
2&3) Thank you, though these buildings weren't meant to be buildable / acquirable anyway - but still, for cleanness' sake. :)
4) Ah, it's HOLY_ROMAN, thank you!

The Alt pop ups worked - but as far as I can tell, that's the only thing. 'Course, that is probably related to my weird crash; a true infinite turn bug would allow me to move the camera around the map fluently, and the game wouldn't crash, in my experience.

6) I really should get around to modifying the Earth map to my liking, as opposed to edit the map before I start playing every single time. :p
 
1) I put that tag, and similar ones, at 10 now. I wonder if there's a reason for not putting them that high - why not have them at 100, for example?
Disclaimer: I didn't actually read the code to answer this question. It's more like a general answer to the concept.

If the code demands a fixed global limit like that, then it is because it will allocate an array of that size regardless of how many bonuses it will add. This mean setting it to 100 will allocate 100 bonus slots, which is a waste of memory. Much worse is it if there is code to loop the array and it will spend CPU time checking each element in the array.

I think that is a likely worst case scenario for using a higher number than absolutely required. Personally I would have asked xml for the number of children (in this case bonuses) and then made an array of that size and possibly add a variable to store the length. This way the dll will figure out which number to use rather than reading it from globalDefineALT, making it easier to use for xml modders. However it would likely make the dll code more complex as in harder to write and having a global value might benefit in some other way as well (though I can't think of such a benefit right now).

Conclusion: keep asserts on and use as small numbers as possible. That is likely the best solution, at least for now.
 
I set up the project files to use makefile 2.5 and attached them to this post. Notice that the path is set to place the project files directory in the mod root dir, not inside the source dir itself. I had to rename the source dir to RevDCM_source due to apparently not supporting spaces in the path to the source.

The perl script is optional to use. It sets up the project to open the game and load the mod from inside MSVC. This mean F5 becomes a hotkey to run the game where it automatically attaches the debugger. This only works if the mod is placed inside the game mods (not my documents) and naturally only if perl is installed.

I went through a lot of debugging and encountered no less than 3 errors, two of those resulting in crashes. For the rest of the post, bold is added text, while crossed out text is removed.

Assert 5) I leave to Nightinggale :p because I don't get it.

It's in CvXMLLoadUtilitySet.cpp, line 1605: ... bSuccess...OWN TYPE - dependency not found... file NONE.
It turns out to be about the civic xml file. After it loads the last civicinfo, it detects more lines and go into "load next element" mode. However it fails to find it because the end is nothing but comments, which causes it to return false as it didn't load anything, which then triggers the assert. The assert message in this case is absolutely bogus.
Solution: remove the commented out civicinfos in the end of the file.


CvTeamAI::AI_techTrade
Problem: fails to set eCurrentResearch if team 0 is the strongest team, resulting in undefined behavior.
Solution: (warning, not really tested)
Code:
TechTypes eCurrentResearch [B]= NO_TECH[/B];
[S]TeamTypes eStrongestTeam = (TeamTypes)0;[/S]
[B] int iMaxPower = MIN_INT;[/B]
for (iI = 0; iI < MAX_PLAYERS; iI++)
{
    if (GET_PLAYER((PlayerTypes)iI).getTeam() == getID())
    {
        if (GET_TEAM((GET_PLAYER((PlayerTypes)iI)).getTeam()).getPower(true) > [S]GET_TEAM(eStrongestTeam).getPower(true)[/S])
        if (GET_TEAM((GET_PLAYER((PlayerTypes)iI)).getTeam()).getPower(true) > [B]iMaxPower[/B])
        {
            eCurrentResearch = GET_PLAYER((PlayerTypes)iI).getCurrentResearch();
            [S]eStrongestTeam = GET_PLAYER((PlayerTypes)iI).getTeam();[/S]
            [B] iMaxPower = GET_TEAM(GET_PLAYER((PlayerTypes)iI).getTeam()).getPower(true);[/B]
        }
    }
}


CvCityAI::AI_trainInquisitor
Problem: reading unitinfo using index -1 (crash)
Solution:
Code:
for (iI = 0; iI < GC.getNumUnitClassInfos(); iI++)
{
	eLoopUnit = (UnitTypes)kCivilization.getCivilizationUnits((UnitClassTypes)iI);
	if ([B]eLoopUnit == NO_UNIT ||[/B] !GC.getUnitInfo(eLoopUnit).isInquisitor())


AI_doTurnUnitsPosts
CvPlayerAI.cpp line 796 (big function)
Problem: reading combat type with type -1 (crash)
Solution:
Code:
iCityExp += pPlotCity->getFreeExperience();
iCityExp += pPlotCity->getDomainFreeExperience(pLoopUnit->getDomainType());
[B]if (pLoopUnit->getUnitCombatType() != NO_UNITCOMBAT)
{[/B]
	iCityExp += pPlotCity->getUnitCombatFreeExperience(pLoopUnit->getUnitCombatType());
[B]}[/B]


I sure hope to use git next time I have to help. Keeping track of what I'm doing without having git to assist me is... well it's a burden compared to how I usually work.
 

Attachments

Interesting! :goodjob:

But it looks to me that you found loopholes in codes and as long as nobody tries to enter through these doors, it's all fine.

Considering that need my speed is probably playing with the RevDCM dll since years (without being able to modify it) and that the problems really only happened recently, then he must have found the doors by doing something unusual in the XML. I'd like to understand what exactly.

A few things to clarify:

- Assert 7 is still there (might be minor but we should know for sure).

- I'm not sure if the source files we have match the dll he is using. I guess there's no real way to find out except play and play again with both dlls!

For me, after recompiling a debug dll with Nightinggale's three changes, the two saves still give an error in the MainInterface.py. It has to do with:
Code:
CyInterface().getActionsToShow()
. With that, I can't test them further.

need my speed: do you remember having done something recently about these "Actions"?

A last note on the Inquisitor (since CvCityAI::AI_trainInquisitor was mentioned): it's currently the last unit, but before it there is the UNIT_GOVERNOR which is commented out and has a UNITCLASS_GOVERNOR with a default UNIT_GOVERNOR wich exists only in green! I would correct this before starting a new game. I think it's best to just put iCost as -1 if you are not ready to use it but want to keep it just in case.

Conclusion: at this stage, I might give you the revised dlls (release and debug) for you to test further. I'm not ready to do anything with Nightinggale's attachment though.

Annex: RevDCM_Revised_DLLs + 3 revised source files as per Nightinggale's suggestions 1 post above [DELETED: obsolete]
 
But it looks to me that you found loopholes in codes and as long as nobody tries to enter through these doors, it's all fine.

Considering that need my speed is probably playing with the RevDCM dll since years (without being able to modify it) and that the problems really only happened recently, then he must have found the doors by doing something unusual in the XML. I'd like to understand what exactly.
They are all sort of related to xml. Trigger conditions are like this:

civic xml: add comments at the end of file. It looks like this limitation applies to all xml files.

AI_techTrade: the code will silently fail and sometimes the AI will not avoid trading to gain the tech it is currently researching. Not a major failure and the player would likely never notice. The debugger triggered on undefined behavior at runtime though.

CvCityAI::AI_trainInquisitor: The civ xml have to set the unitclass to become NO_UNIT to trigger this one.

AI_doTurnUnitsPosts: a unit should have unitcombat set to NONE in xml, though I think there are more demands to the unit than that.


While those require certain xml conditions to trigger, it is still dll bugs and should be fixed in the dll. They are sort of classic bugs in the sense that they request the xml index of the type without considering what should happen if the type is NONE (-1). I debugged another mod not long ago where it crashed because it looped all plots in a city's catchment area and the code forgot that the pointer to the loop plot would be NULL if the plot was outside the map. That's sort of the same thing.

Considering that need my speed is probably playing with the RevDCM dll since years (without being able to modify it)
Wait a minute. You mean to tell me this is an unmodified dll several people used for years and it has undetected crashing bugs? :crazyeye:

If this is the case and need my speed didn't even compile the dll, then it would make the most sense to go back to the source and apply the fixes there. If it's abandoned, it would make sense to "take over" and start a git server to keep track of the code changes. While I don't intend to mod this dll, we should keep track of the bugfixes and distribute them, preferably as a compiled dll.
 
Indeed, I use the unmodified .dll from RevolutionDCM, for I have never, in all these years, managed to compile a new .dll. I really should try that again, at some point. :p

Anyway, by using your .dll, I get this error. It is, however, entirely likely that I'm using it wrongly; I simply replaced my own CvGameCoreDLL.dll with yours.

I haven't yet started on a new game - alas, time! And time will only grow sparser... - but I have removed the Governor, the Empire-related buildings, and the Public School in their entirety. Who knows, perhaps this might help. If I ever get around to testing it. :p

Thank you again, though, as ever! :)
 
Indeed, I use the unmodified .dll from RevolutionDCM, for I have never, in all these years, managed to compile a new .dll. I really should try that again, at some point. :p
Being able to compile is an essential skill, particularly if you ask other people to fix dll bugs for you. The project files I added are designed to be plug n play if you have installed the compiler. Also it requires you to download jom if you want to compile using more than one CPU core (not strictly needed with your compile frequency).

Anyway, by using your .dll, I get this error. It is, however, entirely likely that I'm using it wrongly; I simply replaced my own CvGameCoreDLL.dll with yours.
Replacing should be ok, but I don't make much sense of the error.
Code:
actions = CyInterface().getActionsToShow()
That line calls a function, which is hidden inside the exe. This function fails for an unknown reason and without access to the source code for the function, it is hard to tell precisely what is wrong. Most likely the function calls something in the xml to get data based on the xml files and somehow your xml setup puts the function into some undefined state the programmers didn't consider.

The problem is what precisely triggered this problem and why? The code seems to keep this as a well guarded secret.

I haven't yet started on a new game - alas, time! And time will only grow sparser... - but I have removed the Governor, the Empire-related buildings, and the Public School in their entirety. Who knows, perhaps this might help. If I ever get around to testing it. :p
Adding, removing and/or reordering xml elements have a nasty habit of breaking savegames, particularly those containing the Type tag. Also vanilla (and most mods) tend to save cached version of xml data. If you change something in xml and your savegame become weird or strange error messages starts to appear, then try to start a new game and see if the cleared cache fixes the problem. I know it is annoying, but fixing this modder unfriendly design is rather time consuming.


Also this talk about being unable to compile makes me wonder. What is the history of this mod? Is it personal modifications or is it "real development" as in it is intended for release? and what precisely is the "parent" mod, which delivered the compiled dll? While I tried plenty of mods, I can't recall playing this one. It might be my fault, but I still want to figure out what hornet nest I got my hand into ;)
 
Code:
actions = CyInterface().getActionsToShow()
That line calls a function, which is hidden inside the exe. This function fails for an unknown reason and without access to the source code for the function, it is hard to tell precisely what is wrong. Most likely the function calls something in the xml to get data based on the xml files and somehow your xml setup puts the function into some undefined state the programmers didn't consider.

The problem is what precisely triggered this problem and why? The code seems to keep this as a well guarded secret.

I've stumbled upon this all day. That's why I asked need my speed if he has done something recently about "Actions"
Spoiler :
(see for example CvXMLLoadUtilitySet.cpp(1242): piActionInfoTypeList[iTotalActionInfoCount] =
ACTIONSUBTYPE_COMMAND;
ACTIONSUBTYPE_INTERFACEMODE;
ACTIONSUBTYPE_BUILD;
ACTIONSUBTYPE_PROMOTION;
ACTIONSUBTYPE_UNIT;
ACTIONSUBTYPE_RELIGION;
ACTIONSUBTYPE_CORPORATION;
ACTIONSUBTYPE_SPECIALIST;
ACTIONSUBTYPE_BUILDING;
ACTIONSUBTYPE_CONTROL;
ACTIONSUBTYPE_AUTOMATE;
ACTIONSUBTYPE_MISSION;.

Revolution/DCM is a classic around here.
 
Aye, I cleaned up the XML files and made a few changes here and there. But, using my old .dll, suddenly, I can't even settle a city, nor spawn it in the World Builder.

The new files - the whole mod, source code, and log files - are here: https://dl.dropboxusercontent.com/u/104521722/RevolutionDCM.rar

But this error is a curious one.
When I settle a city, I get this: http://i.imgur.com/rRhBybP.png
When I place a city in the World Builder, I get this: http://i.imgur.com/6xwCaum.png

And I wonder if this is related to it: http://i.imgur.com/NKyi2B3.png

Nightinggale, what would you want me to do? Compile a .dll, from the source code? But, isn't that simply the currently existing CvGameCoreDLL.dll?

And, you know you can just quit this, right? I wouldn't want to impose myself upon you! :)

EDIT: Not as far as I know, no... I do not believe I changed any kind of action like that? Could it have to do with me adding so many new civics?
 
No worries, I solved the city-settling issue; turns out you can't put the maximum amount of trade routes to -1 (and neither probably the maximum amount of distance maintenance).

EDIT: And by the way, this is a private mod, intended for private use - I mean, I could release it, but, to what end? I might share some civilisations I've made at some point in the future I guess... Perhaps the civic system would be of interest... The diplomacy text, too, as I more than doubled it... But it'd all be so much effort to put that as a stand-alone package :p - but, it's based on RevolutionDCM.
 
I found the culprit! :D

My question remains the same: do the source files provided correspond to the dll you had? (no one will/can answer - I even re-downloaded Revolution DCM 2.90 to no avail).

In the meantime, I followed a thin trail that led me to compare the "depends" file of the source files and mine. Strangely, CvPopupinfo.h was absent from CvSelectionGroup.cpp. After 123 different tests, I replaced CvSelectionGroup.cpp in the source files by one found in Legends of Revolution (LoR - a ModMod of Revolution, that did not integrate all final changes made to RevolutionDCM).

And it works (probably with some collateral damages) - no more MainInterface problem!

Collateral damages (missing in the new version so far): (Advanced Automation) MISSION_SHADOW & INTERFACEMODE_SHADOW_UNIT.

need my speed, does it ring a bell to you? Do you use that?

Repairing it might be above my league.

Meanwile, I attach a new pack comprising:

- a new Release dll
- a new Debug dll

- 5 source files (3 with Nightinggale's changes, CvSelectionGroup.cpp from LoR with one change of mine (indentation - look for isenchine) and CvEnums.h where I changed a comma from one position to another (no impact - just for aesthetism...))

- 2 WBS versions of your 2 files if you want to continue your games with new XML/DLL: TestRoyBC320 and TestRoyAD060_Later. The last one was screwed due to missing Players (because of Teams - quite complicate) - so I rearranged them but without fully restoring pacts etc.

Nightinggale: I do suppose that you encountered the same MainInterface error and that you don't have different source files? :eek:

need my speed, you now have two options: play with your old dll or adapt to the new one - which might one day become perfect! :D I'm pretty sure that you will be shortly compiling your own dlls! If the environment is correctly set in place, the rest will follow. When you are ready, just ask your questions.

One last piece of advice: be careful with your comments in the XML. The parser can become very touchy about some, as we have seen. I noticed you even placed comments inside the tags of some civics: you are pushing your luck! ;)

PS: don't worry about the time I spent on this: I learned a lot in the process!
 
We have an increasing amount of dll fixes. It might be a good idea to start a git server and publish the fixes and use that one for whatever bugs we might encounter in the future. Surprisingly it seems that there is quite a number of bugs in an otherwise widely used and assumed stable dll :crazyeye:

Nightinggale: I do suppose that you encountered the same MainInterface error and that you don't have different source files? :eek:
A bug involving MainInterface and SelectionGroup sounds like a bug, which triggers if you select a unit. All I did was load and then hit next turn and look into the code the debugger encountered problems with. Once it reached next turn, I quit the game. After all the reported bug was about crashing during the AI's turn. I didn't think that I should have kept playing as unreported bugs would keep showing up.
 
Well, it might not be an increasing amount of dll fixes if the MainInterface problem was the result of my stupidity!

Please share the source files that you compiled OR tell me that your dll also prompts the MainInterface problem (probably not, as the problem was appearing as soon as the game would open the UI, creating tons of popups per second and rendering the game unplayable).

The way I see it is that I eliminated the direct source of the problem. But the MISSION_SHADOW is still in different files and as such, the mechanism is broken. Either we repair it in full or we desintegrate it in full, but that intermediary solution is not good. That's why I would like to be able to check my source files with yours as I'm more and more convinced that I'm responsible for this problem, not RevDCM.

If need my speed wants to open a git server for his private mod, fine. But he might also want to simply add more dll mod components afterwards and deviate from RevDCM.
 
That's OK, I think I spotted where the problem was.

I have no time anymore to look at it today and I won't work on it during the WE, but I'll try to sort it out on Monday. Just consider the above package as temporary.
 
Monday: that's not OK, I give up, sorry.

For what it's worth, I include a package with 2 up to date DLLs and Source files. It still includes the CvSelectionGroup.cpp from LoR as it is the one that works for me, not the one included in RevDCM files. Still then, MISSION_SHADOW & INTERFACEMODE_SHADOW_UNIT are excluded from CvSelectionGroup.cpp.
 
Back
Top Bottom