Strange CTD on cityconquest

[OrionVeteran]

If you want to add a little more safety to the function AI_unitUpdate, may I suggest:

def AI_unitUpdate(self,argsList):
	pUnit = argsList[0]
	# Absinthe: start Inquisitor AI from Charlemagne (based on SoI)
	iOwner = pUnit.getOwner()
	[COLOR="Blue"]AIpPlayer = gc.getPlayer(iOwner)
	if not AIpPlayer.isNone() and not AIpPlayer.isBarbarian() and not AIpPlayer.isHuman() and AIpPlayer.isAlive():[/COLOR]
		if pUnit.getUnitType() == xml.iProsecutor:
			return self.doInquisitorCore_AI(pUnit)
			# Absinthe: end
	return False

Not making those exclusions can cause problems under certain situations.



As for the function doInquisitorCore_AI, may I suggest you check out the OIM.py file in OGI 3.09 to get a few ideas on how to improve the safety on it as well.

 

[AbsintheRed]

I must have an extremely durable install of civ4. I still can't get it to crash regardless of what I do

The kind of errors I get on load makes me stop. They are a clear indication that what I read isn't what was written. Even if I can get the game to load, I would end up with a buggy savegame because something went wrong with the samegame format.

There are 2 more savegames from yesterday:
http://forums.civfanatics.com/showthread.php?p=14247189#post14247189
The first one is from SVN revision 1193, the second is from 1195.
Both starts perfectly for me without those errors, but both crash on my OS after pressing end turn (under some situations).

The second one might be more interesting as this is the comment for it:
"Here's my contribution: SVN 1195 as Venice, Windows 10.
Just conquered Tangeri, plague hit Firense. CTD at the end of turn.
Fortunately no CTD under Windows 7, I could continue this game."
So it works perfectly on Win 7, while there are issues with it on Win 10 (the original uploader) and also on Win 8.1 (for me and another playtester)

If you want to add a little more safety to the function AI_unitUpdate, may I suggest:

def AI_unitUpdate(self,argsList):
	pUnit = argsList[0]
	# Absinthe: start Inquisitor AI from Charlemagne (based on SoI)
	iOwner = pUnit.getOwner()
	[COLOR="Blue"]AIpPlayer = gc.getPlayer(iOwner)
	if not AIpPlayer.isNone() and not AIpPlayer.isBarbarian() and not AIpPlayer.isHuman() and AIpPlayer.isAlive():[/COLOR]
		if pUnit.getUnitType() == xml.iProsecutor:
			return self.doInquisitorCore_AI(pUnit)
			# Absinthe: end
	return False

Not making those exclusions can cause problems under certain situations.



As for the function doInquisitorCore_AI, may I suggest you check out the OIM.py file in OGI 3.09 to get a few ideas on how to improve the safety on it as well.

Thanks for the help!
Will look into that part of OGI too

 

[AbsintheRed]

@Nightinggale:
Here is another one, from the same user who tests with both win7 and win10:
http://forums.civfanatics.com/showthread.php?p=14248944#post14248944
Revision 1195 again. But I guess it's not really helpful if the previous one wasn't.
None of the saves crash for you on Windows 7?

 

[Nightinggale]

I will post once I have something new to report. Some event worth reporting would be to actually test the savegames to see if they crash. I haven't tested the last 3 yet

 

[AbsintheRed]

Don't get me wrong, I don't want to hurry you at all
Appreciate every minute you spend on this
Just trying to help you with information, if nothing else.

 

[Nightinggale]

Don't get me wrong, I don't want to hurry you at all

Lyin' Red

Let's be honest. You do want to get this bug fixed ASAP. What you don't want is to put pressure on me. I too would like this fixed. Or to be more precise I really want to understand what is going on. Unless we know the cause of this bug, we might very well be doomed to repeat it over and over. If it is possible to write a line of code, which works in Windows 7, but now Windows 10, then I would really want to know what that line is.

The main reason why I haven't tested it yet isn't lack of will. It's more like lack of outside disturbance. It's pointless to look at debugger output if I know I'm not alert enough to spot the bug.

I just started wondering about something. What if the bug is in the compiler optimization? If it is, then it will never crash with the debug DLL, but the compiler's eager to speed up stuff corrupts something. It's rare, but it does happen and I have tracked down a problem to this very problem (though it wasn't for modding). It might be something to consider if I keep failing to crash the game with the debug DLL.

Maybe it would be a good idea to give a debug DLL to people, who experience the crash and see if it still crashes and take note of the asserts (if any). Related note: you can copy paste the assert text, which works much better than screenshots of the assert window. Also looking through asserts works better in window mode.

 

[AbsintheRed]

Lyin' Red

Let's be honest. You do want to get this bug fixed ASAP. What you don't want is to put pressure on me.

Of course I want this fixed ASAP, that goes without a question
Still, there is no real hurry. Development for RFCE goes strongly in the meantime.
Also the bug is there for at least 1/2 a year, so while it's annoying, it doesn't discourage too many people from playing the mod. At least I hope it's true.

On the other hand, you are probably also right.
Since RFC mods are still quite popular (fortunately), this does affect lots of people.
I got around 500 individual downloads in the first 2 weeks after the release of version 1.3, and this is not counting any dls connected to various SVN versions.
So all the players are eagerly waiting for a fix, to get rid of this annoyance for good.

I too would like this fixed. Or to be more precise I really want to understand what is going on. Unless we know the cause of this bug, we might very well be doomed to repeat it over and over. If it is possible to write a line of code, which works in Windows 7, but now Windows 10, then I would really want to know what that line is.

It's great that you are also enthusiastic about it
I'm sure it will be fairly easy to find it (when you manage to reach a CTD) with your experience connected to the .dll.
Second your curiosity too, now it's increasingly sure that the crash always never appears on any Win 7 systems, while it occurs fairly often on both Win 8.1 and Win 10.

The main reason why I haven't tested it yet isn't lack of will. It's more like lack of outside disturbance. It's pointless to look at debugger output if I know I'm not alert enough to spot the bug.

Sure, take your time.
You surely know by now when you are in a debugging mood

I just started wondering about something. What if the bug is in the compiler optimization? If it is, then it will never crash with the debug DLL, but the compiler's eager to speed up stuff corrupts something. It's rare, but it does happen and I have tracked down a problem to this very problem (though it wasn't for modding). It might be something to consider if I keep failing to crash the game with the debug DLL.

Maybe it would be a good idea to give a debug DLL to people, who experience the crash and see if it still crashes and take note of the asserts (if any). Related note: you can copy paste the assert text, which works much better than screenshots of the assert window. Also looking through asserts works better in window mode.

I test most savegames with a debug .dll too.
It crashes for me with that dll too.
And I never experiened any special assert failures close to the crash, only the ones which appear in other situations too.

 

[AbsintheRed]

Maybe it would be a good idea to give a debug DLL to people, who experience the crash and see if it still crashes and take note of the asserts (if any).

Still thinking about this:
If you think it would be better to have additional information from other people too I will try to ask a couple playtesters.
For me it always crashes with the debug dll too, as I said in my last post. Could there be a difference if other people test it too?

Related note: you can copy paste the assert text, which works much better than screenshots of the assert window. Also looking through asserts works better in window mode.

Ohh, I didn't answer to this part before.
I already copy all the assert to have notes about them, and always run the game in windowed mode when trying to debug, even when simply running it with the debug .dll.

These are all the currently appearing asserts I'm aware of:

Spoiler :

# very common, appears right on start
Assert Failed

File: C:\Dev\CvGameCoreDLL_RFCE\CvGameCoreDLL\.\CvUnitAI.cpp
Line: 13134
Expression: !atPlot(pBestPlot)
Message:

# very common, appears right on start
# info from Nightinggale in the CTD thread
Assert Failed

File: C:\Dev\CvGameCoreDLL_RFCE\CvGameCoreDLL\.\CvArea.cpp
Line: 644
Expression: getPower(eIndex) >= 0
Message:

#appears right on load for the 500AD scenario
Assert Failed

File: C:\Dev\CvGameCoreDLL_RFCE\CvGameCoreDLL\.\CvTeam.cpp
Line: 2409
Expression: iCount >= getAtWarCount(bIgnoreMinors)
Message:

#Vassalage problem, very rare
Assert Failed

File: C:\Dev\CvGameCoreDLL_RFCE\CvGameCoreDLL\.\CvDeal.cpp
Line: 1000
Expression: false
Message:

#rare - only one instance in each game. usually around 740 AD? or around 980 AD?
#change setAtWar to declareWar?
Assert Failed

File: C:\Dev\CvGameCoreDLL_RFCE\CvGameCoreDLL\.\CvTeam.cpp
Line: 1279
Expression: !(AI_isSneakAttackPreparing(eTeam))
Message:

#rare - only one instance in each game? new in revision 1195??
Assert Failed

File: C:\Dev\CvGameCoreDLL_RFCE\CvGameCoreDLL\.\CvUnitAI.cpp
Line: 11742
Expression: !(pBestCity->at(pBestPlot))
Message:

#this one is definitely harmless
#only stating that you have more units from a given type than the max you should have (probably through events like reformation or crusades)
Assert Failed

File: C:\Dev\CvGameCoreDLL_RFCE\CvGameCoreDLL\.\CvPlayer.cpp
Line: 11980
Expression: getUnitClassCount(eIndex) <= GC.getUnitClassInfo(eIndex).getMaxPlayerInstances()
Message: getUnitClassCount is expected to be less than maximum bound of MaxPlayerInstances (invalid index)

There are also 5 other assert for which the cause was fixed recently.
2 based on your findings, and 3 which I found in the last couple weeks.

 

[Nightinggale]

It would seem that my reputation as a bug killer precedes me. Whenever I go near that bug, it hides very well. I still can't get any of the savegames to crash

 

[retired]

I just started wondering about something. What if the bug is in the compiler optimization? If it is, then it will never crash with the debug DLL, but the compiler's eager to speed up stuff corrupts something. It's rare, but it does happen and I have tracked down a problem to this very problem (though it wasn't for modding). It might be something to consider if I keep failing to crash the game with the debug DLL.

Maybe it would be a good idea to give a debug DLL to people, who experience the crash and see if it still crashes and take note of the asserts (if any). Related note: you can copy paste the assert text, which works much better than screenshots of the assert window. Also looking through asserts works better in window mode.

There are a few things I learned in the past like.
Not doing a clean compile after making changes can cause odd bugs because of the compiler optimization in non debug builds.

Or code like

CvPlayer* pPlayer;

......

if (pPlayer != NULL )
    pPlayer->Somefunction();

works fine in the Debug build but it can sometimes fail in a Release build. It is necessary to initialize such pointers to NULL to avoid odd crashes.

CvPlayer* pPlayer[B] = NULL[/B];

........

if (pPlayer != NULL )
    pPlayer->Somefunction();

 

[Nightinggale]

Not doing a clean compile after making changes can cause odd bugs because of the compiler optimization in non debug builds.

Makefile 2 keeps a set of temp files for each target (release, assert, debug, profile) to avoid this issue. Makefile 1 doesn't have this separation and yes it can under some circumstances mix release and debug temp files.

It is necessary to initialize such pointers to NULL to avoid odd crashes.

Ideally the debug build should warn about using uninitialized variables. The same goes for the compiler. One thing, which would avoid a bunch of bugs would be to use warning level 4 and fix all warnings. The problem is that boost has a bunch of warnings, meaning warning level should be changed in CvGameCoreDLL.h after including the library headers, but before including the DLL headers. Vanilla has a bunch of warnings as well, which can be fixed.

Maybe I will make this as a ModComp at some point.

 

[AbsintheRed]

@ alberts2:
I don't think there might be any issues connected to that, because I always rebuild the dll.
Btw I use the newer makefile. I wasn't sure before, but Nightinggale confirmed that there shouldn't be any compiling issues anyway with makefile 2 when switching between different options.
Also, those savegames crash with both the release and the debug dll.

The difference is between different OSs, interestingly.
There are even reports that on Windows 10 running Civ IV in a Windows 7 compatibility mode results in less crashes. One of the playtesters even claimed that it totally prevents crashes, some others said that while some CTDs were indeed gone, some still happened.
(we were testing with some specific savegames which we knew were connected to this bug/crash)

 

[AbsintheRed]

It would seem that my reputation as a bug killer precedes me. Whenever I go near that bug, it hides very well. I still can't get any of the savegames to crash

Is there any other way I can help you?
Other than pointing out newer crashing savegames, and hoping that you also reach a crash with one of them somehow

 

[retired]

@ alberts2:
I don't think there might be any issues connected to that, because I always rebuild the dll.
Btw I use the newer makefile. I wasn't sure before, but Nightinggale confirmed that there shouldn't be any compiling issues anyway with makefile 2 when switching between different options.
Also, those savegames crash with both the release and the debug dll.

The difference is between different OSs, interestingly.
There are even reports that on Windows 10 running Civ IV in a Windows 7 compatibility mode results in less crashes. One of the playtesters even claimed that it totally prevents crashes, some others said that while some CTDs were indeed gone, some still happened.
(we were testing with some specific savegames which we knew were connected to this bug/crash)

C2C had a few pieces of code like the one that i posted, but they only caused crashes for very few people. I wasn't able to replicate some of those crashes on my system. Things like another OS, CPU or overclocking can make a difference in such situations. I'am not surprised that the Windows7 compatibility mode fixes some of those crashes on Windows10 since it was built into Windows to do that exactly. But obviously Ms didn't fix all compatibility issues.

 

[retired]

I took a look at your problem.

The crash is in CvSelectionGroup::groupMove and your mod isn't the first one that had that problem. I added the marked piece of code from C2C and the crash is gone. This is proof that it has nothing to do with Windows 10 directly because Koshling made that change in C2C almost 3 years ago.

void CvSelectionGroup::groupMove(CvPlot* pPlot, bool bCombat, CvUnit* pCombatUnit, bool bEndMove)
{
	PROFILE_FUNC();

	CLLNode<IDInfo>* pUnitNode;
	CvUnit* pLoopUnit;

	pUnitNode = headUnitNode();

	while (pUnitNode != NULL)
	{
		pLoopUnit = ::getUnit(pUnitNode->m_data);
		pUnitNode = nextUnitNode(pUnitNode);

[B][COLOR="Red"]		if (pLoopUnit == NULL)
		{
			//	Koshling  - this really isn't a valid condition I think but it's been seen
			//	and until we can understand the underlying cause, mitigate by ignoring the
			//	invalid unit
			continue;
		}[/COLOR][/B]

		if ((pLoopUnit->canMove() && ((bCombat && (!(pLoopUnit->isNoCapture()) || !(pPlot->isEnemyCity(*pLoopUnit)))) ? pLoopUnit->canMoveOrAttackInto(pPlot) : pLoopUnit->canMoveInto(pPlot))) || (pLoopUnit == pCombatUnit))
		{
			pLoopUnit->move(pPlot, true);
		}

 

[AbsintheRed]

Wow, wonderful news
Will try it out ASAP

Btw we didn't thought that it's connected to Win 10 directly
It's really strange that it never crashes on Win 7, just on 8.1 and 10, but I guess it's just connected to different memory handling between them.
I really wonder what can be the real cause, how the pLoopUnit can be NULL in the first place.

 

[AbsintheRed]

I too would like this fixed. Or to be more precise I really want to understand what is going on. Unless we know the cause of this bug, we might very well be doomed to repeat it over and over. If it is possible to write a line of code, which works in Windows 7, but now Windows 10, then I would really want to know what that line is.

If you still want to get a crashing savegame, as you would like to take some further looks on this, here are a couple more.
These 3 are from revision 1174. Tested all 3 with both a release and debug dll from that revision. All of them start without any issues, and crashes for me after hitting end turn with both dlls.
http://forums.civfanatics.com/showthread.php?p=14251782#post14251782
http://forums.civfanatics.com/showthread.php?p=14249691#post14249691
http://forums.civfanatics.com/showthread.php?p=14250435#post14250435

And here is one from revision 1197
But this didn't even crash for me, so didn't test it with a debug dll
http://forums.civfanatics.com/showthread.php?p=14249475#post14249475

 

[retired]

Wow, wonderful news
Will try it out ASAP

Btw we didn't thought that it's connected to Win 10 directly
It's really strange that it never crashes on Win 7, just on 8.1 and 10, but I guess it's just connected to different memory handling between them.
I really wonder what can be the real cause, how the pLoopUnit can be 0 in the first place.

There can be a few different things causing that and there are alot places in the dll like this one, everyone of them can crash the same way. In one of the saves that i tried the nextUnitNode call should have returned NULL but it didn't for some reason.

 

[AbsintheRed]

Added that check in CvSelectionGroup::groupMove to revision 1174 to test the 3 savegames above.
It seems to have fixed the crash itself.
On the other hand, there are various issues appearing in those games.
Surely it has to do something with the real cause of the CTD, causing problems in the background.
(EDIT: As i have no clue how that pLoopUnit can be NULL, there is also a slight chance that adding the check is the real cause for these issues. But I find that highly unlikely)

It is the most noticable with the Scottish save from 1550
http://forums.civfanatics.com/showthread.php?p=14251782#post14251782
The crash occured on the Swedish conquest of Varberg (Scandinavia)
Now it proceeds to the next turn, but strange units appear there.
These units are not even constantly the same, the number greatly varies from time-to-time, also the civ which are these units for.
Also the city changes ownership at least 3-7 times, at least it was always between that when I checked in the changelog.
The ownership of the city in the end also differs.

Units also appear under odd circumstances in this one:
http://forums.civfanatics.com/showthread.php?p=14250435#post14250435
The crash was on Spain conquering Sevilla.
Had only 2 tries, once 4-5 independent units appeared out of nowhere, the other time the same amount of barbarians.
Conquered/razed the now Spanish city on the next turn.

You guys didn't notice any strange behaviour after Koshling added that check to C2C?

 

[retired]

I didn't look too closely this morning so I missed those issues. It sounds like the memory was somehow corrupted and that made the nextUnitNode call return something instead of NULL. I take another look later today.