Single Player bugs and crashes v36 plus (SVN) - After the 24th of October 2015

[Thunderbrd]

Memory allocation failure:

I know this is an old problem, but I am experiencing MAFs every other turn or so with my current game, always at unpredictable moments. The last one occurred when I wanted to save the turn - maf as soon as the save dialogue came up.
I am playing a giant world, 12 civs, normal speed. I am on the latest SVN; I am running the game on Windows 7-64, i5 4690K (normal speed), 16G RAM, Nvidia GTX950 2G.

MAFs are almost impossible to resolve from the point they begin happening. It's enough to know that its taking place and that we need to find a memory leak somewhere. Last I saw this game it was fairly far along.

I'll ask for Alberts2's advisement on how to go about finding the problem.

 

[strategyonly]

The concept of moving all the attack AI units out is sound (they are theoretically preparing to attack once you take the city), but not when it's the last city. This is a known flaw. The code causing this is something I'll be looking to work on fairly soon but it can't be addressed with an 'emergency fix' approach. Needs a lot of deep efforts there to resolve numerous things, including this.

thx, yeah this didnt make any sense at all, why move out if last city, your dead for sure then? weird .

 

[Thunderbrd]

thx, yeah this didnt make any sense at all, why move out if last city, your dead for sure then? weird .

The programming never considered the possibility that it might be looking at the last city of the defending player. The AI is artificial indeed. If we don't consider a possibility when imagining the scenario then the AI won't consider it when facing the scenario.

 

[Pit2015]

Look in my scenario threat please there are some problems, i need to come forward with version 2.5, dormant volcanos disapear in WB after first turn or save game reload? And a problem with oil/gas fields in far away ocean plots, any idear to get that work? Is the dormant volcano problem normal or a bug? Works on first load in WB when no game turn happend, after loading a save game and open the worldbuilder again all dormant volcanos hidden or removed...

Please answer the questions and help to fix the problems for finishing UWM 2.5 in my threat asap: http://forums.civfanatics.com/showthread.php?t=552901

 

[dummy1929]

Please zip and add MiniDump.dmp file for this last CTD/MAF. You will find it in the Main BtS directory. If miniDump is greater than 0 KB it will help. If 0 kb then just say it was a 0 KB because that points to a different problem.

What Difficulty level is your game? And the more Game set up details you can give also helps.

Thanks.

JosEPh

Minidump is attached. My game settings:
Giant Map
Normal Speed
Prehistoric era start
Mastery victory OFF (all other VCs turned on)
Deity level
Nightmare mode
Terrain damage
No revolutions
No barbarian civs
raging barbarians
barbarian world
barbarian generals
aggressive AI
Additional costs for buildings
Surround and destroy
Fight or flight
usable mountains
terrain damage
unlimited wonders
realistic corporations
early buildings
advanced economy
new camouflage mod (don't remember the name)

Thanks!

 

[wowa30]

8920 my save game file after the "end of turn" comes out in windows. You can see all the settings when the save file is loaded

 

[Pit2015]

Look into this savegame file "TEST-GW BC-18750.7z" in http://forums.civfanatics.com/showthread.php?t=552901 page 2, i made all AI civs visible, looks like AI sucks a bit on education now... just klick the civs city and look into the propertys.

 

[Thunderbrd]

8920 my save game file after the "end of turn" comes out in windows. You can see all the settings when the save file is loaded

@any who can help:

void CvUnit::joinGroup(CvSelectionGroup* pSelectionGroup, bool bRemoveSelected, bool bRejoin)
{
	PROFILE_FUNC();

	CvSelectionGroup* pOldSelectionGroup;
	CvSelectionGroup* pNewSelectionGroup;
	CvPlot* pPlot;

	pOldSelectionGroup = GET_PLAYER(getOwnerINLINE()).getSelectionGroup(getGroupID());

	if ((pSelectionGroup != pOldSelectionGroup) || (pOldSelectionGroup == NULL))
	{
		pPlot = plot();

		if (pSelectionGroup != NULL)
		{
			pNewSelectionGroup = pSelectionGroup;
		}
		else
		{
			if (bRejoin)
			{
				pNewSelectionGroup = GET_PLAYER(getOwnerINLINE()).addSelectionGroup();
				FAssert(pNewSelectionGroup != NULL);
				pNewSelectionGroup->init(pNewSelectionGroup->getID(), getOwnerINLINE());

This comes up in the process of initializing a unit. The last line is where it crashes. The only way this happens, if I'm not reading it wrong somehow, is if GET_PLAYER(getOwnerINLINE()).addSelectionGroup():

CvSelectionGroup* CvPlayer::addSelectionGroup()
{
	return ((CvSelectionGroup *)(m_selectionGroups.add()));
}

is coming up with no further available spaces to add another group definition here.

This IS a very very large map size - not sure what size unless I go back and reload and check first but very very large. And the player is of course the Barbarian. So if there's a limit to hit they'd be the ones to hit it.

Now I can think of a few ways it could be addressed:
1) Split animals and barbs - This would destroy save compatibility but maybe it is indeed time to do this.

or

2) Reuse old groups that were initialized but are now destroyed. Much of the AI code runs through new group inits like sheets of toiletpaper and if they're never recycled this will be a problem in any and all games at some point. So we MUST figure out a way to do this and tbh, I'm not sure how to do this myself.

This is the first time I've seen anything suggesting a unit limit being reached... is this what Koshling was indicating? Because this isn't a unit limit itself but a group limit being reached and exceeded. There's a fairly big difference as a unit can take on many differing group definition clothings throughout its time in the game. That said, I'm sure units will eventually have similar issues.


Anyhow, suffice it to say, this game exhibits something other than a basic crash syntax error but rather, something much more. I'd be interested to know how to solve this.

 

[Thunderbrd]

Minidump is attached. My game settings:
Giant Map
Normal Speed
Prehistoric era start
Mastery victory OFF (all other VCs turned on)
Deity level
Nightmare mode
Terrain damage
No revolutions
No barbarian civs
raging barbarians
barbarian world
barbarian generals
aggressive AI
Additional costs for buildings
Surround and destroy
Fight or flight
usable mountains
terrain damage
unlimited wonders
realistic corporations
early buildings
advanced economy
new camouflage mod (don't remember the name)

Thanks!

I'm past the point of being able to use the mini here. Maybe the savegame will prove helpful.

 

[Thunderbrd]

Memory allocation failure:

I know this is an old problem, but I am experiencing MAFs every other turn or so with my current game, always at unpredictable moments. The last one occurred when I wanted to save the turn - maf as soon as the save dialogue came up.
I am playing a giant world, 12 civs, normal speed. I am on the latest SVN; I am running the game on Windows 7-64, i5 4690K (normal speed), 16G RAM, Nvidia GTX950 2G.

The save from the last round is corrupt and doesn't open.

However, this game does exhibit an unusual problem and I don't know what to make of it.

I get this assert:

Assert Failed

File:  CvGameCoreDLL.cpp
Line:  721
SVN-Rev:  8922
Expression:  bIsMainThread || iThreadSlot == -1
Message:  

----------------------------------------------------------

leading to a line in:

bool IFPPythonCall(const char* callerFn, const char* moduleName, const char* fxnName, void* fxnArg, long* result)
{
	PROFILE("IFPPythonCall2");

#ifdef USE_INTERNAL_PROFILER
	FAssert(bIsMainThread || iThreadSlot == -1);
	if ( !bIsMainThread )
	{
		::MessageBox(NULL, "Illegal use of Python on background thread", "CvGameCoreDLL", MB_OK); 
	}
#endif

#ifdef FP_PROFILE_ENABLE				// Turn Profiling On or Off .. 
#ifdef USE_INTERNAL_PROFILER
	static	std::map<int,ProfileSample*>*	g_pythonProfiles = NULL;

	if ( g_pythonProfiles == NULL )
	{
		g_pythonProfiles = new std::map<int,ProfileSample*>();
	}

	CvChecksum xSum;
	const char* ptr;
	ProfileSample* pSample;

	for(ptr = moduleName; *ptr != '\0'; ptr++)
	{
		xSum.add((byte)*ptr);
	}
	for(ptr = fxnName; *ptr != '\0'; ptr++)
	{
		xSum.add((byte)*ptr);
	}

	std::map<int,ProfileSample*>::const_iterator itr = g_pythonProfiles->find(xSum.get());

	if ( itr == g_pythonProfiles->end() )
	{
		char profileName[256];

		sprintf(profileName, "IFPPythonCall2.%s::%s", moduleName, fxnName);
		pSample = new ProfileSample(profileName);

		g_pythonProfiles->insert(std::make_pair(xSum.get(), pSample));
	}
	else
	{
		pSample = itr->second;
	}

	CProfileScope detailedScope(pSample);		
#endif
#endif

	//OutputDebugString(CvString::format("Python call to %s::%s [%d]\n", moduleName, fxnName, pythonDepth++).c_str());

	bool bResult = gDLL->getPythonIFace()->callFunction(moduleName, fxnName, fxnArg, result);

	//OutputDebugString("...complete\n");
	pythonDepth--;

	return bResult;
}

And each time I process through it I get an Illegal use of python in background thread message. It happens 3 times and I have to wonder if it's the issue causing trouble.

I'm not personally skilled enough to figure out what this is pointing at. At all. For one thing, I'm no good at all at working with multi-threading issues and for a second I'm not the greatest with python still. I wouldn't even know how to begin tracking down the issue - and I've no idea if its a dll or python problem in reality.

That's going to require a more master level guy than myself.

We're building up a backlog of these 'over my head' matters aren't we? (Koshling... AIAndy? Can a brother get a hand here?) I know this is a lot for alberts to be looking into all of it.

EDIT: the Illegal use of python on a background thread issue comes up many many times during an end turn.

EDIT2: It's happening after the output reads something like this:

[7516] Note processed item c1d635e0 on stage CompleteProdcution (9 items remain)
[7516] Enqueue item c1d635e0 on stage PostProduction (now 19 items)
[7516] Process item c1d63688
[8568] Note processed item c1d63618 on stage CompleteProdcution (8 items remain)
[8568] Enqueue item c1d63618 on stage PostProduction (now 20 items)
[8568] Process item c1d636c0

EDIT: Mind... the game doesn't crash right away. I can only offer suspicion as to the above noted issue being a big enough problem to generate further crashes down the line. It does get to the next turn so there's not much else I can do here.

 

[Pit2015]

@Dancing Look screeni, this wolf still spawn and stay there in every game, terrain bug? Hes never attacking. Maybe because the wolf cant move in this terrain? I dont checked the wolf unit movemnt...

 

[Toffer90]

@Dancing Look screeni, this wolf still spawn and stay there in every game, terrain bug? Hes never attacking. Maybe because the wolf cant move in this terrain? I dont checked the wolf unit movemnt...

Wolfs can't move to desert tiles. It should be able to move to the plains/hill to its right.

I think it's possible to stop the wolf from spawning on horses if the horse bonus is on a desert terrain, that would be the only bug to fix here imo.

 

[strategyonly]

@ TB

its happening alot lately, forces leave their city with alot of troops and only leave 1 unit in city when i am attacking with really less than 2 units??

 

[alberts8]

Kp

@any who can help:

void CvUnit::joinGroup(CvSelectionGroup* pSelectionGroup, bool bRemoveSelected, bool bRejoin)
{
	PROFILE_FUNC();

	CvSelectionGroup* pOldSelectionGroup;
	CvSelectionGroup* pNewSelectionGroup;
	CvPlot* pPlot;

	pOldSelectionGroup = GET_PLAYER(getOwnerINLINE()).getSelectionGroup(getGroupID());

	if ((pSelectionGroup != pOldSelectionGroup) || (pOldSelectionGroup == NULL))
	{
		pPlot = plot();

		if (pSelectionGroup != NULL)
		{
			pNewSelectionGroup = pSelectionGroup;
		}
		else
		{
			if (bRejoin)
			{
				pNewSelectionGroup = GET_PLAYER(getOwnerINLINE()).addSelectionGroup();
				FAssert(pNewSelectionGroup != NULL);
				pNewSelectionGroup->init(pNewSelectionGroup->getID(), getOwnerINLINE());

This comes up in the process of initializing a unit. The last line is where it crashes. The only way this happens, if I'm not reading it wrong somehow, is if GET_PLAYER(getOwnerINLINE()).addSelectionGroup():

CvSelectionGroup* CvPlayer::addSelectionGroup()
{
	return ((CvSelectionGroup *)(m_selectionGroups.add()));
}

is coming up with no further available spaces to add another group definition here.

This IS a very very large map size - not sure what size unless I go back and reload and check first but very very large. And the player is of course the Barbarian. So if there's a limit to hit they'd be the ones to hit it.

Now I can think of a few ways it could be addressed:
1) Split animals and barbs - This would destroy save compatibility but maybe it is indeed time to do this.

or

2) Reuse old groups that were initialized but are now destroyed. Much of the AI code runs through new group inits like sheets of toiletpaper and if they're never recycled this will be a problem in any and all games at some point. So we MUST figure out a way to do this and tbh, I'm not sure how to do this myself.

This is the first time I've seen anything suggesting a unit limit being reached... is this what Koshling was indicating? Because this isn't a unit limit itself but a group limit being reached and exceeded. There's a fairly big difference as a unit can take on many differing group definition clothings throughout its time in the game. That said, I'm sure units will eventually have similar issues.


Anyhow, suffice it to say, this game exhibits something other than a basic crash syntax error but rather, something much more. I'd be interested to know how to solve this.

There is a limit of 8192 for things like SelectionGroups.
Reusing old Id's is a bad idea, there could be old references to them stored somewhere and reusing them could lead to odd bugs.

There are a few ways to fix that but most of them could increase turn times and/or memory usage. But it has been a long time since that issue came up so it doesn't have a high priority.

Splitting barbs and animals should have been done long ago and i personally don't care if it breaks compatibility. You and me made changes which could cause odd behavior in old saves but luckily nobody really noticed it. That basically means compatibility should be broken more often to avoid odd bugs in old saves.

 

[tmv]

In #353, I mentioned a problem with Felis Superior. The problem still exists with SVN 8921.

To elaborate, Felis Superior requires Effect - Big Cat Trainer, which obsoletes at Aviation. Felis Superior also requires Homo Superior.

I wanted to find out if it was possible to build Felis Superior if you actually didn't research Aviation until you were halfway through TH, but Aviation is a prerequisite for Homo Superior. One of the research chains goes as follows:

Aviation -> Rocketry -> Jet Propulsion -> Advanced Rocketry -> Guided Weapons -> Communication Networks -> Neural Networks -> Machine Learning -> Augmented Reality -> Ubiquitous Computing -> Gene Enhancement -> Mainstream Cloning -> Rapid Recuperation -> Bionics -> Cybernetics -> Species Uplifting -> Homo Superior

And another thing: In the Civilopedia, you cannot open a promotion page anymore. If you click on a promotion, the main page stays blank and you get back to the top of the promotion list. This happens in SVN 8921 as well.

 

[Toffer90]

In #353, I mentioned a problem with Felis Superior. The problem still exists with SVN 8921.

Spoiler :

To elaborate, Felis Superior requires Effect - Big Cat Trainer, which obsoletes at Aviation. Felis Superior also requires Homo Superior.

I wanted to find out if it was possible to build Felis Superior if you actually didn't research Aviation until you were halfway through TH, but Aviation is a prerequisite for Homo Superior. One of the research chains goes as follows:

Aviation -> Rocketry -> Jet Propulsion -> Advanced Rocketry -> Guided Weapons -> Communication Networks -> Neural Networks -> Machine Learning -> Augmented Reality -> Ubiquitous Computing -> Gene Enhancement -> Mainstream Cloning -> Rapid Recuperation -> Bionics -> Cybernetics -> Species Uplifting -> Homo Superior

And another thing: In the Civilopedia, you cannot open a promotion page anymore. If you click on a promotion, the main page stays blank and you get back to the top of the promotion list. This happens in SVN 8921 as well.

The issue has been noted.

@DH: What is the reason behind the chosen obsolete point?
We could add a building that requires big cat trainer but does not obsolete with it using <ConstructCondition> tag. This building would be required for training the felis superior and could be called something like Big Cat Tradition/Heritage.

 

[alberts8]

The issue has been noted.

@DH: What is the reason behind the chosen obsolete point?
We could add a building that requires big cat trainer but does not obsolete with it using <ConstructCondition> tag. This building would be required for training the felis superior and could be called something like Big Cat Tradition/Heritage.

Don't use the ConstructCondition tag because the AI has no knowledge about it.

 

[Dancing Hoskuld]

@Dancing Look screeni, this wolf still spawn and stay there in every game, terrain bug? Hes never attacking. Maybe because the wolf cant move in this terrain? I dont checked the wolf unit movemnt...

Wolfs can't move to desert tiles. It should be able to move to the plains/hill to its right.

I think it's possible to stop the wolf from spawning on horses if the horse bonus is on a desert terrain, that would be the only bug to fix here imo.

If the hills to the right has a resource on it that has not yet been revealed then the animal can't move there. Wild animals can't move into plots with resources on them - standard BtS behaviour.

Yes we can stop wolves spawning on horse plots in deserts but why would we. A similar thing happens in one of the polar terrains.

The issue has been noted.

@DH: What is the reason behind the chosen obsolete point?

Someone asked for the big cat line and I made the propotype to match what they wanted expecting them to finish it off. It appears no one has.

 

[Toffer90]

Don't use the ConstructCondition tag because the AI has no knowledge about it.

Oh, so they wouldn't build it even if they could? Does the same hold true for <TrainCondition>?

It doesn't need to require any building at all if only big cat units can build it.

If the hills to the right has a resource on it that has not yet been revealed then the animal can't move there. Wild animals can't move into plots with resources on them - standard BtS behaviour.

Huh, weird that I've never noticed this rule in my games.

 

[Koshling]

The save from the last round is corrupt and doesn't open.

However, this game does exhibit an unusual problem and I don't know what to make of it.

I get this assert:

Assert Failed

File:  CvGameCoreDLL.cpp
Line:  721
SVN-Rev:  8922
Expression:  bIsMainThread || iThreadSlot == -1
Message:  

----------------------------------------------------------

leading to a line in:

bool IFPPythonCall(const char* callerFn, const char* moduleName, const char* fxnName, void* fxnArg, long* result)
{
	PROFILE("IFPPythonCall2");

#ifdef USE_INTERNAL_PROFILER
	FAssert(bIsMainThread || iThreadSlot == -1);
	if ( !bIsMainThread )
	{
		::MessageBox(NULL, "Illegal use of Python on background thread", "CvGameCoreDLL", MB_OK); 
	}
#endif

#ifdef FP_PROFILE_ENABLE				// Turn Profiling On or Off .. 
#ifdef USE_INTERNAL_PROFILER
	static	std::map<int,ProfileSample*>*	g_pythonProfiles = NULL;

	if ( g_pythonProfiles == NULL )
	{
		g_pythonProfiles = new std::map<int,ProfileSample*>();
	}

	CvChecksum xSum;
	const char* ptr;
	ProfileSample* pSample;

	for(ptr = moduleName; *ptr != '\0'; ptr++)
	{
		xSum.add((byte)*ptr);
	}
	for(ptr = fxnName; *ptr != '\0'; ptr++)
	{
		xSum.add((byte)*ptr);
	}

	std::map<int,ProfileSample*>::const_iterator itr = g_pythonProfiles->find(xSum.get());

	if ( itr == g_pythonProfiles->end() )
	{
		char profileName[256];

		sprintf(profileName, "IFPPythonCall2.%s::%s", moduleName, fxnName);
		pSample = new ProfileSample(profileName);

		g_pythonProfiles->insert(std::make_pair(xSum.get(), pSample));
	}
	else
	{
		pSample = itr->second;
	}

	CProfileScope detailedScope(pSample);		
#endif
#endif

	//OutputDebugString(CvString::format("Python call to %s::%s [%d]\n", moduleName, fxnName, pythonDepth++).c_str());

	bool bResult = gDLL->getPythonIFace()->callFunction(moduleName, fxnName, fxnArg, result);

	//OutputDebugString("...complete\n");
	pythonDepth--;

	return bResult;
}

And each time I process through it I get an Illegal use of python in background thread message. It happens 3 times and I have to wonder if it's the issue causing trouble.

I'm not personally skilled enough to figure out what this is pointing at. At all. For one thing, I'm no good at all at working with multi-threading issues and for a second I'm not the greatest with python still. I wouldn't even know how to begin tracking down the issue - and I've no idea if its a dll or python problem in reality.

That's going to require a more master level guy than myself.

We're building up a backlog of these 'over my head' matters aren't we? (Koshling... AIAndy? Can a brother get a hand here?) I know this is a lot for alberts to be looking into all of it.

EDIT: the Illegal use of python on a background thread issue comes up many many times during an end turn.

EDIT2: It's happening after the output reads something like this:

[7516] Note processed item c1d635e0 on stage CompleteProdcution (9 items remain)
[7516] Enqueue item c1d635e0 on stage PostProduction (now 19 items)
[7516] Process item c1d63688
[8568] Note processed item c1d63618 on stage CompleteProdcution (8 items remain)
[8568] Enqueue item c1d63618 on stage PostProduction (now 20 items)
[8568] Process item c1d636c0

EDIT: Mind... the game doesn't crash right away. I can only offer suspicion as to the above noted issue being a big enough problem to generate further crashes down the line. It does get to the next turn so there's not much else I can do here.

If Python ever gets invoked on any but the main thread a crash is likely to result because the game engine does not have any multi-threading support. This means that all multi-thread support has to be 'hidden' strictly inside the code we run in the DLL or else all hell breaks loose (usually manifesting as a heap, corruption which fits these symptoms). Since we have no control over what happens in a Python call (what the invoked Python will itself then call) it means it is always unsafe to invoke a Python call on any but the main thread. The DLL code goes to some lengths to avoid this happening (stuff gets queued from the background threads to be invoked on the main thread explicitly), but it may well be we missed a few cases. To address this there is a workaround, and a debugging process (so workaround for a user experiencing it, process to follow for someone trying to fix it):

1) To work around it disable multi-threading. There's a one line setting in the config XML to achieve this, which I'm sure someone can look up in the XML and report back here (set the thread count to 1)

2) To try to get to the bottom of it put a debug breakpoint on the assertion failure that spots non-main-thread Python invocation to find out where in the code its making a Python call from where it shouldn't. Having found that point stop it happening, either by removing the need to call Python there somehow, or by queueing it onto the work queue to be executed by the main thread (there is already code to do this in various places, but I forget the exact details)