Assistance with K-Mod/RevDCM Merge

ripple01 · Jul 28, 2012

Hello all,

I've been working on an ambitious project to merge two large mods, K-Mod and RevDCM. So far, it has been going pretty well, and I have been able to get a .DLL compiled, and the game loads and plays pretty much without issue. However, I do have a couple of problems stumping me that I am hoping someone can assist with. I have not modified either of these files, so if you have RevDCM the line #'s match the original files.

1)

Traceback (most recent call last):
File "BugEventManager", line 361, in _handleDefaultEvent
File "BarbarianCiv", line 151, in onBeginGameTurn
File "BarbarianCiv", line 917, in checkBarbCities
File "BarbarianCiv", line 1000, in createMinorCiv
RuntimeError: unidentifiable C++ exception

Here is an excerpt from the createMinorCiv function: (I have bolded and italicized the line generating the unidentifiable C++ exception)

Code:

		newTeam.setIsMinorCiv( True, False )

		if( not newPlayer.isAlive() ) :
			if( self.LOG_DEBUG ) : CvUtil.pyPrint("  BC - Setting new player alive")
			newPlayer.setNewPlayerAlive(True)

		civName = newPlayer.getCivilizationDescription(0)
		leadName = newPlayer.getName()

		if( self.LOG_DEBUG ) : CvUtil.pyPrint("  BC - Minor civ %s (lead by %s) has formed in %s!  Turn: %d, Year: %d"%(civName,leadName,cityString,game.getElapsedGameTurns(),game.getTurnYear(game.getElapsedGameTurns())))

		# Add replay message
		mess = localText.getText("TXT_KEY_BARBCIV_FORM_MINOR", ())%(civName,cityString)
		mess = mess[0].capitalize() + mess[1:]
		game.addReplayMessage( ReplayMessageTypes.REPLAY_MESSAGE_MAJOR_EVENT, newPlayer.getID(), mess, cityList[0].getX(), cityList[0].getY(), gc.getInfoTypeForString("COLOR_HIGHLIGHT_TEXT"))

		for i,giftCity in enumerate(cityList) :
			#giftCity.changePopulation(1)
			# Using following method to acquire city produces 'revolted and joined' replay messages
			[B][I]giftCity.plot().setOwner( newPlayer.getID() )[/I][/B]

		# Note: city acquisition may invalidate previous city pointer, so have to create new list of cities
		cityList = pyNewPlayer.getCityList()
		capital = newPlayer.getCapitalCity()

		# Now setup generic new civ, no special types for minors
		# Gold
		newPlayer.changeGold( 70 + game.getSorenRandNum(100,'BarbarianCiv: give money') )

		# Tech
		self.giveTechs( iNewPlayer, closeTeams, bNewWorldScenario )

		# Units
		iNumBarbDefenders = gc.getHandicapInfo( game.getHandicapType() ).getBarbarianInitialDefenders()

		[iWorker, iSettler, iScout, iBestDefender, iCounter, iAttack, iMobile, iAttackCity, iAssaultShip] = self.getUnitsForPlayer( iNewPlayer )

		# Buildings
		[iLibrary, iGranary, iBarracks, iMarket, iWalls, iLighthouse, iForge, iMonument] = self.getBuildingsForPlayer( iNewPlayer )

		# Put stuff in cities
		for i,city in enumerate(cityList) :
			cityX = city.getX()
			cityY = city.getY()
			pCity = city.GetCy()

			self.setupFormerBarbCity(pCity, newPlayer.getID(), iBestDefender, int(iNumBarbDefenders*self.militaryStrength + 0.5))

			# Adjust Population
			iPop = pCity.getPopulation()
			pCity.setPopulation( max([self.minPopulation, min([iPop, self.minPopulation + 1 + 2*newPlayer.getCurrentEra() - i])]) )
			if( bNewWorldScenario and self.iNewWorldPolicy > 0 ) :
				if( self.iNewWorldPolicy > 1 and not bNonBarbCivUnits ) :
					if( len(closeTeams) < 2 and iEra < 2 ) :
						iNewPop = max([min([self.minPopulation,iPop]),iPop - 1])
						pCity.setPopulation(max([iNewPop,self.minPopulation]))
			if( self.LOG_DEBUG and bVerbose ) : CvUtil.pyPrint("  BC - %s has population %d (adj from %d)"%(pCity.getName(),pCity.getPopulation(),iPop))

			# Extra units
			if(iWorker != -1):
				pyNewPlayer.initUnit(iWorker,cityX,cityY,1)

			attackUnits = list()

			if( iEra > 0 ) :
				if( iCounter == iBestDefender ) :
					attackUnits.append( newPlayer.initUnit(iAttack,cityX,cityY,UnitAITypes.NO_UNITAI, DirectionTypes.DIRECTION_SOUTH ) )
				else :
					attackUnits.append( newPlayer.initUnit(iCounter,cityX,cityY,UnitAITypes.NO_UNITAI, DirectionTypes.DIRECTION_SOUTH ) )

			# Buildings
			if( i == 0 ) :

				if( not bNewWorldScenario or bForeignCivCities or self.iNewWorldPolicy == 0 ) :
					if( city.canConstruct( iBarracks ) ) :
						if( self.LOG_DEBUG and bVerbose ) : CvUtil.pyPrint("  BC - Constructing %s in %s"%(PyInfo.BuildingInfo(iBarracks).getDescription(),city.getName()))
						city.setNumRealBuildingIdx(iBarracks,1)

					if( iEra > 0 and city.canConstruct( iGranary ) ) :
						if( self.LOG_DEBUG and bVerbose ) : CvUtil.pyPrint("  BC - Constructing %s in %s"%(PyInfo.BuildingInfo(iGranary).getDescription(),city.getName()))
						city.setNumRealBuildingIdx(iGranary,1)

					if( iEra > 1 and city.canConstruct( iWalls ) ) :
						if( self.LOG_DEBUG and bVerbose ) : CvUtil.pyPrint("  BC - Constructing %s in %s"%(PyInfo.BuildingInfo(iWalls).getDescription(),city.getName()))
						city.setNumRealBuildingIdx(iWalls,1)

					if( iEra > 1 and city.canConstruct( iForge ) ) :
						if( self.LOG_DEBUG and bVerbose ) : CvUtil.pyPrint("  BC - Constructing %s in %s"%(PyInfo.BuildingInfo(iForge).getDescription(),city.getName()))
						city.setNumRealBuildingIdx(iForge,1)

2)

Traceback (most recent call last):
File "BugEventManager", line 361, in _handleDefaultEvent
File "Revolution", line 7002, in onModNetMessage
File "Revolution", line 4996, in revolutionNetworkPopupHandler
File "Revolution", line 5545, in processRevolution
RuntimeError: unidentifiable C++ exception

Here is an excerpt from the processRevolution function: (I have bolded and italicized the line causing the unidentifiable C++ exception)

Code:

#***********************************
						# Acquire city
						#joinPlayer.acquireCity( pCity, False, True )

						if( self.LOG_DEBUG ) : CvUtil.pyPrint("  Revolt - Population of %s before is %d"%(pCity.getName(),pCity.getPopulation()))
						if( self.LOG_DEBUG ) : CvUtil.pyPrint("  Revolt - Check city culture is %d, at %d, %d"%(pCity.getCulture(pPlayer.getID()), pCity.getX(),pCity.getY()))
						cityPlot = pCity.plot()
						if( pCity.getCulture( pPlayer.getID() ) == 0 ) :
							if( self.LOG_DEBUG ) : CvUtil.pyPrint("  Revolt - Forcing culture > 0")
							pCity.setCulture( pPlayer.getID(), 1, True )

						try :
							pCity.plot().setOwner( pRevPlayer.getID() )
						except :
							print "ERROR in grant independence"
							print "ERROR:  Failed to set owner of city, %s at plot %d, %d"%(pCity.getName(),cityPlot.getX(),cityPlot.getY())
							#print "City culture is %d"%(pCity.getCulture(pPlayer.getID()))

							#pCity = cityPlot.getPlotCity()
							#print "Post culture in %s is %d"%(pCity.getName(),pCity.getCulture(pPlayer.getID()))
							#pRevPlayer.acquireCity( pCity, False, False )
							#RevData.initCity(pCity)
							# City has become invalid, will cause game to crash if left
							print "Destroying city so game can continue"
							[B][I]pCity.kill()[/I][/B]
							continue


						pCity = cityPlot.getPlotCity()
						if( self.LOG_DEBUG ) : CvUtil.pyPrint("  Revolt - Population of %s after is %d"%(pCity.getName(),pCity.getPopulation()))

						if( pCity.getPopulation() < 1 ) :
							if( self.LOG_DEBUG ) : CvUtil.pyPrint("  Revolt - Error!  City %s is empty"%(pCity.getName()))
#*************************************

						# Save most buildings - should some be destroyed?
						for [buildingClass,iNum] in buildingClassList :
							buildingType = gc.getCivilizationInfo(pRevPlayer.getCivilizationType()).getCivilizationBuildings(buildingClass)
							if( pCity.getNumRealBuilding(buildingType) < iNum ) :
								buildingInfo = gc.getBuildingInfo(buildingType)
								if( not buildingInfo.isGovernmentCenter() ) :
									if( self.LOG_DEBUG ) : CvUtil.pyPrint("  Revolt - Building %s saved"%(buildingInfo.getDescription()))
									pCity.setNumRealBuilding( buildingType, iNum )

						#if( self.LOG_DEBUG ) : CvUtil.pyPrint("  Revolt - %s at %d, %d"%(pCity.getName(),pCity.getX(),pCity.getY()))

						if( self.LOG_DEBUG ) : CvUtil.pyPrint("  Revolt - Culture in %s: %d, plot %d"%(pCity.getName(),pCity.getCulture(pPlayer.getID()),pCity.plot().getCulture(pPlayer.getID())))

						newCulVal = int( self.revCultureModifier*max([1.0*pCity.getCulture(pPlayer.getID()),pCity.countTotalCultureTimes100()/200]) )
						newPlotVal = int( self.revCultureModifier*max([1.2*pCity.plot().getCulture(pPlayer.getID()),pCity.plot().countTotalCulture()/2]) )
						RevUtils.giveCityCulture( pCity, pRevPlayer.getID(), newCulVal, newPlotVal, overwriteHigher = False )

						ix = pCity.getX()
						iy = pCity.getY()

The major issue I've noticed so far is that sometimes when a minor civ is created, the city will have 0 population and as such will never grow. This is obviously not supposed to happen. I suspect it has something to do with the first exception.

When I've got the major bugs sorted out, I will release this merge for interested modders. If you can help me sort this out, you'll be helping out a # of other modders down the road. Many thanks for looking.

The_J · Jul 28, 2012

1) I don't really see where #1 could go wrong.
Did you do some tests to see that the ID of the player is?
Might have something to do with the civ limit, I'd guess.

2) No real idea here either.
But I think I remember that the kill() command produced also some problems for me when I used it. I then used the raze command from the CyPlayer class, and it worked again, so I suggest that you try to replace it.

Cruel · Jul 28, 2012

I hope you make available your work. I've seen here in CFC at least three other people working in the same merge, and you might want to open an mod thread to discuss. :goodjob:

ripple01 · Jul 30, 2012

Thanks for the input, The J. I'll check into your suggestions.

I have posted the first public beta of my merge between K-Mod and RevolutionDCM. It is called Evolution and is available here: http://forums.civfanatics.com/showthread.php?t=470980

Opera · Oct 4, 2012

Bumping this since I mentioned it in Evolution's thread.

I took a look at CvPlot.cpp and it seems that the function fails between line 4899 and 4946, since the city flip actually occurs (line 4898) but the erasing of enemy units (barbarians) does not (line 4927). Since it doesn't happen, the city starts with barbs inside it, they go outside the next turn and take it the turn afterwards.

Not sure what's wrong in there though and that's all guessing. I'd say try to set up a debug dll with break points in this function and see what happens. I had 1 civ survive in my test out of all the barbarian cities that grew. Not sure how it did.

Highwayhoss · Nov 18, 2012

I've been playing with Evolution and did a merge with the Neoteric mod plus some additional minor mods. I got it working just fine but came across a problem...I enabled the Archery barrage option but for some reason I can't get my Archers to fire! Anyone else have this problem?

ripple01 · Dec 6, 2012

Opera said:
Bumping this since I mentioned it in Evolution's thread.

I took a look at CvPlot.cpp and it seems that the function fails between line 4899 and 4946, since the city flip actually occurs (line 4898) but the erasing of enemy units (barbarians) does not (line 4927). Since it doesn't happen, the city starts with barbs inside it, they go outside the next turn and take it the turn afterwards.

Not sure what's wrong in there though and that's all guessing. I'd say try to set up a debug dll with break points in this function and see what happens. I had 1 civ survive in my test out of all the barbarian cities that grew. Not sure how it did.

I've quoted the relevant code here for anyone who may be able to lend a hand.

Code:

FAssertMsg(pOldCity->getOwnerINLINE() != eNewValue, "pOldCity->getOwnerINLINE() is not expected to be equal with eNewValue");
			GET_PLAYER(eNewValue).acquireCity(pOldCity, false, false, bUpdatePlotGroup); // will delete the pointer
			pOldCity = NULL;
			pNewCity = getPlotCity();
			FAssertMsg(pNewCity != NULL, "NewCity is not assigned a valid value");

			if (pNewCity != NULL)
			{
				CLinkList<IDInfo> oldUnits;

				pUnitNode = headUnitNode();

				while (pUnitNode != NULL)
				{
					oldUnits.insertAtEnd(pUnitNode->m_data);
					pUnitNode = nextUnitNode(pUnitNode);
				}

				pUnitNode = oldUnits.head();

				while (pUnitNode != NULL)
				{
					pLoopUnit = ::getUnit(pUnitNode->m_data);
					pUnitNode = oldUnits.next(pUnitNode);

					if (pLoopUnit)
					{
						if (pLoopUnit->isEnemy(GET_PLAYER(eNewValue).getTeam(), this))
						{
							FAssert(pLoopUnit->getTeam() != GET_PLAYER(eNewValue).getTeam());
							pLoopUnit->kill(false, eNewValue);
						}
					}
				}

				eBestUnit = pNewCity->AI_bestUnitAI(UNITAI_CITY_DEFENSE);

				if (eBestUnit == NO_UNIT)
				{
					eBestUnit = pNewCity->AI_bestUnitAI(UNITAI_ATTACK);
				}

				if (eBestUnit != NO_UNIT)
				{
					iFreeUnits = (GC.getDefineINT("BASE_REVOLT_FREE_UNITS") + ((pNewCity->getHighestPopulation() * GC.getDefineINT("REVOLT_FREE_UNITS_PERCENT")) / 100));

					for (iI = 0; iI < iFreeUnits; ++iI)
					{
						GET_PLAYER(eNewValue).initUnit(eBestUnit, getX_INLINE(), getY_INLINE(), UNITAI_CITY_DEFENSE);
					}
				}
			}
		}

I'm going to build a debug dll and try to investigate when I have a free moment. I have a new baby, so it's tough!

Highwayhoss said:
I've been playing with Evolution and did a merge with the Neoteric mod plus some additional minor mods. I got it working just fine but came across a problem...I enabled the Archery barrage option but for some reason I can't get my Archers to fire! Anyone else have this problem?

I can confirm that Archery Barrage is broken. I'm updating the DCM components from stolenrays's update. We'll see if this fixes it.

karadoc · Dec 9, 2012

I second Opera's suggestion of running this will a debug version of the dll. In particular, if you run it while attached to the VC++ debugger, it will probably be able to tell you exactly which line in the C++ code is failing.

But I do have rough ideas:

At first glance, the first case looks like it is crashing inside the CvPlot::setOwner of the dll. But it could also be in 'newPlayer.getID', or giftCity.plot().

Judging by the python code leading up to the second case, the Rev authors were having the same problem. Notice the problematic line in the second example is only ever run if `pCity.plot().setOwner( pRevPlayer.getID() )` fails. ie. Both cases are essentially the same problem. The block of code with pCity.kill() is just an error report to say that the thing they actually wanted to do had failed.

In the second case, since pCity.kill() also fails, my guess is that pCity is invalid. And giftCity is probably invalid in the first case. So I suggest you check on what's happening to those objects in the code leading up to this stuff.

I reckon this bug has probably been in RevDCM the whole time, and has nothing to do with the K-Mod merge.

Here's an important bit of info which may be relevant to the problem: when a city changes owners, the city is actually deleted and a new city is created for the new owner. So changing owners actually invalidates city pointers.

ripple01 · Dec 13, 2012

karadoc said:
I second Opera's suggestion of running this will a debug version of the dll. In particular, if you run it while attached to the VC++ debugger, it will probably be able to tell you exactly which line in the C++ code is failing.

But I do have rough ideas:

At first glance, the first case looks like it is crashing inside the CvPlot::setOwner of the dll. But it could also be in 'newPlayer.getID', or giftCity.plot().

Judging by the python code leading up to the second case, the Rev authors were having the same problem. Notice the problematic line in the second example is only ever run if `pCity.plot().setOwner( pRevPlayer.getID() )` fails. ie. Both cases are essentially the same problem. The block of code with pCity.kill() is just an error report to say that the thing they actually wanted to do had failed.

In the second case, since pCity.kill() also fails, my guess is that pCity is invalid. And giftCity is probably invalid in the first case. So I suggest you check on what's happening to those objects in the code leading up to this stuff.

I reckon this bug has probably been in RevDCM the whole time, and has nothing to do with the K-Mod merge.

Here's an important bit of info which may be relevant to the problem: when a city changes owners, the city is actually deleted and a new city is created for the new owner. So changing owners actually invalidates city pointers.

Thanks for taking a look at this, karadoc! I'll take your suggestions and see what I can come up with.

ripple01 · Apr 28, 2013

I'm coming back to this problem, and have been using a debug .DLL. I'm going to post some of my musings here, and see if with the help of the community I can figure this out.

So, looking at CvPlot: setowner, I've noticed that when this is called on existing civilizations, it makes it through the whole function. However, when it is called on a newly created civ, that it doesn't make it past this line:

GET_PLAYER(eNewValue).acquireCity(pOldCity, false, false, bUpdatePlotGroup); // will delete the pointer

Also, is that bUpdatePlotGroup correct in that line? Everywhere else in the code I see acquireCity called, it is either a true or false there.

Additionally, looking at the 1st issue in the top post, I can't find any other reference to giftCity anywhere in the Python or DLL except for the one line it is mentioned. Where is it defined, I wonder?

Opera · Apr 28, 2013

bUpdatePlotGroup should be a boolean, so it should pass either true or false. I don't know how the value of bUpdatePlotGroup is determined however, you should try backtracking through the code to find out.

giftCity is defined through the enumerate function in the python code. I'm not sure how it works though, you should check this out in some kind of python documentation. What it does is it uses the list of cities and produces a giftCity object using the cityList list. You should maybe check out how this list is produced.

glider1 · Apr 30, 2013

I spent a few hours on this bug yesterday. I drilled deeper into acquireCity and beyond but could not find the exact problem yet. The problem almost certainly did not exist in RevolutionDCM since so many mods have used Barbciv for years without myself or anyone else ever reporting this behavior.

What about this for a basic question.

Do cities flip correctly when there is a revolution? I assume the answer is yes. So then, what is the difference in the mechanic between flipping from a barbarian city to a civ (broken), compared to flipping from a civ to a civ (working)?

Cheers

Tholal · Apr 30, 2013

ripple01 said:
So, looking at CvPlot: setowner, I've noticed that when this is called on existing civilizations, it makes it through the whole function. However, when it is called on a newly created civ, that it doesn't make it past this line:

GET_PLAYER(eNewValue).acquireCity(pOldCity, false, false, bUpdatePlotGroup); // will delete the pointer

Add a bunch of logging in the acquireCity function so you can see what the code is going when it gets there (you could also add in a breakpoint so that you can manually step through the code when the problem section is hit, but that's not always as easy as it sounds).

Also, taking a quick glance at the code snippet you posted a few posts back, I see that it checks isEnemy. Perhaps the newly created civ hasn't yet declared war on the barbarians?

ls612 · Apr 30, 2013

Conditional Breakpoints are your friend here. Try setting a breakpoint on the start of acquireCity, and then right click it and in the context menu select "Condition". Then go ingame and use the debug overlay (if you don't have the cheat code on turn it on, it makes life far easier) to figure out the game's ID of the city in question, then write something like this in the condition box;

Code:

pOldCity->getID() == Something (whatever the game said it was)

That will make it only break when the problem occurs, making the problem easier to debug.

glider1 · May 3, 2013

Thanks for the tips ls612 and Tholal. In the end I had to go against my commonsense to fix this bug! The only thing that gave it away was an FASSERT that was raised here:

Code:

int CvPlayerAI::AI_averageYieldMultiplier(YieldTypes eYield) const
{
	FAssert(eYield > -1);
	FAssert(eYield < NUM_YIELD_TYPES);
	// Glider1 - barbciv fix	
	if (m_iAveragesCacheTurn != GC.getGameINLINE().getGameTurn())
	{
		AI_calculateAverages();
	}
	//

For the life of me I do not actually understand how such an innocuous unrelated piece of code that is merely computational in nature could cause this. Note that re-enabling AI_calculateAverages() meant having to revert back to the BTS implementation of mutable int CvPlayerAI.h.

If someone can explain why it fixed it. That would be great!
Cheers

Assistance with K-Mod/RevDCM Merge

ripple01

Emperor

The_J

Say No 2 Net Validations

Cruel

King

ripple01

Emperor

Opera

Deity

Highwayhoss

Red Lantern of Civ4 Earth

ripple01

Emperor

karadoc

AI programmer

ripple01

Emperor

ripple01

Emperor

Opera

Deity

glider1

Deity

Tholal

Emperor

ls612

Deity

glider1

Deity

Similar threads