Making CvGameUtils Modular

EmperorFool

Deity
Joined
Mar 2, 2007
Messages
9,633
Location
Mountain View, California
As a result of a conversation with davidlallen in this thread, I've started working on a method to make CvGameUtils modular as part of BUG. I'd like to get some feedback on my initial design from the modders that will be using it.

First a little background. CvGameUtils defines several callback functions used by the SDK via CvGameInterface. As a modder, you have a few options when overriding how CvGameUtils behaves.

Note: I have written a separate tutorial for those unfamiliar with the process. My design depends on method 1 from that tutorial, so it will help to read that post.[/quote]

My design is similar to BugEventManager which is based on CvCustomEventManager by Dr. Elmer Jiggles. I have written a replacement for CvGameUtils called BugGameUtils which can dispatch callbacks to multiple subordinate modules that get registered by XML configuration files during the BUG initialization phase. As with BugEventManager, modders would never modify BugGameUtils nor CvGameInterfaceFile which connects it to BTS.

Example: AcquireCityGoldGameUtils

This example will change BTS so that when a player captures a city they always gain 1000:gold:. This will be very similar to method 1 from my tutorial.

1. As in method 1, create a new module containing your game utils class and add your modified callback functions as above. It's okay to subclass CvGameUtils, but it's no longer necessary. It will make no difference to BUG which will make merging mods built this way even easier.

Code:
## AcquireCityGoldGameUtils.py

class AcquireCityGoldGameUtils:

	def doCityCaptureGold(self, argsList):
		return 1000

2. Register your module with BUG. In BUG you create an XML file containing your options, events, shortcuts, etc. This step will require one additional line:

Code:
<mod id="AcquireCityGold">
	[B]<game-utils module="AcquireCityGoldGameUtils"/>[/B]
</mod>

Because the names of the callback functions are determined by CvGameInterface.py and cannot be changed by the modder, BUG is able to detect all of your callback functions automatically.​
 
For anyone merging multiple game utils modules created in this manner, you just need to specify each in your XML:

Code:
<mod id="AcquireCityGold">
	<game-utils module="AcquireCityGoldGameUtils"/>
	<game-utils module="PillageGoldGameUtils"/>
	<game-utils module="FreeUpgradeGameUtils"/>
</mod>

Question: What happens when two modules override the same callback function?

Answer: My first design dispatches callbacks in the order they are defined in XML.

For example, if the first two modules above both provided a doGoody() callback function, the one from AcquireCityGoldGameUtils would be used.

A simplistic method is to add an "override" attribute that contain one or more callback function names separated by spaces. It would tell BUG which callbacks should override previously-defined ones. The special value "all" would tell it to override all callbacks being added.

Code:
<game-utils module="PillageGoldGameUtils" [B]override="doGoody"[/B]/>

To provide more fine-grained control, I could add two attributes called "include" and "exclude". If you specify "include", only those callbacks would be registered, automatically overriding previous callbacks (?). Alternatively, if you provide "exclude", all but those would be processed without automatically overriding previous callbacks.

Code:
<game-utils module="AcquireCityGoldGameUtils" [B]exclude="doGoody"[/B]/>
... or ...
<game-utils module="PillageGoldGameUtils" [B]include="all"[/B]/>

Question: Can this design allow two callbacks from different modules to interact?

Answer: Yes, with modification. Using the doGoody() example, say one module gives human players a 1 in 1000 chance to win 5000:gold: instead of the normal prize.

Code:
def doGoody(self, argsList):
	ePlayer = argsList[0]
	pPlot = argsList[1]
	pUnit = argsList[2]
	
	pPlayer = gc.getPlayer(ePlayer)
	if pPlayer.isHuman():
		if CyGame().getSorenRandNum(1000, "Bonus Zenny!") == 666:
			pPlayer.changeGold(5000)
			# skip normal prize determination
			return True
	return False

The other module isn't so fancy. It simply gives 50:gold: in addition to the normal prize.

Code:
def doGoody(self, argsList):
	ePlayer = argsList[0]
	pPlot = argsList[1]
	pUnit = argsList[2]
	
	pPlayer = gc.getPlayer(ePlayer)
	pPlayer.changeGold(50)
	return False

Using the above design, the only way to combine these two callbacks is to write a new one that has all the logic. Yuck! I think we can do better. [Question: do we need to?]

There are a few common flavors of callbacks in CvGameUtils. This is one of the "return True to skip the default behavior, False otherwise" kind. There are quite a few like this. One method to combine functions like this is to call them in the order they were registered with BUG until one of them returns True or we run out of callbacks. In short, the first module to declare that it has handled the callback stops the process.

If you registered the second module first in the above example, a human player could get 50:gold: and 5000:gold:, but they wouldn't get the normal prize in that case. If you reverse the modules, they would get either the 5000:gold: or the 50:gold: plus normal prize.

Most all of the other callbacks function in a similar manner. The only one I spot that's different in a quick scan is getWidgetHelp(). This callback allows you to add text to the normal hover text for a widget (or set it for ones that don't have any). I could use the same method as above, but if two mods want to append text to the same widget hover, it may be better to call all callbacks and concatenate the text from each one.

Where does that leave our two friends doAcquireCityGold() and doPillageGold()? These callbacks have no return value used to signify "normal processing"; all the coding is in the callbacks. One option is to add the return values from all callbacks; another is to make up our own signal value (-1) that you'd return to tell BUG to use stop processing callbacks and use 0.
 
Question: How is this design affected by the PythonCallbacks.xml file?

Answer: It isn't. That file is checked by the SDK before calling the callback function. BUG won't alter that process.

That being said, I think a nice feature would be to have BUG warn when a mod provides a callback function that isn't setup to be called by the SDK. It looks like those settings are exposed through CyGlobalContext.getDefineINT() just like values in GlobalDefines.xml and GlobalDefinesAlt.xml.

Question: How will this design handle mods that define their own callbacks?

Answer: Good question. Do any mods currently do this? I would think that a mod wishing to do so would just define a normal module-level function and call that itself from its own DLL since a new callback function in CvGameUtils would require a new DLL anyway.
 
Interesting idea.

I can't say much about it, because i don't use BUG in my mod, but the look from my position says to me, that it could be a userfriendly addition for modders who don't know python.
For modders with python knowledge it will be a bit confusing, just because loss of the whole control.
 
For modders with python knowledge it will be a bit confusing, just because loss of the whole control.

I see your point since BugGameUtils will be driving the process, but for most mods the effect would be negligible. I'm thinking that I could even release a CvCustomGameUtils a la CvCustomEventManager that would be separate from BUG. For the original modders, you'd use this to plug in your game utils class just like people do with their event managers today.
 
As they say, necessity is the mother of invention, and I suddenly needed to override one of the functions from CvGameUtils. Couple that with an off-hand request from another modder and bam, new toy! :D It wasn't that hard and was a fun exercise.

I now have a nice utility for adding hover text to your own widgets in BUG. The text can be static, from XML, or a function. I'll commit it in a few days after the 3.19 dust has settled.
 
2. Register your module with BUG. In BUG you create an XML file containing your options, events, shortcuts, etc. This step will require one additional line:

Code:
<mod id="AcquireCityGold">
	[B]<game-utils module="AcquireCityGoldGameUtils"/>[/B]
</mod>

Question. How do we do #2? Where do I put this code? Does it go in init.xml or something? Can you expand a bit about where to plug the additional line?
 
Allow me to try to explain what I'm trying to do. I'm trying to add teh machu pichu mod's code, it goes under cannot construct. Well I can construct it so I'm trying to figure out why.

Code:
	def cannotConstruct(self,argsList):
		pCity = argsList[0]
		eBuilding = argsList[1]
		bContinue = argsList[2]
		bTestVisible = argsList[3]
		bIgnoreCost = argsList[4]

		### MachuPicchu Mod begins ###
		###########################################################################################

		if ( eBuilding == gc.getInfoTypeForString("BUILDING_MACHU_PICCHU") ):

			### find peaks within the city radius controlled by your team ###
			pPlayer = gc.getPlayer(pCity.plot().getOwner())
			iPID = pPlayer.getID()
			iTID = pPlayer.getTeam()
			iX = pCity.getX()
			iY = pCity.getY()
			for iXLoop in range(iX - 2, iX + 3, 1):
				for iYLoop in range(iY - 2, iY + 3, 1):
					pPlot = CyMap().plot(iXLoop, iYLoop)
					if ( pPlot.isPlayerCityRadius(iPID)==true ):
						if ( pPlot.getTeam()==iTID ):
							if ( pPlot.isPeak()==true  ):
								return False
			return True

		###########################################################################################
		### MachuPicchu Mod ends ###
 
If you follow from the beginning of the tutorial it will show you all the steps. Did you do this? If so, maybe something is missing or unclear.

1. Create your own XML file a la init.xml. It will look just like what you posted with the names inside the ""s changed to your module's name and such.

2. Load your XML file from init.xml using <load>. This ensures you touch init.xml in a small way so it's easier to upgrade later.
 
1. I have already edited into my xml that is called from init. The xml calls eventmodules currently. I cut and pasted the <game-utils module="AcquireCityGoldGameUtils"/> in there.

When I run my mod, Bug mentions it has errors loading my xml after I do this.

Here's what I'm trying, works fine until I add the thing I'm trying to test (acquirecitygoldgameutils)

here's my destiny.xml from the config folder

Code:
<?xml version="1.0" encoding="ISO-8859-1" ?>
<mod id="DESTINY" 
	 name="Destiny"
	 author="Smeagolheart" 
	 version="2.00" 
	 date="16 Aug 10" 
	 url="http://forums.civfanatics.com/showthread.php?t=226568">
	
	<game-utils module="AcquireCityGoldGameUtils"/>
	<events module="CvEnhancedTechConquestEventManager" />
	<events module="CvPiratesModEventManager" />
	<events module="CvMachuPicchuEventManager" />

</mod>
 
AcquireCityGoldGameUtils is the name of the example module in my tutorial. You must replace that with the name of the module containing your game utils class that has the cannotConstruct() callback.
 
When you have your callback functions in a class you must specify the class name in the <gameutils> element. Yes, this is different from events where it will look for the class by default.

Code:
<gameutils class="DestinyCvGameUtils"/>

Also, remove the references (import and superclass) to CvGameUtils:

Code:
[s][COLOR="Red"]import CvGameUtils[/COLOR][/s]

from CvPythonExtensions import *
gc = CyGlobalContext()

class DestinyCvGameUtils[s][COLOR="Red"](CvGameUtils.CvGameUtils)[/COLOR][/s]:
 
When you have your callback functions in a class you must specify the class name in the <gameutils> element. Yes, this is different from events where it will look for the class by default.

Code:
<gameutils class="DestinyCvGameUtils"/>

Also, remove the references (import and superclass) to CvGameUtils:

Code:
[s][COLOR="Red"]import CvGameUtils[/COLOR][/s]

from CvPythonExtensions import *
gc = CyGlobalContext()

class DestinyCvGameUtils[s][COLOR="Red"](CvGameUtils.CvGameUtils)[/COLOR][/s]:

I made the changes you suggested and it's working.

For anyone who may be reading this later, I did have to call my gameutils module like this:
Code:
<gameutils module="DestinyCvGameUtils" class="DestinyCvGameUtils"/>

I tested out some of the functions I had in there like the sphinx wonder's 80% wonder cost and it's working.

So thank you very much again for the assistance!

One last question I have because I'm not 100% sure is can you confirm that I don't need everything in my gameutils module, I just need things that are changed? ie if I'm not modding 'canTrain', I don't need to have it in there?

edit: And while it seems to be working, I checked my python error log and it's giving these errors (neither of the files mentioned have been modified from the version in the latest RevDCM):
Code:
Traceback (most recent call last):

  File "BugUtil", line 690, in <lambda>

  File "CvDiplomacyInterface", line 33, in handleUserResponse

  File "CvDiplomacy", line 495, in handleUserResponse

  File "CvDiplomacy", line 338, in setAIComment

  File "CvDiplomacy", line 312, in determineResponses

  File "CvDiplomacy", line 324, in addUserComment

  File "CvDiplomacy", line 403, in getDiplomacyComment

RuntimeError: Access violation - no RTTI data!
ERR: Python function handleUserResponse failed, module CvDiplomacyInterface
 
For anyone who may be reading this later, I did have to call my gameutils module like this:
Code:
<gameutils module="DestinyCvGameUtils" class="DestinyCvGameUtils"/>

Correct. I had thought you put the module attribute in your <mod> element. When you do that all of the elements within it will use it as the value when they need a module.

Can you confirm that I don't need everything in my gameutils module, I just need things that are changed?

Correct. If your class doesn't define canTrain() BUG will use the one from CvGameUtils. Actually, BUG will use the registered default if there is one to avoid calling the functions in CvGameUtils. This is why it is so very important that you do not modify the original CvGameUtils as your modifications will be ignored.

Code:
RuntimeError: Access violation - no RTTI data!
ERR: Python function handleUserResponse failed, module CvDiplomacyInterface

That's very strange. The <lambda> in BugUtil is where it grabs translated text from the XML files using CyTraslator. If you're using BUG 4.3 or 4.4, make sure you started from new files and did not just copy 4.3/4 over 4.2. BUG 4.3 removed some files, including one related to the diplomacy screen where this error occurs, and copying over files will not delete them.
 
That's very strange. The <lambda> in BugUtil is where it grabs translated text from the XML files using CyTraslator. If you're using BUG 4.3 or 4.4, make sure you started from new files and did not just copy 4.3/4 over 4.2. BUG 4.3 removed some files, including one related to the diplomacy screen where this error occurs, and copying over files will not delete them.

I've done a lot of crazy stuff to try and get things to work. I haven't used Bug 4.2 in here, I started with the RevDCM which has 4.3 I believe. I attempted to integrate code from QuotCapita to use the GreatPerson mod (popup screen) but that's not working I think it has an earlier version of Bug going on there.

I'll look into the files to ensure I'm based of the RevDCM files. You are referring to CvDiplomacyInterface? I'm using the one from RevDCM... Can I just change it out with what Bug 4.4 has?
 
Okay, it was in 4.4 that I removed CvDiplomacyInterface.py from BUG entirely. I added a feature to BugUtil that allows me to add/change functions from BTS modules from Python, i.e. without supplying a modified file.

That means you should leave it in since RevDCM is based on 4.3. In that case, I have no idea why you'd get that error. When that error happens again, please post PythonDbg.log so I can see what's happening at the same time, hopefully the cause.
 
Top Bottom