updating cvgameutils to new BUG

[davidlallen]

(I started this investigation by PM and it is probably here in the forum somewhere; but if I could not find it with some searching, perhaps others will benefit from this also.)

Problem: updated from old version of BUG (3.6) to new version (4.2) and the callbacks in my python/CvGameUtils.py stopped working. I added print statements and showed that they were no longer being called.

Solution:
1. Rename CvGameUtils.py to something else, and delete all the unmodified routines out of it. It usually contains dozens of "default" routines and a couple which you have modified. Keep all yours and delete all the rest. So if you had this file:

## Sid Meier's Civilization 4
## Copyright Firaxis Games 2005
##
## Implementaion of miscellaneous game functions

from CvPythonExtensions import *
import CvEventInterface
import GodsOfOld
import Popup as PyPopup
import PyHelpers

# globals
gc = CyGlobalContext()
PyPlayer = PyHelpers.PyPlayer

class CvGameUtils:
	"Miscellaneous game functions"
	
	def __init__(self):
		#RevolutionDCM - Inquisition Mod
		self.revModEnabled = gc.getGame().isOption(GameOptionTypes.GAMEOPTION_REVOLUTION)

	def isVictoryTest(self):
		# Mymod: changed 10 to 100 (it is an example)
		if ( gc.getGame().getElapsedGameTurns() > 10 ):
			return True
		else:
			return False
	def isVictory(self, argsList):
		eVictory = argsList[0]
		return True

Rename it to MyModGameUtils.py and remove most of the other code:

## Sid Meier's Civilization 4
## Copyright Firaxis Games 2005
##
## Implementaion of miscellaneous game functions

from CvPythonExtensions import *
import CvEventInterface
import GodsOfOld
import Popup as PyPopup
import PyHelpers

# globals
gc = CyGlobalContext()
PyPlayer = PyHelpers.PyPlayer

class [COLOR="Red"]MyModGameUtils[/COLOR]:
	"Miscellaneous game functions"
	
	def __init__(self):
		#RevolutionDCM - Inquisition Mod
		self.revModEnabled = gc.getGame().isOption(GameOptionTypes.GAMEOPTION_REVOLUTION)

	def isVictoryTest(self):
		# Mymod: changed 10 to 100 (it is an example)
		if ( gc.getGame().getElapsedGameTurns() > 100 ):
			return True
		else:
			return False

Note I removed the unmodified function isVictory and changed the name of the module.

2. You need to tell BUG where to look for your modified functions. The key line is:

<gameutils module="MyModGameUtils" class="MyModGameUtils" override="True"/>

Most mods using BUG already have a file like Mods/<modname>/assets/Config/MyMod.xml which BUG uses to register things. If not, create it. Add this line. The smallest example I can see is FavoriteCivicDetector.xml; delete the "events" line and add the "gameutils" line.

Now, I still have some questions which hopefully others will find useful also.

a. File assets/xml/PythonCallbacks.xml disables a number of callbacks which are called millions of times, such as cannotMoveInto. Does BUG respect this, or do all of these callbacks get called?

b. The amount of python in my mod is "medium sized", less than 2000 lines. So far I have kept it all in one file and one class. It has a few dozen functions, of which about five are calls for CvGameUtils functions. Can I supply my main file + class to the gameutils call? In other words, will BUG get upset if there are functions in the gameutils class which have nothing to do with gameutils?

c. (This is more of a RevDCM question). My mod is built on top of other modcomps including RevDCM. A number of functions in CvGameUtils.py are modified by the RevDCM team and *not* further modified by me. I assume that I should just delete these functions from my MyModGameUtils.py file, because RevDCM has also updated to use this new BUG. True?

d. On the other hand, if there is some other small python modcomp which I have incorporated into my mod, I should probably keep their functions in MyModGameUtils. Or, at least find out if they have updated to BUG and get their files if so. True?

e. Suppose I have overridden five of the CvGameUtil functions and I want to use the default functions for the rest, and also my class contains many extra functions. How does BUG decide what to do? Does it search all the function names in my mod for matches with the default functions? Or do I have to give more information about which functions to use?

 

[davidlallen]

I have updated my mod to work, and my experience is slightly different than the instructions.

a. Rather than creating assets/Config/<modname>.xml, I edited assets/Config/init.xml. This already had a gameutils line, so I replaced it with my own.

b. The original CvGameUtils.py has all its functions within "class CvGameUtils". When I copied this style as shown in post #1, I got an error message, "Module DuneWarsGameUtils must define a function cannotTrain". However, when I removed the class statement and defined all the functions at the top level of DuneWarsGameUtils.py, then it worked. So simply renaming the class definition from CvGameUtils to DuneWarsGameUtils appears to be the wrong approach; instead remove any class definition from the file.

c. The "handlers" field in the gameutils command in init.xml is critical to activate the handlers. If you want to override CvGameUtils functions a,b,c then your init.xml line would be:

<gameutils module="MyModGameUtils" handlers="a,b,c" override="True"/>

If you list "c" in this line but there is no function "c" in your file, you will get an error. If you also intended to override function "d", and you have "d" in your file, but "d" is not listed in the handlers line, then "d" will not be overridden. This answers most of my questions regarding *how* this works; it uses the names in the handlers option.

It is working for me now, I guess I do not need any answers to the questions in the previous post.

 

[EmperorFool]

First, thanks for a) moving this here from PMs because it's far easier to discuss here, and others can benefit from it; and b) for the nice sample code.

It is working for me now, I guess I do not need any answers to the questions in the previous post.

Well, you're getting them anyway. You ended up switching from the class method to the module function method, and pick the way that's easier for you. I prefer the module function method because BUG only uses a few callbacks and doesn't need to share state among them. Using a class is easier when converting a mod using CvGameUtils already, at least that was my intention.

It sounds like you got it working as a class and then switched. I missed the part where things weren't working as a class.

a. File assets/xml/PythonCallbacks.xml disables a number of callbacks which are called millions of times, such as cannotMoveInto. Does BUG respect this, or do all of these callbacks get called?

This is handled by the DLL. The C++ code checks the XML file to decide whether or not to call out to Python. BUG has no control over this. It would be helpful if BUG set callbacks for which you had added handlers for you, but for now you need to activate whichever callbacks you want in that XML file.

In other words, will BUG get upset if there are functions in the gameutils class which have nothing to do with gameutils?

When you use the class method and don't tell BUG which callbacks to use, it matches all of the functions by name to the names of the callbacks. A callback name is any function in CvGameUtils (the originals) and any that you have added using <callback>. Any functions that don't match are ignored.

A number of functions in CvGameUtils.py are modified by the RevDCM team and *not* further modified by me. I assume that I should just delete these functions from my MyModGameUtils.py file, because RevDCM has also updated to use this new BUG. True?

Correct. The goal was to allow for multiple mods to each have their own callback handler class without needing to modify a shared Python file.

d. On the other hand, if there is some other small python modcomp which I have incorporated into my mod, I should probably keep their functions in MyModGameUtils. Or, at least find out if they have updated to BUG and get their files if so. True?

Correct. You can convert that other modcomp to use its own Python file with a gameutils class or module functions, or you can leave them in your gameutils class or module.

e. Suppose I have overridden five of the CvGameUtil functions and I want to use the default functions for the rest, and also my class contains many extra functions. How does BUG decide what to do? Does it search all the function names in my mod for matches with the default functions? Or do I have to give more information about which functions to use?

If the callback handler functions match those in CvGameUtils, they are detected automatically. If you either a) don't have matching names or b) wish to disable some handlers without removing/renaming the functions, you need to specify exactly which handlers to use with the "handlers" attribute.

I have updated my mod to work, and my experience is slightly different than the instructions.

Can you briefly explain what wasn't working? It sounded to me like you had everything setup and working. At least, your guesses to your questions were correct.

a. Rather than creating assets/Config/<modname>.xml, I edited assets/Config/init.xml. This already had a gameutils line, so I replaced it with my own.

Do you mean this line?

<gameutils module="WidgetUtil" handler="getWidgetHelp"/>

If so, then some of BUG's hover help text will stop functioning. That's the only <gameutils> in BUG's original init.xml, so if it was some other <gameutils> line it must have been a part of RevDCM and I would expect it to start failing in some way.

You should be able to have any number of <gameutils> lines, so just add your own instead of replacing an existing one.

b. The original CvGameUtils.py has all its functions within "class CvGameUtils". When I copied this style as shown in post #1, I got an error message, "Module DuneWarsGameUtils must define a function cannotTrain".

I would only expect to see that if you did something like this:

<gameutils module="DuneWarsGameUtils" handler="cannotTrain"/>

or

<gameutils module="DuneWarsGameUtils" class="cannotTrain"/>

If you post your PythonErr.log and PythonDbg.log files in the future, I'd be happy to explain error messages and even happier to make them clearer in BUG for others.

c. The "handlers" field in the gameutils command in init.xml is critical to activate the handlers.

Only as mentioned above or if you are using module-level functions instead of a class. IIRC I didn't have it search modules for handlers, but I should. My goal was to minimize the amount of work needed. If something isn't working, I'd rather fix it for everyone than have you need to do a bunch of extra work. I'm glad you got it working, so obviously stick with what works, but I'd like to get more information (logs are best) when things don't work so I can fix them.

If you list "c" in this line but there is no function "c" in your file, you will get an error.

This is intentional. Not displaying an error would leave modders scratching their heads wondering why their spiffy callback wasn't doing anything when they had a typo in the XML.

If you also intended to override function "d", and you have "d" in your file, but "d" is not listed in the handlers line, then "d" will not be overridden.

Using the "handlers" attribute is designed to allow you total control over what gets registered. If you insist on this control but omit a handler, there's not really much I can do. I suppose I could look for other handler names and issue a warning, "Did you mean to omit cannotTrain?", but then users would see this.

 

[davidlallen]

Is there a reference guide which explains the fields of the commands such as gameutils? I think if I had seen a definition of the "module", "class" and "handler" options, I probably could have figured it out without any posts.

I suspect I was misled slightly by having "module" but not "class" in the gameutils line, while putting my functions into a class. Based on your explanation, either of these should work:

a. gameutils "class" and putting all the functions into a class, *or*
b. no gameutils "class" and putting all the functions at global scope.

I was mixing them.

Thanks for also clarifying that BUG will search for all handlers by name. That means I don't need to put "handler". I suppose the only reason for using "handler" is if I have functions a,b,c with names matching CvGameUtils functions, but for some reason I do not want my c to be executed. Then I would write "handlers=a,b". Actually, if I wanted to do that for some reason, I would probably edit my python and comment out or rename the function.

 

[EmperorFool]

Is there a reference guide which explains the fields of the commands such as gameutils?

The BUG Core XML Reference is what you seek. It describes the usage for each attribute.

I suspect I was misled slightly by having "module" but not "class" in the gameutils line, while putting my functions into a class.

The <gameutils> element differs from the others in BUG in that it will not assume a class with the same name as the module if you omit the "class" attribute. This is because omitting that attribute tells BUG your handlers are module-level functions.

That means I don't need to put "handler".

Correct. I needed handler for the module-level functions case, but I included it for the class function method to cover the case where someone had a function that didn't match the callback name. It also allows you to disable callbacks, but as you said better to just rename the function or remove it entirely.

 

[davidlallen]

I needed handler for the module-level functions case

Perhaps I missed something. For the class case, you can search the defined function names so there is no need for the modder to specify any names. For the module case, is there no way to do the same search?

 

[EmperorFool]

There is, and if I didn't do that already (I may have; I'll check tonight), I can add it. I mentioned this in my previous post.

 

[davidlallen]

Here are two related questions.

1. Since many modcomps by different authors may be incorporated into a mod, is there a convenient way to find out a list of all the registrations? For example, I would like to use PythonCallbackDefines.xml to turn off calling the cannotTrain callback to speed up the mod. But, I am not sure if one of the various modcomps I am using may register a handler for it. Worse, I believe they may register and associate some random function name, so I cannot simply "grep cannotTrain `find . -name '*.py'`" (Unix/linux shorthand, search for cannotTrain in all python files in the tree)

2. I thought I had gotten up to the point where I did not even need to modify CvEventManager.py so I removed my copy from the distribution. A bug appeared -- the vanilla function onCityRazed is doing something I do not want, related to EVENTTRIGGER_PARTISANS. Is there a way to disable one or all of the vanilla functions? I solved it by putting back my local file and making the one small edit. But it would be cleaner to disable the vanilla one. Perhaps a global flag to disable all the vanilla ones would work, they are mostly there only for logging.

 

[EmperorFool]

Those are all good ideas, and you are correct that there is no easier way to do what you're trying to do. Both GameUtils and Events have a logging facility separate from normal debug logging that will log each event/callback as it is dispatched. This only tells you the functions for things as they happen--it doesn't tell you up front everything that is registered.

However, it wouldn't be too hard to make a function to dump that information. It's all in the BugEventManager and/or BugGameUtils classes. Sometimes it's a direct function that is registered, so you'll need to use Python to extract its module and name, but the functions (not classes) registered using config XML files are done using a Function class in BugUtil that holds the module and name directly. However, you can get those by grepping for "<gameutils module" and "<event type".