[PYTHONCOMP] Python utility library (includes INI file reading)

[TheLopez]

I understand. What I'm asking, more specifically, is would the code shown below work just as well? If not, why? I don't have a good test case to run through this, but just from looking at it, it seems like they would both ultimately do the same thing.

        # removed beginEvent and addEvent overloads

	def addCustomEventDefinition(self, eventType, eventDefinition):
                # changed CustomEvents to Events
		self.Events[eventType] = eventDefinition

	def removeCustomEventDefinition(self, eventType):
                # changed CustomEvents to Events
		del self.Events[eventType]

The basic idea is that the Events dictionary already exists, and it appears to do exactly what you want from CustomEvents. Adding more entries to that dictionary for custom events shouldn't break the built in events, and the default popup handling code should still work with the custom events.

Underneath the complexity of CvEventManager, Events is really just a dictionary that maps integer event ID's to popup handler functions. Your CustomEvents is the same thing. There's really no difference that I can see between a "built in" event and a "custom" event.

That should work. Could you see the way that I am doing it check out the Great Statesman Mod and let me know if you see any issues with using the above methods?

 

[Dr Elmer Jiggle]

That should work. Could you see the way that I am doing it check out the Great Statesman Mod and let me know if you see any issues with using the above methods?

Note: I didn't actually try install and run the mod, but I looked at the code.

As far as I can tell, it should work fine. The only way I see a problem is if the integer you use as an event ID (ex. DIPLOMATIC_RELATIONS_POPUP = 4203) isn't unique, but that's actually a problem with your original implementation too. I don't think there's any obvious way around that requirement. You just need to be careful about your choice of integer.

When I implement this, I might stick a check in the add method to make sure the key is unique. That should be easy to do, and that way you'd at least get a warning if you did something stupid.

 

[jdog5000]

First off, a very useful event manager mod! I use it now in my TechCost Mod, and it works great.

I am definitely looking forward to your implementation for adding new popup event handlers, as for an upcoming mod I currently have modified you CEM similar to how TheLopez originally did. I have a question stemming from this new mod as well.

I want to create a new keyboard command (currently using shift-ctrl-x). Is there a way to access whether the shift and ctrl keys are down from my own file? Like the normal event manager, you copy their state into some private bits:

self.bDbg, self.bMultiPlayer, self.bAlt, self.bCtrl, self.bShift, self.bAllowCheats = argsList[flagsIndex:]

Currently the _handleConsumableEvent doesn't pass these on to the event handler:

result = eventHandler(argsList[:len(argsList) - 6])

I assume this is done to play well with existing handlers. Is there a good way to access those bits from my own function, let's call it AIAutoPlay.onKbdEvent()? My current hack is to add getbShift() etc to the CEM code ...

Thanks for your help!

 

[Dr Elmer Jiggle]

Is there a way to access whether the shift and ctrl keys are down from my own file? Like the normal event manager, you copy their state into some private bits:

Currently the _handleConsumableEvent doesn't pass these on to the event handler

Hmm. I'm not using any keyboard handlers, so I hadn't thought about that issue.

My first inclination would be the same hack you used. I don't think Python even has any real public/private access control, so you could probably just access the "bShift" variable directly, but it's obviously cleaner and more object oriented to use a getter function.

I also think it's legal to pass extra, unused arguments to the event handlers. If so, then all those little extra fiddly bits could be added to the arguments for onKbdEvent, and it wouldn't break handlers that don't expect or use those values.

I'm not really certain this will work, but it seems like it has to if you think about how the event handlers are declared. They're all some form of "f(self, argsList)" but they all take different arguments. It seems like it ought to be legal to just jam as much extra garbage as you feel like onto the end of the list.

Can you try that and let me know if it works? Something like:

• change _handleConsumableEvent to pass the whole list (get rid of the -6)
• change your handler function to get bShift and so forth from the argsList
  eventType,key,mx,my,px,py,bDbg,bDummy,... = argsList

If that works, it I'll stick something like that into my version to fix this problem.

 

[TheLopez]

Actually, the answer is easier than that DEJ. Look at how I handle this case in the mercenaries mod, the important code is in bold:

class CvMercEventManager:

	mercenaryManager = None
	
	def __init__(self, eventManager):
	
		self.EventKeyDown=6
		[B]self.eventManager = eventManager[/B]
...
...

	# This method handles the key input and will bring up the mercenary manager screen if the 
	# player has at least one city and presses the 'M' key.
	def onKbdEvent(self, argsList):
		'keypress handler - return 1 if the event was consumed'
		# TO DO: REMOVE THE FOLLOWING LINE BEFORE RELEASE.
		#gc.getPlayer(0).setGold(20000)
		eventType,key,mx,my,px,py = argsList
			
		theKey=int(key)

		if ( eventType == self.EventKeyDown and theKey == int(InputTypes.KB_M) and [B]self.eventManager.bAlt[/B] and gc.getActivePlayer().getNumCities() > 0 and gc.getActivePlayer().getCurrentEra() >= g_iStartingEra):
			self.mercenaryManager.interfaceScreen()
...

 

[jdog5000]

@ DrEJ

I will try that out and report back, although I think it may fail when the uninitiated handlers try and unpack the argList with something like

iTechType, iPlayer = argsList

If that suddenly has a extra entry ... seems like it wouldn't work. I'll check it later. But regardless ... I think TheLopez's idea works. Might add some comments to your CEM to help out future people.

@ TheLopez

Thanks! I think that takes care of it.

 

[Dr Elmer Jiggle]

I think it may fail when the uninitiated handlers try and unpack the argList

Yeah, it does fail. I thought the extra values would be silently ignored, but it's an error. Oh well.

 

[Denniz]

I want to create a new keyboard command (currently using shift-ctrl-x). Is there a way to access whether the shift and ctrl keys are down from my own file?

The below is from eotinb's AutoLog which I modified to run as an eventmanager out of the CEM for the HOF Mod. The basic handling of the Alt-E has been passed unchanged from the original.

def onKbdEvent(self, argsList):
	if hof.get_boolean('AUTOLOG', 'Enabled', False):
		eventType,key,mx,my,px,py = argsList
		if ( eventType == self.eventMgr.EventKeyDown ):
			theKey=int(key)
			'Check if ALT + E was hit == echoes to text log and in-game log'
			if (theKey == int(InputTypes.KB_E) and self.eventMgr.bAlt):
				self.eventMgr.beginEvent(CvUtil.EventCustomLogEntry)
				return 1
	return 0

As you can see there are booleans for Ctrl and Shift as well. So I would think you could do something similar to AutoLog. The main thing seems to be to stay away from Civ4's hot keys.

class CvEventManager:
	def __init__(self):
		#################### ON EVENT MAP ######################
		#print "EVENTMANAGER INIT"
		
		self.bCtrl = False
		self.bShift = False
		self.bAlt = False
		self.bAllowCheats = False
		
		# OnEvent Enums
		self.EventLButtonDown=1
		self.EventLcButtonDblClick=2
		self.EventRButtonDown=3
		self.EventBack=4
		self.EventForward=5
		self.EventKeyDown=6
		self.EventKeyUp=7

If you are looking for popup handling, AutoLog has working examples of that too.

Or am I missing something?

 

[Dr Elmer Jiggle]

If you are looking for popup handling, AutoLog has working examples of that too.

Or am I missing something?

No, I don't think you're missing anything. In fact, I think this validates my suggestion that the build in Events dictionary can be used to do this. Including add/remove methods in the CEM class isn't necessary (as you've shown), but it's probably a nice convenience.

For the sake of clarification about what we're talking about, here's a clip from Denniz' code. You can see that he's adding his dialog handlers directly to the Events dictionary which then gets handled automatically by the built in CvEventManager dialog logic.

	def __init__(self, eventManager):

		AutoLogEvent(eventManager)

		# additions to self.Events
		moreEvents = {
			CvUtil.EventLogOpen : ('LogOpenPopup', self.__eventLogOpenApply, self.__eventLogOpenBegin),
			CvUtil.EventCustomLogEntry : ('', self.__eventCustomLogEntryApply, self.__eventCustomLogEntryBegin),
		}
		eventManager.Events.update(moreEvents)

I'm also thinking about another idea for the keyboard handlers. This approach of saving a reference to the event manager obviously works, but it feels like a bit of a hack to me. I'm not sure why, but it does. Probably partly the cyclic reference with the event manager pointing to your handler object and your handler object also pointing back to the event manager.

Anyway, my thought is that I could implement some kind of new, custom, alternative keyboard handler type. So instead of "em.addEventHandler("kbdEvent", self.onKbdEvent)" you would do something like "em.addEventHandler("newKbdEvent", self.onKbdEvent)." Then there would be some special logic in the CEM so "newKbdEvent" handlers get the additional boolean flags passed to them.

I'm going to experiment with this, but I'm not sure I'll end up liking it. It might turn out that all the special case handling for this winds up being uglier than any other alternative. But the advantage is that I think it would internalize the hack, so it could be done once in the CEM and then never again. The way it works now, anyone that wants to implement a keyboard handler needs to repeat the hack.

 

[Jeckel]

I'm having a problem and no matter how hard I bang my head against the wall, I just can't figure it out.

So I decided my first python mod attempt would be a very simple Fort mod based off of Bhuric's RealFort. I want my mod to be easy as possible ot combine with other mods(even if mostly by me) so I'm using this CustomEvent structure from the start. I managed to make some progress, but I have hit a snag.

Here is the pertinate part from the CvCustomEventManager.py

import CvEventManager
import CvGreatStatesmanEventManager
import CvGDEventManager
import CvFieldHospitalEventManager
import CvGreatGeneralEventManager
import CvGPTrickleEventManager
[B]import CvJFortEventManager[/B]

class CvCustomEventManager(CvEventManager.CvEventManager, object):

	# < NEW CODE START >
	CustomEvents = {}
	# < NEW CODE END >
		
	def __init__(self, *args, **kwargs):
		super(CvCustomEventManager, self).__init__(*args, **kwargs)
		# map the initial EventHandlerMap values into the new data structure
		for eventType, eventHandler in self.EventHandlerMap.iteritems():
			self.setEventHandler(eventType, eventHandler)
		# --> INSERT EVENT HANDLER INITIALIZATION HERE <--
		CvGreatStatesmanEventManager.CvGreatStatesmanEventManager(self)
		CvGDEventManager.CvGDEventManager(self)
		CvFieldHospitalEventManager.CvFieldHospitalEventManager(self)
		CvGreatGeneralEventManager.CvGreatGeneralEventManager(self)
		CvGPTrickleEventManager.CvGPTrickleEventManager(self)
		[B]CvJFortEventManager.CvJFortEventManager(self)[/B]

Just to note, this is from TheLopez's GP Trickle Plus Mod(I don't that matters, but doesn't hurt to mention it). The only thing I altered was the addition of the two bolded lines.

Here is my entire CvJFortEventManager.py

from CvPythonExtensions import *

import Popup as PyPopup
import CvUtil
import sys
import CvMainInterface
import CvGameInterface

import CvJFortGameUtils


# globals
gc = CyGlobalContext()

jf = CvJFortGameUtils.JFort()

# globals
###################################################
class CvJFortEventManager:
	def __init__(self, eventManager):
                # initialize base class

		eventManager.addEventHandler("unitMove", self.onUnitMove)
		[B]eventManager.addEventHandler("ImprovementBuilt", self.onImprovementBuilt)[/B]

	def onUnitMove(self, argsList):
		jf.onUnitMove(argsList)
		
	def onImprovementBuilt(self, argsList):
		jf.onImprovementBuilt(argsList)

And this is my entire CvJFortGameUtils.py

from CvPythonExtensions import *

import Popup as PyPopup
import CvUtil
import sys

# globals
gc = CyGlobalContext()

class JFort:
	def onUnitMove(self, argsList):
		'unit move'
		pPlot,pUnit = argsList

		impFort = gc.getInfoTypeForString("IMPROVEMENT_FORT")
		promFort1 = gc.getInfoTypeForString("PROMOTION_FORT1")
		
		if (pPlot.getImprovementType() == impFort):
                        pUnit.setHasPromotion(promFort1, true)
	
	
	def onImprovementBuilt(self, argsList):
		'Improvement Built'
		iImprovement, iX, iY = argsList

		impFort = gc.getInfoTypeForString("IMPROVEMENT_FORT")
		promFort1 = gc.getInfoTypeForString("PROMOTION_FORT1")

		pMap = CyMap()
		pPlot = pMap.plot(iX, iY)
		if (iImprovement == impFort):
			for iUnit in range(pPlot.getNumUnits()):
				pUnit = pPlot.getUnit(iUnit)
				pUnit.setHasPromotion(promFort1, true)
		else:
			for iUnit in range(pPlot.getNumUnits()):
				pUnit = pPlot.getUnit(iUnit)
				if (pUnit.isHasPromotion(promFort1)):
					pUnit.setHasPromotion(promFort1, false)

My problem is in the CvJFortEventManager.py, specifically the bolded line. When I load Civ4 with that code I get all kinds of unhelpful error messages when it loads the python stuff. But if I comment that line out it runs, except the Fort1 promotion doesn't get put on the units until they move the first time.

I at first figured the onImprovementBuilt method was changed in the last patch or somthing, but I check in the CvEventManager.py and it seemed to be there and to work just as I'm calling it.

I don't know if I'm not setting this up right or maybe I'm just overlooking somthing simple, but any help would be appriceated.

 

[Denniz]

It's case sensitive. You have to copy the string from the original exactly.

'improvementBuilt' 		: self.onImprovementBuilt,

eventManager.addEventHandler("[COLOR="Red"][B]i[/B][/COLOR]mprovementBuilt", self.onImprovementBuilt)

I did that a few times before I figured it out.

 

[Jeckel]

*grins* I figured it was some little syntax error, if I had a nickle for every hour I've spent fighting with code when the only problem was a misspelling or incorrect case. Thanks for the quick response, I will now sleep peacefully tonight.

 

[Dr Elmer Jiggle]

It's case sensitive. You have to copy the string from the original exactly.

'improvementBuilt' 		: self.onImprovementBuilt,

eventManager.addEventHandler("[COLOR="Red"][B]i[/B][/COLOR]mprovementBuilt", self.onImprovementBuilt)

I did that a few times before I figured it out.

If I remember to look back through this thread for ideas when I get around to making improvements, I'm going to add something in the addEventHandler function to test the validity of the name. This is just too common a mistake, and it's way too easy to make it. I've done it a few times myself.

 

[Padmewan]

Sorry to bring the conversation here down a notch. I am trying to implement my custom coding the "right" way using this component (you'll be horrified at what I was doing before ) but I now find myself in a maze of twisty passages all alike.

I am doing my best to learn from other mods that utilize this framework, e.g. Civ4Alerts and Great Statesman, but they are all a tad too complicated for me to deconstruct. Partly I'm not a great OOP thinker, and partly I'm still getting my head around Python syntax, and partly I'm trying to learn the hierarchy of Civ Python classes/functions/etc.

What is the minimum that I need to do to get this component to "work"? To the best of my knowledge, it is:

1) Put D.E.G.'s CvCustomEventManager.py file in the root Python directory.

2) Assuming my custom .py file will be called TestMod, change the CvCustomEventManager.py code to read:

import CvEventManager
import TestMod

    def __init__(self, *args, **kwargs):
        super(CvCustomEventManager, self).__init__(*args, **kwargs)
        # map the initial EventHandlerMap values into the new data structure
        for eventType, eventHandler in self.EventHandlerMap.iteritems():
            self.setEventHandler(eventType, eventHandler)
        # --> INSERT EVENT HANDLER INITIALIZATION HERE <--
        TestMod.TestMod(self)

3) Create TestMod.py . This file can be created and put anywhere under the root Python directory, yes, and "import TestMod" will know where to find it? For example, I see that CvGreatStatesmanEventManager.py is in a subdirectory called "GreatStatesman."

4) Now what must TestMod.py import in order to function properly? It seems funny to me that TestMod must now import CvEventManager again, but that seems to be the case across all the mods I've seen. Presumably you also need to import all functions called by this package?


Now that I've asked my general question, here is one specific example which I hope you can help me with to understand how to do more like it. This is the exact code that I inserted straight into CvEventManager (not even CvCustomEventManager -- yes, I know, do not pass Go, do not collect $200!):

	def onImprovementBuilt(self, argsList):
		'Improvement Built'
		iImprovement, iX, iY = argsList

#################### BEGIN CUSTOM TERRAFORM ##################
# Moves terrain "up" one level (relies on terrain being sequential)
# Replaces the fake terraform improvement with a real improvement
		pPlot = CyMap().plot(iX,iY)		
		iTerraform = gc.getInfoTypeForString('IMPROVEMENT_TERRAFORM1')
		iHothouse = gc.getInfoTypeForString('IMPROVEMENT_HOTHOUSE')
		iFTerran = gc.getInfoTypeForString('FEATURE_TERRAN1')

		if (iImprovement==iHothouse):
			pPlot.setFeatureType(iFTerran, -1)
		elif (iImprovement==iTerraform):
			iOldTerrain = CyMap().plot(iX, iY).getTerrainType()
			pPlot.setTerrainType(iOldTerrain+1, 1, 1)
			pPlot.setImprovementType(iHothouse)
			pPlot.setFeatureType(-1, -1)

#################### END TERRAFORM ##################

		if (not self.__LOG_IMPROVEMENT):
			return
		CvUtil.pyPrint('Improvement %s was built at %d, %d'
			%(PyInfo.ImprovementInfo(iImprovement).getDescription(), iX, iY))

OK, so how would I refactor this the "right" way?

Thanks for all of your efforts to make Civ Modding a safe and healthy environment for everyone

 

[Padmewan]

At the risk of making myself sound even more ignorant, how do you turn on Python debugging and where does the log file go? Also, is there some thread, wiki, whatever where all of this information is collected that I just am not tapped into?

 

[Dr Elmer Jiggle]

4) Now what must TestMod.py import in order to function properly? It seems funny to me that TestMod must now import CvEventManager again, but that seems to be the case across all the mods I've seen. Presumably you also need to import all functions called by this package?

I don't think you should need to. Civ4lerts doesn't do that.

The rest of the steps you wrote all look right up to this point. Then your final job is to write TestMod.py. Something like this should do the job (I might have some syntax errors here, I'm just typing this from memory).

class TestMod:
    def __init__(self, eventManager):
        eventManager.addEventHandler("improvementBuilt", self.onImprovementBuilt)

    def onImprovementBuilt(self, argsList):
        # no changes required

 

[Padmewan]

Then your final job is to write TestMod.py. Something like this should do the job (I might have some syntax errors here, I'm just typing this from memory)...

OK, then just pay attention to case and all that, per the rest of this thread? Whew.... this is going to be arduous but I think worthwhile

And just to confirm: I can put my custom functions ANYWHERE in the Python directory and as long as I name (1) the file AND/OR (2) the main class in that file correctly, the Civ4 engine will find it?

THANKS A MILLION.

 

[Dr Elmer Jiggle]

OK, then just pay attention to case and all that, per the rest of this thread?

Right. Later today or tomorrow I'm going to post a new version that will throw an exception if you make a mistake in the event name, but that still means you need to get it right. It just makes it easier to notice when you've made a mistake.

And just to confirm: I can put my custom functions ANYWHERE in the Python directory and as long as I name (1) the file AND/OR (2) the main class in that file correctly, the Civ4 engine will find it?

Right. That's actually sort of strange. Normally with Python, if you put a module in foo/bar/Zap.py, you need to import it as "import foo.bar.Zap", but Civ4 uses a custom class loader that ignores the directory paths.

As far as the logging goes, look in your personal Civ4 directory (not the main game installation directory), typically something like C:\Documents and Settings\User\My Documents\My Games\Sid Meier's Civilization 4. You should see a file called CivilizationIV.ini. If you search through that file, you'll find a bunch of options related to logging. They aren't well documented, and I don't know exactly what all of them do, but basically you want to set LoggingEnabled to 1. You can fiddle with the other options too if you like.

In that same directory, you'll find a subdirectory called Logs which contains the various log files. PythonDbg.log, PythonErr.log, and PythonErr2.log seem to be the most useful. I don't know what the difference is between each one of those, especially Err and Err2.

There are a few sites with information on modding, but honestly I haven't found many of them to be of much help. The main one I've used is http://civilization4.net/files/modding/PythonAPI which lists most of the functions available in the API. There are some other sites that I don't seem to have bookmarked that have descriptive articles about what some of the files are for, but they all seemed extremely incomplete last time I looked. They were mostly just full of empty stub articles.

 

[Denniz]

A couple of notes:

Locutus' PythonAPI link is out. You can use this one instead: http://civilization4.net/files/modding/PythonAPI_v160/

Must have for Python modding:

; Set to 1 for no python exception popups
HidePythonExceptions = 0

Logging options:

; Enable the logging system
LoggingEnabled = 1

; Enable synchronization logging
SynchLog = 0

; Overwrite old network and message logs
OverwriteLogs = 1

; Enable rand event logging
RandLog = 0

; Enable message logging
MessageLog = 0

I like to turn on Overwrite logs so I don't have to search around for the last error and they don't grow continuously.

Watch out if you have McAfee anti-virus. Logging with it on seems to slow things down for some people.

 

[Dr Elmer Jiggle]

Finally, at long last, the much anticipated changes for popup dialogs have been uploaded!!! Sorry about the delays. I also put in simple checking for the event type strings passed to addEventHandler.