[MOD] Modular Python

I have a question... and forgive my ignorance, because I didn't read all the post in the thread.... but: Does this just make CvCustomEventsManger modular? Because if it only works on events, then FF has that ability with the BUG Mod. How to Merge Mods with BUG.. You need to modify just a couple of files within FF to make this work, and no need for the .txt file method.
 
I have a question... and forgive my ignorance, because I didn't read all the post in the thread.... but: Does this just make CvCustomEventsManger modular? Because if it only works on events, then FF has that ability with the BUG Mod. How to Merge Mods with BUG.. You need to modify just a couple of files within FF to make this work, and no need for the .txt file method.

In addition to CvEventManager it also makes CvGameUtils modular. Replaces them actually. Also CvSpellInterface and CvRandomEventInterface.

But the hard work is really making all the modifications FFH has made into event handlers, and that work should be usable for BUG as well.

Also I don't know how easy it is to overide parts of other modules with BUG. It's needs to be easy, or micro modules, the ones that only alter some tiny thing, become to complicated. Another advantage is that all modmod files go in the modmod directory.

And the txt marker file is there so that there is no need to modify any files. The problem is that there is no way to get at the current mod directory from python. BUG forces the user to modify a file whenever he renames the mod. Modular python looks for a particular file instead, at the cost of not being able to have more than one modular python mod per civilization install.

The next version of modular python will enable the user to choose either method, with the marker file being the default. I've been having hardware problems, so it's been delayed though.
 
In addition to CvEventManager it also makes CvGameUtils modular. Replaces them actually. Also CvSpellInterface and CvRandomEventInterface.

BUG 4.0 adds support for modular CvGameUtils. One of the major driving factors in my design of BUG was to make it as easy as possible to take an existing mod and install it into BUG. You can take a modified CvEventManager or any event manager that works with CvCustomEventManager and use it directly with BUG. All that's needed is a single line in an XML file to hook it up.

Code:
<events module="MyEventManager"/>

Auto-discovery is great when designing code from scratch, but it can be difficult to make it support older mods. I've tried to strike a balance in BUG in favor of supporting older mods, but there are clearly advantages to other ways too.

After a cursory glance over the source of this mod, it appears that you alter the function signatures required for events. Instead of taking a single list of arguments, a dictionary is created. Am I correct? Is there a need for this? Also, it looks like the return from normal event handlers is a list of the returns from each handler. Have you verified that this is okay to do? Most likely the SDK ignores the return result of those handlers, but I try to be more cautious about changing a function's contract.

BTW, when dispatching a normal event I see that you open the error log, dispatch to each handler, and close the log. This may cause a slowdown since the normal operation is for there to be no errors, and it happens on every event. I suggest opening and closing it for each error that occurs. Errors will run slightly slower, but working code will run slightly faster.

Also I don't know how easy it is to overide parts of other modules with BUG. It's needs to be easy, or micro modules, the ones that only alter some tiny thing, become to complicated.

There are several extension points in BUG, but I definitely designed BUG more for other modders to build upon. For example, if you add a new type of great person, you need to modify GPUtil to give it an icon so it shows up correctly in the GP Progress Bar. While I could write code to make this possible within XML, I doubt too many mods add new GP types.

I made adding event handlers and game util callbacks a snap since that's a very common need by modders.

Another advantage is that all modmod files go in the modmod directory.

Does this include XML and Art? If a modmod requires a CIV4GameText.xml file, can it be placed with the Python files? That would be very nice!

And the txt marker file is there so that there is no need to modify any files.

For BUG 4.2 I plan to make the config XML files auto-load so you don't have to add your mod's file to init.xml. I think I have a design that will be easy to use but still allow for dependencies (ordering the file loading) in a simple fashion.

The problem is that there is no way to get at the current mod directory from python. BUG forces the user to modify a file whenever he renames the mod.

Dresden found a way to get that folder, and BUG no longer requires modifying any Python files to handle this. Yay! It's in BugPath and involves grabbing it out of the CyReplayInfo object if you'd like to scavenge the code.

I'm very happy to see new work on the Python framework side of things. :goodjob:
 
BUG 4.0 adds support for modular CvGameUtils.

I read about support for CvGameUtils after I posted. And the code for spells and random events is only about 15 lines.

I've tried to strike a balance in BUG in favor of supporting older mods, but there are clearly advantages to other ways too.

Yes, I have no support for old mods. Any merging will need a complete rewrite; not a difficult job, but boring.

It forces any changes to actually be modular. That is if I like most of a mod, but there is just one tiny thing that annoys me, I can easily change that one thing.

I like to think it allows modular python to be faster too.

After a cursory glance over the source of this mod, it appears that you alter the function signatures required for events. Instead of taking a single list of arguments, a dictionary is created. Am I correct? Is there a need for this?

It's correct. It allows the arguments to event handlers to be easily extended, without breaking old code.

For instance, the original arguments to onUnitBuilt are just a city and a unit. I add unit type, unit class, the owner and a few more pieces of information.

The benefit is that the event handlers needn't look up those values themselves, saving a few miroseconds.

Also, it's part of my attempt to make the system coherent. If you get a 'pUnit' key in the dictionary, you know what it is.

Also, it looks like the return from normal event handlers is a list of the returns from each handler. Have you verified that this is okay to do? Most likely the SDK ignores the return result of those handlers, but I try to be more cautious about changing a function's contract.

I'm a bit unclear about what you are thinking. Possibly about the various _handle<something>EventKey<sometime>, but they are internal functions, as evidenced by startig with underscore.

The values returned to the SDK is the same as before, None in most cases.

BTW, when dispatching a normal event I see that you open the error log, dispatch to each handler, and close the log. This may cause a slowdown since the normal operation is for there to be no errors, and it happens on every event. I suggest opening and closing it for each error that occurs. Errors will run slightly slower, but working code will run slightly faster.

Yes, I'll change that.

Does this include XML and Art? If a modmod requires a CIV4GameText.xml file, can it be placed with the Python files? That would be very nice!

Yes. Actually that is thanks to xienwolfs and World of Civilization, and it would be more accurate to say that one can place the python with the XML and art.

Essentially Fall Further 0.51 was released and one could make changes simply by placing art and xml in a subfolder of Assets\Modules. Only python was left unmodular, so I thought "It can't be that hard to bash the event manager to load from there too.".

Dresden found a way to get that folder, and BUG no longer requires modifying any Python files to handle this. Yay! It's in BugPath and involves grabbing it out of the CyReplayInfo object if you'd like to scavenge the code.

Excellent.
 
It forces any changes to actually be modular. That is if I like most of a mod, but there is just one tiny thing that annoys me, I can easily change that one thing.

Can't you still create a bunch of events that all depend on each other? True modularity requires a shift in design--not a change in function signature, unless we're talking about two different things here.

It's correct. It allows the arguments to event handlers to be easily extended, without breaking old code.

Yes, if you alter the SDK that fires the event by adding more parameters, old events can ignore them. I don't really know how often that would be useful as changing the parameters to an event typically means the old parameters weren't sufficient.

What you lose, however, is interoperability. If someone really likes an event using one system, they have to rewrite it significantly to get it to work in the other system. The beauty of CvCustomEventManager is that it didn't require you to alter how you write events; you just put them in a different module.

While not significant, the conversion from a list to dictionary along with dictionary lookups to access parameters adds a small time penalty. I would be very cautious about increasing the runtime cost. Have modders been asking for the ability to have events whose parameters changed over time?

For instance, the original arguments to onUnitBuilt are just a city and a unit. I add unit type, unit class, the owner and a few more pieces of information.

The benefit is that the event handlers needn't look up those values themselves, saving a few miroseconds.

You save time only if you have multiple onUnitBuilt event handlers that need this information. If the event handlers don't need it, you pay the extra cost of looking it up needlessly. Here is where I favor clean coherent helper modules that make the coder's job easier.

The values returned to the SDK is the same as before, None in most cases.

This is what I was talking about. I see that it created a list to hold the return values from each handler, appending each onto the list as it called them. I didn't check what it returned to the SDK, but I assume it returned something from the list otherwise why build it? Again, I only glanced over this part.

Yes. Actually that is thanks to xienwolfs and World of Civilization, and it would be more accurate to say that one can place the python with the XML and art.

Ah, that would be handy. BUG cannot require a custom DLL so that solution is out sadly. I could put the Python and config XML files into a single folder for each module, but the XML and art would have to live in the normal folders, so that's probably pointless and would merely cause confusion. Darn you Civ4! :mad:
 
Can't you still create a bunch of events that all depend on each other? True modularity requires a shift in design--not a change in function signature, unless we're talking about two different things here.

I think we are a little bit...

As an example, the onUnitBuilt event in FFH does a boatload of things. It duplicates units for warrens, grants dwarves extra xp from brewerys, gives amurite units magic promotions and more.

Just adding all that to the onUnitBuilt event is simple. However, If I want to change warrens to produce three goblins instead of two, I'll then have to replace all of it, amurites, dwarves, golems et cetera.

In modular python the event is broken apart. There are several handlers for the onUnitBuilt event, and one can replace only the handler that duplicates units.

I'm sure that's possible in BUG too, just not mandatory. And I could have added the entire original onUnitBuilt to modular python, too.

The most added value is the refactoring of the monolithic CvEventManager into several tiny modular pieces.

There's also the specialized events, to replace multipronged if statements with fast dictionary lookups.

Yes, if you alter the SDK that fires the event by adding more parameters, old events can ignore them. I don't really know how often that would be useful as changing the parameters to an event typically means the old parameters weren't sufficient.

What you lose, however, is interoperability. If someone really likes an event using one system, they have to rewrite it significantly to get it to work in the other system. The beauty of CvCustomEventManager is that it didn't require you to alter how you write events; you just put them in a different module.

The original events are somewhat quirky, differing capitalization and so forth, and I wanted to add some more specialized events and make things easier to remember. By using dictionaries instead of tuples, one can remember a mnemonic key instead of an essentially random integer.

The part about adding parameters to a handler call is more about backwards compatibility for future versions of modular python.

And I think you are overstating the rewrite needed. Change the function signature, extract the arguments from a dict at the start instead of a tuple; ensure proper indentation, slap a decorator on the function and you're done.

While not significant, the conversion from a list to dictionary along with dictionary lookups to access parameters adds a small time penalty. I would be very cautious about increasing the runtime cost. Have modders been asking for the ability to have events whose parameters changed over time?

You save time only if you have multiple onUnitBuilt event handlers that need this information. If the event handlers don't need it, you pay the extra cost of looking it up needlessly.

Dictionary lookups are fast and hard to avoid; every time you access a variable or function or almost anything, there is at least one dictionary lookup, often more. I doubt one more explicit one is going to hurt.

And most, probably all, of the extra arguments in the data passed to the handlers were added because there already were multiple handlers that needed them. Ditto for the specialized event I mentioned earlier.

Here is where I favor clean coherent helper modules that make the coder's job easier.

I too favour clean coherent code, but I fail to see what that has to do with Fall From Heaven. (Not to denigrate the modders of Fall from Heaven, Fall Further et al. I much prefer working code over clean code.)

Seriously though, I don't know how helper modules would help with that.

This is what I was talking about. I see that it created a list to hold the return values from each handler, appending each onto the list as it called them. I didn't check what it returned to the SDK, but I assume it returned something from the list otherwise why build it? Again, I only glanced over this part.

Sometimes it's used, other times not. Depends on what the dll expects.

It hasn't seemed worth the bother of making two different helper funtions for the two cases. Needs profiling to tell for sure.
 
Just adding all that to the onUnitBuilt event is simple. However, If I want to change warrens to produce three goblins instead of two, I'll then have to replace all of it, amurites, dwarves, golems et cetera.

Your onUnitBuilt event example is exactly the type of modularity CvCustomEventManager and BUG allow, so we're on the same page here. What I thought you were saying was that a single modmod with many events would itself be modular, and anyone would be able to pull a single event out and it would magically work without the other events it depended on. Glad to see neither of us is that crazy. :)

Your example also demonstrates the usefulness of the added parameters to the events. In BUG, since most of the events service interface enhancements, the features that fire off the same event--mostly BeginActivePlayerTurn and endTurnReady--only care that the event fired. They don't even need to know the game turn that comes along with the events. Our use cases are very different, and I'm unfamiliar with the FF code, so your example helped a lot.

The original events are somewhat quirky, differing capitalization and so forth.

Don't get me started! I'm constantly having to open CvEventManager to check the correct capitalization of an event key. If you've changed the event keys to be more uniform, you may annoy long-time modders who have memorized the old keys. Given that all events have to be rewritten to use the new system, this isn't such a big deal. But it's the internet, so be prepared for some haters. ;)

By using dictionaries instead of tuples, one can remember a mnemonic key instead of an essentially random integer.

True, but I always just copy the function out of CvEventManager or anywhere else I've defined a handler to remove the possibility of error. I'm not saying a different scheme isn't better, only that a different scheme requires change. In FF's case, it may be worth it given how many events there are that can benefit from it.

The part about adding parameters to a handler call is more about backwards compatibility for future versions of modular python.

Yes, I see that now. You're not considering the SDK changing the parameters it passes but the extra parameters you add. That makes sense.

And I think you are overstating the rewrite needed. Change the function signature, extract the arguments from a dict at the start instead of a tuple; ensure proper indentation, slap a decorator on the function and you're done.

I suppose when dealing with modmodders that are writing original code this is true. With BUG I mostly work with modders that are good with WinMerge to merge in different mods but aren't writing code from scratch.

Dictionary lookups are fast and hard to avoid; every time you access a variable or function or almost anything, there is at least one dictionary lookup, often more. I doubt one more explicit one is going to hurt.

I agree, that's the cost of using Python. Lookup time is O(1) anyway; it's the creation of the dictionary from the list that I'm talking about. This cost will be very, very small. But without the ability to profile the code, I'm being very thorough in my analysis given that you want the FF guys to incorporate it.

Seriously though, I don't know how helper modules would help with that.

I was suggesting writing helper modules that would access that extra information (unit type, class type, etc) in one fell swoop, but given that most of the events require that information it makes sense to pass it in under this system. I haven't read any FFH/FF code, so I don't have a clue about the use cases. I wasn't implying that the FFH/FF or your code isn't clean.
 
v.6 is uploaded.

Some bugfixes, updated to patch C.

The restriction on number of installations is lifted but not quite removed, see the second post for details.
 
The base code was almost untouched by patch C. However the templatemod breaks things.

Any xml errors upon upgrading are likely to be due to the templatemod v.2 , upgrade to v.3 .
 
Any chance you'd be willing to put a version out for FFPlus after I release the patch? Sometime next week, looks like at the moment.

There is some chance... I suppose I'll have to if I want to play with the Mechanos fortresses without having the early game ruined by dawn of man popups.

On a related note, v.6+ has been added to the first post.
 
Undercouncil gambling houses appear to be bugged.

I think this is the fix:
Code:
@event.getBuildingCostMod('BUILDING_GAMBLING_HOUSE')
def getGamblingHouseCost( data ):
    pPlayer = data['pPlayer']
[b]    iBuilding = data['iBuilding'][/b]
    if pPlayer.isGamblingRing():
        return gc.getBuildingInfo(iBuilding).getProductionCost() / 4
 
Version .7beta fixes the three bugs found since version .6beta.
 
Back
Top Bottom