Modular python possible?
- Thread starter deadliver
- Start date
EmperorFool
Deity
I don't believe so, but I'm not 100% sure. I'd be really surprised if that was the case. First, it's called "Modular XML Loading" in the INI file. Second, It was originally added by a community coder in the SDK, and the SDK is not responsible for loading Python. Third, Python works quite differently from XML.
davidlallen
Deity
There is no way to drop in new python without some kind of merging of source files. There are several ways to do this. BUG has a nice way where you must make a one line addition to their event manager file, and then you can do pretty much anything you want with events such as onEndGameTurn. However, if you want to hook into the functions in CvGameUtils such as AI_unitUpdate, you still have to change that file. There is no handy callback mechanism for that.
EmperorFool
Deity
This would be pretty easy to build into BUG, however I didn't expect multiple mods that work together to operate through this class. Is that actually happening?
I would essentially make the class function similar to CvEventManager where you register the callbacks you want. Given that the function names are fixed, I could even make it auto-discover the linkages--you'd just have to tell BUG about your class/module.
davidlallen
Deity
I thought that was the whole purpose of that part of the bug event manager. If you look at GodsOfOld.py, they put all their own hooks, which makes it a mess to merge multiple python mods. There is an "old way" by subclassing CvEventManager, but what you (was it you?) did in bug is better. Using callbacks is definitely the way to go.
I am currently working on Dune Wars, which is based on RevolutionDCM, which is (I guess) written on top of BUG. I lose track of which mod contains which other mod. Anyway, RevolutionDCM defines RevolutionInit, which is called by the BUG event manager. So I just added one line into BUGEventManager, to call my own DuneWarsInit. Then I can hook onto whatever events I want, in my own file, and nobody else needs to know. So somebody else could write a mod on top of DuneWars, and add their own line into BUGEventManager.
I also hook into a few functions in CvGameUtils. In Fury Road I hooked about eight, in Dune Wars so far I am only hooking AI_unitUpdate. This is another area where mods may often collide. Having a similar callback registration mechanism would be nice; but I would hardly call it "important".
I am currently working on Dune Wars, which is based on RevolutionDCM, which is (I guess) written on top of BUG. I lose track of which mod contains which other mod. Anyway, RevolutionDCM defines RevolutionInit, which is called by the BUG event manager. So I just added one line into BUGEventManager, to call my own DuneWarsInit. Then I can hook onto whatever events I want, in my own file, and nobody else needs to know. So somebody else could write a mod on top of DuneWars, and add their own line into BUGEventManager.
I also hook into a few functions in CvGameUtils. In Fury Road I hooked about eight, in Dune Wars so far I am only hooking AI_unitUpdate. This is another area where mods may often collide. Having a similar callback registration mechanism would be nice; but I would hardly call it "important".
EmperorFool
Deity
Yes, it is. I was proposing doing the same thing to CvGameUtils.
Dr. Elmer Jiggles created CvCustomEventManager before I got into modding Civ4, and it used a method where you added two lines to it for your mod: one to import your module and another to hook it into the manager.
I improved on this design by building the XML configuration so you don't have to modify BugEventManager. Instead, you add a line to your mod's XML file that tells BUG to hook in your event manager or individual events. If you're modifying BugEventManager.py, you should definitely read this page as you're making your life harder.
The key consideration would be if as a modder you ever merge two mods that modify this file. Your two examples make it sound like only you are adding the hooks to that file; is that right?
davidlallen
Deity
I will check out the xml thing. In Dune Wars, which keldath started and I am just helping out, we started with a pretty big pile of mods already merged together. Read the first post of the Dune Wars thread, it lists like 15 mods already merged in. There are multiple changes by different parties in CvEventManager which did not use either the BUG method or the DrJiggles method. There are also multiple changes by different parties in CvGameUtils, including at least terraforming, inquisitor (GodsOfOld), and RevolutionDCM. To be honest, it's kind of a mess in there.
It is at least possible to combine event mods neatly with the callback method. Having a callback mechanism on the CvGameUtils entry points would be helpful for the future, although the old mods may be unlikely to be revised to use it.
The other place in python where I see merged mods colliding is CvMainInterface.py; this is where you go to add new action buttons. GodsOfOld got here first, and others have left their own trails. This one might be hard to add an abstract interface, and probably fewer modders reach this level. But it can still get messy.
It is at least possible to combine event mods neatly with the callback method. Having a callback mechanism on the CvGameUtils entry points would be helpful for the future, although the old mods may be unlikely to be revised to use it.
The other place in python where I see merged mods colliding is CvMainInterface.py; this is where you go to add new action buttons. GodsOfOld got here first, and others have left their own trails. This one might be hard to add an abstract interface, and probably fewer modders reach this level. But it can still get messy.
Nice. my module has a tiny amount of python dealing with events included specifically for a civic. All it is really is an expanded version of SlaveRevolt in the randomeventsinterface file. So I could make a py file specifically for my new events?
EmperorFool
Deity
The "events" in this discussion are not related to the "random events" added in BTS. Civ4 uses an event-based system to allow mods to perform actions when various things happen in the game: a player is met, war is declared, a city is founded, a unit is killed, etc. These are triggered by the SDK and sent to CvEventManager via CvEventInterface.onEvent().
Note that the BUG method is essentially the same as Jiggles's method. BUG has two differences. The first is that you no longer have to modify the event manager class itself; you hook up your events in XML. Doing it this way is strongly recommended so you don't have to re-merge to upgrade BUG in your mod.
The second difference is that you can use module-level event handlers without the need of an "event manager" class yourself. This method is a little easier if you don't mind being tied to BUG. If you want to release your mod without BUG, don't use this method. Stick with what that wiki page calls "old-style" event managers using the <events module="FooEventManager"/> XML element.
Okay, I'll take a look at that file (I have DuneWars installed) and see if I can get a sense of how I could make it more modular. It's always easier to have an example to go from. Am I correct that those individual mods each had their own CvGameUtils modules, and you merged them all into one file? Fairly easy as long as no two mods override the same callback. Did any do that?
True, but if one modder does it, the others can use their version instead. My goal was to create a forum for modders to post updates of other people's mods they BUGified. Looks like it's time for that.
This could certainly use a lot of work. The main issue is that because it's building a huge screen where each part works with the other parts, it's difficult to make it modular. I was tempted early on to split it apart into multiple modules; that would have helped. At this point, however, it's such a hodgepodge that it would be a lot of effort, though it would have been a lot back then too.
It's possible that we could build some hooks into it for a few things that get modded frequently, such as action buttons as you said. How exactly does that module get modified to handle a new action button? I thought that was done automatically by modifying the SDK.
davidlallen
Deity
davidlallen said:
EmperorFool said:
I have rechecked my facts. I think I have accidentally clobbered some stuff, anyway. In version 1.1 of DuneWars, you will see terraforming and dunewars changes in CvEventManager. This is before I cleaned up my own approach, in patch 1.1.1 of DuneWars. In doing that I apparently pulled out a vanilla CvEventManager, which clobbered the terraform stuff. Both terraform and dunewars could have avoided any collision using the BUG stuff we have discussed. So, no infrastructure improvement needed here, just better modders
In CvGameUtils, RevolutionDCM mods a few routines such as isVictory, canTrain. I have modded AI_unitUpdate. There are no overlaps in the same functions. I have seen a number of other mods with more extensive changes in this file, such as Final Frontier and my own Fury Road. Merging requires carefully putting your changes into the same file as the other guy's changes. So, having callbacks would make this more maintainable.
In CvMainInterface, there are dozens of different BUG components. I think we agree this is beyond help.
EmperorFool said:
Ah, there is magic. I am not sure if anybody besides me, and apparently Gods Of Old, has mastered adding action buttons purely in python. Most people have probably done this in sdk. I do not work in the sdk, so I can't comment if it is harder or easier there. I never wrote a real tutorial, but this post lists the steps.
EmperorFool
Deity
I've started a new thread about making GameUtils modular.
Yeah looks awesome and seems simple enough I could use it. I have one question though. You mentioned BUG. Does that mean that I would still need to modify the python of the mod I am making a module for?
davidlallen
Deity
deadliver said:
Sorry, it seems we totally hijacked your original thread and went way beyond the level of detail you requested. I haven't used randomeventsinterface much, and I do not see a slave revolt entry in the vanilla bts file. The comments at the beginning of the file read:
Code:
# These functions are App Entry Points from C++
# WARNING: These function names should not be changed
# WARNING: These functions can not be placed into a classSo I am guessing no, there is no way to "drop in" changes for this. The only way you can do it is to supply your own randomeventsinterface.py file. If your mod, and some other mod, have both changed this file then there will be a problem. You or the other modder will have to manually merge your changes into a single file.
A good approach would be to write your own python file which defines a class and contains your new function. Then in randomeventsinterface, import your python, and make a *one line* change to the file which calls your new function. Add comments around your changes which can be easily searched for. This will make the merge as easy as possible, and gives you the best protection from other modders accidentally stomping your change.
EmperorFool
Deity
That thread I linked is specifically for those writing mods to work with BUG or merging existing mods into BUG. If you use method 1 that I proposed in the tutorial linked from that thread you would not have to modify your Python code to use it with BUG later.
