Questions about Python callbacks

General Tso · Jan 13, 2010

Many of the Python callbacks come in two flavors. For example for Civics there is a canDoCivic and cannotDoCivic. I've been doing some tests and now I am confused (nothing new there :crazyeye:

). cannotDoCivic seems to work as expected - returning True causes the AI to stop changing civs, returning False allows them to do so. However, canDoCivic appears to do nothing, the AI always changes civs no matter what the return value is.

I know that some callbacks need to be enabled in PythonCallbackDefines.xml, am I missing something else?

What is the difference between the can and cannot callbacks? If I want to disable a given item like civics is either one of the can or cannot callbacks quicker and more efficient?

Afforess · Jan 13, 2010

General Tso said:
Many of the Python callbacks come in two flavors. For example for Civics there is a canDoCivic and cannotDoCivic. I've been doing some tests and now I am confused (nothing new there ). cannotDoCivic seems to work as expected - returning True causes the AI to stop changing civs, returning False allows them to do so. However, canDoCivic appears to do nothing, the AI always changes civs no matter what the return value is.

I know that some callbacks need to be enabled in PythonCallbackDefines.xml, am I missing something else?

What is the difference between the can and cannot callbacks? If I want to disable a given item like civics is either one of the can or cannot callbacks quicker and more efficient?

Can Do Civic over rides the regular false settings, so that you could potentially use a civic without the requirements. Can Not Do overrides the regular true setting, so that you can not use a civic that you normally could.

Python callbacks are inherently inefficient, because, as I understand it, the inter-language interpreter for C++/python is very inefficient and slow. Since that is handled in the .exe, there is nothing modders can do about it except avoid callbacks at all costs.

If you need callbacks for a lot of things, I highly, highly recommend editing the SDK. The SDK is much much faster and more powerful than python. I started SDK modding, and I had no professional or amatuer C experience, it isn't terribly hard once you get the jist of things.

General Tso · Jan 13, 2010

Can Do Civic over rides the regular false settings, so that you could potentially use a civic without the requirements. Can Not Do overrides the regular true setting, so that you can not use a civic that you normally could.

Thanks, that explains it. The Can Do method works differently than I thought. As far as the SDK - I have a mod that I have been working on for a long time and I'm spending all of my time struggling to get it in to version that's ready to release, so I'm not prepared for a new project right now, but thanks for the answer.

EmperorFool · Jan 15, 2010

You need to consider how frequently a callback is being called to decide whether or not it's going to kill your mod's performance. All of the events are done using the same mechanism (C++ calling out to Python), and Python code runs quite fast. The problem is when you use a callback that will be called to decide how a unit should act as this will be called thousands of times for the AIs' game turn. I doubt the AI's decide on civics changes thousands of times each turn.

The_J · Jan 15, 2010

I guess, the call itself is at this point not so performance intensive, but the new AI decissions are.
That's what i've experienced with the CannotMoveInto callback. It leads to a serious lag in performance, when the AI has to recalculate the route. If that's not the case, the callback itself shouldn't cause bigger issues.

davidlallen · Jan 15, 2010

Some callbacks are called literally millions of times. In my experience cannotMoveInto is the worst, because it is used in all pathfinding calls. Also can/cannotTrain and can/cannotConstruct are used very frequently as each city decides which unit or building to work on. A lot of the other ones are very infrequent, such as onImprovementBuilt, onTechAcquired, etc.

My rule of thumb is, never use python cannotMoveInto; if you *have* to use the train/construct, try very hard to precompute everything, and the other callbacks are probably OK to do something more compute-intensive.

BTW, regarding cannotMoveInto, many things can be covered by using JRouteNative.

The_J · Jan 15, 2010

davidlallen said:
Some callbacks are called literally millions of times. In my experience cannotMoveInto is the worst

You haven't tried "CannotHandleAction", right

?
That's even more horrible, because it's called for every unit and every action (including movement, i guess).

Afforess · Jan 15, 2010

The_J said:
You haven't tried "CannotHandleAction", right ?
That's even more horrible, because it's called for every unit and every action (including movement, i guess).

*shudder*

All...that..lag...

Let's just say if you are using python to change unit actions, you're doing it wrong. :lol:

About python callbacks though, I got some nice speed boosts when I enclosed all the 15 other callbacks in the SDK that weren't in the PythonCallbackDefines.xml, and added them there, then turned them off... Like a 25% speed boost.

. SDK FTW!

davidlallen · Jan 15, 2010

Afforess said:
About python callbacks though, I got some nice speed boosts when I enclosed all the 15 other callbacks in the SDK that weren't in the PythonCallbackDefines.xml, and added them there, then turned them off... Like a 25% speed boost. . SDK FTW!

I find that surprising. When you did this experiment, were any of these callbacks being used to do some actual work? Or were the python callbacks only stubs? If they weren't being used, then I recommend you publish this result and get it included into CAR or BBAI or whatever would be appropriate.

Afforess · Jan 15, 2010

davidlallen said:
I find that surprising. When you did this experiment, were any of these callbacks being used to do some actual work? Or were the python callbacks only stubs? If they weren't being used, then I recommend you publish this result and get it included into CAR or BBAI or whatever would be appropriate.

I did. I posted on the CAR thread, but no one besides Phungus seemed interested. 14/15 of them got pushed into RevDCM's SVN. They were silly things that most people didn't use like AI_doProduction() (Who uses python to tell the AI not to build something instead of using CanNotConstruct!?) and some Unit Upgrade ones... oh and canDoVictory. Heck just go grab the SVN's latest pythoncallbackdefines.xml, I commented the new ones I added at the bottom.

I made sure I wasn't getting rid of ones that did work. I've been playing with these for a few weeks now, they seem to be working fine.

davidlallen · Jan 15, 2010

Afforess said:
They were silly things that most people didn't use like AI_doProduction() (Who uses python to tell the AI not to build something instead of using CanNotConstruct!?)

My understanding of AI_doProduction is different. It can check some conditions and force a particular AI to build a certain building next. This is useful in many situations.

Heck just go grab the SVN's latest pythoncallbackdefines.xml

That appears to be one small part of the solution. You have also changed all of the places in the cpp files where these callbacks are called, to check the associated globaldefine first. I am a little afraid of different revisions of revdcm svn; is this specific change available somewhere else?

Afforess · Jan 15, 2010

davidlallen said:
My understanding of AI_doProduction is different. It can check some conditions and force a particular AI to build a certain building next. This is useful in many situations.

Yes, the function does that, but what about this callback nestled in there:

Code:

CyCity* pyCity = new CyCity(this);
    CyArgsList argsList;
    argsList.add(gDLL->getPythonIFace()->makePythonObject(pyCity));    // pass in city class
    long lResult=0;
    gDLL->getPythonIFace()->callFunction(PYGameModule, "AI_chooseProduction", argsList.makeFunctionArgs(), &lResult);
    delete pyCity;    // python fxn must not hold on to this pointer
    if (lResult == 1)
    {
        return;
    }

If the python returns 1, the function quits, the AI chooses no production next, since this is near the beginning of the function. I can't even imagine a use for that... I know it's not used in any mods I know of. So I put a callback around it, added the callback to Globals cpp/h and the XML. Works like a charm.

davidlallen said:
That appears to be one small part of the solution. You have also changed all of the places in the cpp files where these callbacks are called, to check the associated globaldefine first. I am a little afraid of different revisions of revdcm svn; is this specific change available somewhere else?

In my Mod. Other than that, no.

deanej · Jan 15, 2010

Afforess said:
If the python returns 1, the function quits, the AI chooses no production next, since this is near the beginning of the function. I can't even imagine a use for that... I know it's not used in any mods I know of. So I put a callback around it, added the callback to Globals cpp/h and the XML. Works like a charm.

You can push building orders to cities via python, so that's where it's used. Final Frontier uses it to get the AI to build some needed buildings more often.

For some reason Firaxis seems to have a phobia of SDK modding.

Afforess · Jan 15, 2010

deanej said:
You can push building orders to cities via python, so that's where it's used. Final Frontier uses it to get the AI to build some needed buildings more often.

Wouldn't it be better to actually teach the AI about the building rather then just randomly tell them to build it every so often?

deanej said:
For some reason Firaxis seems to have a phobia of SDK modding.

Not just Firaxis, most modders on CFC are phobic about SDK modding. Their loss...

deanej · Jan 15, 2010

I don't recall if the buildings use anthing other than XML, but Final Frontier changes so much from the base game that the AI might need the help anyways. The function also has a complex weighting system so it's not entirely random (though there is a small chance of letting the AI pick something to build on its own). Believe it or not, but base Final Frontier has no SDK components.

I would think everyone who made official mods at Firaxis has done some SDK work, so I don't know why they avoid it when they make their official mods.

Afforess · Jan 15, 2010

deanej said:
I would think everyone who made official mods at Firaxis has done some SDK work, so I don't know why they avoid it when they make their official mods.

For the patches. If the mod updates the SDK, then they have to patch that, and the base game too. :eek:

That's double the QA time. If every official mod had their own SDK... :mischief:

The_J · Jan 15, 2010

Afforess said:
*shudder*

All...that..lag...

Let's just say if you are using python to change unit actions, you're doing it wrong.

I don't use it, but i've tested it, after i've seen, that the canBuild callback is also performance intensive, and i've searched for a better alternative.
But luckily, i got my other (hacky) XML + python solution to work

.

Afforess said:
Not just Firaxis, most modders on CFC are phobic about SDK modding. Their loss...

It's not only a question of "fear", it's a question of:
- ability
- time
- cross plattform compatibility
- and luck

It's not the case, that everbody has the ability to programm...

Afforess · Jan 15, 2010

The_J said:
It's not only a question of "fear", it's a question of:
- ability
- time
- cross plattform compatibility
- and luck

It's not the case, that everbody has the ability to programm...

Everyone has the ability, but time I understand. For those poor souls with Mac versions of BTS.......

Luck is no doubt a major factor in my SDK success. If at first I don't succeed, I may get lucky the next try.

Questions about Python callbacks

General Tso

Panzer General

Afforess

The White Wizard

General Tso

Panzer General

EmperorFool

Deity

The_J

Say No 2 Net Validations

davidlallen

Deity

The_J

Say No 2 Net Validations

Afforess

The White Wizard

davidlallen

Deity

Afforess

The White Wizard

davidlallen

Deity

Afforess

The White Wizard

deanej

Deity

Afforess

The White Wizard

deanej

Deity

Afforess

The White Wizard

The_J

Say No 2 Net Validations

Afforess

The White Wizard

Similar threads