Efficiency

[Rystic]

This topic is dedicated to the effort of optimizing the python (eliminating unnecessary checks, generating fewer random numbers, etc). Sometimes, Fall Further runs a bit slow. Here, we can brainstorm different ways to make the code more efficient.

Here are a few ideas of my own:

1) Call Golem-Promotion function when Barnaxus promotes, not every turn.
Reasoning: Fantastic drag on speed, especially when Barnaxus is only promoted roughly 5-15 per game. One check per promoted unit is substantially less than checking each Luchuirp unit at the beginning of every turn.

2)For the Khazad, make multiple palaces with "Counts as Vault(X) in every city".
Reasoning: This eliminates cycling through every city each turn.

3)Units getting haunted lands promotions should check with UnitMove, and not every turn.
Reasoning: Stops double-checking. There's no reason a unit that already has haunted-lands promotions should be looked at again.

4) Reformat Lizardman conversion. Have a 10% per turn that conversion will happen, and multiply probabilities of conversion by 10 (eg Swamp is 1% --> 10%)
Reasoning: A lot less random number generation per turn with the same probability something will convert (1 event with 1.5% = 2 events with 10% and 15%). This will lead to "clumping", so that terrain converts more in chucks. Overall, it should be a good choice efficiency-wise.

5) Let deep jungles remain, even if unowned.
Reasoning: This removes an entire check per plot, could save hundreds of checks per turn on a larger map.

6) For deep jungles, check whenever a Lizardman city expands or is founded, instead of each turn.
Reasoning: Saves a good number of checks on large maps. Also, jungles will convert to deep jungle the turn you obtain them!

7) Convert road to trail automatically in Lizardman territory.
Reasoning: This will eliminate a random number check.

8) Wetlands should be replaced with Marsh.
Reasoning: A marsh is a wetland.

9) Make a button to toggle the information gathering (eg, "X will grow to size 10 next turn")
Reasoning: This feature is a drag on speed. Does a bunch of checks that have no effect on gameplay. If toggled, it would count be 1 bool check per turn.

These are just a few ideas I came up with off the bat. Any feedback is welcome.

 

[Honor]

Cool. So which file do we check out to have ideas of our own?

 

[Iceciro]

I love this post. With tounge. Give me more of this stuff and I'll verify/import it into Fall Flat, because one of my regular alpha-testers has a pretty crap CPU.

 

[Vehem]

1) Call Golem-Promotion function when Barnaxus promotes, not every turn.
Reasoning: Fantastic drag on speed, especially when Barnaxus is only promoted roughly 5-15 per game. One check per promoted unit is substantially less than checking each Luchuirp unit at the beginning of every turn.

Would also need it on unit creation, which then requires for each individual unit created a loop over potentially all units to find Barnaxus and look at his promotions. That is potentially a lot of separate loops over all units per turn, as opposed to the current single loop over all units (which finds Barnaxus and creates an optimized list of only Golems) followed by a second loop over a reduced set of units.

Some turns it'd be quicker this way, but others could be a *lot* slower. Also doesn't provide any maintenance - if a unit doesn't get the promotion for whatever reason (created in an obscure way or captured from another civ), they wouldn't be "fixed" until Barnaxus next leveled.

2)For the Khazad, make multiple palaces with "Counts as Vault(X) in every city".
Reasoning: This eliminates cycling through every city each turn.

Not a bad idea, though I don't think it's a particularly slow loop to carry out anyway. Cities tend to be in the dozens and the check/action is simple, whilst slow loops tend to be in the hundreds or thousands of entries or have a complex action to perform.

Might be worth look at anyway though...

3)Units getting haunted lands promotions should check with UnitMove, and not every turn.
Reasoning: Stops double-checking. There's no reason a unit that already has haunted-lands promotions should be looked at again.

Haunted Lands have a chance of causing other things to happen once per turn (Creeper spawning/Ghostwalkers, Estranged units etc) beyond the promotion effects. Standing still doesn't prevent that from happening.

4) Reformat Lizardman conversion. Have a 10% per turn that conversion will happen, and multiply probabilities of conversion by 10 (eg Swamp is 1% --> 10%)
Reasoning: A lot less random number generation per turn with the same probability something will convert (1 event with 1.5% = 2 events with 10% and 15%). This will lead to "clumping", so that terrain converts more in chucks. Overall, it should be a good choice efficiency-wise.

Efficiencywise it would work, but it would just mean that you'd get "Nothing Changes, Nothing Changes, Nothing Changes, Nothing Changes, Everything Changes" if I'm understanding you right - which isn't what is needed at all.

Or do you mean that the whole routine works as currently and is only checked on 10% of all turns (with the individual chances scaled up)?

5) Let deep jungles remain, even if unowned.
Reasoning: This removes an entire check per plot, could save hundreds of checks per turn on a larger map.

AI would then consider the terrain to be "good" and settle in it. Assuming that Deep Jungle owned by non-lizards would revert back to standard Jungle, they'd find themselves with a terrible city where they expected a good one.

6) For deep jungles, check whenever a Lizardman city expands or is founded, instead of each turn.
Reasoning: Saves a good number of checks on large maps. Also, jungles will convert to deep jungle the turn you obtain them!

Feasible, though won't handle foreign culture encroaching on Lizard lands, or Lizards pushing back a neighbour's culture after the city already "expanded". The only way to hook that is to call it on any change of cultural ownership, which is going to be far worse.

7) Convert road to trail automatically in Lizardman territory.
Reasoning: This will eliminate a random number check.

Eliminates a check, also eliminates the purpose of the check...

8) Wetlands should be replaced with Marsh.
Reasoning: A marsh is a wetland.

Already been stated to happen once an SDK change has been made to allow *both* the Mazatl and Cualli to treat Marsh as beneficial terrain (can currently do it for one or the other, but not both)

9) Make a button to toggle the information gathering (eg, "X will grow to size 10 next turn")
Reasoning: This feature is a drag on speed. Does a bunch of checks that have no effect on gameplay. If toggled, it would count be 1 bool check per turn.

I'll look at re-enabling the config file for this - but it's a fairly standard Civ4 mod and hasn't been reported as causing any significant slowdown.

 

[xienwolf]

A while ago someone had run a profiler on the python for us and found that (remarkably), very little of our turn delay is in python. I always loved to blame python for being the slowdown, but had to stop after that.

 

[Thunder_Gr]

Would also need it on unit creation, which then requires for each individual unit created a loop over potentially all units to find Barnaxus and look at his promotions. That is potentially a lot of separate loops over all units per turn, as opposed to the current single loop over all units (which finds Barnaxus and creates an optimized list of only Golems) followed by a second loop over a reduced set of units.

Some turns it'd be quicker this way, but others could be a *lot* slower. Also doesn't provide any maintenance - if a unit doesn't get the promotion for whatever reason (created in an obscure way or captured from another civ), they wouldn't be "fixed" until Barnaxus next leveled.

I have an objection here. Since this type of checking is really time-consuming.
What if you had a pointer to Barnaxus for the Lurchuip. At all times. If barnaxus is dead the pointer is null. So, at the OnCreate(or whatever is the name of the event), for the Lurchuip units, if the unit is golem, you apply the promotions readily, without having to iterate through everything to find Barnaxus and then again to apply the promotions...Just an idea.

 

[odalrick]

Efficiencywise it would work, but it would just mean that you'd get "Nothing Changes, Nothing Changes, Nothing Changes, Nothing Changes, Everything Changes" if I'm understanding you right - which isn't what is needed at all.

Or do you mean that the whole routine works as currently and is only checked on 10% of all turns (with the individual chances scaled up)?

I assumed he meant something like only checking for tiles where tile_index % 10 == turn_number % 10 .

The problem though is that without profiling, all you're actually doing is replace tested/bugfree code with untested/buggy code.

And as xienwolf said, profiling tells us that the python isn't particularly expensive.

 

[Iceciro]

Crud. I really wanted a fix-it for the issues, but I guess in general Fall Further is more CPU-intensive than Fall From Heaven, which is then more CPU intensive than BTS.

 

[Rystic]

I assumed he meant something like only checking for tiles where tile_index % 10 == turn_number % 10 .

The problem though is that without profiling, all you're actually doing is replace tested/bugfree code with untested/buggy code.

And as xienwolf said, profiling tells us that the python isn't particularly expensive.

Maybe we should look into some of the added C++ code?

 

[Rystic]

A while ago someone had run a profiler on the python for us and found that (remarkably), very little of our turn delay is in python. I always loved to blame python for being the slowdown, but had to stop after that.

How long ago was it?

 

[Rystic]

Cool. So which file do we check out to have ideas of our own?

CustomFunctions.py and CvEventManager.py

 

[Vehem]

I assumed he meant something like only checking for tiles where tile_index % 10 == turn_number % 10 .

I think Kael actually used that method to handle lair spawning up until an early version of Shadow.

The problem though is that without profiling, all you're actually doing is replace tested/bugfree code with untested/buggy code.

And as xienwolf said, profiling tells us that the python isn't particularly expensive.

Aye - any major "optimization" project will need some evidence that indicates that it's worthwhile...

How long ago was it?

Just either just prior to or just after the release of 050 as I recall - a couple of the performance tweaks regarding terrain alteration was prompted by that.

 

[xienwolf]

We run a profiler on the C++ already, I just need to figure out which chunks of code might be causing significant slowdown and ensure that they are included in the Profile. So far with everything I would suspect to be an issue added I still fail to account for even 5% of the total turn time in what is displayed, which would imply it is .exe stuff, or just a group of functions I haven't found yet (if it is .exe stuff, that means it is mostly graphical in nature. But I run my laptop with absolute minimal graphics, so what I log is the quickest possible for the code)

 

[MagisterCultuum]

Wouldn't it make more sense to give Barnaxus a PyPerTurn call that to give other golems their free promotions, and a PythonOnDeath call to remove them?

 

[[to_xp]Gekko]

ah, this is a genius thread. FF really needs to get faster, or else I'll need a new computer

 

[Rystic]

Wouldn't it make more sense to give Barnaxus a PyPerTurn call that to give other golems their free promotions, and a PythonOnDeath call to remove them?

I don't think a per turn function is that efficient. I'm trying to come up with a function that doesn't double-check golems that have the promotions already.

 

[Mylon]

The most troublesome part I've found with this game is not that it's so CPU-intensive between turns, but the game only seems to use only one of my four cores. Which kinda means my computer's strength doesn't come into play when playing this.

 

[Vehem]

Wouldn't it make more sense to give Barnaxus a PyPerTurn call that to give other golems their free promotions, and a PythonOnDeath call to remove them?

That would eliminate the lesser of the two loops and should work well. Don't think it'll do anything amazing in terms of improvement, but it does sound a better way to do it in theory at least.

 

[Azhral]

The biggest drain on resources I've seen is the automated AI pathfinding. Just watch Crazed units run around, or put a unit on Auto Explore on an island and watch them go. Not such a big issue in itself, but imagine this happening with 50% of all units under AI control. Just build a dozen Lunatics on an island and watch how much processing time they are using.

 

[xienwolf]

Only better approach would be to get some C++ control going by holding an array on the player which states which free promotions he offers, but since you have a limited target, you would have to specify that as well, and then it gets complicated quickly to make it decently generalized and of use elsewhere.


AI movement is one huge area that needs drastic improvement. But it is not one easily tackled.