Some C++ Tips to make CvGameCoreDLL to run faster

[mediv01]

Recently I am working on a Mod -Mod base on the Dawn of Civilization, and I figure out some tips to optimize the speed of the game and the usage of the RAM. Anyone who is interest in CIV4 DLL compiler can discuss it.
Tips1: Optimize the use of the globalDefineAlt
Do not direct use GC.getDefineINT("XXX") in hot function, because it is very slow when it is used over 100000 times
You should use int variable cache in CvGlobal.h as

and use GC.XXX instead of GC.getDefineINT("XXX") ; Or use GC.getXXX(） { return XXX；}

it may improve the speed about 5% - 10% time faster

 

[keldath]

HI
very true.

this was done in other mods, Advanced Civ, Kmod, Realism Invictus and more.
along many many other changed.
i highly recommend to use Kmod/kek mod/Advanced Civ as a mod basis.

 

[mediv01]

HI
very true.

this was done in other mods, Advanced Civ, Kmod, Realism Invictus and more.
along many many other changed.
i highly recommend to use Kmod/kek mod/Advanced Civ as a mod basis.

Yes if I want to write a new Mod, I will choose these mods as Base
However, My Mod base is Dawn of Civilization 1.16.2, and further RFC MOD
it is hard to rewrite all existed code.

 

[mediv01]

Tips2: if you want to write doTurn logic , write it in DLL , not in Python


use CvGlobals::doTurn() or CvPlayer::doTurn in C++
instead of checkTurn() in Python::CvEventHandler

 

[mediv01]

Tips3: updatePlotGroups in CvPlayer::doTurn, not in CvPlot::doTurn

if updatePlotGroups () in CvPlot::doTurn, it will be calculated over 40000 times per turn.
if we update updatePlotGroups () in CvPlayer::doTurn(), it can save much more time

 

[mediv01]

Tips4: Use cache can accelerate the speed of do turn

I have read some C2C code, I found many cache in CvCity and CvPlayer

 

[mediv01]

Tips5: Use stdext::hash_map instead of std::map
In vc7.1(vc 2003), the performance of std::map is poor because it needs sort, and it's speed is O(N)
use the stdext::hash_map can have the O(1) speed to search because it use hash function.

 

[mediv01]

Tip6: Do not store huge data in Python, store it in C++

In DOC, there are huge data in SettlerMap WarMap and CityNameData
I calculate the RAM usage, it reach almost 300MB in game
I introduce it into C++ , and the data of RAM is less than 30MB

 

[mediv01]

Tip7: Use proper struct in cpp can speed up the checkturn
I use delay queue in CPP, it can do some temp bonus or punishment
use delayQueue struct will be faster

 

[mediv01]

Tips8: less use gc.getInfoTypeForString("XXX")

gc.getInfoTypeForString("BUILDING_UNIVERSITY") is very slow
use the enum BUILDING_UNIVERSITY can be faster

 

[mediv01]

Tips9: Use Inline in hot function

can both inline the CvPlot::getOwner() and CvPlot::getOwnerINLINE() to speed up

 

[mediv01]

Tips10: rewrite time cost function in cpp

for example, if I want to know which player control most lands , I can loop the plot in Python
However, it is very slow and it cost almost 1-2 seconds to calculate
I rewrite it in c++, and it works 10 - 50 MS to calculate

if you need to calculate it twice per turn, you can use the DoTurnCache to cache the calculation result per turn.

 

[keldath]

wow very nice!
i will look for places that the mods i use did not apply these.

thank you very much, please keep the tips coming!


))

 

[Nightinggale]

Tips1: Optimize the use of the globalDefineAlt
Do not direct use GC.getDefineINT("XXX") in hot function, because it is very slow when it is used over 100000 times

Tips8: less use gc.getInfoTypeForString("XXX")

Tip 1 is so true. In We The People (Religion and revolutions back then) I managed to lower the AI CPU usage by 1% by caching a single call to getDefineINT. 1% may not sound like a whole lot, but I think it is considering that it was a single line of code.

I have gone all in to get rid of getDefineINT and getInfoTypeForString in We The People. What I have done is written some perl scripts, which the makefile will run prior to compiling. Those will (among other things) loop through all xml files to harvest Type tags and then build enum header files. This way if somebody adds something new to xml, next time the dll is compiled, the enum will be updated to match.

Using enum values will always be the fastest solution because the compiler will see them as ints. If you write UNIT_WARRIOR and it is 4 in the xml file (didn't check, example), then the compiler will compile like you wrote 4 in the code meaning it will skip any code to look up the value entirely.

That is if the compiler is set to hardcode. If set to dynamic (adapt to changes in xml without recompiling), it will script making global const references to variables, which are set in a cpp file, which is called once right after loading xml and then never again. This time UNIT_WARRIOR is set at startup and can be used like an enum, hence same code.

Same treatment of globalDefine(ALT).

If getInfoTypeForString is needed, I made it type specific meaning if I need a UnitClassTypes, it will loop just CvUnitClassInfos to look for a match and return the result as UnitClassTypes. No need to waste time scanning strings from multiple files and no risk of name clashes between files.

CyEnumsInterface.cpp will set up python enums. It seems like vanilla haven't discovered this, but whatever c++ value is given to the enum doesn't have to be known at compile time in c++ despite the enum name. It's perfectly fine to write a loop to provide all the types from an xml file. That is you should be aware that not all xml data has been read prior to initializing python, so there might be a problem there if you aren't careful. I can't remember what has been read at this point because I modded the read order in WTP.
Long story short: with enough modding, it is possible to add UnitTypes.UNIT_WARRIOR and all the others to python and that way completely skip the need for python using getInfoTypeForString.

One huge benefit from using enum constants rather than looking up the value from a string is that if the xml entry is removed or renamed, then the enum constant is renamed, which in turn will make the compiler fail on the line using the old name. If it's a call to getInfoTypeForString, then the compiler will have no way of knowing that it will fail. I have caught multiple bugs this way. I also recently added a script to scan the python files to see when enum types are used where it checks if the enum value exist. That too has managed to catch renamed enum values.

I'm not sure how much of this is easy to copy as I haven't tried to make it modular. However I do feel like this has considerations worth adding here.

 

[keldath]

how do you guys do the tests to see what takes long/high memory ?

 

[mediv01]

how do you guys do the tests to see what takes long/high memory ?

Run the game in Civ4 to test speed & memory

 

[keldath]

humm...
what do you mean?

i thought you use some profiler or something to measure it.

 

[Nightinggale]

EDIT: to clear up a misunderstanding: this is NOT me in that video. It's a semi random video I found on youtube on this topic.

 

[keldath]

thanks for the video.

what do you recommend to do to replace gc.getInfoTypeForString(.. ?

i haven't quite understood the alternative, some placed its better then running a loop on all values no?

 

[Nightinggale]

what do you recommend to do to replace gc.getInfoTypeForString(.. ?

CyEnumsInterface.cpp

    python::enum_<BuildingTypes>("BuildingTypes")
        .value("NO_BUILDING", NO_BUILDING)
        .value("BUILDING_CHICKEE", BUILDING_CHICKEE)
        ;

Adding values here allows using them in python (in this example BuildingTypes.BUILDING_CHICKEE).
It should be noted that value takes two arguments (string, int) meaning you can add say

.value("NUM_BUILDING_TYPES", (BuildingTypes)GC.getNumBuildingInfos())

Vanilla relies on enums for this, but that's not a requirement and runtime calculations are allowed. Just be aware that it's fairly limited which xml files have been read when CyEnumsInferface.cpp is executed unless you mod xml read order.

gc.getInfoTypeForString("BUILDING_CHICKEE")

will be perfectly happy staying error free in the code and return -1 if the type doesn't exist. The c++ value call will fail if BUILDING_CHICKEE isn't present, hence better error detection in case xml changes.

boost::python::enum_<BuildingTypes> BuildingTable = python::enum_<BuildingTypes>("BuildingTypes")
        .value(NO_BUILDING, NO_BUILDING)
        .value(NUM_BUILDING_TYPES, (BuildingTypes)GC.getNumBuildingTypes());

    for (int i = 0; i < GC.getNumBuildingTypes(); i++)
    {
       BuildingTypes eBuilding = (BuildingTypes)i;
       BuildingTable.value(GC.getBuildingInfo(eBuilding).getType(), eBuilding);
    }

It's possible to add values in a loop like this.

EDIT: I think building info xml haven't been read yet when exposing enums to python. That is an issue there to look into.