1. We have added a Gift Upgrades feature that allows you to gift an account upgrade to another member, just in time for the holiday season. You can see the gift option when going to the Account Upgrades screen, or on any user profile screen.
    Dismiss Notice

Python Performance and Interface Overhaul (PPIO)

Discussion in 'C2C mod mods' started by Toffer90, Jul 7, 2017.

  1. KaTiON_PT

    KaTiON_PT Emperor

    Joined:
    Jan 2, 2010
    Messages:
    1,090
    Location:
    Portugal
    Interestingly I cannot seem to make CvPlayer::calculateScore show up in the profiler...
     

    Attached Files:

  2. Toffer90

    Toffer90 C2C Modder

    Joined:
    Oct 16, 2011
    Messages:
    7,204
    Location:
    Norway
    Do you think the code is not processed at all, or do you think the profiler is simply not detailed enough?

    Can you see this one in the profiler?
    CvGame::updateScore(...)
    (The main place where CvPlayer::calculateScore is called from)

    Or these?
    CvGame::doTurn()
    CvGame::update()
    (The main places where CvGame::updateScore(...) is called from)
     
    Last edited: Jan 10, 2019
  3. KaTiON_PT

    KaTiON_PT Emperor

    Joined:
    Jan 2, 2010
    Messages:
    1,090
    Location:
    Portugal
    @Toffer90 I could find CvGame::doTurn() and CvGame::update(), like you've said the profiler might not be detailed enough to see CvPlayer::calculateScore nor CvGame::updateScore...

    First thing of notice: in vanilla the score doesn't even update (stays at 0) if using the python call. :lol:

    Five repetitions were done for each scenario in a single save. Before measuring each scenario a recalculation followed by a save and a load was made.

    So the results:

    Vanilla with C++ code: 3464 ms
    Vanilla with Python call: 3453 ms < -- Could be taking less time because it isn't used... the score wasn't getting updated.
    PPIO with C++ code: 3434 ms
    PPIO with Python call: 3417 ms

    This show that your changes have indeed made it faster.

    +1.38% faster vs. Vanilla with C++ code
    +1.05% faster vs. Vanilla with Python call
    +0.5% faster vs. PPIO with C++ code
     

    Attached Files:

    Last edited: Jan 11, 2019
  4. Toffer90

    Toffer90 C2C Modder

    Joined:
    Oct 16, 2011
    Messages:
    7,204
    Location:
    Norway
    3 and a half second, you are measuring too much code to make me believe the results about the score code can be conclusive.

    Did you measure a complete run of one of these functions: CvGame::doTurn() or CvGame::update() ?

    Can't you use some standard library function like clock()

    clock_t startTime = clock();
    ... ... ... // Any code
    clock_t endTime = clock();
    clock_t clockTicksTaken = endTime - startTime;

    print/log/check clockTicksTaken

    Thanks for entertaining my curiosity btw, and if you are going to be a good C2C modder you will need to learn how to check code performance reliably at any rate. We need performance minded coders more than anything atm. ^^
     
    Last edited: Jan 11, 2019
  5. KaTiON_PT

    KaTiON_PT Emperor

    Joined:
    Jan 2, 2010
    Messages:
    1,090
    Location:
    Portugal
    Oh, I think I indirectly found a way to access the functions, take a look in the attachments.

    If that is what I think it is, then CvPlayer::calculateScore takes 6 ms to run with PPIO+Python call.
     

    Attached Files:

    • 1.png
      1.png
      File size:
      103.2 KB
      Views:
      49
    • 2.png
      2.png
      File size:
      103 KB
      Views:
      36
    • 3.png
      3.png
      File size:
      104.8 KB
      Views:
      44
  6. Toffer90

    Toffer90 C2C Modder

    Joined:
    Oct 16, 2011
    Messages:
    7,204
    Location:
    Norway
    I found this C2C dll code used in a couple of places:

    XMLPlatformUtils::Initialize()
    unsigned long duration;
    const unsigned long startMillis = XMLPlatformUtils::getCurrentMillis();
    ... ... // some code
    const unsigned long endMillis = XMLPlatformUtils::getCurrentMillis();
    duration = endMillis - startMillis;
    XMLPlatformUtils::Terminate();​

    It looks like a more reliable timer than the standard clock() function which returns amount of CPU clock ticks elapsed since program started, which can be a bit unreliable as a time measurement.

    This is how it is printed, though I don't know where it prints it:

    XERCES_STD_QUALIFIER cout << xmlFile << ": " << duration << " ms" << XERCES_STD_QUALIFIER endl;
    I don't know enough about VS to confirm or refute it.
     
    Last edited: Jan 11, 2019
    KaTiON_PT likes this.
  7. Thunderbrd

    Thunderbrd C2C War Dog

    Joined:
    Jan 2, 2010
    Messages:
    28,064
    Gender:
    Male
    Location:
    Las Vegas
  8. Toffer90

    Toffer90 C2C Modder

    Joined:
    Oct 16, 2011
    Messages:
    7,204
    Location:
    Norway
    In the logs folder I looked at bull.log, it contained a single line:
    [24388.094] isBug: true​

    Searching for "isBug:" in sources lead me to this:

    const char* format = "isBug: %s", bIsBug ? "true" : "false"
    static char buf[2048];
    _vsnprintf(buf, 2048 - 4, format, (char*)(&format + 1));
    gDLL->logMsg("bull.log", buf);​

    That may be a better way to log stuff. ^^
     
  9. HorseshoeHermit

    HorseshoeHermit 20% accurate as usual, Morty

    Joined:
    Apr 5, 2013
    Messages:
    1,306
    Location:
    Canada
    I attempted to just paste into the C2C folder with overwrites, but that caused a system failure after "GFC" failed to initialize a "control theme" ?

    This was on trying to load C2C as a mod. Latest SVN revision. I attempted twice. (Also, it is a fresh C2C with no UserSettings.) I exported the SVN fresh, and was able to generate a new game.

    If I brave reproducing the crash again, I will have Python exceptions unhidden.
     
  10. GiuseppeIII

    GiuseppeIII The Third Giuseppe

    Joined:
    Jul 12, 2011
    Messages:
    648
    Location:
    Chicago IL
  11. HorseshoeHermit

    HorseshoeHermit 20% accurate as usual, Morty

    Joined:
    Apr 5, 2013
    Messages:
    1,306
    Location:
    Canada
    It's Caveman2Cosmos
     
  12. Toffer90

    Toffer90 C2C Modder

    Joined:
    Oct 16, 2011
    Messages:
    7,204
    Location:
    Norway
    What is your full folder path for C2C?
     
  13. HorseshoeHermit

    HorseshoeHermit 20% accurate as usual, Morty

    Joined:
    Apr 5, 2013
    Messages:
    1,306
    Location:
    Canada
    [My Documents]\My Games\beyond the sword\MODS\Caveman2Cosmos (steam installation).

    My Documents is in a different drive from my steam folder, and neither one is C:
     
  14. Toffer90

    Toffer90 C2C Modder

    Joined:
    Oct 16, 2011
    Messages:
    7,204
    Location:
    Norway
    That explains it then, C2C should not be in My Documents.

    C2C should be in the "mods" folder that is in the same folder as the Civ4BeyondSword.exe file is in.

    A lot of errors can occur if C2C is installed in My documents even without PPIO.

    Here's my path as an example:
    C:\SteamLibrary\steamapps\common\Sid Meier's Civilization IV Beyond the Sword\Beyond the Sword\Mods\Caveman2Cosmos
    The bold is the important part

    That's uninportant, here's my path for My Documents.
    D:\My Stuff\Documents\My Games\Beyond the Sword​
    I changed it to save space on my small SSD C system drive.
    A lot of software throw a lot of junk files into "My Documents" all the time. Especially games whom almost all put game settings and save files in there, I play a lot of games and I like to have many saves, it can amount to GB of saves.
     
    Last edited: Jan 13, 2019
    HorseshoeHermit likes this.
  15. Toffer90

    Toffer90 C2C Modder

    Joined:
    Oct 16, 2011
    Messages:
    7,204
    Location:
    Norway
    Have you concluded your involvement in this analysis?
    Are you certain my python code is faster than the current dll code? If you are, I will add my code to the SVN, and you can follow up by removing the dll score calculation code and un-comment the python call.

    Example of a good way to measure code performance (that may work without any altercations, I'm not too sure about the logging code I added):

    int CvPlayer::calculateScore(bool bFinal, bool bVictory) const
    {
    PROFILE_FUNC();

    if (!isAlive())
    {
    return 0;​
    }
    if (GET_TEAM(getTeam()).getNumMembers() == 0)
    {
    return 0;​
    }​

    XMLPlatformUtils::Initialize() \\ This seems necessary from the other code I looked at.
    unsigned long duration;
    // Measure python performance
    unsigned long startMillis = XMLPlatformUtils::getCurrentMillis();


    long lScore = 0;
    CyArgsList argsList;
    argsList.add((int) getID());
    argsList.add(bFinal);
    argsList.add(bVictory);
    PYTHON_CALL_FUNCTION(__FUNCTION__, PYGameModule, "calculateScore", argsList.makeFunctionArgs(), &lScore);

    lScore = (int)lScore
    // return ((int)lScore); // commented out because we want the rest of the code here to run.

    unsigned long endMillis = XMLPlatformUtils::getCurrentMillis();
    duration = endMillis - startMillis;
    // Python performance measured.

    // log result. Could you perhaps look at the variable value for duration in VS at this point?
    TCHAR szOut[2048];
    sprintf(szOut, "Dll duration: %d ms.\n", duration);
    gDLL->logMsg("Test.log", szOut);

    // Measure dll performance
    startMillis = XMLPlatformUtils::getCurrentMillis();


    int iPopulationScore = getScoreComponent(getPopScore(), GC.getGameINLINE().getInitPopulation(), GC.getGameINLINE().getMaxPopulation(), GC.getSCORE_POPULATION_FACTOR(), true, bFinal, bVictory);

    int iLandScore = getScoreComponent(getLandScore(), GC.getGameINLINE().getInitLand(), GC.getGameINLINE().getMaxLand(), GC.getSCORE_LAND_FACTOR(), true, bFinal, bVictory);

    int iTechScore = getScoreComponent(getTechScore(), GC.getGameINLINE().getInitTech(), GC.getGameINLINE().getMaxTech(), GC.getSCORE_TECH_FACTOR(), true, bFinal, bVictory);

    int iWondersScore = getScoreComponent(getWondersScore(), GC.getGameINLINE().getInitWonders(), GC.getGameINLINE().getMaxWonders(), GC.getSCORE_WONDER_FACTOR(), false, bFinal, bVictory);

    lScore = iPopulationScore + iLandScore + iTechScore + iWondersScore; // include what is in the return statement even though it wouldn't take any time at all really.

    endMillis = XMLPlatformUtils::getCurrentMillis();
    duration = endMillis - startMillis;
    // dll performance measured.

    // log result. Could you perhaps look at the variable value for duration in VS at this point?
    sprintf(szOut, "Dll duration: %d ms.", duration);
    gDLL->logMsg("Test.log", szOut);


    XMLPlatformUtils::Terminate(); \\ This seems necessary from the other code I looked at.


    return iPopulationScore + iLandScore + iTechScore + iWondersScore;​
    }
    Bold text is what differs from the current code, excluding the un-commenting of the python call.
     
    Last edited: Jan 18, 2019
    KaTiON_PT likes this.
  16. KaTiON_PT

    KaTiON_PT Emperor

    Joined:
    Jan 2, 2010
    Messages:
    1,090
    Location:
    Portugal
    I will on Friday. :) Thanks for supplying the code.
     
  17. raxo2222

    raxo2222 Time Traveller

    Joined:
    Jun 10, 2011
    Messages:
    7,823
    Location:
    Poland
    @Toffer90 there is bug in pedia:
    Resource isn't displayed as being used by vicinity building in some cases.

    Spoiler :

    Civ4BeyondSword 2019-01-15 19-09-41-84.jpg Civ4BeyondSword 2019-01-15 19-10-01-28.jpg Civ4BeyondSword 2019-01-15 19-10-13-22.jpg
     
  18. Toffer90

    Toffer90 C2C Modder

    Joined:
    Oct 16, 2011
    Messages:
    7,204
    Location:
    Norway
    K, I'll look into it, the issue is probably present without PPIO too, so it's more a missing feature than a bug I guess.
     
    raxo2222 likes this.
  19. code99

    code99 Warlord

    Joined:
    Apr 28, 2010
    Messages:
    274
    Location:
    Romania
    Stupid question incoming

    What should I do when updating c2c svn to a new revision? Do I delete everything in C2C mod folder, put in the updated c2c and then follow the first install instructions of this mod?


    I recently updated c2c to a new revision and forgot about this mod. I loaded up my game and I couldnt figure out why it all looked different lol ... took me a day before I remembered that I had this before

    EDIT: Oh and maybe im crazy but does this mod mess about with the camera? It feels to me its a bit tilted downwards... kinda like increasing the field of view ... am i crazy? Can this be removed somehow? I prefer the vanilla camera ...
     
    Last edited: Jan 16, 2019
  20. Toffer90

    Toffer90 C2C Modder

    Joined:
    Oct 16, 2011
    Messages:
    7,204
    Location:
    Norway
    Pretty much, yeah, though you only have to delete the UserSettings folder if you play the game without PPIO before installing PPIO.
    What I recommend, if you only play with PPIO, is to delete everything except your userSettings folder before exporting in the latest SVN version and installing PPIO, that way you retain your BUG option preference between updates.

    If you switch between PPIO and non-PPIO C2C, then make a backup of your PPIO UserSettings folder and your non-PPIO UserSettings folder.
    It messes with the camera in several ways, e.g. it increases the distance you can zoom out before entering "globe view", and the default FoV has been changed from 42 to 50.
    You could always reduce FoV with the slider.

    PPIO v0.5.9.9
    SVN rev.10382
    • Updated to latest SVN.
    • Misc tweaks, don't remember the details so it was probably not very noteworthy.
    PPIO v0.5.9.10
    SVN rev.10384
    • Code cleaning/optimization.
     

Share This Page