Python Performance and Interface Overhaul (PPIO)
- Thread starter Toffer90
- Start date
Do you think the code is not processed at all, or do you think the profiler is simply not detailed enough?
Can you see this one in the profiler?
CvGame::updateScore(...)
(The main place where CvPlayer::calculateScore is called from)
Or these?
CvGame::doTurn()
CvGame::update()
(The main places where CvGame::updateScore(...) is called from)
Can you see this one in the profiler?
CvGame::updateScore(...)
(The main place where CvPlayer::calculateScore is called from)
Or these?
CvGame::doTurn()
CvGame::update()
(The main places where CvGame::updateScore(...) is called from)
Last edited:
@Toffer90 I could find CvGame::doTurn() and CvGame::update(), like you've said the profiler might not be detailed enough to see CvPlayer::calculateScore nor CvGame::updateScore...
First thing of notice: in vanilla the score doesn't even update (stays at 0) if using the python call.
Five repetitions were done for each scenario in a single save. Before measuring each scenario a recalculation followed by a save and a load was made.
So the results:
Vanilla with C++ code: 3464 ms
Vanilla with Python call: 3453 ms < -- Could be taking less time because it isn't used... the score wasn't getting updated.
PPIO with C++ code: 3434 ms
PPIO with Python call: 3417 ms
This show that your changes have indeed made it faster.
+1.38% faster vs. Vanilla with C++ code
+1.05% faster vs. Vanilla with Python call
+0.5% faster vs. PPIO with C++ code
First thing of notice: in vanilla the score doesn't even update (stays at 0) if using the python call.
Five repetitions were done for each scenario in a single save. Before measuring each scenario a recalculation followed by a save and a load was made.
So the results:
Vanilla with C++ code: 3464 ms
Vanilla with Python call: 3453 ms < -- Could be taking less time because it isn't used... the score wasn't getting updated.
PPIO with C++ code: 3434 ms
PPIO with Python call: 3417 ms
This show that your changes have indeed made it faster.
+1.38% faster vs. Vanilla with C++ code
+1.05% faster vs. Vanilla with Python call
+0.5% faster vs. PPIO with C++ code
Attachments
Last edited:
3 and a half second, you are measuring too much code to make me believe the results about the score code can be conclusive.@Toffer90 I could find CvGame::doTurn() and CvGame::update(), like you've said the profiler might not be detailed enough to see CvPlayer::calculateScore nor CvGame::updateScore...
First thing of notice: in vanilla the score doesn't even update (stays at 0) if using the python call.
Five repetitions were done for each scenario in a single save. Before measuring each scenario a recalculation followed by a save and a load was made.
So the results:
Vanilla with C++ code: 3464 ms
Vanilla with Python call: 3453 ms < -- Could be taking less time because it isn't used... the score wasn't getting updated.
PPIO with C++ code: 3434 ms
PPIO with Python call: 3417 ms
This show that your changes have indeed made it faster.
+1.38% faster vs. Vanilla with C++ code
+1.05% faster vs. Vanilla with Python call
+0.5% faster vs. PPIO with C++ code
Did you measure a complete run of one of these functions: CvGame::doTurn() or CvGame::update() ?
Can't you use some standard library function like clock()
clock_t startTime = clock();
... ... ... // Any code
clock_t endTime = clock();
clock_t clockTicksTaken = endTime - startTime;
print/log/check clockTicksTaken
Thanks for entertaining my curiosity btw, and if you are going to be a good C2C modder you will need to learn how to check code performance reliably at any rate. We need performance minded coders more than anything atm. ^^
Last edited:
I found this C2C dll code used in a couple of places:
It looks like a more reliable timer than the standard clock() function which returns amount of CPU clock ticks elapsed since program started, which can be a bit unreliable as a time measurement.
This is how it is printed, though I don't know where it prints it:
XMLPlatformUtils::Initialize()
unsigned long duration;
const unsigned long startMillis = XMLPlatformUtils::getCurrentMillis();
... ... // some code
const unsigned long endMillis = XMLPlatformUtils::getCurrentMillis();
duration = endMillis - startMillis;
XMLPlatformUtils::Terminate();
unsigned long duration;
const unsigned long startMillis = XMLPlatformUtils::getCurrentMillis();
... ... // some code
const unsigned long endMillis = XMLPlatformUtils::getCurrentMillis();
duration = endMillis - startMillis;
XMLPlatformUtils::Terminate();
It looks like a more reliable timer than the standard clock() function which returns amount of CPU clock ticks elapsed since program started, which can be a bit unreliable as a time measurement.
This is how it is printed, though I don't know where it prints it:
XERCES_STD_QUALIFIER cout << xmlFile << ": " << duration << " ms" << XERCES_STD_QUALIFIER endl;
I don't know enough about VS to confirm or refute it.
Last edited:
Thunderbrd
C2C War Dog
Well... I'm completely useless here. Sorry.
In the logs folder I looked at bull.log, it contained a single line:
[24388.094] isBug: true
Searching for "isBug:" in sources lead me to this:
const char* format = "isBug: %s", bIsBug ? "true" : "false"
static char buf[2048];
_vsnprintf(buf, 2048 - 4, format, (char*)(&format + 1));
gDLL->logMsg("bull.log", buf);
static char buf[2048];
_vsnprintf(buf, 2048 - 4, format, (char*)(&format + 1));
gDLL->logMsg("bull.log", buf);
That may be a better way to log stuff. ^^
HorseshoeHermit
20% accurate as usual, Morty
I attempted to just paste into the C2C folder with overwrites, but that caused a system failure after "GFC" failed to initialize a "control theme" ?
This was on trying to load C2C as a mod. Latest SVN revision. I attempted twice. (Also, it is a fresh C2C with no UserSettings.) I exported the SVN fresh, and was able to generate a new game.
If I brave reproducing the crash again, I will have Python exceptions unhidden.
This was on trying to load C2C as a mod. Latest SVN revision. I attempted twice. (Also, it is a fresh C2C with no UserSettings.) I exported the SVN fresh, and was able to generate a new game.
If I brave reproducing the crash again, I will have Python exceptions unhidden.
GiuseppeIII
The Third Giuseppe
@HorseshoeHermit Is your mod folder called Caveman2Cosmos or C2C
HorseshoeHermit
20% accurate as usual, Morty
It's Caveman2Cosmos
What is your full folder path for C2C?
HorseshoeHermit
20% accurate as usual, Morty
[My Documents]\My Games\beyond the sword\MODS\Caveman2Cosmos (steam installation).
My Documents is in a different drive from my steam folder, and neither one is C:
My Documents is in a different drive from my steam folder, and neither one is C:
That explains it then, C2C should not be in My Documents.
C2C should be in the "mods" folder that is in the same folder as the Civ4BeyondSword.exe file is in.
A lot of errors can occur if C2C is installed in My documents even without PPIO.
Here's my path as an example:
C:\SteamLibrary\steamapps\common\Sid Meier's Civilization IV Beyond the Sword\Beyond the Sword\Mods\Caveman2Cosmos
The bold is the important part
That's uninportant, here's my path for My Documents.
D:\My Stuff\Documents\My Games\Beyond the Sword
I changed it to save space on my small SSD C system drive.A lot of software throw a lot of junk files into "My Documents" all the time. Especially games whom almost all put game settings and save files in there, I play a lot of games and I like to have many saves, it can amount to GB of saves.
Last edited:
Have you concluded your involvement in this analysis?
Are you certain my python code is faster than the current dll code? If you are, I will add my code to the SVN, and you can follow up by removing the dll score calculation code and un-comment the python call.
Example of a good way to measure code performance (that may work without any altercations, I'm not too sure about the logging code I added):
int CvPlayer::calculateScore(bool bFinal, bool bVictory) const
{
PROFILE_FUNC();
if (!isAlive())
{
if (GET_TEAM(getTeam()).getNumMembers() == 0)
{
if (!isAlive())
{
return 0;
}if (GET_TEAM(getTeam()).getNumMembers() == 0)
{
return 0;
}XMLPlatformUtils::Initialize() \\ This seems necessary from the other code I looked at.
unsigned long duration;
// Measure python performance
unsigned long startMillis = XMLPlatformUtils::getCurrentMillis();
long lScore = 0;
CyArgsList argsList;
argsList.add((int) getID());
argsList.add(bFinal);
argsList.add(bVictory);
PYTHON_CALL_FUNCTION(__FUNCTION__, PYGameModule, "calculateScore", argsList.makeFunctionArgs(), &lScore);
lScore = (int)lScore
// return ((int)lScore); // commented out because we want the rest of the code here to run.
unsigned long endMillis = XMLPlatformUtils::getCurrentMillis();
duration = endMillis - startMillis;
// Python performance measured.
// log result. Could you perhaps look at the variable value for duration in VS at this point?
TCHAR szOut[2048];
sprintf(szOut, "Dll duration: %d ms.\n", duration);
gDLL->logMsg("Test.log", szOut);
// Measure dll performance
startMillis = XMLPlatformUtils::getCurrentMillis();
int iPopulationScore = getScoreComponent(getPopScore(), GC.getGameINLINE().getInitPopulation(), GC.getGameINLINE().getMaxPopulation(), GC.getSCORE_POPULATION_FACTOR(), true, bFinal, bVictory);
int iLandScore = getScoreComponent(getLandScore(), GC.getGameINLINE().getInitLand(), GC.getGameINLINE().getMaxLand(), GC.getSCORE_LAND_FACTOR(), true, bFinal, bVictory);
int iTechScore = getScoreComponent(getTechScore(), GC.getGameINLINE().getInitTech(), GC.getGameINLINE().getMaxTech(), GC.getSCORE_TECH_FACTOR(), true, bFinal, bVictory);
int iWondersScore = getScoreComponent(getWondersScore(), GC.getGameINLINE().getInitWonders(), GC.getGameINLINE().getMaxWonders(), GC.getSCORE_WONDER_FACTOR(), false, bFinal, bVictory);
lScore = iPopulationScore + iLandScore + iTechScore + iWondersScore; // include what is in the return statement even though it wouldn't take any time at all really.
endMillis = XMLPlatformUtils::getCurrentMillis();
duration = endMillis - startMillis;
// dll performance measured.
// log result. Could you perhaps look at the variable value for duration in VS at this point?
sprintf(szOut, "Dll duration: %d ms.", duration);
gDLL->logMsg("Test.log", szOut);
XMLPlatformUtils::Terminate(); \\ This seems necessary from the other code I looked at.
return iPopulationScore + iLandScore + iTechScore + iWondersScore;
}unsigned long duration;
// Measure python performance
unsigned long startMillis = XMLPlatformUtils::getCurrentMillis();
long lScore = 0;
CyArgsList argsList;
argsList.add((int) getID());
argsList.add(bFinal);
argsList.add(bVictory);
PYTHON_CALL_FUNCTION(__FUNCTION__, PYGameModule, "calculateScore", argsList.makeFunctionArgs(), &lScore);
lScore = (int)lScore
// return ((int)lScore); // commented out because we want the rest of the code here to run.
unsigned long endMillis = XMLPlatformUtils::getCurrentMillis();
duration = endMillis - startMillis;
// Python performance measured.
// log result. Could you perhaps look at the variable value for duration in VS at this point?
TCHAR szOut[2048];
sprintf(szOut, "Dll duration: %d ms.\n", duration);
gDLL->logMsg("Test.log", szOut);
// Measure dll performance
startMillis = XMLPlatformUtils::getCurrentMillis();
int iPopulationScore = getScoreComponent(getPopScore(), GC.getGameINLINE().getInitPopulation(), GC.getGameINLINE().getMaxPopulation(), GC.getSCORE_POPULATION_FACTOR(), true, bFinal, bVictory);
int iLandScore = getScoreComponent(getLandScore(), GC.getGameINLINE().getInitLand(), GC.getGameINLINE().getMaxLand(), GC.getSCORE_LAND_FACTOR(), true, bFinal, bVictory);
int iTechScore = getScoreComponent(getTechScore(), GC.getGameINLINE().getInitTech(), GC.getGameINLINE().getMaxTech(), GC.getSCORE_TECH_FACTOR(), true, bFinal, bVictory);
int iWondersScore = getScoreComponent(getWondersScore(), GC.getGameINLINE().getInitWonders(), GC.getGameINLINE().getMaxWonders(), GC.getSCORE_WONDER_FACTOR(), false, bFinal, bVictory);
lScore = iPopulationScore + iLandScore + iTechScore + iWondersScore; // include what is in the return statement even though it wouldn't take any time at all really.
endMillis = XMLPlatformUtils::getCurrentMillis();
duration = endMillis - startMillis;
// dll performance measured.
// log result. Could you perhaps look at the variable value for duration in VS at this point?
sprintf(szOut, "Dll duration: %d ms.", duration);
gDLL->logMsg("Test.log", szOut);
XMLPlatformUtils::Terminate(); \\ This seems necessary from the other code I looked at.
return iPopulationScore + iLandScore + iTechScore + iWondersScore;
Bold text is what differs from the current code, excluding the un-commenting of the python call.
Last edited:
Thanks for supplying the code.
K, I'll look into it, the issue is probably present without PPIO too, so it's more a missing feature than a bug I guess.
Stupid question incoming
What should I do when updating c2c svn to a new revision? Do I delete everything in C2C mod folder, put in the updated c2c and then follow the first install instructions of this mod?
I recently updated c2c to a new revision and forgot about this mod. I loaded up my game and I couldnt figure out why it all looked different lol ... took me a day before I remembered that I had this before
EDIT: Oh and maybe im crazy but does this mod mess about with the camera? It feels to me its a bit tilted downwards... kinda like increasing the field of view ... am i crazy? Can this be removed somehow? I prefer the vanilla camera ...
What should I do when updating c2c svn to a new revision? Do I delete everything in C2C mod folder, put in the updated c2c and then follow the first install instructions of this mod?
I recently updated c2c to a new revision and forgot about this mod. I loaded up my game and I couldnt figure out why it all looked different lol ... took me a day before I remembered that I had this before
EDIT: Oh and maybe im crazy but does this mod mess about with the camera? It feels to me its a bit tilted downwards... kinda like increasing the field of view ... am i crazy? Can this be removed somehow? I prefer the vanilla camera ...
Last edited:
Pretty much, yeah, though you only have to delete the UserSettings folder if you play the game without PPIO before installing PPIO.
What I recommend, if you only play with PPIO, is to delete everything except your userSettings folder before exporting in the latest SVN version and installing PPIO, that way you retain your BUG option preference between updates.
If you switch between PPIO and non-PPIO C2C, then make a backup of your PPIO UserSettings folder and your non-PPIO UserSettings folder.
It messes with the camera in several ways, e.g. it increases the distance you can zoom out before entering "globe view", and the default FoV has been changed from 42 to 50.
You could always reduce FoV with the slider.
PPIO v0.5.9.9
SVN rev.10382
- Updated to latest SVN.
- Misc tweaks, don't remember the details so it was probably not very noteworthy.
SVN rev.10384
- Code cleaning/optimization.
