Laggy City Interface

The WIDGET_CONSTRUCT was a dead end, there didn't seem to be any problematic code there; I haven't edited it at all, and it all seemed logical to me.

One question, when I click on a building to construct in the city interface, what code is executed? It takes an awfully long time from clicking on the construct button to it appearing in the city queue as well.
 
One function is called for the hover; the other for the click. It should be obvious from the name or the code which is which.

When the building is added to the queue I think it uses a different WidgetTypes, right?
 
So... Since we can't seem to find any way to speed this up, are there alternatives? Could I break things apart?
 
I would suggest doing a little more forensics. While it seems that your DLL isn't the problem, it would be good to eliminate it. Can your mod start without the DLL? If not, is there a small set of changes you could make to a vanilla 3.19 DLL to allow your mod to launch?

Vanilla BTS doesn't take 2 seconds to draw the building list, and you haven't added that many more buildings (that show up at one time). We have eliminated the artwork and CvBuildingInfos from the equation with no effect.

There is one bit of Python that gets executed "out of band": the "gameUpdate" event is fired regularly while the game runs. I do not know if it fires while other Python code is running, but it's possible that the EXE could fire it while the Python code is waiting for a EXE call to return (adding a button to the UI control). Normally this event isn't used, but I use it for BUG and expose it for use by mods.

1. BugEventManager.onGameUpdate() uses it for firing the "BeginActivePlayerTurn" and "endTurnReady" events. The code that does is very short and quick to execute, but it does make a call to the EXE to check the end turn state.

2. BugUtil.deferCall() allow modders to schedule functions to be called on the next available update or after a delay. Again the code that checks for awaiting tasks is short and quick. [I just made a small change to make it even quicker when the queue is empty.] It logs when a call is executed, and we weren't seeing any calls, but the function still runs to check for calls. I don't see how this could be it, though.
 
No chance of getting my mod to work without a custom DLL. I tried though. After fixing all the python errors, it still crashed, because a lot of my code uses the WoC loading system, which just crashes spectacularly without it.

I could always try the RevDCM dll, that's as close to pure as I can ever get, really.
 
Maybe there is a completely external factor to the construct queue. I was playing around with the Emphasis buttons, and they have a 1-2 second lag too, in BTS they are pretty much instant.

Something must be running while in the city screen...
 
When you click an emphasis button, doesn't the full city AI run in order to decide which tiles should be worked and change them all? I am not surprised that this takes a little time.

I wish I had some good solution to offer. It would be interesting to know whether the revdcm mod by itself has this lag problem; at least that would narrow down the search area.

Is it possible that constructing the building help/hover text happens when the button is drawn, and that is where the time is being lost? Perhaps there is some time consuming lookup for buildings, but not for units.
 
davidlallen said:
Is it possible that constructing the building help/hover text happens when the button is drawn, and that is where the time is being lost? Perhaps there is some time consuming lookup for buildings, but not for units.
Click to expand...

You might be right on th emphasis buttons.


I really don't know how all that hover text works. Is it drawn as needed, or all drawn at once and stored in a buffer?
 
The hover text is calculated on-the-fly every time you hover over a widget. I've verified this in the past using logging.

The emphasis buttons do indeed tell the city AI to reassign citizens. I doubt the construct buttons are doing the same thing. If they are, I'd expect that to happen when you call canConstruct() and not when you happen to draw a button on the screen.

Oh, Afforess, you mentioned that perhaps the city screen is doing something extra. I hadn't even considered asking you this basic question: have you timed the code by simply clicking a city bar on the main interface rather than opening the city screen??
 
Some closure on this topic. I created a quick and dirty cache for CanConstruct calls to CvCity; so the game was not calculating, only accessing stored values. The speed of the interface was identical. I suspect in the EXE the way the screen is coded is takes O(n) time, or something ridiculous like that...

Oh Firaxis, why have thou deserted me? :rolleyes:
 
I stumbled onto something that might be related to this problem. I profiled the time in the DLL for how long the worldbuilder was taking to open (It opens painfully slow in my mod) and found that, lo and behold, buildings were taking 10000ms to create. A bit more digging, and it was the loop over the building descriptions that was taking that long. Each building was taking between 10-20ms for their description to load. (That number sounds familiar...). I guess I need to break the function up and see where the culprit is, but do you have any idea on how it could be taking so long, off the top of your head?
 
I would wonder first why the descriptions were being loaded. Those descriptions are only needed when you hover over a building, and i thought they were all loaded when the XML was parsed during game startup.

As for why it takes so long if it is indeed loading them, I would say it probably loads a whole file at once, and it may need to scan all files until it finds that description. That doesn't seem like something they'd do. Can you be more specific about the part of the SDK you're talking? Where precisely did you put your timings?
 
EmperorFool said:
I would wonder first why the descriptions were being loaded. Those descriptions are only needed when you hover over a building, and i thought they were all loaded when the XML was parsed during game startup.

As for why it takes so long if it is indeed loading them, I would say it probably loads a whole file at once, and it may need to scan all files until it finds that description. That doesn't seem like something they'd do. Can you be more specific about the part of the SDK you're talking? Where precisely did you put your timings?
Click to expand...

Here is the function I logged for the total time:
Code: 
void CvGameTextMgr::getBuildingDataForWB(bool bStickyButton, std::vector<CvWBData>& mapBuildingData)
{
	int iCount = 0;
	if (!bStickyButton)
	{
		mapBuildingData.push_back(CvWBData(iCount++, GC.getMissionInfo(MISSION_FOUND).getDescription(), GC.getMissionInfo(MISSION_FOUND).getButton()));
	}

	[COLOR="Red"]CvWStringBuffer szBuffer;[/COLOR]
	for (int i=0; i < GC.getNumBuildingInfos(); i++)
	{
		szBuffer.clear();
		setBuildingHelp(szBuffer, (BuildingTypes)i);
		mapBuildingData.push_back(CvWBData(iCount++, szBuffer.getCString(), GC.getBuildingInfo((BuildingTypes)i).getButton()));
	[COLOR="red"]}[/COLOR]
}
I only logged from the first red line, to the last red line. I assumed the clear call, and appending information took no time.
 
If you simply comment out the setBuildingHelp line, what happens to the runtime? Perhaps one of the calls involved in setBuildingHelp is doing an unexpectedly large amount of lookups of something. The help text will now be blank, but the point is to travel down the right part of the calling stack, to find where the large runtime is. If that removes all the runtime, then dig underneath setBuildingHelp and comment out parts of that.
 
davidlallen said:
If you simply comment out the setBuildingHelp line, what happens to the runtime? Perhaps one of the calls involved in setBuildingHelp is doing an unexpectedly large amount of lookups of something. The help text will now be blank, but the point is to travel down the right part of the calling stack, to find where the large runtime is. If that removes all the runtime, then dig underneath setBuildingHelp and comment out parts of that.
Click to expand...

I did that. When I did that, the time it took was 5ms. I then profiled the descriptions one by one, and that's how I got the 10-20ms number.
 
Are you saying that by commenting out this line the total time went from 10,000 msec to 5 msec? Or each building went from 10-20 msec to 5 msec?

Sadly setBuildingHelp is a 1200 line function. But, you can use binary search a couple of times. ifdef out out the first 600 lines, obviously ending someplace where you are not between curly braces. What is the runtime? Ifdef out the other 600 lines. One of them will make a big difference in the runtime. Then break that into two 300 line blocks and repeat. Now it is probably reasonable to inspect the remaining code to see which subroutines "might" have a long runtime. If all of them "look" like they are fast then do the binary subdivision again. It is painful, but there is no better way I know of, and the "ah-ha!" moment is always worthwhile.
 
davidlallen said:
Are you saying that by commenting out this line the total time went from 10,000 msec to 5 msec? Or each building went from 10-20 msec to 5 msec?
Click to expand...

Total time, so 10s to .005s.

davidlallen said:
Sadly setBuildingHelp is a 1200 line function. But, you can use binary search a couple of times. ifdef out out the first 600 lines, obviously ending someplace where you are not between curly braces. What is the runtime? Ifdef out the other 600 lines. One of them will make a big difference in the runtime. Then break that into two 300 line blocks and repeat. Now it is probably reasonable to inspect the remaining code to see which subroutines "might" have a long runtime. If all of them "look" like they are fast then do the binary subdivision again. It is painful, but there is no better way I know of, and the "ah-ha!" moment is always worthwhile.
Click to expand...

Heh, try 3k line function for me. :(
 
3000 lines only means 1 more iteration of a binary search compared to a 1500 line function, so it's not much worse. You could probably skip the original bits since you say the vanilla time was tiny (or am I not recalling what you wrote above correctly?). Look especially for nested loops.
 
I did finally pick through my changes with a fine tooth comb. I found that Multidimensional arrays were using > 90% of the time, despite being only a small part of the code. Even the multidimensional arrays used in BTS for buildings (SpecialistYieldChanges - Used for Angor Wat & BonusYieldChanges - Used on quite a few buildings) were horribly inefficient, consuming 700ms of time alone. With my 7 new 2d arrays, it was no surprise to see the function take 6000ms...

The fix is simple, and doesn't break saves. I added a new public boolean variable for each 2d array, like m_bAnySpecialistYieldChanges, and set it to true when the game loaded the XML for the building and encountered the SpecialistYieldChange portion. (yes, there is an opportunity for false positives here, when all the <iYield> tags are present, but zero.) Anyway, I just skipped the loops if there were no SpecialistYieldChanges detected. My function time went from 6000 ms total to 200ms. 800ms was the BTS average. In case you care...
 
You must log in or register to reply here.

Similar threads

Ferocca
"not a member" problem
Replies
2
Views
199
Ferocca
Ferocca
CSD
[TUTORIAL] How to Set Persistent Variables in Pure Python
Replies
4
Views
471
CSD
CSD
P
Potential Python Optimizations for Civ4 on Intel and AMD CPUs
Replies
11
Views
583
PurpleMelbourne
P
L. Vern
  • Locked
  • Poll
[Vote] (7-22) Allow Non-Recon Units to Pick Up Ancient Ruins
2
Replies
37
Views
2K
CppMaster
CppMaster
Spillsandstains
Getting the XML to accept a double-move doesn't work with custom features
Replies
4
Views
242
Nightinggale
Nightinggale
Top Bottom