Excessive Load Times after Adding New Techs Problem - Solved

[sman1975]

Hello!

I am trying to build a mod, and it's quite simple at this point: I only add an Era (Enlightenment - inspired by EE, but widely divergent/credit will be given), and 50 new technologies. The new techs are added to the new Era and inter-woven between the normal Ancient-Renaissance Era techs. A quick screenshot of the design spreadsheet shows the layout:



The green backgrounds (more or less) are the normal Civ5 techs, the blue backgrounds (more or less) are the new techs. The peach backgrounds are also new techs, belonging to the new Enlightenment Era.

OK - a simple, database-only mod. Everything executes correctly and the logs are clean. All the requisite database changes are made and everything looks like it's supposed to.

Then - what's the problem?


This simple mod takes 15+ minutes to load!


This performance took place on a new map, Duel sized.

I've read a lot of posts about similar problems, but as this mod doesn't remove any techs (just shifts their locations), most of those problems don't seem related to mine.

The Technology_PrereqTechs entries are minimal - basically 1-2 connections between each new tech on the tree, nothing reaching back more than the previous column. Previously, the prereqs were fairly complex, and I was thinking this might be slowing down the game load time - resolving a complicated tech tree piping, etc. Dumbing the connections down to this level didn't have any appreciable affect to the load time problem.

During testing, I noticed if I eliminated Column #20 (Humanism, et al; changing the erstwhile Column #21 prereqs to Column #19 entries), the load time was all the way down to about 6 minutes. If I then eliminated Column #19 (Sextant, et al), the load time was down to about 4.5 minutes. This still seems pretty slow for such a simple mod, adding 3 dozen techs on a Duel-sized map.


Anyone have any suggestions about how to speed the load time up???


Mod is attached. Appreciate any/all suggestions. Thanks!

*****************************************************************************************************************

Here are the screenshots of the Tech Tree, to show the limited number of prerequisite links.

 

[Nomad or What]

Gotta love CiV: Every new idea is like a puzzle box to solve...from Hellraiser.

 

[Ja Mes]

To add for reference for people who know what they're talking about, SMAN and I have been going over that it might be like a similar problem discovered in Enlightenment Era + Vox Populi 2 years back: https://forums.civfanatics.com/threads/enlightenment-era-for-vox-populi.610282/page-16, and could be a problem with VictoryProgress.Lua.

We aren't very convinced on this, but maybe it's a lead.

 

[sman1975]

@Ja Mes - I've looked at both the post you referenced. I know it's a VP file, and my mod isn't VP-compatible, but then I thought, "What the hay..."

I added the replacement file, rebuilt the mod, then fired up a new game. Any guess on how long it took to load the game then???????????

Spoiler New Load Time... :

Less than 5 seconds!!!!!

You, my friend, are an F'in genius...

*******************************************************

For anyone looking in the distant google future, if you're experiencing dramatic load times on a mod adding a few techs to the game, please follow @Ja Mes' recommendation and look at that post.

Originally, I conducted a thorough search through the forums for similar problems. The reason I initially didn't pay attention to the link that @Ja Mes referenced was that it dealt with Vox Populi. Since my mod isn't VP compatible, I ignored the post, thinking the solution wouldn't work for me. Bad Idea...

After discussing it with him on Discord (a superb modder tool, and a great way to get help on modding issues), I decided to give it a shot. And replacing the game's "Victory Progress.lua" with the file in the linked post definitely fixed the problem for my mod!

BTW, if you're not building a Vox Populi-compatible mod, you'll have to comment out lines 683 & 684 in the replacement file, as those lines are VP-related content that will error out in a non-VP environment whenever the Victory Progress UI is called.



P.s. Technical discussion: the reason for the long load times is related to the fact that in game setup, the Victory Progress.Lua file executes a function that tries to determine how many techs need to be discovered before a civ can complete the Apollo Program. It uses this count to determine how to fill up the "bubble" on the Victory Progress popup to show a player how close they are to reaching that goal.

The function relies on a recursive algorithm (a design where a function calls itself to drill down through data). It's an elegant way to solve a problem and is used frequently when iteratively processing data of unknown size. For some reason (probably related to stack management), the game's LUA interpreter doesn't handle recursion very well, resulting in exponential delays a more and more techs are added. So, a data set that has only 225 members takes over 15 minutes to tabulate.

Recursion may be elegant, but in Civ5's LUA, it can be lethal...

 

[Nomad or What]

That is very good information to know, indeed! I no doubt will come across it in my mod. Thanks!