Utilities -- Modular Script Loading

[Lord Shadow]

I've received a couple of reports that quests stop working in Alien Strains v3, but in truth the problem seems to be related to the most recent version of your utility's QuestSystem.lua.

It's an issue I haven't perceived myself as I didn't update that Lua for my latest release. However, people are seeing the bug because other mods they've installed resort to the newest incarnation (and override mine).

 

[Vicevirtuoso]

You can have your users delete the QuestSystem.lua from my mod, and it will load yours instead. There's no dynamic way to choose which copy of a core game Lua replacement is loaded, so this is unfortunately the only way.

I'm also getting a lot of bug reports about users being unable to receive quests, though. Never experienced it myself, so I'm asking users to send me their Lua.log. If you could do the same for your mod's users, that could help me get to the root of the problem more quickly.

 

[Lord Shadow]

Yeah, I've told them about manual deletion to make the game load the right file. Thing is that while my older (and unmodified) version of QuestSystem.lua (Nov 3) doesn't seem to have the bug they describe, at least in my experience, neither does it have the improvements you made in the newer releases, which might cause other problems.

Anyway, yeah, I'm asking around for the errors Lua.log produces.

I have this one so far, but I think it concerns my older file and an issue you fixed recently about file paths. The user in question was also using the Affinity Quests Fix mod.

Quests: Adding modded QuestObjective from modID ef6b3f14-804c-46eb-a27f-8711e6144bd7
Runtime Error: cannot open assets\Gameplay\Lua\Quests\QuestObjectives\ClearForestFromPlotsQuestObjective_fix.lua: No such file or directory
stack traceback:
        [C]: in function 'dofile'
        C:\Users\USER\Documents\My Games\Sid Meier's Civilization Beyond Earth\MODS\Alien Strains (v 3)\Lua\QuestSystem.lua:199: in function '(main chunk)'
        [C]: in function '(anonymous)'
Runtime Error: Error loading C:\Users\USER\Documents\My Games\Sid Meier's Civilization Beyond Earth\MODS\Alien Strains (v 3)\Lua\QuestSystem.lua.
stack traceback:
        [C]: in function '(anonymous)'

I'll try to get you a log of the quest disappearance bug issue as soon as possible.

EDIT: Here's a fairly extensive log from TPangolin, again presenting a conflict between Alien Strains and Affinity Quests Fix, from what I can see. I hope the bug reports you're getting aren't from Alien Strains players who are using its outdated QuestSystem.lua. If that proves to be the case and the new version is actually fine, I'll update ASAP.

 

[Vicevirtuoso]

I went in your QuestSystem.lua, pasted everything relevant from my newest version into it, and tried with MSL's QuestSystem.lua renamed to prevent it from loading. It was a success:

Spoiler :

 Quests: Loaded QuestObjective script for ''QUEST_OBJECTIVE_BUILD_BUILDING''
 Quests: Added modded QuestObjective script ''VV_BuildBuildingQuestObjective'' with ModID 21c3216d-1a90-466a-a95f-7a86cef95c5f
 Quests: addinpath: C:\Users\Vice\Documents\My Games\Sid Meier's Civilization Beyond Earth\MODS\Quests -- Resource Building Quest Fix (v 2)\VV_BuildBuildingQuestObjective.lua
 Quests: Loaded QuestObjective script for ''QUEST_OBJECTIVE_BUILD_PROJECT''
 Quests: Script loaded from game's Assets folder
 Quests: Loaded QuestObjective script for ''QUEST_OBJECTIVE_BUILD_UNITS''
 Quests: Script loaded from game's Assets folder
 Quests: Loaded QuestObjective script for ''QUEST_OBJECTIVE_BUILD_IMPROVEMENTS''
 Quests: Script loaded from game's Assets folder
 Quests: Loaded QuestObjective script for ''QUEST_OBJECTIVE_DISCOVER_SIGNAL''
 Quests: Script loaded from game's Assets folder
 Quests: Loaded QuestObjective script for ''QUEST_OBJECTIVE_FIND_RESOURCE_PODS''
 Quests: Script loaded from game's Assets folder
 Quests: Loaded QuestObjective script for ''QUEST_OBJECTIVE_LANDMARK_ACTION''
 Quests: Script loaded from game's Assets folder
 Quests: Loaded QuestObjective script for ''QUEST_OBJECTIVE_LAUNCH_SATELLITES''
 Quests: Script loaded from game's Assets folder
 Quests: Loaded QuestObjective script for ''QUEST_OBJECTIVE_RESEARCH_TECH''
 Quests: Script loaded from game's Assets folder
 Quests: Loaded QuestObjective script for ''QUEST_OBJECTIVE_SETTLE_EARTHLINGS''
 Quests: Script loaded from game's Assets folder
 Quests: Loaded QuestObjective script for ''QUEST_OBJECTIVE_COMMIT_UNITS_TO_GATE''
 Quests: Script loaded from game's Assets folder
 Quests: Loaded QuestObjective script for ''QUEST_OBJECTIVE_WAIT''
 Quests: Script loaded from game's Assets folder
 Quests: Loaded QuestObjective script for ''QUEST_OBJECTIVE_KILL_UNIT_TYPE''
 Quests: Added modded QuestObjective script ''Kill Unit Type'' with ModID ef6b3f14-804c-46eb-a27f-8711e6144bd7
 Quests: addinpath: C:\Users\Vice\Documents\My Games\Sid Meier's Civilization Beyond Earth\MODS\Alien Strains (v 3)\KillUnitTypeQuestObjective.lua
 Quests: Loaded QuestObjective script for ''QUEST_OBJECTIVE_KILL_PLAYER_UNITS''
 Quests: Script loaded from game's Assets folder

Since my newest code can load scripts from subfolders, whereas my old one could not, issues are happening when users use both Alien Strains and Affinity Quest Fix such as in TPangolin's example. AQF has its scripts in subfolders, so the latest code is necessary for it to work. Just pasting over the "Load Scripts" section with my new code is all you should need to do. This will also need to be done in PlotBonusSystem.lua.

 

[Ryoga]

All right I checked how Firaxis adjusted QuestSystem.lua and it turns out they used a system that is very similar to yours.

Quests and Objectives still need to be set as added content, but the tags are respectively "Quest" and "QuestObjective".

Moreover the check is done between the script name specified in the XML and the "name" in the content tab. So people need to to make sure those two are the same.

I tested it with a file in a subdirectory so that should work fine in any case. It looks like "Modding.GetEvaluatedFilePath" works now.

 

[Vicevirtuoso]

Yeah, I've already updated my version to be compatible with both my old implementation and the Firaxis implementation.

Unfortunately, Firaxis neglected to add the changes to any files other than QuestSystem.lua, so this mod will have to live on a bit longer.

 

[Ryoga]

Yeah, I've already updated my version to be compatible with both my old implementation and the Firaxis implementation.

Unfortunately, Firaxis neglected to add the changes to any files other than QuestSystem.lua, so this mod will have to live on a bit longer.

Actually just look what I found

This is in the do file for Quests:

	-- Load quest scripts
	for questInfo in GameInfo.Quests() do
		local path;

		-- Check mods first to see if there's a matching scriptname.
		for questScript in Modding.GetActivatedModEntryPoints("Quest") do

			[COLOR="Red"]if(questScript.Name == questInfo.ScriptName) then[/COLOR]

and this is in the do file for Quest objectives:

	-- Load objective scripts
	for objectiveInfo in GameInfo.QuestObjectives() do

		local path;

		-- Check mods first to see if there's a matching scriptname.
		for questObjectiveScript in Modding.GetActivatedModEntryPoints("QuestObjective") do
			[COLOR="Red"]if(questScript.Name == questInfo.ScriptName) then[/COLOR]

It looks like someone made a copy paste and then forgot to update all the variables.

In other words... we still need a fix to QuestSystem.lua

Way to go Firaxis!

 

[Vicevirtuoso]

So it appears that Modular Script Loading is still quite necessary for all of the systems.

It's okay, Firaxis, you'll get it next time.

Spoiler :

 

[Ryoga]

I received a notification that my current "Affinity Quests Fix" mod conflicts with your recent version of this snippet. I tested it with the version currently on steam and I did notice that the conflict exists.

Looking at the code I see that in the section of the "Compatibility with new patch" for Quests there is this suspicious line:

for addin in Modding.GetActivatedModEntryPoints("[B]QuestAddin[/B]") do

Shouldn't that be "Quest"?

 

[Vicevirtuoso]

Whoops. I'll fix it.

I have to admit that I'm not proofreading my fixes much. The new patch makes the game horrendously boring. I actually found the broken things like trade routes, the powerful T1 affinity units, and intrigue exploits fun. Now that that stuff is gone, all which remains is essentially vanilla Civ 5 in space. So I'm really trying to make my fixes to them quick so I can go back to my Civ 5 mods...

 

[Ryoga]

Whoops. I'll fix it.

I have to admit that I'm not proofreading my fixes much. The new patch makes the game horrendously boring. I actually found the broken things like trade routes, the powerful T1 affinity units, and intrigue exploits fun. Now that that stuff is gone, all which remains is essentially vanilla Civ 5 in space. So I'm really trying to make my fixes to them quick so I can go back to my Civ 5 mods...

Yeah I understand that well, I also don't quite feel motivated to make mods for this game anymore, especially after seeing the sloppy job they did for this patch.

 

[whoward69]

I have to admit that I'm not proofreading my fixes much. The new patch makes the game horrendously boring.

Unfortunately Firaxis seem to be doing the same ... possibly for the same reasons

 

[Ryoga]

Bad news
It seems that there's a major problem with the whole "dofile" function.

Someone reported that the process fails when the program attempts to use a path that contains non standard characters (such as "à"). I was able to reproduce the error by creating an account with such characters in the name.

The Firetuner console didn't report any error until the "dofile" line. The path appears correctly in the print, but then the non standard characters are clearly not recognized in the error report.

Where is the "dofile" function located? I hope it's not a C++ function, because in that case I fear there won't be any viable solution to fix this problem.

 

[Vicevirtuoso]

dofile() is a function inherent to Lua, so this issue wouldn't even be Firaxis's fault, but that of the Lua group itself.

Now, I haven't actually opened up QuestSystem again to look at it, but moving away from dofile() to include() is most likely going to require an actual rewrite of the system. Last time I tried it, you couldn't declare an include as a variable like you can with a dofile. We might be looking at the need to replace every quest file in that case..

And I'll be blunt: I don't feel like doing that at all. All my modding desires for BE are fulfilled, the patch was awful, and I've already got a backlog of Civ 5 mod fixes and features..

 

[whoward69]

but that of the Lua group itself.

As CivBE uses Havok Script (Lua compatible replacement/enhancement) it would be Havok's issue - or possibly not, as Havok have non-US offices they must have hit this problem in the past, could be that Firaxis are incorrectly using utf-8 characters in the APIs.

Either way, it's almost certainly not something modders can fix

 

[Ryoga]

As CivBE uses Havok Script (Lua compatible replacement/enhancement) it would be Havok's issue - or possibly not, as Havok have non-US offices they must have hit this problem in the past, could be that Firaxis are incorrectly using utf-8 characters in the APIs.

It's just plainly a bad design since its inception. The program created by Civ V coders never contemplated the possibility to retrieve files directly from the mod directory upon starting a game. Rather the system was clearly designed to upload all the mods before the game setup screen.

The coders of Civ BE must have designed this new system with the idea that this would allow for modularity, but they only showed how little understanding they had of the original code (since their system didn't even work) and the changes they have made (half-assed and untested since they left a blatant error for quest objectives) after the patch are now still crashing against a wall.

Either way, it's almost certainly not something modders can fix

Yeah that's pretty much it.

But theoretically speaking, is there a better method to allow modularity for quests? And I mean a method that doesn't make use of the "dofile" function and that doesn't require the lua code to retrieve files from their full path in the mod directory?

 

[whoward69]

Not necessarily better, but you could use the same approach as the InfoCorner drop-down. That is a custom Addin type and a couple of Lua events. That way only the VFS is involved, and not the hard disk

 

[Vicevirtuoso]

So, having looked a little bit more into it, what the Quest files are doing when the dofile() is executed is returning an "hmake" (I assume this is related to Havok Script) containing all of the quest's functions, such as OnInit(), OnStart(), etc. This returned hmake is then put in QuestSystem's g_QuestScripts table.

This could possibly be worked around. How I'd see it:

• Use include() to include every script file instead of dofile()
• Every quest script must be manually edited to add its hmake QuestScript to the g_QuestScripts table itself, along with its metatable, Info, and Type

Would certainly need to be tested first. May not be unworkable, but it *will* require more work than I'm willing to do on this half-game right now. I just can't conjure up the willpower to copy and edit every single quest, quest objective, covert operation, and plot bonus until Firaxis can demonstrate that they're going to attempt to make the game fun again at some point.

At the risk of sounding quite lazy and apathetic, my current solution for this issue is "Rename your user directory to remove diacritics."

 

[Ryoga]

Hi, I know I've been bothering you a lot lately, and I know that you no longer care about this mod and civ BE in general, but I thought you might want to know.

It turns out that this utility as it is has a critical error. I have confirmed that simply by launching a lasercom satellite the quest scripts for the Emancipation and Promised Land victories will crash.
Firetuner returns an error on the line 583 of your modified QuestSystem.lua.

I am not completely positive that I found the real reason this happens, but a simple comparison between your file and the patched one shows that yours is entirely missing a whole function "BroadcastVictoryProgressNotification", which is probably the function that the victory quest scripts are calling.

Anyway any mods that is using any version of your script will make impossible for those victories to be completed (not a small issue). The question is, which mod is still using it at this point?

My affinity Quest Fix doesn't use that, and I have already included a fixed version of the patched QuestSystem.lua in my "True Domination" mod.

Alien Strains hasn't been updated in months, and it simply can't be used in its current version.

 

[Vicevirtuoso]

That seems like a much easier issue to correct than the aforementioned one, so I can fix it sometime in the next few days.