LuaJIT for Civilization V (Faster Turns)

[AntmanMike]

EDIT: Have updated for LuaJIT 2.0.1

Not entirely sure where to put this thread since it isn't quite a mod...

After reading http://forums.civfanatics.com/showthread.php?t=388670, I realized that the arguments about Lua were exactly right - an Engineer myself, Lua is a fully interpreted language and can be slow for games such as this.

I therefore took it upon myself to make a fast build of LuaJIT (no fastcall, sorry) that should be drop in compatible, as LuaJIT is ABI and API compatible with Lua. I've tested it myself, it seems marginally faster.

You will need 7zip or WinRAR to extract the archive - just overwrite your current DLL (I recommend backing it up, but you can always download a more recent, standard version from the Lua site), and you should see benefits the next time you start the game.

I also do not take credit for LuaJIT, I merely built it. I don't take much credit for this - a chimp with training can execute a compile (though I also made some changes to it so it should be marginally faster). If anyone else has already done this, sorry!

Edit: I have absolutely NO idea if Civ V has any kind of cheat protection - if it does, it may detect that the DLL is different (different checksum, obviously). Though as per the EULA they have no grounds to ban on this, it may still occur. You've been forewarned! Not entirely why they'd care if the Lua DLL is different... or even the client. It's a server-side (IE, hosting client) managed game, so you really can't "cheat"...

[Shmike]

Definitely going to try this. Is it available through the mod browser? I'd rather get it from there.

[AntmanMike]

It's a replacement DLL for the Lua library, it's not quite a mod.

[The_J]

Moderator Action: Moved to C&C utilities.
I guess modders will have the best use for it .

[AntmanMike]

Originally Posted by The_J

Moderator Action: Moved to C&C utilities.
I guess modders will have the best use for it .

Most likely modders will have the least use of it. It's meant for the average user who wants the game to run faster. From the modders perspective, it would change nothing as LuaJIT and Lua have the same output, LuaJIT is simply faster.

[PawelS]

I installed it, and starting a new game seems to be faster (I suppose it's because the map generation uses LUA). I haven't yet checked how it affects the late game turn times.

But there is one problem: when I exit the game, it "hangs" - I can still see the game menu, but the cursor changes to the standard Windows one. Pressing Ctrl-Alt-Del, and then Cancel helps to see the Desktop again (I use Win7 64-bit). Do you have an idea why it happens?

[Kruelgor]

Originally Posted by PawelS

I installed it, and starting a new game seems to be faster (I suppose it's because the map generation uses LUA). I haven't yet checked how it affects the late game turn times.

But there is one problem: when I exit the game, it "hangs" - I can still see the game menu, but the cursor changes to the standard Windows one. Pressing Ctrl-Alt-Del, and then Cancel helps to see the Desktop again (I use Win7 64-bit). Do you have an idea why it happens?

I was going to try it, it sounds great, but the problem you're having there is a bit discouraging.

[PawelS]

You can try it, it's nothing harmful, and possibly you won't have this problem.

[AntmanMike]

Off hand, no, but I experienced the same delay in testing and rather ignored it. When I get home from work I will pop in a debug build of LuaJIT and see where it may be hanging. For me, it did close in the end but just took longer. It sounds like the window or rendering context isn't being fully closed in time but control is shifted back to Explorer. Could also be a bug in Civ's Lua that doesn't show using Lua but shows using LuaJIT (which, given that it must output machine code, is stricter). Or vice-versa.

[The_J]

Originally Posted by AntmanMike

Most likely modders will have the least use of it. It's meant for the average user who wants the game to run faster. From the modders perspective, it would change nothing as LuaJIT and Lua have the same output, LuaJIT is simply faster.

With this point of view, you are sure right .
But will most users install a mod which "just" runs faster? No, they all want the additional stuff -> bigger mods are likely to incorporate this one here (unless i'm wrong here and it can't be used with mods, but then it's still right here in the utilities section).

[alpaca]

Originally Posted by The_J

With this point of view, you are sure right .
But will most users install a mod which "just" runs faster? No, they all want the additional stuff -> bigger mods are likely to incorporate this one here (unless i'm wrong here and it can't be used with mods, but then it's still right here in the utilities section).

Not really because it's a DLL replacement rather than something a mod can switch on. People have to install it manually, but you can sure tell them how to.

[AntmanMike]

Not sure on the policy of resurrection, but since I'm not quite resurrecting, I've updated the DLL to be LuaJIT 2.0.1. It has been built with practically all optimizations enabled, and requires an SSE2-capable CPU (as does Civilization V). The link is in the original post.

[Bobert13]

You should mention that the Microsoft Visual C++ 2012 Redistributable runtime is required for this to operate.

Missing msvcr110.dll is no fun to troubleshoot.

Edit: Well, installing the redistributable and restarting my pc didn't fix the issue.

Edit Edit: I copied the .dll from \System32\ to the game's root directory \Steam\steamapps\common\Sid Meier's blah blah blah\ and I got the following error instead of the missing .dll message.

---------------------------
CivilizationV_DX11.exe - Application Error
---------------------------
The application was unable to start correctly (0xc000007b). Click OK to close the application.
---------------------------
OK
---------------------------

The game runs fine if I use the original Lua51_Win32.dll

[AntmanMike]

Originally Posted by Bobert13

You should mention that the Microsoft Visual C++ 2012 Redistributable runtime is required for this to operate.

Missing msvcr110.dll is no fun to troubleshoot.

Edit: Well, installing the redistributable and restarting my pc didn't fix the issue.

Edit Edit: I copied the .dll from \System32\ to the game's root directory \Steam\steamapps\common\Sid Meier's blah blah blah\ and I got the following error instead of the missing .dll message.

---------------------------
CivilizationV_DX11.exe - Application Error
---------------------------
The application was unable to start correctly (0xc000007b). Click OK to close the application.
---------------------------
OK
---------------------------

The game runs fine if I use the original Lua51_Win32.dll

I've uploaded a statically-linked version which ideally should not require the MSVC11 redistributable runtimes.

[Bobert13]

Originally Posted by AntmanMike

I've uploaded a statically-linked version which ideally should not require the MSVC11 redistributable runtimes.

The new version works.

And Holy Crap!

From FireTuner's .lua console:

Generated Huge Map in 6.718 seconds.

With the original .dll, I was running anywhere from 18.5 to 19 seconds to generate a Huge Map with my modified version of PerfectWorld 3.

[Bobert13]

I know there probably isn't anything you can do about this since you just packed LuaJIT, but it's breaking some mods. I haven't really tested most of the mods I have installed but I know Reseed! v.7 doesn't play well with JIT at all. In fact, it doesn't load at all. It errors out during the loading process with the following:

 RSD_UI: 'Reseed!'... loading
 Runtime Error: [string "C:\Users\Boberto\Documents\My Games\Sid Meier..."]:6: bad argument #1 to 'unpack' (table expected, got no value)
 Runtime Error: Error loading C:\Users\Boberto\Documents\My Games\Sid Meier's Civilization 5\MODS\Reseed! (v 7)\RSD_Loader.lua.

I'm not going to pretend I know what I'm talking about with strings and XML references and whatnot, but I did find the function in RSD_Utils.lua that contains the problem and the line where RSD_Loader.lua calls said function. I'll probably try contact the author and see if he wouldn't mind taking a look.

Edit: If anybody else runs into this particular issue, adding the following line to function L() at the top of RSD_Utils.lua fixed it for me:

if arg == nil then arg = {} end

Anywhere in that function above the line that calls unpack(arg) works.

[DonQuiche]

Hello Bobert13 and thank you for your messages on Steam. I fixed the bug on my local copy and I am going to upload new versions of Reseed and IGE (both should be affected). Note that the same bug should also cause the map preview to fail in Reseed (I didn't check but it should), the new version will also fix that.

BUG DETAILS FOR OTHER DEVS (corrected)

• Before 5.1, Lua exposed a table named "arg" to retrieve the variable arguments in a function. Ex : local firstVarArg = arg[1]
• Starting with 5.1, the "arg" table is deprecated and one should use "..." to retrieve variable arguments. Ex : local firstVarArg= select(1, ...)
• The standard interpreter is backward compatible and supports both forms. But LuaJIT only supports the newest form.
• Solution for both LuaJIT and standard interpreter : uses "..." instead of "arg" everywhere.

Thanks to Whoward69's for correcting me about this (see message below).

• Let f be a varargs function (a function with a "..." argument). Ex: function L(str, ...)
• Now f is called at some point without any optional argument. Ex: L("abc")
• According to the Lua specs, the "arg" parameter in f's body should be assigned with an empty table.
• But LuaJIT instead assigns a nil value.

[whoward69]

Lua 5.1 deprecated the "arg" variable in a varargs function (see http://www.lua.org/manual/5.1/manual.html#7.1).

You should be able to get the same effect with
local arg = {...}
or
local arg = table.pack(...)
(various web sources)

[DonQuiche]

Ahhh, thank you Whoward. Then I guess "arg" is not supported at all and I still have hidden bugs.

Well, I hope that the standard Lua interpreter supports both the pre-5.1 and post-5.1 ways.

[whoward69]

Originally Posted by AntmanMike

I've uploaded a statically-linked version which ideally should not require the MSVC11 redistributable runtimes.

Would it be possible to add "JIT" into the version string?

[13139.465] Initializing Lua 5.1.4 JIT

Then it's obvious in lua.log and the FireTuner console which dll is in use