LuaJIT for Civilization V (Faster Turns)

Is it something that can be included in a mod and "activated" through the VFS like a custom CivCore dll? Faster Lua would sure help some mods that make heavy use of Lua between turns (GEM comes to my mind, but there are probably others).
 
The question I have is, if I update the DLL and make use of said changes in my code, will it work with the old DLL? If not, it wouldn't make much sense for a modder to use this, as they wouldn't be able to upload to steam without a bunch of manual DLL install instructions I am not going to trust Joe User to make.
 
The question I have is, if I update the DLL and make use of said changes in my code, will it work with the old DLL? If not, it wouldn't make much sense for a modder to use this, as they wouldn't be able to upload to steam without a bunch of manual DLL install instructions I am not going to trust Joe User to make.
I was wrong in this message,n as pointed by Whoward69 in message #18. I just updated the message you quoted where I answer your question (uses "..." to support both the standard interpreter and LuaJIT). Could you please remove the content of the quote so that other people are not lured into a mistake?
 
You'll need to find someone with the Mac skills and tools to recompile the source code as a library usable on the mac. Suggest you try asking in the Mac sub-forum
 
Can luajit be reason of crash with next lua.log end

TurnProcessing: Hiding TurnProcessing
TopPanel: LoadData() used 0 sec for ForeignTradeRoutes
...
TopPanel: -------------------------------------
DiscussionDialog: Handling LeaderMessage: 30, Kamehameha has become a nuisance. One that can only be dealt with through war. I seek allies - can I count you among them?
TopPanel: LoadData() used 0 sec for ForeignTradeRoutes
TopPanel: -------------------------------------
TopPanel: LoadData() used 0 sec for ForeignTradeRoutes
TopPanel: -------------------------------------
DiscussionDialog: Handling LeaderMessage: 12, That is disappointing.
Cannot cast from WorldAnchorControl to ControlBase. Using NULL instead.
ControlBase::SetAlpha could not be called: Call cannot be made on NULL
Cannot cast from BarControl to ControlBase. Using NULL instead.
ControlBase::SetAlpha could not be called: Call cannot be made on NULL
Cannot cast from WorldAnchorControl to ControlBase. Using NULL instead.
ControlBase::SetAlpha could not be called: Call cannot be made on NULL
Cannot cast from BarControl to ControlBase. Using NULL instead.
ControlBase::SetAlpha could not be called: Call cannot be made on NULL
Cannot cast from WorldAnchorControl to ControlBase. Using NULL instead.
ControlBase::SetHide could not be called: Call cannot be made on NULL
Cannot cast from WorldAnchorControl to ControlBase. Using NULL instead.
ControlBase::SetAlpha could not be called: Call cannot be made on NULL
Cannot cast from BarControl to ControlBase. Using NULL instead.
ControlBase::SetAlpha could not be called: Call cannot be made on NULL
Cannot cast from ImageControl to ControlBase. Using NULL instead.
ControlBase::SetOffset could not be called: Call cannot be made on NULL
Cannot cast from WorldAnchorControl to ControlBase. Using NULL instead.
ControlBase::SetHide could not be called: Call cannot be made on NULL
Cannot cast from WorldAnchorControl to ControlBase. Using NULL instead.
...lots of same errors
ControlBase::SetOffset could not be called: Call cannot be made on NULL
Cannot cast from WorldAnchorControl to ControlBase. Using NULL instead.
ControlBase::SetHide could not be called: Call cannot be made on NULL
Cannot cast from ImageControl to ControlBase. Using NULL instead.

only one possible problem in log:
NotificationPanel: Unknown Notification, Adding generic ***
Which happened in very diffrent places of log about 20 times. Probably no connection to the crash.

Crash is in Civ exe, stack trace unclear, but shows lua library in previous calls.

It's not because of debug or log game settings. (Had crash with both them on & off)

Seems, crash happened once even without mods, but not sure. Sure that it happens once per ~70 turns in quck small map game with ~30 AI with CultureDiffusion, ForeignTradeRoutes and xml policy changes.

Thanks for any help.
 
I built some new dlls for Luajit 2.0.2. These were built with the compilers from Visual Studio 2012 update 2, MSVCRT110 statically linked, arch:SSE2, O2, etc.

Link (Mediafire)

Moderator Action: File seems clean, according to VirusTotal.
 
My question is: Why isn't Firaxis doing this? Is there any downside for them to use LuaJIT?
 
My question is: Why isn't Firaxis doing this? Is there any downside for them to use LuaJIT?
Security: the official lua implementation is well known and verified, while luajit is not and a full review would be very expensive and not as secure. Besides Firaxis modified the official implementation to disable things such as file I/O in order to wrap mods in a sandbox.

The idea is that the average non-pirate user installs mostly renowned applications or games from "trustable" editors (but who is trustable in the days of Prism?), so mods could be a weakness since they come from unknown amateurs.
 
If they can afford to modify Lua, I don't think reviewing and modifying LuaJIT is that far off. Performance is a key matter for games - just look at what length other game makers go to in order to improve performance.

BTW, how much faster (in percentage) is LuaJIT subjectively (user perception) in Civ5? (I haven't tried it myself.)
 
If they can afford to modify Lua, I don't think reviewing and modifying LuaJIT is that far off.
I have to disagree with that. The modifications they made likely amount to a couple of days, maybe a single day. On the other hand, a security review of the whole lua implementation is a matter of weeks or months.

Performance is a key matter for games - just look at what length other game makers go to in order to improve performance.

BTW, how much faster (in percentage) is LuaJIT subjectively (user perception) in Civ5? (I haven't tried it myself.)
When it comes to possible optimizations, I guess that lua is far to be a priority since it is only used on the UI and map generation, so again it's probably not a good way to invest development time. Personally I never saw that much difference with luajit and I am pretty sure that Firaxis made measures to ensure that lua was fast enough to not be a hindrance during the prototyping stage, especially if you consider that they left python out for this very reason.
 
Personally I never saw that much difference with luajit and I am pretty sure that Firaxis made measures to ensure that lua was fast enough to not be a hindrance during the prototyping stage, especially if you consider that they left python out for this very reason.

Yeah, about that: I never could understand it - and I couldn't find any first-hand (be it official or otherwiese) explanation from Firaxis for their reasoning on dropping Python. I've been programming in Python since version 1.x and I know it to be one of the best optimized interpreting languages out there. I can hardly imagine it being "too slow" - assuming the code is properly written. Which, sadly, the Civ4 python code was anything but - in fact, the code was so primitive (as in: coding Python as if it were Basic) it was painful to read. The Lua code of Civ5 is marginally better, but I'd say it can still be classified as "primitive".
 
I think the matter of performances was actually regarding the startup time since the Python VM is heavier and its sources are slower to parse because of its features. So lua is faster on that point, although at the expense of richness and convenience.

And it makes sense since, again, I do not expect the script part to take a significant amount of CPU time, while the initialization time is still too long for my taste. One more reason for Firaxis to not use luaJit since it likely has a slower startup time.

Note that all of that is pure speculation from me.
 
LuaJit will only make noticeable speed improvements if your mod executes pages and pages of Lua frequently - eg for total conversion mods.

If you're just using it to add a trait (say), the main consumer of time will be the context switches between the C++ DLL and the Lua sub-system as the events trigger.
 
My mod has pages and pages of Lua. It's adding about 50% to run time.

I still have a lot of potential for Lua optimization, which is part of the reason I haven't looked hard at LuaJit yet.

The other is that I don't quite understand how LuaJit would save time during game run, as opposed to only at game init. From a very cursory look, my understanding (probably incorrect) is that it is just giving you pre-compiled Lua. If so, then won't it only save you time during game init and not per turn? Can someone please tell me where I'm misunderstanding?
 
LuaJit will only make noticeable speed improvements if your mod executes pages and pages of Lua frequently - eg for total conversion mods.
Of course : the improvements from luaJit will be significant only if the slow down from the mod was significant in the first place. But I wonder whether there are really such mods. Unless someone writes his own pathfinding or other computationally intensive tasks. Pazyryk reports a high but says that he has a lot of room for improvements.

The other is that I don't quite understand how LuaJit would save time during game run, as opposed to only at game init. From a very cursory look, my understanding (probably incorrect) is that it is just giving you pre-compiled Lua. If so, then won't it only save you time during game init and not per turn? Can someone please tell me where I'm misunderstanding?
A standard interpreter parses the file then builds a bytecode representation: typically you have opcodes ("add", "read index", "push", "invoke", etc) acting on an value stack ("add" pops the two topmost values and push one). On execution, the VM looks for the next opcode, performs the operation, then look at the next opcode, performs the operation, etc. It is like emulating a CPU (although the image is wrong in many places).

A jit compiler, however, transforms the bytecode stream into a x86 stream similar to the program's stream itself. That way you get rid of the CPU emulation and directly run the code on the CPU like any program. So it adds a lag the first time a function is ran (since the bytecode needs to be compiled) but subsequent calls are faster. The JIT introduces a latency at startup but is faster after that.

Finally, the CPU emulation image is mostly wrong because a CPU model and a VM model are very different:
* The x86 instructions acts on registers while the bytecodes act on a value stack.
* The x86 instructions only handles numeric values while the bytecodes act on complex values (objects, strings, etc) and include high-level operations such as member accesses.
 
Of course : the improvements from luaJit will be significant only if the slow down from the mod was significant in the first place. But I wonder whether there are really such mods.

There are a few candidates - those that replace all the core unit movement/attack code with their own, those that have to calculate re-supply of units from many possible cites along ever changing road/rail routes - but not many.

For most Lua files the gain in performance will be less than the time it took the user to click the mouse to execute the file in the first place. I only noticed an improvement in one file in all my mods by using LuaJIT - and hand-optimisation (by caching into a table) even wiped that out, so for most files, careful study of the code will gain better improvements than reliance on a "better compiler" (as was always the case since Fortran 4 and before!)
 
Top Bottom