[TUTORIAL] Compiling the DLL on Linux

[bluepotato]

Getting the DLL to compile using wine isn't nearly as trivial as it is on Windows. But in the process of getting rid of my previous hacky Makefile setup I mostly figured this out, so I decided to write a guide for it.

Prerequisites
- A PC with a Linux distribution. I've done this on several distributions and haven't found one that this didn't work on.
- Civ 4 installed with wine. This is trivial if you have the GOG version, just winetrick msxml3 and d3dx9. See the winehq page for further information.
- Basic Linux knowledge. You should at least know stuff like how paths work and how to use wine in a terminal emulator.

Step 0: Getting the source code
You should probably have it by now, but if you don't, and want to start from scratch, clone Leoreth's repo at https://github.com/dguenms/beyond-the-sword-sdk.

Step 1: Getting wine 1.7
Why do we need such an ancient version of wine?, you might ask - and rightly so, wine 1.7 was released 5 years ago. Well as it turns out, most newer versions don't work with the VC++ 2003 compiler and will cause an internal compiler error. 1.8 or 1.9 might work as well, I just haven't tried. 2.0 definitely didn't work for me.

Luckily, PlayOnLinux provides an easy way to get wine 1.7. After installing it, go to Tools->Manage Wine Versions. Then select wine 1.7.55 (on the x86 tab), and install it. You won't need PlayOnLinux after this.
You should now be able to locate your wine executable in ~/.PlayOnLinux/wine/linux-x86/1.7.55/bin. I like to move this to a ~/.wine_versions folder so that its path is ~/.wine_versions/linux-x86/1.7.55/bin/wine.

Now create a new wineprefix using

$ export WINEPREFIX=~/compile_linux; ~/.wine_versions/linux-x86/1.7.55/bin/wine wineboot

After this, you should have a 32-bit wineprefix in the ~/compile_linux folder.

Step 2: Getting the Visual C++ 2003 Toolkit and the Platform SDK
For the VC++ 2003 Toolkit I like to use Leoreth's installer. Install this in your regular wineprefix (presumably in .wine, using your system-wide wine command), then move the .wine/drive_c/Program Files/Microsoft Visual C++ Toolkit 2003 folder to ~/compile_linux/drive_c/Program Files.

Delete the PSDK folder, it won't work. Instead, download this installer from Microsoft. Mount and install it in your regular wineprefix (next next finish, though you may want to select the Core SDK only in the custom install option). Now you should have a Program Files/Microsoft Platform SDK folder; move this to ~/compile_linux/drive_c/Program Files as well.

Step 3: Getting fastdep
You will need a fastdep binary compatible with the build script. For this you'll have to clone my patched version like this:

$ svn checkout https://github.com/bptato/RFC-Greek-World/trunk/CvGameCoreDLL/bin/fastdep-0.16

Place the resulting folder in your CvGameCoreDLL/bin folder, and optionally compile fastdep using make if you can't/don't want to use the binary I compiled.

Step 4: Compiling the DLL
This should be the easiest step if you've done everything correctly. I've made a shell script which functions like the Makefile (probably not perfect but it works; suggestions are welcome); just save https://raw.githubusercontent.com/bptato/RFC-Greek-World/master/CvGameCoreDLL/compile.sh in your DLL folder.

You will also have to download compile_settings.sh and save it in the same folder: https://raw.githubusercontent.com/bptato/RFC-Greek-World/master/CvGameCoreDLL/compile_settings.sh.

Then make sure the variables in compile_settings.sh match your setup. Specifically, PYTHON and BOOST should point to an existing folder; I placed these in my DLL folder, since this version of wine can behave weird with absolute paths outside of the wineprefix. If you really want to use absolute paths (to e.g. put your Python/Boost folders in your civ folder as recommended in Nightinggale's makefile thread), use something like "Z:$HOME/your/path". Also make sure the FASTDEP variable points to your fastdep binary.

Finally, make compile.sh executable:

$ chmod +x compile.sh

and launch it like this:

$ ./compile.sh [release/debug/final_release]

You can also use [clean] to remove the Release/, Debug/, Final_release/, or if no target was specified, all of these folders.
And that's it!

Further notes
The final_release target (idea from advciv's makefile) performs whole program optimization. This means that linking takes ages, but you supposedly get a somewhat faster DLL.

Also, by default, the script will spawn a child process for checking and compiling every single cpp file to make use of all available processing power. While this speeds up compilation on my (hexa-core) system, it may actually be slower than running a single process on systems with a low amount of cores. Change PARALLEL=true to PARALLEL=false in compile_settings.sh to disable this.

For editing the sources, any text editor works. Those that I've found so far with somewhat usable code completion support are Geany, and vim/neovim with the OmniCppComplete plugin. CodeBlocks's completion also works nicely, but it's a huge pain to set up (an ancient windows version is required, which of course can't use compile.sh). And I haven't the slightest clue how to get the clangd language server to cooperate, unfortunately.

Debugging can be done with winedbg, see this page for further information. Most of the time you'll want to use the info proc (get civ 4's process id), attach (attach to a running instance of civ 4), cont (after attaching) and bt (get backtrace) commands.

 

[bluepotato]

I'm pretty sure that I'm still the only person who is modding the civ4 DLL on Linux, but just in case I'm wrong, a slight update:
I've compiled the fastdep sources for Linux (with some small modifications), so that I could get rid of some hacks I had to use to fit its output into a variable (also, it's faster when running natively). So you'll need this fastdep version for the script to work, which you can get without downloading the entire mod like this:
svn checkout https://github.com/bptato/RFC-Greek-World/trunk/CvGameCoreDLL/bin/fastdep-0.16

 

[Nightinggale]

I'm pretty sure that I'm still the only person who is modding the civ4 DLL on Linux, but just in case I'm wrong

I know you aren't completely alone and it's possible that there might be more in the future. Just keep up the good work and post whatever you figure out in regards to working on linux.

 

[KeeperOT7Keys]

thanks, your scripts were very helpful

 

[Ramkhamhaeng]

Hi, I was able to compile the DLL due your instructions and compile script. Thanks.
For other readers I want comment a few things because the setup is a little bit easier as described.


• For me it wasn't required to take the ancient wine version. I've used wine-8.0 without problems during the installation/building.
• I've installed the SDK, etc. directly in this new created Environment/Wineprefix.

• The default install path for Leoreth's installer is ../drive_c/Program Files (x86)/Civ4SDK/Microsoft Visual C++ Toolkit 2003/
Do not wonder if it's not found in 'Program Files'

• If you get following error…

CvTextScreens.cpp(5) : fatal error C1083: Cannot open include file: 'CvTextMgr.h': No such file 
or directory

…simply remove 'CvTextScreens.cpp'. The default Windows-Makefile blacklists this non-required file during the build, too.

• If you adapt the Makefile in your project you has to transfer the changes into compile.sh, too.

 

[Ramkhamhaeng]

I've created a bash script to automate the setup process a little bit more:

pbmod/setup_dll_build_with_wine.sh at master · civ4-mp/pbmod

Mods for Civilization 4 Pitboss. Contribute to civ4-mp/pbmod development by creating an account on GitHub.

github.com

It's part of my mod, but the script should work as standalone, too. Probabley a few paths has to be altered. (e.g. to your CvGameCoreDLL/compile.sh file.)

Variant A) The following would download the Microsoft SDK and Toolkit and starting the update. You just need to mount the image file and clicking through the installers manually.

./setup_dll_build_with_wine.sh install

Variant B) This variant trying to avoid the manual steps of variant A.

./setup_dll_build_with_wine.sh unattended

Tested with wine 8.0 on Manjaro Linux. Hope the script didn't break sooe because of the used download links.

 

[bluepotato]

Thanks for the updates. Glad to see my post helped

• The default install path for Leoreth's installer is ../drive_c/Program Files (x86)/Civ4SDK/Microsoft Visual C++ Toolkit 2003/

Yeah, that is expected if you install in a 64-bit wineprefix. 32-bit only has Program Files, and I couldn't get wine64 of wine 1.7 working.

…simply remove 'CvTextScreens.cpp'. The default Windows-Makefile blacklists this non-required file during the build, too.

Ah right, I forgot about that, thanks for mentioning it.

• For me it wasn't required to take the ancient wine version. I've used wine-8.0 without problems during the installation/building.

Interesting, it seems to be working for me too. Though when I do

./compile.sh debug
./compile.sh release

with wine 8.2, this results in an error during linking, and only works if I insert a ./compile.sh clean between the two. Using the above two commands, wine 1.7 links without troubles. No clue why this happens, so I think I'll stick with 1.7 for now.

If you adapt the Makefile in your project you has to transfer the changes into compile.sh, too.

Sure, depending on what you change... I think the settings corresponding to the Makefile's settings are all in compile_settings.sh, so changing that should be enough in most cases.