New makefile (faster compilation, profiling and more)

Nightinggale

Deity
Joined
Feb 2, 2009
Messages
5,263
I started modding recently and like everybody else I used Danny's makefile. However it didn't live up to my needs and little by little it changed into something else. Now it's full of new features and I decided to share it with all of you.

First of all, credits for the original makefile goes to DannyDaemonic. You can see his makefile in this thread. While most of my makefile changed, the core idea and setup is almost unchanged. It's still the same compiler with almost the same options.

This makefile is written for Civ4 colonization, but it should work for all Civ4.

What my new makefile changes:
  • Multi core CPU support
  • A total of 4 different DLL files
  • Faster precompile setup (though it wasn't that slow to begin with)
  • Different (easier?) setup
  • Smaller file as code is reused

Userguide
A total of 4 different DLL files are now possible.
  • Assert (Release optimization with asserts enabled)
  • Debug
  • Profile (Release optimization with symbols added)
  • Release

Important notice for steam users:
It has come to my attention that civ4col crashes when it is debugged. Switching to the CD version appears to fix the problem. The cause is unknown and I'm unaware of how to get it to work. I haven't investigated as I don't have the steam version myself.

Released version 2.5
Changelog:
Spoiler :

2.1 and 2.2 was never uploaded as a standalone release. However they exist in some project commit history and are added here for completeness.
2.5:
Added SOURCE_DIR, allowing project files and source files to be in different locations
Added the ability to compile cpp files in subdirectories for the source dir
Fixed a number of bugs in fastdep to allow this to work
Added Makefile.project

2.3:
Fixed fastdep (broken in 2.2)

2.2:
Makefile now generates a file called Makefile.settings
Changes in this file overwrites changes written in Makefile. Useful is the makefile is part of a project in svn/git.
Makefile.settings can also use CUSTOM_CFLAGS and CUSTOM_LDFLAGS.

TEMP_FILE_DIR (default temp_files) can also be changed in Makefile.settings. Disk speed is more important for temp files than source files. You can compile a source on a network drive with decent speed if you store temp files locally. Pointless for most users.

Example:
Code:
CUSTOM_CFLAGS= -DMY_FLAG
CIV4_PATH=c:\Program Files (x86)\Steam\steamapps\common\civilization iv colonization\CvGameCoreDLL
This will define MY_FLAG in C++ and search for python and boost in the default install dir for colonization installed by steam.

2.1:
Uses $(PROGRAMFILES) by default, which converts to "C:\Program Files (x86)" or whatever it is called on your system (32/64 bit support in any language?)

Added new directory temp_files. All temp files are placed here instead of together with the source itself.



Assert executes almost as fast as a release build. However it will trigger asserts, meaning it is useful to play normally at decent speed to get into cases where asserts trigger. You will likely need a Debug build to investigate the asserts though. Autosave each turn when using this is likely a good idea and random random seed is a bad idea.

Profile provides info regarding the source files. This mean profiling tools like Very Sleepy and AMD Code Analyst can tell the amount of time spend in each line of the C++ code. Personally I had the best experience with Very Sleepy, possibly due to how simple it is to set up.

The calls to the makefile is a bit different now. You set up the type of DLL file and then what to do with it.

Project file setup
The attached project files are written for Visual Studio 2010. This section is for people who for one reason or another will make their own rather than using the attached one.

The place to set up build options is Project->(mod name) Properties->Configuration Properties->Nmake
Here you edit your targets.

Edit Command lines for build, rebuild and clean like this:
Code:
set TARGET=Release
nmake clean (not for build)
nmake source_list
nmake fastdep
nmake dll
Changing TARGET will give access to all 4 types of DLL files. You likely need to add more targets in Configuration Manager. Additional setup is needed if you want to use multiple CPU cores (see below)

The clean command only needs nmake clean.

Using multiple cores
Nmake works fine, but Microsoft decided for some reason that one core is enough. With most people using dual or quad cores today this makes compiling slower than it have to. To get around that limitation you have to use the 3rd party nmake clone called jom. Using that instead of nmake will automatically use all cores for compilation. However it can't always run the same makefile as nmake, which is why it has to be called a bit different.

Example of rebuild Release with multiple cores
Code:
set TARGET=Release
nmake clean
nmake source_list
nmake fastdep
nmake precompile
jom build
Notice how dll is split into precompile and build. That's because jom can't handle the setup before the cpp files are compiled.
jom needs to be in the project directory or in the path for this to work. You can write something like
Code:
bin\jom build
This will work if jom is in the bin directory (presumably together with fastdep.exe).

Makefile setup
The first time the compiler uses the makefile, it will create Makefile.settings and Makefile.project.

Makefile.settings is intended to personal use and can contain setup like path to boost and similar computer specific data. It uses the flags CUSTOM_CFLAGS and CUSTOM_LDFLAGS.

Makefile.project is intended to be added to svn/git and contains data, which will be needed by everybody in order to compile the code. In addition to PROJECT_CFLAGS and PROJECT_LDFLAGS, it has the flags:
    • PRECOMPILE_DEPENDENCIES - list of files, which forces the precompiled header to be generated again. Fastdep should take care of this in most cases
    • SOURCE_DIR - relative path from the makefile to the source dir. Can use and even start with ..
    • SUBDIR1 - name of a directory in the source dir, which contains cpp files, which should be included
    • SUBDIRPATH1 - the path from SUBDIR1 to SOURCE_DIR. Defaults to ..\, which mean it should only be used if a directory tree is included
It's possible to use multiple subdirs, in which case they are named SUBDIR1, SUBDIR2 and so on. The highest allowed number is 15, though copy pasting in the Makefile can add even more if needed.

Note on speed: while it would be logical to assume 4 cores makes compilation 4 times as fast it isn't the case. Precompile and linking is still using only one core. Also when compiling 4 files at once it also mean a lot more file I/O at the same time. Waiting for disk access can become an issue. Still you will get noteworthy faster compilation. Defraging your HD could improve speed as well.

IMPORTANT FOR SINGLECORE CPUS: jom adds a little bit of extra overhead to gain support for multiple cores. This mean if you only have one, then DON'T use jom as it will likely just slow you down a little.

Jom can't build Profile or Debug targets
This is because Microsoft decided that symbols are written to vc70.pdb by the compiler for each cpp file. This means jom tries to have multiple threads write to the same file at the same time. The makefile will give a proper error message for this instead of trying, which gives hard to read error messages.

I recommend having two Release build rules, one with jom and one without it. The reason is that jom handles files in parallel, but they also write output in parallel. This makes reading error messages tricky as it goes
Code:
compile A
compile B
compile C
compile D
error B
error A
compile E
error C
error E
While it is possible, it does take a little work to see which error belongs to which file. Luckily if the two build rules are written like in the examples (but without nmake clean), nmake will start where jom encountered an error as they share object files. The object files depends only on TARGET.

Shared boost and python libs
The makefile can use boost and python in the Civ install dir instead of in the project directory. EmperorFool and Asaf came up with the idea, though I implemented it a bit different.

At the start of the makefile is a variable called CIV4_PATH (commented out by default). If this is set, the compiler will look for boost and python in the directory stated. This mean it is the path to CvGameCoreDLL including CvGameCoreDLL if you use your installed Civ.

Firaxis didn't change or update either boost or python. This mean linking to warlords CvGameCoreDLL is fine even if you compile for say colonization or vice versa.

Environmental variables[\B]
The makefile now listens to environmental variables set in windows. They aren't needed, but they can be used to overwrite variables in the makefile. They are as follows:
CIV4_LIB_INSTALL_PATH overwrites CIV4_PATH
FASTDEP_FULL_PATH overwrites FD
This is useful if the makefile is added to svn/git because then the user can set this to the local system without modifying the file.

Guilde on how to set environmental variables in Windows 7.

Path to YOURMOD
I changed the default path to YOURMOD to" ..". This works for the majority of MODs with files and source distributed in the same directory, which appears to be common, specially for svn/git sources.

I added Visual C++ 2010 project files with a setup for all 4 TARGETs with fast option for Assert and Release. It expects jom to be in the bin directory.

Guide for writing fast compiling code:
Firaxis screwed up including headers and included everything in all files. Using fastdep allows us to recompile just the files including the modified header. This guide will tell how to remove unneeded includes with a great reduction in compilation time to follow. I measured a reduction of 66% when changing CvPlayer.h.
Spoiler :
Rule 1: Only include headers you really need.

Rule 2: try to avoid including headers inside headers.
This can't be completely avoided.

Rule 3: Declare class instead of include.
You don't need to include the header to make a pointer to a class. Say you need a pointer to CvUnit, you can write
Code:
class CvUnit;
instead of
Code:
#include "CvUnit.h"

Rule 4: adding headers is easy. Consider adding a new header if it is only included in two files instead of writing the change in a header included in 20 files.
Once you are done tweaking the new code and don't plan to modify it anytime soon again, you can copy it into the big header if you like.
Also be careful not to add 50 header files with less than 10 lines in each. Code organization shouldn't suffer just because you don't want to wait on the compiler.

Fix Firaxis mess
Apply the rules to CvGameCoreDLL.h
Firaxis wrote this file to include a lot of libraries and the headers in this file are precompiled. This is great for compilation performance.
This file should be the first one to include in any cpp file (top of the list)
What is not so great is the fact that it forces a recompilation of all files if any one of the included files is modified. Because of this it would be beneficial to remove likely to be modified headers from this file and include those files directly in the cpp files needing those.
I managed to remove:
#include "CvPlayerAI.h"
#include "CvMap.h"
#include "CvGameAI.h"
#include "CvPlot.h"
#include "CvUnit.h"
#include "CvCity.h"
#include "CyCity.h"
#include "CvInfos.h"
#include "CvTeamAI.h"
#include "CvArea.h"
#include "CvDeal.h"
#include "CvGame.h"
#include "CyGlobalContext.h"
#include "CvSelectionGroup.h"
#include "CvTalkingHeadMessage.h"
#include "CvCityAI.h"
#include "CvSelectionGroupAI.h"
#include "CvUnitAI.h"

CvGameCoreUtils.h is still included and that one includes CvMap.h, which includes CvArea.h and CvPlot.h.
I solved this by removing CvMap.h and moved all map related functions from CvGameCoreUtils.h to CvMap.h.

Once I was done testing these changes I committed the changes to Medieval Conquest. The changes can be seen here: https://github.com/Nightinggale/Medieval_Tech/commit/e4c77edc9667498ee0f97270c4db87afc4d5ea41
Sadly it appears to have modified too many lines. I have had a line ending issue, which appears to have hit again :(
Most of the changes are readable and the approach should be readable.


Starting the mod from the project itself
If the project is started using the default settings, it will try to execute the dll file, which is useless. However it is possible to set up the project to start the game exe and tell it to start the mod in question.

Benefits:
  • Allows starting the game by hitting shift-F5
  • F5 starts with the debugger attached
  • The dll is recompiled if needed when started from the project
The ability to start a debug build with the debugger attached from the start makes debugging a whole lot easier than attaching to an existing process. It's good for debugging in general, but it is critical if you plan on debugging anything happening prior to the Main Menu as you do not have to quickly attach before the game reach the line you plan to debug.

Here is how you set up the project to start the mod
You open the Project menu and select References.
Under Configuration Properties, you select Debugging.

line|value|explanation
Command|C:\...\Civ4BeyondSword.exe|Full path to the game exe
Command arguments|mod=" modname "| modname is the name of the mod you want to start
Working Directory|Full path to the dir with the exe|copy paste the line from Command and remove Civ4BeyondSword.exe
This works best if the makefile is set to place the DLL in your mod. This mean YOURMOD should point to the root of your mod. The default setting is "..", but it can be set to anything in Makefile.settings. The default setting is good for most cases as it assumes a source/project directory in the root of the mod. This makes the source distributable and works well with svn/git.

Note: even though Civ4BeyondSword.exe is used in the example, it works with any version of the civ4 engine, including Colonization.
 

Attachments

  • Makefile 2.0+fastdep.zip
    104.5 KB · Views: 387
  • Makefile 2.0.zip
    2.6 KB · Views: 316
  • VC++ 2010 Project files.zip
    1.7 KB · Views: 333
  • Makefile_2.3.zip
    3.3 KB · Views: 381
  • makefile-2.5+project+fastdep-0.17.rar
    34 KB · Views: 402
Last edited:
Looks great! :goodjob:
 
Hi Nightingale!

First, thanks for providing this makefile :goodjob: which once :mischief: has fitted me very well.
The term "once" indicates the problem:
Since quite some time I can't compile anymore. :cry:

In the spoiler you will see the error message I'm getting over and over again:
Spoiler :
1> Datei nicht gefunden (en: File not found)
1> "D:\050_Spiele\010_Strat\Col\Mods\990_Create_DLL\VisualC++Toolkit2003\bin\cl.exe" /nologo /MD /O2 /Oy /Oi /G7 /DWIN32 /D_WINDOWS /D_USRDLL /DCVGAMECOREDLL_EXPORTS /DNDEBUG /DFINAL_RELEASE /GR /Gy /W3 /EHsc /Gd /Gm- /Yu"CvGameCoreDLL.h" /Fp"Release\CvGameCoreDLL.pch" /IBoost-1.32.0/include /IPython24/include /I"D:\050_Spiele\010_Strat\Col\Mods\990_Create_DLL\VisualC++Toolkit2003/include" /I"C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A/Include" /I"C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A/Include/mfc" /YcCvGameCoreDLL.h /Fo"Release\_precompile.obj" /c _precompile.cpp
1> _precompile.cpp
1>c1xx : fatal error C1083: Cannot open source file: '_precompile.cpp': No such file or directory
1>NMAKE : fatal error U1077: "D:\050_Spiele\010_Strat\Col\Mods\990_Create_DLL\VisualC++Toolkit2003\bin\cl.exe": R³ckgabe-Code "0x2" (return code "0x2")
1> Stop.
I have marked the parts which I think are important in red.

Although I have deinstalled and reinstalled VC++2010 Express, my folder "C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Include" does not contain a subfolder "mfc", but the "include" folder of the Microsoft SDK 6.0 (which I am making use of) does, as you can see in the spoiler (and just to make it obvious, of course I have included the "_precompile.cpp" in the source files):
Spoiler :





Just for your information, these are the changes I did to your version of the makefile:
Spoiler :
#### Paths ####
## TOOLKIT=C:\Program Files (x86)\Microsoft Visual C++ Toolkit 2003
TOOLKIT=D:\050_Spiele\010_Strat\Col\Mods\990_Create_DLL\VisualC++Toolkit2003
## PSDK=C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A
PSDK=D:\050_Spiele\010_Strat\Col\Mods\990_Create_DLL\WindowsPlatformSDKv60
## Uncomment to have newly compiled dlls copied to your mod's Assets directory
## YOURMOD=..

I was able to compile with your makefile for exactly one time (actually, I compiled the DLL in my signature by using it...). Why it doesn't work any more, is completely beyond me.
Any idea?
 

Attachments

  • Ashampoo_Snap_2013.11.05_18h34m59s_004_.png
    Ashampoo_Snap_2013.11.05_18h34m59s_004_.png
    79.2 KB · Views: 3,128
  • Ashampoo_Snap_2013.11.05_18h37m17s_005_.png
    Ashampoo_Snap_2013.11.05_18h37m17s_005_.png
    10.9 KB · Views: 3,145
c1xx : fatal error C1083: Cannot open source file: '_precompile.cpp': No such file or directory
This looks suspiciously like it fails to find _precompile.cpp. It looks in the same directory as Makefile is placed. Make sure you have the full source code together with the makefile. Some mods/modcomps only supply the files, which are modified compared to vanilla. Failure to merge them with vanilla will cause errors like this one.

Make sure you merge with a copy of vanilla. Also remember that Firaxis wrote CvStr and that it will not compile until you change it to CvString (or something like that).
 
EDIT: PROBLEM SOLVED! I was using the wrong version of windows SDK for VS 2012. Changing it to 8.0 A made it build although I'm getting other errors. I've posted about it here:

Having problems using VS 2012 and .NET framework 4 on Win 7 using windows SDK v8.0A as well. It seems to be building most of the .cpp files but I'm getting the following error:
"C:\Program Files (x86)\Microsoft SDKs\Windows\v8.0A\bin\rc.exe" /FoDebug\CvGameCoreDLL.res /I"F:\Games\Civ4 Colonization\CvGameCoreDLL/Boost-1.32.0/include" /I"F:\Games\Civ4 Colonization\CvGameCoreDLL/Python24/include" /I"C:\Program Files (x86)\Microsoft Visual C++ Toolkit 2003/include" /I"C:\Program Files (x86)\Microsoft SDKs\Windows\v8.0A/Include" /I"C:\Program Files (x86)\Microsoft SDKs\Windows\v8.0A/Include/mfc" CvGameCoreDLL.rc
1> 'C:\Program' is not recognized as an internal or external command,
1> operable program or batch file.

Original problem:
Spoiler :

I tried using your makefile and project settings but I'm getting the following errors while building on WinNT.h, WinBase.h and WinSvc.h. (These are Windows SDK files)

Errors snapshot (Image)
Spoiler :

Complete Error Text (Warning, very long :whew:)
Spoiler :

Error 3 error C2144: syntax error : 'unsigned int' should be preceded by ';' C:\Dev\WindowsSDK\include\WinNT.h 797 1 CvGameCoreDLL
Error 4 error C2501: '__inner_checkReturn' : missing storage-class or type specifiers C:\Dev\WindowsSDK\include\WinNT.h 797 1 CvGameCoreDLL
Error 5 error C2144: syntax error : 'unsigned int' should be preceded by ';' C:\Dev\WindowsSDK\include\WinNT.h 812 1 CvGameCoreDLL
Error 6 error C2501: '__inner_checkReturn' : missing storage-class or type specifiers C:\Dev\WindowsSDK\include\WinNT.h 812 1 CvGameCoreDLL
Error 7 error C2086: 'int __inner_checkReturn' : redefinition C:\Dev\WindowsSDK\include\WinNT.h 812 1 CvGameCoreDLL
Error 8 error C2146: syntax error : missing ';' before identifier 'DWORD' C:\Dev\WindowsSDK\include\WinBase.h 3839 1 CvGameCoreDLL
Error 9 error C2501: '__inner_checkReturn' : missing storage-class or type specifiers C:\Dev\WindowsSDK\include\WinBase.h 3839 1 CvGameCoreDLL
Error 10 error C2370: '__inner_checkReturn' : redefinition; different storage class C:\Dev\WindowsSDK\include\WinBase.h 3839 1 CvGameCoreDLL
Error 11 error C2146: syntax error : missing ';' before identifier 'BOOL' C:\Dev\WindowsSDK\include\WinBase.h 12156 1 CvGameCoreDLL
Error 12 error C2501: '__inner_checkReturn' : missing storage-class or type specifiers C:\Dev\WindowsSDK\include\WinBase.h 12156 1 CvGameCoreDLL
Error 13 error C2370: '__inner_checkReturn' : redefinition; different storage class C:\Dev\WindowsSDK\include\WinBase.h 12156 1 CvGameCoreDLL
Error 14 error C2146: syntax error : missing ';' before identifier 'BOOL' C:\Dev\WindowsSDK\include\WinBase.h 12164 1 CvGameCoreDLL
Error 15 error C2501: '__inner_checkReturn' : missing storage-class or type specifiers C:\Dev\WindowsSDK\include\WinBase.h 12164 1 CvGameCoreDLL
Error 16 error C2370: '__inner_checkReturn' : redefinition; different storage class C:\Dev\WindowsSDK\include\WinBase.h 12164 1 CvGameCoreDLL
Error 17 error C2146: syntax error : missing ';' before identifier 'BOOL' C:\Dev\WindowsSDK\include\WinBase.h 12212 1 CvGameCoreDLL
Error 18 error C2501: '__inner_checkReturn' : missing storage-class or type specifiers C:\Dev\WindowsSDK\include\WinBase.h 12212 1 CvGameCoreDLL
Error 19 error C2370: '__inner_checkReturn' : redefinition; different storage class C:\Dev\WindowsSDK\include\WinBase.h 12212 1 CvGameCoreDLL
Error 20 error C2146: syntax error : missing ';' before identifier 'BOOL' C:\Dev\WindowsSDK\include\WinBase.h 12223 1 CvGameCoreDLL
Error 21 error C2501: '__inner_checkReturn' : missing storage-class or type specifiers C:\Dev\WindowsSDK\include\WinBase.h 12223 1 CvGameCoreDLL
Error 22 error C2370: '__inner_checkReturn' : redefinition; different storage class C:\Dev\WindowsSDK\include\WinBase.h 12223 1 CvGameCoreDLL
Error 23 error C2146: syntax error : missing ';' before identifier 'BOOL' C:\Dev\WindowsSDK\include\WinBase.h 12233 1 CvGameCoreDLL
Error 24 error C2501: '__inner_checkReturn' : missing storage-class or type specifiers C:\Dev\WindowsSDK\include\WinBase.h 12233 1 CvGameCoreDLL
Error 25 error C2370: '__inner_checkReturn' : redefinition; different storage class C:\Dev\WindowsSDK\include\WinBase.h 12233 1 CvGameCoreDLL
Error 26 error C2146: syntax error : missing ';' before identifier 'BOOL' C:\Dev\WindowsSDK\include\WinBase.h 12254 1 CvGameCoreDLL
Error 27 error C2501: '__inner_checkReturn' : missing storage-class or type specifiers C:\Dev\WindowsSDK\include\WinBase.h 12254 1 CvGameCoreDLL
Error 28 error C2370: '__inner_checkReturn' : redefinition; different storage class C:\Dev\WindowsSDK\include\WinBase.h 12254 1 CvGameCoreDLL
Error 29 error C2146: syntax error : missing ';' before identifier 'BOOL' C:\Dev\WindowsSDK\include\WinBase.h 12263 1 CvGameCoreDLL
Error 30 error C2501: '__inner_checkReturn' : missing storage-class or type specifiers C:\Dev\WindowsSDK\include\WinBase.h 12263 1 CvGameCoreDLL
Error 31 error C2370: '__inner_checkReturn' : redefinition; different storage class C:\Dev\WindowsSDK\include\WinBase.h 12263 1 CvGameCoreDLL
Error 32 error C2146: syntax error : missing ';' before identifier 'PTP_POOL' C:\Dev\WindowsSDK\include\WinBase.h 12283 1 CvGameCoreDLL
Error 33 error C2501: '__inner_checkReturn' : missing storage-class or type specifiers C:\Dev\WindowsSDK\include\WinBase.h 12283 1 CvGameCoreDLL
Error 34 error C2370: '__inner_checkReturn' : redefinition; different storage class C:\Dev\WindowsSDK\include\WinBase.h 12283 1 CvGameCoreDLL
Error 35 error C2146: syntax error : missing ';' before identifier 'PTP_CLEANUP_GROUP' C:\Dev\WindowsSDK\include\WinBase.h 12315 1 CvGameCoreDLL
Error 36 error C2501: '__inner_checkReturn' : missing storage-class or type specifiers C:\Dev\WindowsSDK\include\WinBase.h 12315 1 CvGameCoreDLL
Error 37 error C2370: '__inner_checkReturn' : redefinition; different storage class C:\Dev\WindowsSDK\include\WinBase.h 12315 1 CvGameCoreDLL
Error 38 error C2146: syntax error : missing ';' before identifier 'BOOL' C:\Dev\WindowsSDK\include\WinBase.h 12456 1 CvGameCoreDLL
Error 39 error C2501: '__inner_checkReturn' : missing storage-class or type specifiers C:\Dev\WindowsSDK\include\WinBase.h 12456 1 CvGameCoreDLL
Error 40 error C2370: '__inner_checkReturn' : redefinition; different storage class C:\Dev\WindowsSDK\include\WinBase.h 12456 1 CvGameCoreDLL
Error 41 error C2146: syntax error : missing ';' before identifier 'PTP_WORK' C:\Dev\WindowsSDK\include\WinBase.h 12467 1 CvGameCoreDLL
Error 42 error C2501: '__inner_checkReturn' : missing storage-class or type specifiers C:\Dev\WindowsSDK\include\WinBase.h 12467 1 CvGameCoreDLL
Error 43 error C2370: '__inner_checkReturn' : redefinition; different storage class C:\Dev\WindowsSDK\include\WinBase.h 12467 1 CvGameCoreDLL
Error 44 error C2146: syntax error : missing ';' before identifier 'PTP_TIMER' C:\Dev\WindowsSDK\include\WinBase.h 12500 1 CvGameCoreDLL
Error 45 error C2501: '__inner_checkReturn' : missing storage-class or type specifiers C:\Dev\WindowsSDK\include\WinBase.h 12500 1 CvGameCoreDLL
Error 46 error C2370: '__inner_checkReturn' : redefinition; different storage class C:\Dev\WindowsSDK\include\WinBase.h 12500 1 CvGameCoreDLL
Error 47 error C2146: syntax error : missing ';' before identifier 'PTP_WAIT' C:\Dev\WindowsSDK\include\WinBase.h 12543 1 CvGameCoreDLL
Error 48 error C2501: '__inner_checkReturn' : missing storage-class or type specifiers C:\Dev\WindowsSDK\include\WinBase.h 12543 1 CvGameCoreDLL
Error 49 error C2370: '__inner_checkReturn' : redefinition; different storage class C:\Dev\WindowsSDK\include\WinBase.h 12543 1 CvGameCoreDLL
Error 50 error C2146: syntax error : missing ';' before identifier 'PTP_IO' C:\Dev\WindowsSDK\include\WinBase.h 12578 1 CvGameCoreDLL
Error 51 error C2501: '__inner_checkReturn' : missing storage-class or type specifiers C:\Dev\WindowsSDK\include\WinBase.h 12578 1 CvGameCoreDLL
Error 52 error C2370: '__inner_checkReturn' : redefinition; different storage class C:\Dev\WindowsSDK\include\WinBase.h 12578 1 CvGameCoreDLL
Error 53 error C2144: syntax error : 'int' should be preceded by ';' C:\Dev\WindowsSDK\include\WinSvc.h 777 1 CvGameCoreDLL
Error 54 error C2501: '__inner_checkReturn' : missing storage-class or type specifiers C:\Dev\WindowsSDK\include\WinSvc.h 777 1 CvGameCoreDLL
Error 55 error C2086: 'int __inner_checkReturn' : redefinition C:\Dev\WindowsSDK\include\WinSvc.h 777 1 CvGameCoreDLL
Error 56 error C2143: syntax error : missing ';' before '__stdcall' C:\Dev\WindowsSDK\include\WinSvc.h 777 1 CvGameCoreDLL
Error 57 error C2501: 'SC_HANDLE' : missing storage-class or type specifiers C:\Dev\WindowsSDK\include\WinSvc.h 777 1 CvGameCoreDLL
Error 58 error C2377: 'SC_HANDLE' : redefinition; typedef cannot be overloaded with any other symbol C:\Dev\WindowsSDK\include\WinSvc.h 777 1 CvGameCoreDLL
Error 59 error C2146: syntax error : missing ')' before identifier 'hSCManager' C:\Dev\WindowsSDK\include\WinSvc.h 779 1 CvGameCoreDLL
Error 61 error C2501: 'CreateServiceA' : missing storage-class or type specifiers C:\Dev\WindowsSDK\include\WinSvc.h 779 1 CvGameCoreDLL
Error 62 error C2059: syntax error : ')' C:\Dev\WindowsSDK\include\WinSvc.h 792 1 CvGameCoreDLL
Error 63 error C2144: syntax error : 'int' should be preceded by ';' C:\Dev\WindowsSDK\include\WinSvc.h 796 1 CvGameCoreDLL
Error 64 error C2501: '__inner_checkReturn' : missing storage-class or type specifiers C:\Dev\WindowsSDK\include\WinSvc.h 796 1 CvGameCoreDLL
Error 65 error C2086: 'int __inner_checkReturn' : redefinition C:\Dev\WindowsSDK\include\WinSvc.h 796 1 CvGameCoreDLL
Error 66 error C2143: syntax error : missing ';' before '__stdcall' C:\Dev\WindowsSDK\include\WinSvc.h 796 1 CvGameCoreDLL
Error 67 error C2501: 'SC_HANDLE' : missing storage-class or type specifiers C:\Dev\WindowsSDK\include\WinSvc.h 796 1 CvGameCoreDLL
Error 68 error C2146: syntax error : missing ')' before identifier 'hSCManager' C:\Dev\WindowsSDK\include\WinSvc.h 798 1 CvGameCoreDLL
Error 70 error C2501: 'CreateServiceW' : missing storage-class or type specifiers C:\Dev\WindowsSDK\include\WinSvc.h 798 1 CvGameCoreDLL
Error 71 error C2059: syntax error : ')' C:\Dev\WindowsSDK\include\WinSvc.h 811 1 CvGameCoreDLL
Error 72 error C2146: syntax error : missing ')' before identifier 'hService' C:\Dev\WindowsSDK\include\WinSvc.h 823 1 CvGameCoreDLL
Error 74 error C2491: 'DeleteService' : definition of dllimport data not allowed C:\Dev\WindowsSDK\include\WinSvc.h 823 1 CvGameCoreDLL
Error 75 error C2059: syntax error : ')' C:\Dev\WindowsSDK\include\WinSvc.h 823 1 CvGameCoreDLL
Error 76 error C2144: syntax error : 'int' should be preceded by ';' C:\Dev\WindowsSDK\include\WinSvc.h 828 1 CvGameCoreDLL
Error 77 error C2501: '__inner_checkReturn' : missing storage-class or type specifiers C:\Dev\WindowsSDK\include\WinSvc.h 828 1 CvGameCoreDLL
Error 78 error C2086: 'int __inner_checkReturn' : redefinition C:\Dev\WindowsSDK\include\WinSvc.h 828 1 CvGameCoreDLL
Error 79 error C2143: syntax error : missing ';' before '__stdcall' C:\Dev\WindowsSDK\include\WinSvc.h 828 1 CvGameCoreDLL
Error 80 error C2501: 'BOOL' : missing storage-class or type specifiers C:\Dev\WindowsSDK\include\WinSvc.h 828 1 CvGameCoreDLL
Error 81 error C2377: 'BOOL' : redefinition; typedef cannot be overloaded with any other symbol C:\Dev\WindowsSDK\include\WinSvc.h 828 1 CvGameCoreDLL
Error 82 error C2146: syntax error : missing ')' before identifier 'hService' C:\Dev\WindowsSDK\include\WinSvc.h 830 1 CvGameCoreDLL
Error 84 error C2501: 'EnumDependentServicesA' : missing storage-class or type specifiers C:\Dev\WindowsSDK\include\WinSvc.h 830 1 CvGameCoreDLL
Error 85 error C2059: syntax error : ')' C:\Dev\WindowsSDK\include\WinSvc.h 837 1 CvGameCoreDLL
Error 86 error C2144: syntax error : 'int' should be preceded by ';' C:\Dev\WindowsSDK\include\WinSvc.h 841 1 CvGameCoreDLL
Error 87 error C2501: '__inner_checkReturn' : missing storage-class or type specifiers C:\Dev\WindowsSDK\include\WinSvc.h 841 1 CvGameCoreDLL
Error 88 error C2086: 'int __inner_checkReturn' : redefinition C:\Dev\WindowsSDK\include\WinSvc.h 841 1 CvGameCoreDLL
Error 89 error C2143: syntax error : missing ';' before '__stdcall' C:\Dev\WindowsSDK\include\WinSvc.h 841 1 CvGameCoreDLL
Error 90 error C2501: 'BOOL' : missing storage-class or type specifiers C:\Dev\WindowsSDK\include\WinSvc.h 841 1 CvGameCoreDLL
Error 91 error C2146: syntax error : missing ')' before identifier 'hService' C:\Dev\WindowsSDK\include\WinSvc.h 843 1 CvGameCoreDLL
Error 93 error C2501: 'EnumDependentServicesW' : missing storage-class or type specifiers C:\Dev\WindowsSDK\include\WinSvc.h 843 1 CvGameCoreDLL
Error 94 error C2059: syntax error : ')' C:\Dev\WindowsSDK\include\WinSvc.h 850 1 CvGameCoreDLL
Error 95 error C2144: syntax error : 'int' should be preceded by ';' C:\Dev\WindowsSDK\include\WinSvc.h 860 1 CvGameCoreDLL
Error 96 error C2501: '__inner_checkReturn' : missing storage-class or type specifiers C:\Dev\WindowsSDK\include\WinSvc.h 860 1 CvGameCoreDLL
Error 97 error C2086: 'int __inner_checkReturn' : redefinition C:\Dev\WindowsSDK\include\WinSvc.h 860 1 CvGameCoreDLL
Error 98 error C2143: syntax error : missing ';' before '__stdcall' C:\Dev\WindowsSDK\include\WinSvc.h 860 1 CvGameCoreDLL
Error 99 error C2501: 'BOOL' : missing storage-class or type specifiers C:\Dev\WindowsSDK\include\WinSvc.h 860 1 CvGameCoreDLL
Error 100 error C2146: syntax error : missing ')' before identifier 'hSCManager' C:\Dev\WindowsSDK\include\WinSvc.h 862 1 CvGameCoreDLL
Error 102 error C2501: 'EnumServicesStatusA' : missing storage-class or type specifiers C:\Dev\WindowsSDK\include\WinSvc.h 862 1 CvGameCoreDLL
Error 103 error C2059: syntax error : ')' C:\Dev\WindowsSDK\include\WinSvc.h 871 1 CvGameCoreDLL
Error 104 error C2144: syntax error : 'int' should be preceded by ';' C:\Dev\WindowsSDK\include\WinSvc.h 875 1 CvGameCoreDLL
Error 105 error C2501: '__inner_checkReturn' : missing storage-class or type specifiers C:\Dev\WindowsSDK\include\WinSvc.h 875 1 CvGameCoreDLL
Error 106 error C2086: 'int __inner_checkReturn' : redefinition C:\Dev\WindowsSDK\include\WinSvc.h 875 1 CvGameCoreDLL
Error 107 error C2143: syntax error : missing ';' before '__stdcall' C:\Dev\WindowsSDK\include\WinSvc.h 875 1 CvGameCoreDLL
Error 108 error C2501: 'BOOL' : missing storage-class or type specifiers C:\Dev\WindowsSDK\include\WinSvc.h 875 1 CvGameCoreDLL
Error 109 error C2146: syntax error : missing ')' before identifier 'hSCManager' C:\Dev\WindowsSDK\include\WinSvc.h 877 1 CvGameCoreDLL
Error 110 error C1003: error count exceeds 100; stopping compilation C:\Dev\WindowsSDK\include\WinSvc.h 877 1 CvGameCoreDLL
Error 111 error U1077: '"C:\Program Files (x86)\Microsoft Visual C++ Toolkit 2003\bin\cl.exe"' : return code '0x2' F:\Games\Civ4 Colonization\Mods\MyMod\DLL_Sources\NMAKE CvGameCoreDLL
Error 112 error MSB3073: The command "set TARGET=Debug
nmake clean
nmake dll" exited with code 2. C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V110\Microsoft.MakeFile.Targets 43 5 CvGameCoreDLL


Project Build Settings
Spoiler :

I double checked if my makefile paths are correct for TOOLKIT, PSDK and CIV4_PATH.

Here are the steps I did:
  1. Installed Microsoft Visual C++ Toolkit 2003 and WindowsSDK
  2. Created a new mod folder named "MyMod".
  3. Copied the vanilla folder CvGameCoreDLL contents onto a folder "MyMod\DLL_Sources" (without boost, ipch, etc.)
  4. Merged the contents of the mod I wanted to modmod
  5. Used the configuration property from your attached VS project files.
  6. Modified makefile to point to correct paths
  7. Ensured my mod's bin file contains fastdep and jom while using jom
I'm getting the same error whether I use nmake with "debug" or with jom on "release".
Could it be because I'm not using Visual C++ instead of Visual Studio 2012? :confused:
 
Thanks for the replies. I take that it works in Civ4 or people would have complained :)

Seeing that people actually use this makefile made me finally take the time to upload the new version. I uploaded version 2.3 of the makefile. See changelog in first post.

Added a guide in first post on how to handle header includes. This will reduce a lot of unneeded recompilation when modifying headers.

@vetiarvind
It looks like "C:\Program Files (x86)" is read as "C:\Program". That is a common issue, but I'm clueless to why you have that problem. I added "" in the makefile to avoid it and if the cause was a faulty makefile I should experience the very same problem. Hopefully it works with the new version.
 
Thank you for the new version! :)

And yes, it works without issues in Civ4 BtS. The makefile.settings addition is quite helpful. I find it desirable to allow the inclusion of different compilation flags for each target. For example, I do want to include BBAI log generation only for the Assert and Debug targets. Since the Assert target does not have its own definition (it just uses Release_DEFINES) I cannot define custom flags only for it by just modifying the makefile.settings file.
 
The makefile.settings addition is quite helpful. I find it desirable to allow the inclusion of different compilation flags for each target. For example, I do want to include BBAI log generation only for the Assert and Debug targets. Since the Assert target does not have its own definition (it just uses Release_DEFINES) I cannot define custom flags only for it by just modifying the makefile.settings file.

Makefile.settings is actually read as a makefile itself meaning you can write code like (untested)
Code:
!IF "$(TARGET)" == "Assert"
CUSTOM_CFLAGS = -DASSERT_TARGET
!ENDIF

However there is a little difference between release and assert targets and you can identity the target in C++ like this:

|ASSERT_ENABLE|_DEBUG|C++ code
Assert|X||#if defined(ASSERT_ENABLE) && !defined(_DEBUG)
Debug|X|X|#if defined(_DEBUG)
Release|||#if !defined(ASSERT_ENABLE)
Profile|||#if !defined(ASSERT_ENABLE)

Checking for ASSERT_ENABLE alone will be true for both Assert and Debug.
There is no way to tell Profile from Release because I didn't consider that to be needed. You can add code in Makefile.settings to add a profile flag if you really want to, but I think it would be best to profile on a DLL as close to release as possible.
 
@vetiarvind It looks like "C:\Program Files (x86)" is read as "C:\Program". That is a common issue, but I'm clueless to why you have that problem. I added "" in the makefile to avoid it and if the cause was a faulty makefile I should experience the very same problem. Hopefully it works with the new version.
It seems like the mistake was because I was missing 3 lib files: msvcrt.lib, msvcrtd.lib, msvcprt.lib. Copying it into VC++ 2003 toolkit\lib fixed the linking issues. :)
 
I updated the Makefile to 2.5. It has a number of changes, but most noteworthy it can handle a relative path to the source dir, meaning project files and Makefile will no longer have to be mixed with the source files. Also the source dir can contain subdirectories with cpp files, which will also be compiled and linked. The idea with the latter is to make a directory with code, which can be applied to multiple mods, kind of like a library approach. Copying and updating is easier as it is a matter of copying everything in the directory and nothing else.

Setup has increased a bit in speed. Building the list of source files and running fastdep is now targets, meaning this will only be done once and not at all when cleaning.

I fixed a number of bugs in fastdep to allow this to work and call it version 0.17. Since other people have released version 0.17 as well, I decided to add (windows) to the version as I think I might have broken the code on non-windows computers. For those who are interested, the code is here: https://sourceforge.net/p/colonizationmodcollection/fastdep/ci/master/tree/

I had reached Makefile 2.4. However seeing that the date is 05/05/15, which happens to be the 125th day of the year, which is 53, then I figured version 2.5 would be better.
 
Picking up on our PM discussion:
Nightinggale said:
First of all, the thread would be a better place or I will have to answer the same questions over and over.

Leoreth said:
I decided to give your makefile a try (don't know why I haven't upgraded long ago).
A number of people stick with Makefile 1.0 because they have a working setup. That's a valid argument, but I think the ability to make assert DLLs alone make it worth it as it makes it easier to spot bugs. What most people don't get is that it actually asserts on "stupid" XML setup as well, which would render the game quite strange.

Leoreth said:
I'm running into a problem though: even though I have it blacklisted as is the standard setting for the makefile, the compiler still attempts to build CvTextScreens and fails at that point.
That's quite odd. I never had problems with that and nobody else ever reported such a problem. However a quick solution is to simply remove CvTextScreens.cpp as missing or ignored results in the same thing: not included when compiling.

I guess I better run some tests to see if I can get the error you came up with and then make a proper fix.

Leoreth said:
Do you have any ideas what might cause this? I've continued with an existing VS project, could that be the issue?
That shouldn't matter at all. The only link between the project and the makefile is the lines where you enter "TARGET=, NMAKE something" and that stuff. If you failed to do that correctly, it should display an error right away warning of invalid setup.
I've made some other modifications on the makefile, maybe some of that is responsible:
- I replaced $(PROGRAM_FILES) with the correct absolute paths, because the location of the toolkit was different and it was resolved incorrectly (this might be a problem with system language)
- I'm using psapi.lib, so I have added it to GLOBAL_LIBS

My version of the makefile and VS console output on clean build are attached, if required.
 

Attachments

  • Makefile.txt
    13.6 KB · Views: 303
  • Output.txt
    17.5 KB · Views: 185
- I replaced $(PROGRAM_FILES) with the correct absolute paths, because the location of the toolkit was different and it was resolved incorrectly (this might be a problem with system language)
- I'm using psapi.lib, so I have added it to GLOBAL_LIBS
The idea is that custom settings like that is placed in Makefile.settings. That way if there is an updated Makefile, you can overwrite your file without overwriting your custom settings. In your case the file should look like this:

Code:
TOOLKIT=C:\Dev\Microsoft Visual C++ Toolkit 2003
PSDK=C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A
CUSTOM_LDFLAGS = psapi.lib
Note that LDFLAGS are given to the linker just like LIBS are. This mean you can add psapi.lib to either of those, but only CUSTOM_LDFLAGS can be used in Makefile.settings.

As for the problem with CvTextScreens.cpp: just try to remove the file from the source code. Note that the makefile will not care if it is present in the project. It only cares if the file is present in explorer, which is why just removing it from the project is not enough.

The warnings in CvGameTextMgr.cpp is just that, warnings. Unless you added very un-vanilla code, it should work just fine. Still it would be good to fix those, because any new warning regarding a real bug could drown in the warning spam.
 
Weird, I thought I had tried to just remove the CvTextScreens without success (compiler complained that it was missing), and I was under the impression it was excluded from the project at that point. Apparently I wasn't thorough enough. I wanted to mention this in my previous post but I was distracted when writing it.

Thanks for the tip with the makefile.settings file, I moved my local stuff there. And yeah, I'm not really worried about the CvGameTextMgr stuff.

Anyway, even a clean build is now pleasantly fast, and I haven't even tried multiple cores. Thanks for your help and writing the makefile in the first place!
 
I also plan to switch to your new 2.5 makefile soon enough
Looks like it has some really nice improvements
 
What version of Windows Platform SDK do you guys use?
Still the version 6, the one Asaf's guide has?
http://forums.civfanatics.com/showthread.php?t=405444#Debugging
I'm using that one for the simple reason that at first I followed the guide on Modiki to compile the first time I managed to compile a DLL file. While I have been thinking about lots of issues and ways to improve a whole lot of stuff, I haven't even considered the SDK until you mentioned it.

Now start to wonder what would happen if I start messing with the SDK and if I could hope to gain anything, where I would enjoy some improvement :think:
 
Top Bottom