New makefile (faster compilation, profiling and more)

[AbsintheRed]

Just noticed again the issue with the CvTextScreens.cpp file, when setting up your new makefile for a different compiling project.
Maybe it won't be bad to add info about it's removal to the opening post.

 

[AbsintheRed]

I have some actual questions as well:
Ran into a compilation error with the new source files I try to test.
It's an error LNK2019: unresolved external symbol
Appears at the very end of the compilation process, in CvPlot.obj

I'm not really sure if it's a mistake in the code itself, or a mistake in the setup I made for it.
Sourcefiles and my makefile settings and projekt files are attached, if you have some time to check it out.

EDIT: Ohh nevermind this post
I forgot to include the .rc file in the source
Also the uploaded CvGameCoreDLL.vcxproj is not your original one which I initially used, it was edited during the tests (probably unnecessarily)
Will try to compile with the .rc added, will post if I still have some issues with it.

EDIT2:
I guess it's very easy to get comfortable with a working setup for compilation and forget everything connected to it...
Thus making silly mistakes when trying to set up a new one
Anyway, now I think all the files are ok, but I probably made a mistake with destinations? or maybe nmake specifications?

Error 940 error MSB3073: The command "set TARGET=Release
nmake clean /NOLOGO
nmake source_list /NOLOGO
nmake fastdep /NOLOGO
nmake dll /NOLOGO" exited with code 2.
C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V110\Microsoft.MakeFile.Targets 43 5 Civ4DLL

This was a rebuild, so nmake clean should be there
All the command lines for release:

Spoiler :

build command line:
set TARGET=Release
nmake source_list /NOLOGO
nmake fastdep /NOLOGO
nmake dll /NOLOGO

rebuild command line:
set TARGET=Release
nmake clean /NOLOGO
nmake source_list /NOLOGO
nmake fastdep /NOLOGO
nmake dll /NOLOGO

clean command line:
set TARGET=Release
nmake clean /NOLOGO

 

[AbsintheRed]

Sry for the 3rd post in a row, didn't want to edit my last post any further.
Turns out the .rc file isn't needed with your new makefile after all.
I have a working compiling environment, at least it works with different source codes. Tested with other mods, and with vanilla BtS source files as well.
Still can't get it to work with the source files I attached above.
(Never mind the other files in the zip, I used a different sln and project file in the end, with your original 2.5 makefile edited with the correct links. I'm also aware that there is no need for the Boost and Python24 libraries there with your new makefile, and used the fastdep from the opening post here)

Same error as I originally reported:
Error 939 error LNK2019: unresolved external symbol _K32GetProcessMemoryInfo@12 referenced in function "bool __cdecl NeedToFreeMemory(void)" (?NeedToFreeMemory@@YA_NXZ)
C:\Dev\CvGameCoreDLL_RFCE\CvPlot.obj CvGameCoreDLL

This is the full line of errors:

Spoiler :

Error 940 error LNK1120: 1 unresolved externals C:\Dev\CvGameCoreDLL_RFCE\temp_files\Release\CvGameCoreDLL.dll CvGameCoreDLL

Error 939 error LNK2019: unresolved external symbol _K32GetProcessMemoryInfo@12 referenced in function "bool __cdecl NeedToFreeMemory(void)" (?NeedToFreeMemory@@YA_NXZ)
C:\Dev\CvGameCoreDLL_RFCE\CvPlot.obj CvGameCoreDLL

Error 942 error MSB3073: The command "set TARGET=Release
nmake clean /NOLOGO
nmake source_list /NOLOGO
nmake fastdep /NOLOGO
nmake dll /NOLOGO" exited with code 2.
C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V110\Microsoft.MakeFile.Targets 43 5 CvGameCoreDLL

Error 941 error U1077: '"C:\Dev\Microsoft Visual C++ Toolkit 2003\bin\link.exe"' : return code '0x460' C:\Dev\CvGameCoreDLL_RFCE\NMAKE CvGameCoreDLL

Do you have any ideas what might cause this?

 

[Leoreth]

What is the NeedToFreeMemory method referenced in the error message? Is RFCE implementing the plot graphics paging from C2C by any chance? It has psapi.h as a dependency which might be missing or not referenced in your settings file. Don't have my sources on hand right now but can look it up later if my assumption is correct.

 

[AbsintheRed]

What is the NeedToFreeMemory method referenced in the error message? Is RFCE implementing the plot graphics paging from C2C by any chance? It has psapi.h as a dependency which might be missing or not referenced in your settings file. Don't have my sources on hand right now but can look it up later if my assumption is correct.

Ohh, thanks for the clue!
Actually this is not for RFCE, I'm experimenting with the Realism Invictus dll, and it does use graphical paging.
Will try to look into it tonight.

EDIT: The DoC source code do result in the very same error with my current compiling setup.
You also use paging, right? So it very well be connected to that.

 

[Nightinggale]

Error 940 error LNK1120: 1 unresolved externals C:\Dev\CvGameCoreDLL_RFCE\temp_files\Release\CvGameCoreDLL.dll CvGameCoreDLL

Error 939 error LNK2019: unresolved external symbol _K32GetProcessMemoryInfo@12 referenced in function "bool __cdecl NeedToFreeMemory(void)" (?NeedToFreeMemory@@YA_NXZ)
C:\Dev\CvGameCoreDLL_RFCE\CvPlot.obj CvGameCoreDLL

"unresolved external symbol" means there is a function call to a non-existing function. In more plain English it means you called NeedToFreeMemory. The compiler accepted it because it was in an included header file. Once reaching the linker, it has to be put together with the code from the cpp file where the function is written, but none of the cpp files contains the function in question. It also tries to read functions from included libraries, but this also failed to provide any useful functions.

The question is what NeedToFreeMemory is. Most likely a requirement for a library, which should to be added to the ld flags in Makefile.project.

 

[AbsintheRed]

@ Leoreth:
Psapi.h is referenced in the cvplot.cpp file, so it does have a dependency
I have added psapi.lib to the makefile:
GLOBAL_LIBS=/LIBPATH:"$(TOOLKIT)/lib" /LIBPATH:"$(PSDK)/Lib" winmm.lib user32.lib psapi.lib

Still the same error.
Where else should I reference it in the settings files?

 

[AbsintheRed]

"unresolved external symbol" means there is a function call to a non-existing function. In more plain English it means you called NeedToFreeMemory. The compiler accepted it because it was in an included header file. Once reaching the linker, it has to be put together with the code from the cpp file where the function is written, but none of the cpp files contains the function in question. It also tries to read functions from included libraries, but this also failed to provide any useful functions.

The question is what NeedToFreeMemory is. Most likely a requirement for a library, which should to be added to the ld flags in Makefile.project.

NeedToFreeMemory is in cvplot.cpp, and has some references as well.
This is the function:

Spoiler :

bool NeedToFreeMemory()
{
	PROCESS_MEMORY_COUNTERS pmc;
	static unsigned int uiMaxMem = 0xFFFFFFFF;

	if ( uiMaxMem == 0xFFFFFFFF )
	{
		MEMORYSTATUSEX memoryStatus;

		memoryStatus.dwLength = sizeof(memoryStatus);
		GlobalMemoryStatusEx(&memoryStatus);

		uiMaxMem = 1024*GC.getDefineINT("MAX_DESIRED_MEMORY_USED", DEFAULT_MAX_WORKING_SET_THRESHOLD_BEFORE_EVICTION/1024);

		DWORDLONG usableMemory = memoryStatus.ullTotalPhys - 1024*(DWORDLONG)GC.getDefineINT("OS_MEMORY_ALLOWANCE", DEFAULT_OS_MEMORY_ALLOWANCE/1024);
		if ( usableMemory < uiMaxMem )
		{
			uiMaxMem = (unsigned int)usableMemory;
		}
	}

	GetProcessMemoryInfo( GetCurrentProcess(), &pmc, sizeof(pmc));
	
	if ( pmc.WorkingSetSize > uiMaxMem )
	{
		OutputDebugString(CvString::format("Found need to free memory: %d used vs %d target\n", pmc.WorkingSetSize, uiMaxMem).c_str());
		return true;
	}
	else
	{
		return false;
	}
}

But yeah, I already added psapi.lib after Leoreth's clue.
Somethings is still missing though.

 

[AbsintheRed]

Turns out the "something still missing" was a silly mistake in my PSDK links, now it compiles perfectly.
Thanks for the help guys!

So if anyone bumps into the same error when trying to compile a source with added graphical paging:
It looks like the graphical paging system calls the psapi function GetProcessMemoryInfo to monitor how much memory the Civ4 process is using
So the only extra change you have to do is to add psapi.lib into the GLOBAL_LIBS of the makefile!

 

[Leoreth]

Good to know that it helped. And I can really sympathise with the situation, had to set up my SDK again a couple of times as well and I always forgot how to do one thing or another.

 

[keldath]

heya,

got an erro upon compile:

>NMAKE : fatal error U1073: don't know how to make 'temp_files\Release\cvcity

any ideas?

 

[Leoreth]

Is that the pasted error message? Does a file with the name cvcity exist?

 

[Nightinggale]

Updated the first post. Looked like something went wrong with the forum migration and most of the post was crossed out. Now it's back to how it was except for the table at the very end of the post. I will fix that one later.

>NMAKE : fatal error U1073: don't know how to make 'temp_files\Release\cvcity

Sorry I missed this post. Is it still a problem? Also it looks like something went wrong with the filename. I would have expected it to say CvCity.o.

 

[keldath]

ty ng.

if i remember right - i just had a file called cvcity.cpp2 or something like that (backups).

 

[Leoreth]

Do you have any idea if it's possible to get CMake to use this Makefile? I'm currently considering to buy CLion because I'm already using IntelliJ IDEs for everything else and I'm getting tired of the relative clunkiness of Visual Studio, but afaict it currently only supports CMake and no automatic integration with custom Makefiles. But I don't really understand how CMake works so it's hard to tell if I'm wasting money on something that won't work for Civ4.

(Apologies if I asked this already, because it kind of feels like I have, even though I can't find CMake when searching these forums.)

 

[Nightinggale]

The easiest way to get a better IDE is to upgrade to Visual Studio 2017. It can read the 2010 project files meaning it just works out of the box. No configuration needed when upgrading.

I don't know CLion, but stack overflow appears to have the question answered. It looks like it should be possible. However you should be aware that due to limitations of NMake the makefile isn't able to finish the job in one go and the project file is actually calling the makefile multiple times. Building looks like this:

set TARGET=Release
nmake source_list /NOLOGO
nmake fastdep /NOLOGO
nmake precompile  /NOLOGO
bin\jom build

This means while it might still be possible to get CLion to work with the makefile it won't be as simple as the examples in the replies.

 

[Leoreth]

I tried VS2017 and it worked, but I still don't really like how it looks and feels, which may be partly because I'm otherwise used to the IntelliJ ecosystem.

Thanks for digging up the Stack Overflow answer, I guess I can at least attempt to get it running with the trial version.

 

[billw2015]

You can get parallel build to work quite easily for all the configurations. Change your Zi to Z7, and it will embed the pdb info direct into the obj files (it calls it "old" format, but it works fine). PCH works fine still with this.

 

[keldath]

hi all,

i have a compile question.
im using make 2/5 + advanced civ and f1rpo alteration.

what does this file mean?
PROJECT_LDFLAGS = /DEF:CvGameCoreDLL.def

also,
i added to this file (x4)
<AdditionalOptions>/DWIN32 /D_WINDOWS /D_USRDLL /D_DEBUG /DUSE_DATA_DIR /D_CODE_EDITOR /KMOD_PATH_FINDER</AdditionalOptions>

this part - /KMOD_PATH_FINDER
in another make file from kek mod i saw ";KMOD_PATH_FINDER" that this existed.

i dunno what that means, but my kek mod dll , compiled well....

just toying with some compiling stuff for learning purposes.

thanks in advance.

 

[Nightinggale]

im using make 2/5 + advanced civ and f1rpo alteration.

The obvious approach would be to ask the person who added the changes

PROJECT_LDFLAGS = /DEF:CvGameCoreDLL.def

Page 728 in the manual. It adds a module definition to the linking. Module definitions are explained on page 783. Looks like it's a way to avoid adding DllExport in the source code.

this part - /KMOD_PATH_FINDER
in another make file from kek mod i saw ";KMOD_PATH_FINDER" that this existed.

All the compiler options are listed on page 568 and none of those are listed. I suspect it was suppose to start with /D as in defining precompiler flags, meaning it should say /DKMOD_PATH_FINDER. Because it's a makefile compilation, <AdditionalOptions> are for the display only meaning it can contain invalid flags without affecting the compiler. What matters are the flags set in the makefile.

You can download the manual here (all 27k pages of it).