1. We have added a Gift Upgrades feature that allows you to gift an account upgrade to another member, just in time for the holiday season. You can see the gift option when going to the Account Upgrades screen, or on any user profile screen.
    Dismiss Notice

Improved CvGameCoreDLL Makefile! AND ~71% faster compilation

Discussion in 'Civ4 - SDK/Python' started by DannyDaemonic, Jul 5, 2010.

  1. DannyDaemonic

    DannyDaemonic Chieftain

    Joined:
    Jun 6, 2010
    Messages:
    25
    Current version: 1.0

    Today's my birthday, but I have a gift for you.

    The makefile everyone has been using is bad. It looks like it started as a codeblocks makefile, but it's not how makefile are suppose to be. You aren't suppose to have a separate target for each source file. I've written a better one. It uses rules instead of listing each cpp file. It also uses nmake preprocessing to build a list of cpp files so you don't have to be editing the makefile.

    I added a BLACKLIST macro so that it won't try to compile CvTextScreens. If you are compiling for older versions of civ4 you might want to comment the BLACKLIST line out.

    This makefile should work with ALL versions of Civ4 CvGameCoreDLL. It works with vanilla, warlords, and bts. It should also work with BULL, ACO, Better Bts AI, etc.

    CvGameCoreDll is set up to use precompiled headers, but the old makefile doesn't take advantage of this. This is a huge speed up, especially when STL is involved.

    Here are the two timings Visual Studio building stock Civ4 BTS CvGameCoreDLL:
    Time Elapsed 00:07:10.55
    Time Elapsed 00:02:04.90

    The old makefile takes 344.7% longer! On larger projects it will be more significant, even 400%+.

    The other improvement I've added is `fastdep` support. `fastdep` is only optional, and is not enabled unless you uncomment the line for it in the makefile (or use the makefile I've included with it). This allows us to mimic the built-in dependency support Visual Studio normally has. For those not in the know, this lets us recompile the source files affected when we make a change, especially a change to a header file. With the old makefile, if you change a header you have to clean and rebuild everything, otherwise things could crash unexpectedly, or simply not link. This saves you time with that step.

    Why use this makefile?
    • You don't have to change the makefile once it's setup. You don't have to edit it to add new files or remove those you no longer use.
    • Did I mention it's 300-400% faster? CvGameCoreDLL was designed to be used with PCH.
    • It handles dependencies automatically. `fastdep` is an amazingly fast dependency generator. Runs in under 0.1 seconds!
    • No knowledge is needed for a simple build user and only basic knowledge of make for a real developer.
    • Generates dependencies in a separate file. You can include this file for people who don't have or want to use fastdep.
    Update
    • Now uses /Zi instead of /ZI (faster/smaller debugging executable).
    • Statically linked fastdep.
    • Added another include path to support different versions of Windows Platform SDK.
    • No longer tries to build a resource file if CvGameCoreDLL.rc is missing.
    • Clearer error message when fastdep is missing but enabled.
    • Modified Makefile to be compatible with all versions of Windows.
    • Added zip archives.
    Previous (erased) total downloads: 67

    Attachments
    I've attached 3 7z and 3 zip archives. The first is just the makefile. The second is the makefile and fastdep. (Just put fastdep.exe in a bin directory in your project folder.) And the 3rd, as required by GPL, is the modified fastdep source code. (I had to fix a couple bugs for it to work with Civ4). I've added a 4th attachment of each type: Visual Studio 2010 project files.
     
  2. DannyDaemonic

    DannyDaemonic Chieftain

    Joined:
    Jun 6, 2010
    Messages:
    25
    I've updated the Makefile to work on all versions of Windows. Let me know if anyone has trouble with it.

    Also, I changed clean_Debug and clean_Release to Debug_clean and Release_clean, so you'll have to update your project to reflect that.
     
  3. deanej

    deanej Deity

    Joined:
    Apr 8, 2006
    Messages:
    4,859
    Location:
    New York State
    That doesn't sound like a good idea! I don't know anything about makefiles, but the ability to create a debug DLL is necessary!
     
  4. DannyDaemonic

    DannyDaemonic Chieftain

    Joined:
    Jun 6, 2010
    Messages:
    25
    I think you are confused, both Debug and Release builds are there. It's just instead of writing clean_Debug and clean_Release when setting up Visual Studio, you write Debug_clean and Release_clean.
     
  5. TC01

    TC01 Deity

    Joined:
    Jun 28, 2009
    Messages:
    2,216
    Location:
    Irregularly Online
    Do you know if this will work with Colonization? It sounds like it should, and the current Colonization makefile is just an adapted version of the standard one, but is there any reason why it wouldn't?

    It sounds like he only changed the name of the commands to clean (delete everything in the release or DLL folder). The command that's ran when you click "Clean Solution".

    This has nothing to do with actually building the debug DLL (or the release DLL for that matter).
     
  6. deanej

    deanej Deity

    Joined:
    Apr 8, 2006
    Messages:
    4,859
    Location:
    New York State
  7. DannyDaemonic

    DannyDaemonic Chieftain

    Joined:
    Jun 6, 2010
    Messages:
    25
    Yes it works fine with Civ 4 Colonization!

    But be warned. There seems to be a bug in the latest colonization SDK sources they provide.

    Line 6621 of CvGame.cpp:
    Code:
    	OutputDebugStr(CvString::format("[CvGame::updateOceanDistances] Plots: %i, Visits: %i\n", GC.getMapINLINE().numPlotsINLINE(), iVisits).GetCString());
    
    Should be
    Code:
    	OutputDebugString(CvString::format("[CvGame::updateOceanDistances] Plots: %i, Visits: %i\n", GC.getMapINLINE().numPlotsINLINE(), iVisits).GetCString());
    
    Also, earlier, I found a bug in my Makefile that could cause problems in some versions of Windows. Re-download the latest version of the Makefile just to make sure it works for you.
     
  8. Cybah

    Cybah Emperor

    Joined:
    Jun 22, 2007
    Messages:
    1,480
    Sounds too good to be true, have to try it out. :D
     
  9. Cybah

    Cybah Emperor

    Joined:
    Jun 22, 2007
    Messages:
    1,480
    Cannot unpack. 7zip says "compression not supported" or something. (Makefile+fastdep.7z)
     
  10. DannyDaemonic

    DannyDaemonic Chieftain

    Joined:
    Jun 6, 2010
    Messages:
    25
    Hmm, works for me. Oh it's using LZMA2, a new but better 7zip compression format. Are you using the latest stable 7Zip? I'll just put zips up there along with the 7z's.
     
  11. Admiral Armada

    Admiral Armada The Admiral of Armadas

    Joined:
    Sep 11, 2004
    Messages:
    462
    Location:
    Terra
    This does seem faster, but i get 2 'The system cannot find the path specified.' errors as soon as building starts. It then goes and runs through everything, so it looks like its building.
    Then at the end I get:
    Code:
    1>NMAKE : fatal error U1073: don't know how to make 'Release\CvGameCoreDLL.res'
    1>Stop.
    1>Project : error PRJ0019: A tool returned an error code from "Performing Makefile project actions"
    How can i fix this?
    edit: also, im confused as to where i put this fasdep program. Do i put it in the bin folder of the 2003 toolkit?
     
  12. DannyDaemonic

    DannyDaemonic Chieftain

    Joined:
    Jun 6, 2010
    Messages:
    25
    This means it can't find fastdep. Tomorrow I'll edit the makefile to check for fastdep's existence and put a better error message in there. By default it just checks your project's bin folder. The CvGameCoreDLL directory has a boost and python directory, just put the bin directory amongst those other directories in the CvGameCoreDLL directory.

    If you'd like to put it in your sdk bin folder, which is a good idea, then edit this line:
    Code:
    FD="$(MAKEDIR)/bin/fastdep.exe"
    To be:
    Code:
    FD="$(TOOLKIT)/bin/fastdep.exe"
    It sounds like CvGameCoreDLL.rc is missing. I forgot a lot of custom projects don't include this file anymore. Maybe no one figured out what it was for? It lets you put your Mod's info directly into the DLL. So when you right click on your CvGameCore.DLL file and put Properties, it can show your mod name, and a lot of other info. I assumed everyone would want this.

    Tomorrow I'll make an easy line you can comment to enable or disable this feature. In the mean time, you can copy the CvGameCoreDLL.rc file over from your stock CvGameCoreDLL directory included with Civilization 4.
     
  13. EmperorFool

    EmperorFool Deity

    Joined:
    Mar 2, 2007
    Messages:
    9,633
    Location:
    Mountain View, California
    I was told the VC++ 2003 Tookit (as opposed to the full version) would not handle the .rc file. I know C++, but I haven't coded in it for a long time so I just deleted it. I would definitely like to have it in the DLL, and I know others would too. The problem is we didn't have anyone who really knew their C++ and make like you to step up. Thanks! :goodjob:
     
  14. avain

    avain (key)

    Joined:
    Jul 29, 2006
    Messages:
    2,770
    Location:
    Budapest, EU
  15. Cybah

    Cybah Emperor

    Joined:
    Jun 22, 2007
    Messages:
    1,480
    ???

    "1>'Final_Release' is up-to-date"

    but there is no dll in my final release folder...

    edit: I have deleted the folder after cut and paste the dll to my mods folder. Can I get all files back without a change?

    Debug dll started fine, but then:

    PHP:
    1>CvGameCoreDLL.rc(10) : fatal error RC1015cannot open include file 'afxres.h'.
    1>NMAKE fatal error U1077'"C:\Programme\Microsoft Platform SDK/bin/rc.exe"' : return code '0x1'
    1>Stop.
    1>Project error PRJ0019A tool returned an error code from "Performing Makefile project actions"
    whats afxres.h?

    My search gave me this:

     
  16. DannyDaemonic

    DannyDaemonic Chieftain

    Joined:
    Jun 6, 2010
    Messages:
    25
    Ah, but that's because it's not part of the toolkit. Rather, it's a part of the Microsoft Platform SDK.

    Oh sorry, I also forgot to mention I that I also renamed the target from Final_Release to just Release. I did this to save a step in the directions on setting up the nmake project. Maybe I should make a how-to for this. Perhaps I can just zip up some completed projects.

    It's an MFC header that contains some DEFINEs.

    Could you try changing line number 53 from:
    Code:
    GLOBAL_INCS=/I"$(TOOLKIT)/include" /I"$(PSDK)/Include"
    To:
    Code:
    GLOBAL_INCS=/I"$(TOOLKIT)/include" /I"$(PSDK)/Include" /I"$(PSDK)/Include/mfc"
    And letting me know if this fixes the problem for you?

    This might be why the resource file has been avoided. The Platform SDKs versions vary quiet a bit more than I had expected.
     
  17. Cybah

    Cybah Emperor

    Joined:
    Jun 22, 2007
    Messages:
    1,480
    So... what to do? :)


    Another error:

    PHP:
    1"C:\Programme\Microsoft Platform SDK/bin/rc.exe" /FoDebug\CvGameCoreDLL.res /IBoost-1.32.0/include /IPython24/include /I"C:\Programme\Microsoft Visual C++ Toolkit 2003/include" /I"C:\Programme\Microsoft Platform SDK/Include /I"C:\Programme\Microsoft Platform SDK/Include/mfc" CvGameCoreDLL.rc
    1>fatal error RC1107: invalid usage; use RC /? for Help
    1>NMAKE : fatal error U1077: '"
    C:\Programme\Microsoft Platform SDK/bin/rc.exe"' : return code '0x1'
    1>Stop.
    1>Project : error PRJ0019: A tool returned an error code from "
    Performing Makefile project actions"
     
  18. DannyDaemonic

    DannyDaemonic Chieftain

    Joined:
    Jun 6, 2010
    Messages:
    25
    Sorry about that, there is a quote missing. You have this:
    Code:
    GLOBAL_INCS=/I"$(TOOLKIT)/include" /I"$(PSDK)/Include /I"$(PSDK)/Include/mfc"
    When you should have this:
    Code:
    GLOBAL_INCS=/I"$(TOOLKIT)/include" /I"$(PSDK)/Include" /I"$(PSDK)/Include/mfc"
     
  19. Cybah

    Cybah Emperor

    Joined:
    Jun 22, 2007
    Messages:
    1,480
    Works. :) Holy crap... this is sooooooooo fast. :D

    For final release, do I have to rename Final_Release into Release at all instances in my project config?
     
  20. DannyDaemonic

    DannyDaemonic Chieftain

    Joined:
    Jun 6, 2010
    Messages:
    25
    Glad to hear it! I'll add the extra include path to the Makefile next update, but I fear this path is going to be different for every version of the Platform SDK.

    Yeah, sorry. I just didn't see the point in calling it all Final_Release. Maybe I should make a compatibility Makefile for people who would like a drop in replacement. I'll have to think about it.
     

Share This Page