[DLL/C++] Adding new columns to the database

Discussion in 'Civ5 - Modding Tutorials & Reference' started by whoward69, Dec 24, 2013.

  1. whoward69

    whoward69 DLL Minion

    Joined:
    May 30, 2011
    Messages:
    8,581
    Location:
    Near Portsmouth, UK
    The aim of this tutorial is to show you how to extend the existing database tables by adding new columns and the required C++ code such that they actually do something useful.

    This tutorial assumes you are familiar with the techniques presented in the "Replacing hard-coded constants with database values" tutorial, specifically adding your own custom header file and the pre-processor techniques. If you haven't already done so, please read that tutorial first.

    This tutorial also assumes that you have
    1. Knowledge of the Civ 5 database tables and how to update them either via XML or SQL
    2. Basic knowledge of SQL - enough to figure out what an ALTER TABLE statement does.
    3. A basic knowledge of C programming and an understanding of object-orientated programming as implemented by C++ (if you don't know the significance of :: and the difference between -> and . then this tutorial is probably not for you)
    4. Successfully built, deployed and loaded the unaltered source as a custom DLL mod
    5. Created your own custom header file and added it to the CvGameCoreDLLPCH.h file

    Some of the most frequently asked questions in the forums are "Can I just add a new column A to table B ..." or "Can I use column X from table Y in table Z ..." "... and the game will know what to do with it". This tutorial explains that while adding columns is possible with simple SQL, without adding the required C++ code to the DLL these additional columns will do nothing.

    In the Improvements table there is a HillsMakesValid column but no MountainsMakesValid column, we will start by adding this.

    The SQL is simple

    Code:
    ALTER TABLE Improvements ADD MountainsMakesValid INTEGER DEFAULT 0;
    
    and we can then create an improvement for mountains

    Code:
    <Improvements>
        <Row>
            <Type>IMPROVEMENT_TUNNEL</Type>
            <Permanent>true</Permanent>
            <OutsideBorders>true</OutsideBorders>
            [COLOR="Red"]<MountainsMakesValid>true</MountainsMakesValid>[/COLOR]
            ...
        </Row>
    </Improvements>
    
    Job done!

    Ummmmm, not quite. Ignoring the not insignificant issues of getting a unit onto the mountain in the first place, keeping it alive while it digs the tunnel, and finally getting other units to use the tunnel, the game core has no idea what "MountainsMakesValid" is supposed to do. While it may be clear to us from its name, we could just as easily have called the column "MyFunkyStuff", in which case nothing (human or computer) would know what it is for.

    We need to add the logic (code) to the DLL such that the game "knows" what this column does.

    To us humans, it's fairly obvious that the column is a counterpart to HillsMakesValid, so we need to find all occurrences of the usage of that in the DLL code and add corresponding code for MountainsMakesValid.

    We need a good search tool to locate the code, use whatever you're comfortable with, personally I use AgentRansack. Searching the code base for "HilllsMakesValid" we find three files - CvImprovementClasses.h, CvImprovementClasses.cpp and CvPlot.cpp A quick scan of these three files shows us that the CvImprovementClasses.h/cpp files relate to retrieving the column's value from the database and making it available for other code to use, while CvPlot.cpp uses the value to make the decision if a specific improvement is valid for the plot. For CvImprovementClasses.h/cpp we just need to duplicate and edit the code from hills to mountains.

    (Note: The sample code comes from the changes in my DLL needed to fully implement tunnels as an improvement, if you're interested in seeing how to solve the issues above, search my source for "MOD_GLOBAL_ALPINE_PASSES".)

    In CvImprovementClasses.h
    Code:
        bool IsHillsMakesValid() const;
    #if defined(MOD_GLOBAL_ALPINE_PASSES)
        bool IsMountainsMakesValid() const;
    #endif
    
    and
    Code:
        bool m_bHillsMakesValid;
    #if defined(MOD_GLOBAL_ALPINE_PASSES)
        bool m_bMountainsMakesValid;
    #endif
    
    In CvImprovementClasses.cpp
    Code:
        m_bHillsMakesValid(false),
    #if defined(MOD_GLOBAL_ALPINE_PASSES)
        m_bMountainsMakesValid(false),
    #endif
    
    and
    Code:
        m_bHillsMakesValid = kResults.GetBool("HillsMakesValid");
    #if defined(MOD_GLOBAL_ALPINE_PASSES)
        m_bMountainsMakesValid = kResults.GetBool("MountainsMakesValid");
    #endif
    
    and
    Code:
    /// Requires hills to be constructed
    bool CvImprovementEntry::IsHillsMakesValid() const
    {
        return m_bHillsMakesValid;
    }
    
    #if defined(MOD_GLOBAL_ALPINE_PASSES)
    /// Requires mountains to be constructed
    bool CvImprovementEntry::IsMountainsMakesValid() const
    {
        return m_bMountainsMakesValid;
    }
    #endif
    
    This adds a new data member to the class, initialises it and defines an accessor for it. The only bit of code which is actually interesting is
    Code:
    m_bMountainsMakesValid = kResults.GetBool("MountainsMakesValid");
    
    which reads the column from the results of the database query (kResults) and copies it into memory. There are also GetText(), GetInt() and GetFloat() methods depending on what you've stored in the column.

    The only use of the IsHillsMakesValid() accessor is in one place in CvPlot.cpp, the CvPlot::canHaveImprovement() method. The hint is in the name, it's this method that "decides" if a plot can have a specific improvement.

    If we were to blindly duplicate the usage of the IsHillsMakesValid() accessor we would run into a problem.

    Code:
    // THIS IS WRONG, DO NOT COPY!!!
        if(pkImprovementInfo->IsHillsMakesValid() && isHills())
        {
            bValid = true;
        }
    
    #if defined(MOD_GLOBAL_ALPINE_PASSES)
        if (pkImprovementInfo->IsMountainsMakesValid() && isMountain()) {
        {
            bValid = true;
        }
    #endif
    
    Mountain plots do NOT have a terrain type, and right up at the start of the CvPlot::canHaveImprovement() method we have
    Code:
        CvAssertMsg(getTerrainType() != NO_TERRAIN, "TerrainType is not assigned a valid value");
    
    and also some of the methods that are called before getting to our new IsMountainsMakesValid() method assume that the plot has a valid terrain - so our final mod would cause a CTD when we move a unit into the mountains to start building a tunnel.

    We need to disable that assert and also do our test for mountains before all other tests that assume the plot has a valid terrain type.

    Code:
        CvAssertMsg(eImprovement != NO_IMPROVEMENT, "Improvement is not assigned a valid value");
    #if !defined(MOD_GLOBAL_ALPINE_PASSES)
        CvAssertMsg(getTerrainType() != NO_TERRAIN, "TerrainType is not assigned a valid value");
    #endif
    
        CvImprovementEntry* pkImprovementInfo = GC.getImprovementInfo(eImprovement);
        if(pkImprovementInfo == NULL)
        {
            return false;
        }
    
    #if defined(MOD_GLOBAL_ALPINE_PASSES)
        if (pkImprovementInfo->IsMountainsMakesValid() && isMountain()) {
            return true;
        }
    
        CvAssertMsg(getTerrainType() != NO_TERRAIN, "TerrainType is not assigned a valid value");
    #endif
    
    Instead of disabling the assert, I've moved it after the test for mountains, and the test for mountains is the very first thing we do (once we know pkImprovementInfo is valid).

    Now our job is done. Just don't forget that in addition to the modded DLL our new improvement mod must also contain the SQL to add the MountainsMakesValid column to the Improvements table, otherwise we will CTD as kResults.GetBool("MountainsMakesValid") executes.

    In summary, adding a new column to an existing database table with functionality similar to an existing column is pretty much a copy-paste-edit exercise for the basics of getting the value out of the database and into the C++ code, and then making sure that you actually understand how the code that uses the new column behaves.

    So how about a column from another table? Slightly harder, but not much.

    Features can "add fresh water", eg the oasis. So what if we want a building (eg a cistern) to do the same? We need to duplicate the AddsFreshWater column from the Features table to the Buildings table, once again the SQL is trivial

    Code:
    ALTER TABLE Buildings ADD AddsFreshWater INTEGER DEFAULT 0;
    
    The Buildings database table already has a FreshWater column, so we'll use that to track down where we need to add the new code to retrieve the value from the database and load it into the C++ code. Searching for "FreshWater" results in a large number of files, but given that the HillsMakesValid changes were restricted to the CvImprovementClasses.h/cpp files, it's a pretty safe bet that the files we want are CvBuildingClasses.h/cpp (but you should check all the files and convince yourself that they don't need editing).

    In CvBuildingClasses.h
    Code:
        bool IsFreshWater() const;
    #if defined(MOD_API_EXTENSIONS)
        bool IsAddsFreshWater() const;
    #endif
    
    and
    Code:
        bool m_bFreshWater;
    #if defined(MOD_API_EXTENSIONS)
        bool m_bAddsFreshWater;
    #endif
    
    In CvBuildingClasses.cpp
    Code:
        m_bFreshWater(false),
    #if defined(MOD_API_EXTENSIONS)
        m_bAddsFreshWater(false),
    #endif
    
    and
    Code:
        m_bFreshWater = kResults.GetBool("FreshWater");
    #if defined(MOD_API_EXTENSIONS)
        m_bAddsFreshWater = kResults.GetBool("AddsFreshWater");
    #endif
    
    and
    Code:
    /// Must this be built in a city next to FreshWater?
    bool CvBuildingEntry::IsFreshWater() const
    {
        return m_bFreshWater;
    }
    
    #if defined(MOD_API_EXTENSIONS)
    /// Does this building add FreshWater?
    bool CvBuildingEntry::IsAddsFreshWater() const
    {
        return m_bAddsFreshWater;
    }
    #endif
    
    None of which should come as a surprise.

    Now we need to track down where to use CvBuildingEntry::IsAddsFreshWater(). As we are replicating an aspect of a feature, we just need to find where the feature code is used. Searching for usages of "IsAddsFreshWater" we find it in only the CvPlot.cpp file, specifically the CvPlot::isFreshWater() method.

    This method, after doing some quick, basic checks, iterates all adjacent plots and checks them for a feature that adds fresh water. We need to extend this check to include cities that contain buildings that add fresh water. Now we could check for a city, and if there is one, iterate all its buildings searching for one that adds fresh water. But that's bad design as it's adding city specific code outside of the CvCity object (and not to mention that it's not reusable). What we should (and will) do is add a method to the CvCity object to ascertain if a city provides fresh water to adjacent plots.

    CvCity has an isCoastal() method, so we'll add our new method after it.

    In CvCity.h
    Code:
        bool isCoastal(int iMinWaterSize = -1) const;
    #if defined(MOD_API_EXTENSIONS)
        bool isAddsFreshWater() const;
    #endif
    
    In CvCity.cpp
    Code:
    //    --------------------------------------------------------------------------------
    bool CvCity::isCoastal(int iMinWaterSize) const
    {
        VALIDATE_OBJECT
        return plot()->isCoastalLand(iMinWaterSize);
    }
    
    #if defined(MOD_API_EXTENSIONS)
    //    --------------------------------------------------------------------------------
    bool CvCity::isAddsFreshWater() const {
        VALIDATE_OBJECT
    
        // Determine if the city has a fresh water building
        
        return false;
    }
    #endif
    
    There are two ways we could code the check for a fresh water building. Firaxis tend to cache a count of the number of buildings in the city that add a feature, and then as buildings are constructed/bought and sold increment/decrement the count. The downside to this is that the count must be persisted into the saved game data, so will break previous saves, the upside is that it is nanoseconds faster. The alternative method is just to iterate the buildings in the city and look for one that adds fresh water. As we only need the first, on average we will check half the buildings in a city, hardly a huge overhead, so we'll adopt that approach.

    Code:
    bool CvCity::isAddsFreshWater() const {
        VALIDATE_OBJECT
    
        for (int iI = 0; iI < GC.getNumBuildingInfos(); iI++) {
            if (m_pCityBuildings->GetNumBuilding((BuildingTypes)iI) > 0) {
                if (GC.getBuildingInfo((BuildingTypes)iI)->IsAddsFreshWater()) {
                    return true;
                }
            }
        }
        
        return false;
    }
    
    Basically for each possible building, do we have that building in the city, and if so, does it add fresh water?

    With out new CvCity::isAddsFreshWater() method, we can now update the CvPlot::isFreshWater() method

    Code:
        if(feature_type != NO_FEATURE)
        {
            if(GC.getFeatureInfo(feature_type)->isAddsFreshWater())
            {
                return true;
            }
        }
    
    #if defined(MOD_API_EXTENSIONS)
        if (pLoopPlot->isCity() && pLoopPlot->getPlotCity()->isAddsFreshWater())
        {
            return true;
        }
    #endif
    
    Nicely OO and nicely reusable. Speaking of reusability, the CvCity::isAddsFreshWater() method should be exposed to Lua - see "Extending the Lua API" for details

    We can now create our Cistern building
    Code:
        <Buildings>
            <Row>
                <Type>BUILDING_CISTERN</Type>
                <BuildingClass>BUILDINGCLASS_AQUEDUCT</BuildingClass>
                <Cost>100</Cost>
                <GoldMaintenance>1</GoldMaintenance>
                <PrereqTech>TECH_ENGINEERING</PrereqTech>
                <FoodKept>40</FoodKept>
                [COLOR="red"]<AddsFreshWater>true</AddsFreshWater>[/COLOR]
                ...
            </Row>
        </Buildings>
    
    don't forget to include the SQL to alter the Buildings table to add the new column!

    In summary, adding a new column to an existing database table with functionality similar to an existing column in another table is pretty much a copy-paste-edit exercise for the basics of getting the value out of the database and into the C++ code, and then replicating the code for the existing column for the new column.

    [END]
     
  2. whoward69

    whoward69 DLL Minion

    Joined:
    May 30, 2011
    Messages:
    8,581
    Location:
    Near Portsmouth, UK
  3. whoward69

    whoward69 DLL Minion

    Joined:
    May 30, 2011
    Messages:
    8,581
    Location:
    Near Portsmouth, UK
    Please use this thread in the SDK / Lua sub-forum for asking questions about this tutorial, I will then update/expand this thread as necessary.

    This is NOT a tutorial on basic C/C++ programming, so please do not ask questions about the syntax, semantics or usage of the C++ language - there are many excellent resources for learning C++ available (most of which can be found in your local bookshop)

    Also, please do NOT ask questions about specific issues for your own mods and/or ask for someone else to write your code for you on these threads - start a new thread either in the main C&C forum or the SDK / Lua sub-forum to ask your specific question/request, as that way you'll get specific answers ;) And finally, please note that I do NOT take requests for writing code for others' mods!
     

Share This Page