Adding an era using relative techniques

whoward69 · Nov 25, 2015

The common way to add a new era is to place both it and its associated technologies directly into the database at absolute positions, eg era 5 and techs at grid X positions 9 and 10.
While this is simple and works when playing with a single era adding mod, it breaks completely if a player attempts to play with two mods adding new eras. The first mod to load will be working with the unmodded base eras
and technologies but the second mod to load is now making false assumptions about what is already in the Eras and Technologies tables and will almost ceratinly break. Worse still, the second mod will probably delete the
entire contents of the Eras table and re-add the base eras (in addition to its new era), without any regard for technologies added by the first mod that now refer to a deleted era, which will almost certainly cause the
game to crash.

There is a way to add both the new era and its associated technologies in a relative manner, but it involves not insignificant amounts of SQL.

First we need to fix the hard-coded era splash screens in NewEraPopup.lua. We'll do this by adding a column to the <Eras> table and rewriting the OnPopup function to read from this new column

The SQL to add the new column needs to be in a file on its own, such that if another mod has already made the change, this file can just fail and everything will still work

Code:

-- The ALTER and UPDATE statements MUST be a file on their own
-- Add the new column
ALTER TABLE Eras ADD SplashScreen TEXT DEFAULT 'ERA_Medievel.dds';

-- And update the base eras with the correct values
UPDATE Eras SET SplashScreen='ERA_Classical.dds'  WHERE Type='ERA_CLASSICAL';
UPDATE Eras SET SplashScreen='ERA_Medievel.dds'   WHERE Type='ERA_MEDIEVAL';
UPDATE Eras SET SplashScreen='ERA_Renissance.dds' WHERE Type='ERA_RENAISSANCE';
UPDATE Eras SET SplashScreen='ERA_Industrial.dds' WHERE Type='ERA_INDUSTRIAL';
UPDATE Eras SET SplashScreen='ERA_Modern.dds'     WHERE Type='ERA_MODERN';
UPDATE Eras SET SplashScreen='ERA_Atomic.dds'     WHERE Type='ERA_POSTMODERN';
UPDATE Eras SET SplashScreen='ERA_Future.dds'     WHERE Type='ERA_FUTURE';

And the new OnPopup function, that'll replace the standard one (don't forget to set NewEraPopup.lua as VFS=true only)

Code:

function OnPopup( popupInfo )
    if( popupInfo.Type ~= ButtonPopupTypes.BUTTONPOPUP_NEW_ERA ) then
        return;
    end

    m_PopupInfo = popupInfo;

    local iEra = popupInfo.Data1;
    Controls.DescriptionLabel:LocalizeAndSetText("TXT_KEY_POP_NEW_ERA_DESCRIPTION", GameInfo.Eras[iEra].Description);
	
    lastBackgroundImage = GameInfo.Eras[iEra].SplashScreen;
    Controls.EraImage:SetTexture(lastBackgroundImage);
	
    UIManager:QueuePopup( ContextPtr, PopupPriority.NewEraPopup );
end
Events.SerialEventGameMessagePopup.Add( OnPopup );

Now, using XML or SQL, create your new era.
* Do use unique TXT_KEY_s for the Description and ShortDescription values
* Don't worry about the Abbreviation value
* Don't forget the new SplashScreen column

For example

Code:

<Eras>
    <Row>
        <Type>ERA_XYZ</Type>
        <Description>TXT_KEY_ERA_XYZ</Description>
        <ShortDescription>TXT_KEY_ERA_XYZ_SHORT</ShortDescription>
        <SplashScreen>ERA_Xyz.dds</SplashScreen>
        <!-- other era specific values in here -->
    </Row>
</Eras>
 
<Language_en_US>
    <Row Tag="TXT_KEY_ERA_XYZ">
        <Text>Xyz Era</Text>
    </Row>
    <Row Tag="TXT_KEY_ERA_XYZ_SHORT">
        <Text>Xyz</Text>
    </Row>
</Language_en_US>

Using XML or SQL make any updates to other era's values to "balance" them, but remember there may be other eras in use other than the base ones!

Now we have to solve two problems for the Eras table

Firstly, the game expects eras to be in ascending order by ID, but we just added our new era to the end of the list. We could use SQL to "make a hole in the IDs" and insert the new era with the correct ID, but then the
natural sort order of the Eras table would still be wrong and there may be core code and/or mods that assume "SELECT * FROM Eras;" returns the eras in the correct sequence. The following SQL fixes the Eras table to allow
for both of these requirements

In this example, we will be inserting our new era after the Renaissance, but we could just as easily pick any other era or even rewrite the SQL to make it insert before the Industrial

Code:

-- Create a temp table holding all the eras before our new era
CREATE TABLE Eras_Temp AS SELECT * FROM Eras WHERE ID <= (SELECT ID FROM Eras WHERE Type='ERA_RENAISSANCE') ORDER BY ID ASC;
 
-- Now add our era into the temp table
INSERT INTO Eras_Temp SELECT * FROM Eras WHERE Type='ERA_XYZ';
 
-- Add all the eras after our new era into the temp table
INSERT INTO Eras_Temp SELECT * FROM Eras WHERE ID > (SELECT ID FROM Eras WHERE Type='ERA_RENAISSANCE') AND Type!='ERA_XYZ' ORDER BY ID ASC;
 
-- Renumber the eras based on their (correct) order in the temp table
UPDATE Eras_Temp SET ID=rowid-1;
 
-- Empty the Eras table
DELETE FROM Eras;
 
-- Copy everything back from the temp table into the Eras table in the correct order
INSERT INTO Eras SELECT * FROM Eras_Temp ORDER BY rowid ASC;
 
-- Finally dispose of the temp table
DROP TABLE Eras_Temp;

Secondly we need to fix the Description, ShortDescription and Abbreviation values to be of the form TXT_KEY_ERA_{ID}...

But first we need to give ourselves some more standard era abbreviations

Code:

-- Give ourselves some more abbrevations, a total of 15 eras should be more than enough!
INSERT OR REPLACE INTO Language_EN_US(Tag, Text) VALUES('TXT_KEY_ERA_7_ABBREV', 'VIII');
INSERT OR REPLACE INTO Language_EN_US(Tag, Text) VALUES('TXT_KEY_ERA_8_ABBREV', 'IX');
INSERT OR REPLACE INTO Language_EN_US(Tag, Text) VALUES('TXT_KEY_ERA_9_ABBREV', 'X');
INSERT OR REPLACE INTO Language_EN_US(Tag, Text) VALUES('TXT_KEY_ERA_10_ABBREV', 'XI');
INSERT OR REPLACE INTO Language_EN_US(Tag, Text) VALUES('TXT_KEY_ERA_11_ABBREV', 'XII');
INSERT OR REPLACE INTO Language_EN_US(Tag, Text) VALUES('TXT_KEY_ERA_12_ABBREV', 'XIII');
INSERT OR REPLACE INTO Language_EN_US(Tag, Text) VALUES('TXT_KEY_ERA_13_ABBREV', 'XIV');
INSERT OR REPLACE INTO Language_EN_US(Tag, Text) VALUES('TXT_KEY_ERA_14_ABBREV', 'XV');

Now we can fix the TXT_KEY_s

Code:

-- Create a temp table to hold the names of the eras  
CREATE TABLE IF NOT EXISTS Eras_Text(
  ID INTEGER NOT NULL,
  Type TEXT NOT NULL,
  Lang TEXT NOT NULL,
  Desc TEXT DEFAULT NULL,
  Short TEXT DEFAULT NULL
);
DELETE FROM Eras_Text;
 
-- Grab all the names of the eras for the EN_US language, repeat this statement for any/all other languages you may care about
INSERT INTO Eras_Text(ID, Type, Lang, Desc, Short)
  SELECT e.ID, e.Type, 'EN_US', t1.Text, t2.Text
  FROM Eras e, Language_EN_US t1, Language_EN_US t2
  WHERE e.Description=t1.Tag AND e.ShortDescription=t2.Tag;
 
-- Update the era names and abbreviations to the required format (as required by the tech tree and 'pedia)
UPDATE Eras SET
  Description='TXT_KEY_ERA_'||ID,
  ShortDescription='TXT_KEY_ERA_'||ID||'_SHORT',
  Abbreviation='TXT_KEY_ERA_'||ID||'_ABBREV';
 
-- Update the text entries corresponding to the new TXT_KEY_s, repeat these statements for any/all other languages you may care about
INSERT OR REPLACE INTO Language_EN_US(Tag, Text) SELECT 'TXT_KEY_ERA_'||e.ID, et.Desc FROM Eras e, Eras_Text et WHERE e.Type = et.Type AND et.Lang='EN_US';
INSERT OR REPLACE INTO Language_EN_US(Tag, Text) SELECT 'TXT_KEY_ERA_'||e.ID||'_SHORT', et.Short FROM Eras e, Eras_Text et WHERE e.Type = et.Type AND et.Lang='EN_US';
 
-- Finally dispose of the temp table
DROP TABLE Eras_Text;

Now we have to solve the issue of the placement of technologies in the tech tree.

Basically, as we can no longer make any assumptions about how many eras there are, we cannot use absolute values for GridX. This means we'll have to add new techs for the era either in the 1st column (GridX=1) or
the 2nd column (GridX=2) and then use SQL to adjust them based on the position of techs in the following era. But first we need to make a hole in the existing tech tree for our new era techs.

Code:

-- Shift all techs in eras after our new era two columns right
UPDATE Technologies SET GridX=GridX+2
  WHERE GridX >= (SELECT GridX FROM Technologies
                    WHERE Era IN (SELECT Type FROM Eras
                                    WHERE ID > (SELECT ID FROM Eras 
                                                  WHERE Type='ERA_RENAISSANCE'))
                    ORDER BY GridX LIMIT 1);

Now we can use XML or SQL to add our new technologies, for example

Code:

<Technologies>
    <!-- New techs for ERA_XYZ -->
    <!-- Note that GridX is used to indicate the 1st or 2nd column in the era, not an absolute position in the tech tree -->
    <Row>
        <Type>TECH_ABC</Type>
        ...
        <Era>ERA_XYZ</Era>
        <GridX>1</GridX>
        ...
    </Row>
    <Row>
        <Type>TECH_DEF</Type>
        ...
        <Era>ERA_XYZ</Era>
        <GridX>2</GridX>
        ...
    </Row>
 
    <!-- New techs for adjacent eras -->
    <!-- Note that GridX is used to indicate relative position to the columns of ERA_XYZ, not an absolute position in the tech tree -->
    <Row>
        <!-- This will end up in the renaissance era, but for now we place it in the Xyz era -->
        <Type>TECH_IJK</Type>
        ...
        <Era>ERA_XYZ</Era>
        <GridX>0</GridX><!-- Column 0 of ERA_XYZ is the 2nd column of the preceeding era -->
        ...
    </Row>
    <Row>
        <!-- This will end up in the industrial era, but for now we place it in the Xyz era -->
        <Type>TECH_LMN</Type>
        ...
        <Era>ERA_XYZ</Era>
        <GridX>3</GridX><!-- Column 3 of ERA_XYZ is the 1st column of the following era -->
        ...
    </Row>
</Technologies>

Finally, with the new techs added, we need to relocate them to their correct absolute position in the tech tree

Code:

-- Relocate techs to their absolute positions
UPDATE Technologies SET GridX=GridX+(SELECT GridX-3 FROM Technologies
                                       WHERE Era IN (SELECT Type FROM Eras
                                                       WHERE ID > (SELECT ID FROM Eras
                                                                      WHERE Type='ERA_RENAISSANCE') AND Type!='ERA_XYZ')
                                       ORDER BY GridX LIMIT 1)
  WHERE Era='ERA_XYZ';

If you are adding techs into adjacent eras, you'll need to treat them as techs of the new era, position them relative to the new era columns (eg GridX=-1 or GridX=3), let them be moved with the SQL above and then correct their era designations, eg for TECH_IJK and TECH_LMN above

Code:

-- Reassign techs to their correct era -->
UPDATE Technologies SET Era='ERA_RENAISSANCE' WHERE Type IN ('TECH_IJK');
UPDATE Technologies SET Era='ERA_INDUSTRIAL' WHERE Type IN ('TECH_LMN');

If you are moving techs, you'll need to use SQL to move them a relative number of columns, eg GridX=GridX-1, and not place them at an absolute position, eg not GridX=15

Or you may just decide that allowing for other era adding mods is just too much like hard work and ignore all of this!

whoward69 · Nov 25, 2015

Using these techniques, I've managed to alter the Prehistory, Enlightenment and Future era mods to make it possible to load them in any combination - see combined tech tree

There are shortcuts that can be used for eras/techs at the start and end of the tech tree.

An era at the start of the tech tree can simply shift the entire tree right to make room for it's techs, while an era at the end of the tech tree can either assume it loads first (hopefully the authors of other era adding mods will add References to your mod) or it can use WHERE EXISTS ... clauses to conditionally shift its techs based on any other era mods that may have loaded first.

Lynnes · Nov 26, 2015

Very nice tutorial, thank you!

HandyVac · Dec 2, 2015

Very helpful, tutorial, thanks!

Keyalha · Feb 12, 2016

whoward69 said:
Using these techniques, I've managed to alter the Prehistory, Enlightenment and Future era mods to make it possible to load them in any combination - see combined tech tree

There are shortcuts that can be used for eras/techs at the start and end of the tech tree.

An era at the start of the tech tree can simply shift the entire tree right to make room for it's techs, while an era at the end of the tech tree can either assume it loads first (hopefully the authors of other era adding mods will add References to your mod) or it can use WHERE EXISTS ... clauses to conditionally shift its techs based on any other era mods that may have loaded first.

Could you elaborate on that for someone that has no clue of modding? I am trying to get Prehistoric, Future Worlds and Enlightenment ERa to play nice with each other to no avail. Any help would be highly appreciated.

whoward69 · Feb 13, 2016

Keyalha said:
Could you elaborate on that for someone that has no clue of modding?

The elaboration is in post #1

Keyalha · Feb 13, 2016

whoward69 said:
The elaboration is in post #1

I was talking about a way a non modder understands.

carnivorejoe · Dec 15, 2019

Great tutorial! I've used it to modify Ultimate Eras Mod to be compatible with Future Worlds and Enlightenment Era. A couple updates I want to add...

I found that the NewEraPopup.lua file needs some extra functions, or else the close button doesn't work on the splash screen. Forunately, the author of Future Worlds also figured this out already, and so I didn't have to go digging through the elusive API to figure out how to fix that problem. Here's the full Lua file we're both using:

Spoiler :

Code:

function OnPopup( popupInfo )
    if( popupInfo.Type ~= ButtonPopupTypes.BUTTONPOPUP_NEW_ERA ) then
        return;
    end

    m_PopupInfo = popupInfo;

    local iEra = popupInfo.Data1;
    Controls.DescriptionLabel:LocalizeAndSetText("TXT_KEY_POP_NEW_ERA_DESCRIPTION", GameInfo.Eras[iEra].Description);
    
    lastBackgroundImage = GameInfo.Eras[iEra].SplashScreen;
    Controls.EraImage:SetTexture(lastBackgroundImage);
    
    UIManager:QueuePopup( ContextPtr, PopupPriority.NewEraPopup );
end
Events.SerialEventGameMessagePopup.Add( OnPopup );

----------------------------------------------------------------       
-- Input processing
----------------------------------------------------------------       

-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
function OnClose()
    UIManager:DequeuePopup( ContextPtr );
    Controls.EraImage:UnloadTexture();
end
Controls.CloseButton:RegisterCallback( Mouse.eLClick, OnClose);

-------------------------------------------------------------------------------
-------------------------------------------------------------------------------

function InputHandler( uiMsg, wParam, lParam )
    if uiMsg == KeyEvents.KeyDown then
        if wParam == Keys.VK_ESCAPE or wParam == Keys.VK_RETURN then
            OnClose();
            return true;
        end
    end
end
ContextPtr:SetInputHandler( InputHandler );


-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
function ShowHideHandler( bIsHide, bInitState )

    if( not bInitState ) then
       Controls.EraImage:UnloadTexture();
       if( not bIsHide ) then
            Controls.EraImage:SetTexture(lastBackgroundImage);
            UI.incTurnTimerSemaphore();
            Events.SerialEventGameMessagePopupShown(m_PopupInfo);
        else
            UI.decTurnTimerSemaphore();
            Events.SerialEventGameMessagePopupProcessed.CallImmediate(ButtonPopupTypes.BUTTONPOPUP_NEW_ERA, 0);
        end
    end
end
ContextPtr:SetShowHideHandler( ShowHideHandler );

----------------------------------------------------------------
-- 'Active' (local human) player has changed
----------------------------------------------------------------
Events.GameplaySetActivePlayer.Add(OnClose);

This does, however, break my custom splash screen for the Ancient Era that was working previously... still trying to sort that out.

The other issue I encountered may be unique to the fact that the Prehistoric Era is at the beginning of the tech tree, but when adding the new Ancient Era techs to the database, I actually ended up having to specify that they're from the Ancient Era in the Technologies.xml file, whereas your guide says to put them in the custom era, then reassign them with SQL. Maybe I'm blind, but this SQL didn't work:

Spoiler :

This resulted in the Ancient Era simply not existing in practice in the game (although it appeared on the game's config options for starting era). The first 7 columns of techs were all classed as part of the Prehistoric Era, which then moved straight into the Classical Era. Any idea why that might be, and are there any repercussions of the fact that I'm setting the techs to their proper eras in the .xml file instead of the .sql file later?

Adding an era using relative techniques

whoward69

DLL Minion

whoward69

DLL Minion

Lynnes

King

HandyVac

Gentleman

Keyalha

Warlord

whoward69

DLL Minion

Keyalha

Warlord

carnivorejoe

Chieftain

Similar threads