Replacing the Custom Game screen (proof of concept)

f1rpo

plastics
Joined
May 22, 2014
Messages
1,702
Location
Germany
Since the potential for modifying the Custom Game screen is very limited, I've written a mod component that overlays the Custom Game screen with this (so far) mostly blank replacement:
custom_custom_game_Screen.jpg

The code, along with a compiled DLL, is available on GitHub. A playable version can be extracted from the ZIP download on GitHub. I'm posting this mainly to make other modders aware that the Custom Game screen could – probably – be replaced pretty cleanly. (Also: to get confirmation that my implementation actually works, and not just on my own system.) Of course someone – probably not me – would have to put in the effort of recreating all the menus. Or, better: implement a more user-friendly redesign. A hybrid approach could also work: place only a small overlay (for some special setting that a mod wishes to make) in an unused area of the original screen.

Implementation: The mod keeps track of certain calls to CvInitCore to detect when the Custom Game screen has been opened; then it opens the replacement screen on top of the original screen through a DLL-to-Python call (inspired by the Full Of Resources map script). When "Launch" is clicked, the replacement screen is closed and a press of the Return key is emulated (through the SendInput Windows function), which causes the "Launch" button of the original screen to be pressed. The default selection for the game speed menu is read from CvInitCore (which gets it from CivilizationIV.ini), and when the selection is changed, the new speed setting is stored at CvInitCore. The data at CvInitCore doesn't seem to get overwritten upon launch, so this is all it takes to bypass the original screen.

It looks a little strange when the two screens open (and close) in quick succession. This can't be helped. And there's a bit of an open issue with the "Go Back" button. Currently, the click on that button is doubled through SendInput so that the original "Go Back" button positioned in exactly the same place also gets clicked. This doesn't work if the user moves the mouse away very quickly or when "Go Back" gets pressed through TAB (highlight next button) and Return. I've tried emulating TAB followed by Return, but that wasn't working reliably at all. Crucially, the "Launch" button does work reliably on my end. (Well, unless the player has TAB'ed through all elements of the replacement screen, causing some element - other than "Launch" - to be highlighted on the original screen).
 
Things that I think could be improved about the original Custom Game screen:

(1) Separate tab for the player list: The original screen kind of has that option, but the maximize button is difficult to click and one needs to click twice to go from the maximized player list to maximized game options. Two tabs seems like a reasonable middle ground between the cramped Custom Game screen and the rigmarole Play Now screen.
(2) If the number of AI players were automatically adjusted to the sea level, there would be one fewer reason to enter the players tab. The automatic addition and removal of players is generally convenient, but can also be unwelcome; might want to tie that to an extra button. At the least, only AI slots with Random civ and Random leader should be removed automatically. Also, the current player count should be shown somewhere.
(3) There should be some way of opening or closing multiple slots at once.
(4) Find some way to make team games easier to set up. E.g. buttons for dividing the players evenly into teams of 2, 3 or 4. Or a button for remembering the team assignment in the INI file (not so easy to implement).
(5) A button for restoring the default settings (everything medium, checkboxes set to the defaults set in XML, recommended number of players, all set to Random leaders) might be handy.
(6) Hover text should be shown for the drop-down menus. Text already exists for most of them, but BtS shows them only on the Play Now screen. The same goes for the map script description text. Would be nice if map scripts could also show hover text for their custom options (which often aren't self-explanatory).
(7) The time-related settings (game speed, start era) should not be wedged in between the standard and custom map options.
(8) Barbarian and Tech Trade options that are mutually exclusive (No Tech Brokering together with No Tech Trading doesn't make sense) should be represented by a drop-down menu or slider. And the screen should somehow warn about contradictory settings, e.g. by showing an error popup instead of launching when "Launch" is clicked.
(9) To break up the tiresome list of option checkboxes further, options that change the game fundamentally (e.g. One-City Challenge) could be moved to a separate panel; there's room under the Victory settings. And the options should be arranged in a sensible order. (Doing so in XML breaks savegames.) Might want to add a <iPosition> tag to the game option XML schema; and maybe <bGameMode> for the special-challenge panel.
(10) Put the Unrestricted Leaders on the tab with the player list – that's where the leaders get assigned. My own mod allows starting points to be assigned unequally; the whole starting point thing should perhaps be handled by the player list.
(11) Let the player set a custom difficulty for AI players. BtS enforces Noble, but scenarios can already have unequal handicaps for AI players, i.e. the game supports this. May have to adjust the Dan Quayle score calculation.
(12) The leader traits and their effects should be spelled out somewhere. That could be handled by a special item in the leader-selection drop-down that opens a popup akin to the leader selection on the Play Now screen. Same deal for the civ selection – would be nice to see the starting techs spelled out. (Note that no hover text can be shown for individual items of a drop-down menu; only for the whole menu.)

(I've made a mockup, but it's pretty messy, and I think, to get someone else interested in reimplementing this screen, it's better to let that person come up with their own design.)
 
Last edited:
A few more ideas:

(2) Have an option to easily change the number of players. E.g. I always reduce the number of starting civs by 25-33% to have more space for early hunting in my mid.

(13) Ban/force civ/leader to be in the game. You may want to play an Eurasia game or one without modern civs and leaders. Or you want to play with Unrestricted Leaders and want to have more than one leaders be present in the game.
(14) Save/Load Settings. I hate it when the game restores everything to default and I need to go through the settings again.
(15) Access to the pedia without leaving the Game Creation screen.
 
(13) Ban/force civ/leader to be in the game. You may want to play an Eurasia game or one without modern civs and leaders. Or you want to play with Unrestricted Leaders and want to have more than one leaders be present in the game.
For the record, the Ashes of Erebus mod implements this, but as a separate screen that pops up when you click on a certain gameoption.
 
For the record, the Ashes of Erebus mod implements this, but as a separate screen that pops up when you click on a certain gameoption.
Sounds like a good solution; pretty resourceful of whoever implemented that.
(14) Save/Load Settings. I hate it when the game restores everything to default and I need to go through the settings again.
When does it do that? The INI is supposed to remember pretty much everything except the players and custom map options. When switching between modds I guess? That can be avoided by loading a different INI for every mod ('ini=' command-line option of the BtS EXE). Well, that whole system somehow isn't great ...
(15) Access to the pedia without leaving the Game Creation screen.
I like this idea; should be easy enough to find a place for that tiny red button.
 
When does it do that? The INI is supposed to remember pretty much everything except the players and custom map options. When switching between modds I guess? That can be avoided by loading a different INI for every mod ('ini=' command-line option of the BtS EXE). Well, that whole system somehow isn't great ...
Yes, especially when switching mods. But AND2 also clears the cache (or does something else) that often resets settings. But I think it would be great if the game would remember the settings, including number of civs preset civs, etc.
I think custom initial files could do that, can't they?
 
[...] But I think it would be great if the game would remember the settings, including number of civs preset civs, etc.
I think custom initial files could do that, can't they?
There is
class CvDLLIniParserIFaceBase
Spoiler :
Code:
{
public:
   virtual FIniParser* create(const char* szFile) = 0;
   virtual void destroy(FIniParser*& pParser, bool bSafeDelete=true) = 0;
   virtual bool SetGroupKey(FIniParser* pParser, const LPCTSTR pGroupKey) = 0;
   virtual bool GetKeyValue(FIniParser* pParser, const LPCTSTR szKey, bool  *iValue) = 0;
   virtual bool GetKeyValue(FIniParser* pParser, const LPCTSTR szKey, short *iValue) = 0;
   virtual bool GetKeyValue(FIniParser* pParser, const LPCTSTR szKey, int   *iValue) = 0;
   virtual bool GetKeyValue(FIniParser* pParser, const LPCTSTR szKey, float *fValue) = 0;
   virtual bool GetKeyValue(FIniParser* pParser, const LPCTSTR szKey, LPTSTR szValue) = 0;
};
but I'm not sure if that's sufficient for creating new INI files. BUG also has code for managing INI files, but the BUG modules aren't loaded until a game has been created or loaded. One can always use the Windows API to create and open files (the Full Of Resources map script does that, for example).

As a modder, creating files outside the mod folder makes me a bit nervous about not accidentally overwriting something and not leaving files behind once my mod has been uninstalled. And there's the hassle of figuring out paths on different OS and Civ versions and dealing with access restrictions.
 
PHP:
CvString filename = gDLL->getModName();
filename.append("my custom file.txt");
Using windows to write to that path will create the file in the root dir of the loaded mod. If people uninstall the mod, it won't leave files behind. Obvoiusly you can pick whatever file name and format you want.

This approach fails if the user installed the mod into the "My Games\Beyond the Sword\Mods" folder, but not the installation directory of Civ4 didn't it?!

I use this to get the path to the custom DLL and derive the mod location from there.
(I found this also in other more popular mods but find no reference ATM.)
 
@f1rpro: Nice project. I've tried to create a similar screen, but without changes of the SDK…

… and failed due some restriction/bugs of handling user input (e.g loosing input focus). (Reference)

Maybe I can adapt your solution and can continue this old project :) I derived my screen from the Pedia screen. Did you use a modal like the options screen dialog?
 
@Ramkhamhaeng: Sorry about the late reply. I see that you've already been experimenting with this when I was only beginning to mod Civ 4 ...
Thanks for making me aware of the setModal function. As you're already aware, the CyGTabCtrl API is, in other regards, much more limited than CyGInterfaceScreen and I don't think one can have it both ways, so, for replacing the Custom Game screen, I'd rather stick with the nonmodal screen. (Specifically, the screen I use is derived from lfgr's GenericAdvisorScreen, which is just a blank Advisor screen with maximized dimensions.) The Tab key moving the focus onto the screen in the background really is a minor issue – if this is indeed the only thing that a modal screen would improve.
Maybe I can adapt your solution and can continue this old project :)
Hm ... Not sure what novel thing you could get out of my approach. The idea with the emulated key presses maybe(?). Distinguishing between the Play Now screen(s) and Custom Game by tracking CvInitCore calls took a bit of experimentation, but I guess that's not a problem you'd need a solution for.
--
(I haven't done anything on the project since my last post.)
 
I've written a mod component that overlays the Custom Game screen ...
This "proof of concept" is a really great job and a breakthrough in Civ4BTS modding. :thumbsup:
(I always thought that was impossible due to dependencies witht the .exe... )

It may not be the "perfect solution" but "overlay approach" does sound interesting though. :think:
(We simply lack anything better ... )

The mod keeps track of certain calls to CvInitCore to detect when the Custom Game screen has been opened; then it opens the replacement screen on top ...
Does the "track keeping" (calls to CvInitCore) affect ingame performance once the actual game started?
Or is the mod not listening to these calls anymore then once the actual game started?
 
Last edited:
The keeping track is just one boolean, and the CvInitCore setter functions don't really get called once the game has been launched.

The only alternative to an overlay (replacing the old screen or augmenting it) that I can think of is to open a replacement screen after the game has been launched. But that would mean that the player first has to launch a game with meaningless settings.
 
General note: My gust tells me that designing/deriving the screen from the Options-Screen wouldn't be a bad solution. The calls of this type (and Pedias, too) of screen are designed to work without a loaded game.

The only alternative to an overlay (replacing the old screen or augmenting it) that I can think of is to open a replacement screen after the game has been launched. But that would mean that the player first has to launch a game with meaningless settings.

Maybe following can help you:
In https://github.com/civ4-mp/mod-updater I've wrote following comment about this topic
• Adding a Screen to the main menu is a bit tricky! You can not use the OnInit-Eventhandler because it fires to early. At this game startup stage, it is not possible to draw anything on the screen.
The only useful event handler is onWindowActivation. Unfortunately, the first call of this event has two pitfalls.

Firstly, some data is still not initialized and i.e.
CyTranslator.getText can throw a C++-Exceptions and the call of
CvModUpdaterScreen.getScreen also fails.

Secondly, the drawing of the main menu begins after onWindowActivation(). This hides everything behind the background image.

Starting a new Python thread for some delayed redrawing command fails because the Cy*-Objects did not live long enough. (?!)

I resolved this problem by starting a new thread over a DLL call.

I do not suggest to copy my code because it is not clean enough.
My overall idea was following to draw a window out of the blue at startup:

1. Add "delayedPythonCall(iSeconds)" to CvEventManager.onWindowActivation to start a new thread in the DLL. (This Threads survives, a Python thread not)
2. After iSeconds use 'gDLL->getPythonIFace()->callFunction(…)' to run the screen drawing.
This probably needs improvement because Civ4 not expecting that a second thread calls some drawing functions?!

Unfortunately this never works as stable for me as wished. Sometimes this let the game crashing so there is probably a race condition I've never founded.
Moreover, the fullscreen mode is troublesome.

Here an other trick I found out as workaround for fullscreen mode:
The getMousePos()-call returns (x,y) = (0,0) if the window isn't ready for GUI drawing calls.
Code:
pt = CyInterface().getMousePos()
if pt.x == 0 and pt.y == 0:
       print("(ModUpdaterScreen) Hey, window not ready for drawing."
        "Wait %s milliseconds..." % (1000,))
 
General note: My gust tells me that designing/deriving the screen from the Options-Screen wouldn't be a bad solution. The calls of this type (and Pedias, too) of screen are designed to work without a loaded game.
Regarding the parenthesis, Pedia being a screen known to work prior to game start is a good point; it does make more sense insofar to use the Pedia screen as a template for a new Custom Game screen than to use an Advisor screen. That said, looking at CvPediaMain.py now, it isn't very different from the in-game Advisor screens. Same interaction with CyGInterfaceScreen, CvScreensEnum and CvInterfaceScreen:
Code:
def getScreen(self):
   return CyGInterfaceScreen(self.PEDIA_MAIN_SCREEN_NAME, CvScreenEnums.PEDIA_MAIN)
(Which is what your CvModUpdaterScreen does as well.) The top and bottom panels and background are set up in the same way too. These lines are interesting to me:
Code:
screen.setRenderInterfaceOnly(True)
screen.setScreenGroup(1)
screen.showScreen(PopupStates.POPUPSTATE_IMMEDIATE, False)
RenderInterfaceOnly causes a delay upon opening the screen, doesn't seem to accomplish anything desirable. I see that you've also decided not to set this for your CvModUpdaterScreen. ScreenGroup – doesn't seem to make any difference. I guess that's for tying together several sub-screens. As for the showScreen call, the Advisor screens use the same parameters as the Pedia.

I've found that I needed to call screen.setPersistent(True), otherwise calls to interfaceScreen weren't bringing my screen back after closing it via the Esc key. Similarly, if I don't set an activation type for my "Close" button,
screen.setActivation(CLOSE_ID, ActivationTypes.ACTIVATE_NORMAL)
then the screen doesn't properly close through Esc. The Pedia screen does it somehow differently I guess. Perhaps worth for me to investigate further because not setting an activation type seems to prevent the Tab key from messing with the focus.
My overall idea was following to draw a window out of the blue at startup: [...]
I think I already get the right timing for the Custom Game screen by opening the screen from CvInitCore::setActivePlayer. I don't do it on the very first call to that function, but, if you want to show a screen that appears along with the opening menu, the first setActivePlayer call could be worth a try (if haven't already tried that).

Your delayedPythonCall technique could perhaps help reliably emulate the key sequence Tab-Return for pressing "Close" on the original Custom Game screen. I've tried using Sleep in the DLL to that end, but it didn't help or at least not much. Well, I'm not sure if there's any real problem with my current solution of duplicating the mouse click on the overlayed "Close" button. One can break it on purpose (Tab key, very fast mouse movement), but I don't know if that would happen by accident.

Just to make sure that there isn't a misunderstanding: My "alternative (idea) to an overlay" was to let the player start a new game or load a savegame by whichever means and then, instead of the Dawn Of Man screen, show a screen for setting up the game properly. Clicking "Launch" on that screen would reset all the game data and generate a new map.
Showing anything directly on the opening menu, i.e. before any of the game setup screens has been opened, isn't going to work for me because a "Launch" button needs to be pressed. I think that's the only way to get the EXE to actually initiate a new game.
 
I think I already get the right timing for the Custom Game screen by opening the screen from CvInitCore::setActivePlayer. I don't do it on the very first call to that function, but, if you want to show a screen that appears along with the opening menu, the first setActivePlayer call could be worth a try (if haven't already tried that).

Good suggestion. It would simplify my code a lot and I can remove this complex threading stuff.
But CvInitCore::setActivePlayer is called one time before the main menu is drawn.
If I draw my Python screen here, it will be hidden behind the background image of the main menu.
It works if I put my stuff into CvGame::getActivePlayer(). Changing this functions isn't as handy as changing the setter because it will be called more often.

I want use this to show my preferred technique to filter out calls of functions with a fixed parent location in the binary. A poor mans variant of hooking functions ;)
(This technique is not required here, because catching the first call is already good enough.)

Note, this only works with a Makefile change: Release_CFLAGS needs to be changed to /Oy-. This could slow down the code a little bit. In my case (Pitboss games) this is no real issue.
Code:
static void *CriticalParent_MainMenu = (void*) 0x00508b8e;
/* If MainMenu has input focus this method will be called in each cycle.
 * and the parent stack pointer is above function.
 */
static int Counter_MainMenu = -1;  // not required

PlayerTypes CvGame::getActivePlayer() const
{
           void ** volatile puEBP = NULL;
           __asm { mov puEBP, ebp };
           void * pvReturn1 = puEBP[1];
           if( pvReturn1 == CriticalParent_MainMenu ){
               CyArgsList argsList;
               argsList.add(-1);
               argsList.add(++Counter_MainMenu);

               long lResult = 0;
               gDLL->getPythonIFace()->callFunction(PYGameModule, "showUpdater", argsList.makeFunctionArgs(), &lResult);
               if (lResult == 1){
                   CriticalParent_MainMenu = (void *) 0;
               }
           }
   return GC.getInitCore().getActivePlayer();
}
 
[...] CvInitCore::setActivePlayer is called one time before the main menu is drawn.
If I draw my Python screen here, it will be hidden behind the background image of the main menu.
That's unfortunate. I think all the other CvInitCore setters get called even earlier than setActivePlayer.
I want use this to show my preferred technique to filter out calls of functions with a fixed parent location in the binary. [...] Note, this only works with a Makefile change: Release_CFLAGS needs to be changed to /Oy-. This could slow down the code a little bit. In my case (Pitboss games) this is no real issue.
Thanks for sharing. I could also imagine this technique being useful somehow for reverse-engineering the EXE, if needs be.

If the updater doesn't need to be shown upon returning to the opening menu, then I'd try the following approach for a smaller performance penalty: Create a separate function CvGame::getActivePlayerExternal, export that function as getActivePlayer through a .def file
Code:
?getActivePlayer@CvGame@@QBE?AW4PlayerTypes@@XZ=?getActivePlayerExternal@CvGame@@QBE?AW4PlayerTypes@@XZ
and remove the DllExport specifier from CvGame::getActivePlayer. The external version would still be fairly frequently called, however, something like
Code:
PlayerTypes CvGame::getActivePlayerExternal() const
{
   if (!m_bUpdaterShown)
   {
       pyShowUpdater();
       m_bUpdaterShown = true;
   }
   return GC.getInitCore().getActivePlayer();
}
is not going to make any appreciable difference for performance.

Showing it upon returning might work through setActivePlayer. Seems to get called when returning from the Single Player, Multiplayer and Advanced submenu (not upon closing the Hall of Fame or Pedia screen though). Must also get called at some point when exiting to the opening menu from inside a game. May then be possible to figure out from the state of CvInitCore that the setActivePlayer call is not coming e.g. from the Custom Game screen.
 
I've implemented your suggestion. At first it didn't work:
Code:
Unhandled exception at 0x75f9a6f2 in Civ4BeyondSword2015.exe: 0xC06D007F: Procedure not found.

Turns out, that the 'const' flag matters and needs to be added?!
So I declared m_bUpdaterShown as mutable and now it works :) Thanks.

For other users here a more detailed description how to use the function renaming by a def-File in Civ4's case:

CvGame.h header
Code:
/* DllExport */ PlayerTypes getActivePlayer() const;
/* DllExport as getActivePlayer() by Def-file */ PlayerTypes getActivePlayerExternal();   // Exposed to Python
mutable bool m_bUpdaterShown;

Put definitions into def file:
Code:
LIBRARY CvGameCoreDLL
EXPORTS
?getActivePlayer@CvGame@@QBE?AW4PlayerTypes@@XZ=?getActivePlayerExternal@CvGame@@QBE?AW4PlayerTypes@@XZ

Added def-File to linker in makefile:
Code:
$(LD) /DEF:Updater.def[/b] /out:$(Debug_BIN)
[…]
$(LD) /DEF:Updater.def /out:$(Release_BIN)
 
Top Bottom