Replacing the Custom Game screen (proof of concept)

[P&nny]

Guys, I profusely apologies for not coming back on the topic earlier but I have been so so excited about the prospects of this since your first message f1rpo !
I've put it on my list to dig in as soon as next version of my mod (beyond the game) is out and well, pfiou, this always takes more time than expected (I should be wrapping up next version in the month and taking break after).

Please tell me you're still keen on this, and I would totally offer help & support !
I do realize you're guys are several levels above my very own skill level but hopefully I can be of use.

What would be first step ? I was simply thinking of just reproducing the current custom game menu on a vanilla BTS to have a common base for everyone ?


I must say I always assumed this would be extremely useful in multiplayer but re-reading the topic I have a doubt "Custome game" is not the same menu as "LAN games" or the lobby games.
Could we hope to use the same underlying code for both ? Or the synchronization in MP would not work ? Full of Ressource map does work for example, not sure if every player could see the menu or just the host though.

What I think would be awesome for MP :
(16) More tabs for option by "genre" via new XML tag in Civ4GameOptionsInfo (I've got so many in my MOD, I name them accordingly but confusing & not clean)
(14) Save/Load Settings. --> Yes totally but for a reason not clearly mentioned here : Because you can play "a scenario". For example you want to play 5v5 Ancient you load the file 5v5 Ancient Teambattleground which selects all good options, or if you want to play 3v3 Renaissance Inland Sea you load that specific scenario file and boom not setting up to do and most importantly no mistake

 

[f1rpo]

I must say I always assumed this would be extremely useful in multiplayer but re-reading the topic I have a doubt "Custome game" is not the same menu as "LAN games" or the lobby games.
Could we hope to use the same underlying code for both ? Or the synchronization in MP would not work ? Full of Ressource map does work for example, not sure if every player could see the menu or just the host though.

Letting the DLL detect when the Staging Room screen (or, for that matter, Custom Scenario) is opened does not seem difficult; same method as for the Custom Game screen. Before launch, the "Ready" box needs to be checked by every participant. Seems that this can simply be done programmatically by the host DLL (through CvInitCore::setReady); at least this has worked for me in a quick test with two instances of Civ running on a single PC (connecting to 127.0.0.1). I've put my code for this test on this Git branch.

I expect that the network communication before launch would have to be reimplemented through gDLL->sendMessageData. E.g. when the game speed is changed by the host (the other participants can't change settings other than their own civ and leader), instead of calling CvInitCore::setGameSpeed (which is what my demo code does), a message would have to be sent that causes setGameSpeed to be called and the (reimplemented) screens to be updated – so that the new game speed setting becomes visible to every participant. Not sure what exactly should happen when a player joins.

Hard to say if the chat panel could be reimplemented. We have gDLL->sendChat, but I suppose that's for chatting within a running game, and I don't know if a suitable widget is available in Python. That could be a big obstacle; the chat seems kind of important.

Using (essentially) the same Python screen for Custom Game, Custom Scenario and the various Staging Room screens (Direct IP map, scenario, Hot Seat) shouldn't be too difficult. Could pass a parameter to the showCustomGameScreen Python function that my DLL currently calls or expose CvInitCore::getType to Python.

I haven't looked into LAN, PBEM and PitBoss.

Please tell me you're still keen on this, and I would totally offer help & support !

That's my plan too – offer help and support. Not keen on writing all that Python code.

What would be first step ? I was simply thinking of just reproducing the current custom game menu on a vanilla BTS to have a common base for everyone ?

Sounds reasonable. Could clone my repository (which is just BtS, a build system and the Custom Game demo stuff). The very first step, if you haven't already, should perhaps be to verify that my approach doesn't somehow only work on my PC. I've just updated the DLL on the main branch – which I had forgotten for the last few commits in May – so just downloading a ZIP archive from GitHub (main branch or mp_test) should suffice for a test. And if MP is a must for you, then we should of course prioritize the open issues mentioned above.

(16) More tabs for option by "genre" via new XML tag in Civ4GameOptionsInfo (I've got so many in my MOD, I name them accordingly but confusing & not clean)

Yes, would be nice to base the arrangement of the options on XML data rather than hardcoding it.

(14) Save/Load Settings. [...] Because you can play "a scenario". [...]

I see – store separate settings for each map script and scenario. Just quite an (extra) effort to implement.

On a related note, I've been wondering if the Python code in the EXE could be decompiled somehow. Since no pyc files for e.g. the game setup screens or diplo screen are anywhere to be found, I suppose that the Python code must've been included in the EXE, through something like py2exe. Python bytecode can generally be decompiled. Don't know if that would be readable enough to be a useful starting point for reimplementing the game setup screens. Or if one could even make small changes to the decompiled code, recompile it and reinsert into the EXE – and not reimplement anything. Well, that all sounds a little unrealistic ...

 

[Toffer90]

On a related note, I've been wondering if the Python code in the EXE could be decompiled somehow. Since no pyc files for e.g. the game setup screens or diplo screen are anywhere to be found, I suppose that the Python code must've been included in the EXE, through something like py2exe. Python bytecode can generally be decompiled. Don't know if that would be readable enough to be a useful starting point for reimplementing the game setup screens. Or if one could even make small changes to the decompiled code, recompile it and reinsert into the EXE – and not reimplement anything. Well, that all sounds a little unrealistic ...

I've decompiled the pyc files bundled with vanilla, not the pyd files though.

 

[f1rpo]

@Toffer90: Thanks, that's educational for me as a Python know-nothing. So all the identifiers are preserved in (those) pyc files and the decompiled scripts are therefore perfectly readable. And pyd would be a very different story. That System folder contains, I gather, simply the Python 2.4.x interpreter plus standard library, wxPython and perhaps a couple of other libraries.

Having run some tools on the EXE (screenshots attached), it seems to be a pretty normal MSVC executable. There is a fairly large resource section, but that appears to be mostly image data for the splash screen. Surely the game setup screens and diplo screen were implemented in Python; still mystifies me what exactly was done to that Python code and where it has ended up.

Edit (several months later): My bet is now that those missing screens have been implemented natively in C++. I'm guessing that the game's widget toolkit (Scaleform GFC) is also implemented in C++, perhaps with an API similar to the one that the EXE exposes to Python.

 

[P&nny]

Coming back to this after a while (but having made some, I think, interesting progress) :

Is there an (easy) way to call a custom Screen with a keyboard shortcut while waiting on the custom game screen ?
CvEventManager is not in action yet so is not obvious to me if it can be. I imagine in C++ for sure you could do it (but I don't know how) but there should be a way in Python only I imagine.

I ask before I spent too much time drilling into your code f1pro

The code, along with a compiled DLL, is available on GitHub. A playable version can be extracted from the ZIP download on GitHub.

Or try to copycat into this which should be similar but probably more difficult to implement (definitely C++?)

For the record, the Ashes of Erebus mod implements this, but as a separate screen that pops up when you click on a certain gameoption.

On my side I've done two of the things discussed here :

(16) More tabs for option by "genre" via new XML tag in Civ4GameOptionsInfo (I've got so many in my MOD, I name them accordingly but confusing & not clean)

Yes, would be nice to base the arrangement of the options on XML data rather than hardcoding it.

I built an option screen you can click at anytime during the game (I made restriction for the player non-host, the turn of the game (not first turn) and if there is a spectator/referee, but whatever),
You can change the options just by clicking it, I've introduce the iCategory in the option XML to have them categorise in tabs (it was quite easy (but C++ required))

(14) Save/Load Settings. --> Yes totally but for a reason not clearly mentioned here : Because you can play "a scenario". For example you want to play 5v5 Ancient you load the file 5v5 Ancient Teambattleground which selects all good options, or if you want to play 3v3 Renaissance Inland Sea you load that specific scenario file and boom not setting up to do and most importantly no mistake

So I did an extension of that screen (you can toggle from one to the other) where I've fed lists of options corresponding to specific scenarios;
On the left side is what the current selection is, on the right handside what the declared list for the scenario is (not clickable), when it's aligned it's green, when not it's red
I also did a sub section for "recommandation", and besides the coloring scheme which is different, there is an "Align all" checkbox which makes all the difference in the main part align (basically changes the options in red).

I'm 95% there, which involves a lot of MODNet communication so that (in multiplayer) when someone changes it on his screen, it changes on everyone screens at the same time...

But the biggest shortcoming of course is what you described @f1rpo , that it comes after the fact of launching the game so everything is already created, pretty much forces a relaunch of map creation.

Hencewhy my question, could it be as easy as calling this screen from the Custom game menu ? I appreciate we'll have to send MODNets to communicate from this screen to main menu, let along other player's main menu, but that'd be a start !

 

[f1rpo]

Regarding net messages, I've tried sending one upon game launch for a different purpose a while ago and was unable to get it to work. My impression is that net messages can be sent - but don't get delivered until the game is fully set up. If that's true, then I guess the Staging Room screen really can't be customized meaningfully, or only with excessive effort. Or perhaps net messages work before launch (obviously the EXE is able to exchange data across the network), just not during launch. Don't know how you could quickly rule out that this going to be a problem.

Let me look up what I was trying to do ... My Settings tab (Victory screen) shows which leaders were selected manually and which were randomly assigned. The DLL can figure that out upon game launch (player clicks the Launch button) by keeping track of random numbers obtained by the EXE from the DLL. In network games, the randomization happens only on the host. I wanted the host to send that info to the guests so that they can display the leader selection info on the Settings tab as well. Doing that right away from CvInitCore, as the game was launching, worked, but the message didn't arrive until later in the game setup process. No real problem for me; I'm sending the info in CvGame::setInitialItems now:

Spoiler :

void CvGame::setInitialItems()
{
   // ...
   /*   <advc.190c> Letting CvInitCore do this would be misleading b/c
       net messages don't get delivered that early in game setup. */
   if (isNetworkMultiPlayer())
   {
       CvInitCore const& kInitCore = GC.getInitCore();
       FOR_EACH_ENUM(Player)
       {
           if (kInitCore.wasCivRandomlyChosen(eLoopPlayer) ||
               kInitCore.wasLeaderRandomlyChosen(eLoopPlayer))
           {   // We're the host
               CvMessageControl::getInstance().sendCivLeaderSetup(kInitCore);
               break;
           }
       }
   } // </advc.190c>
   // ...
}

As for keyboard input, GetKeyState (WinUser.h) could be polled if one can find a DLL function that the EXE calls periodically. My replacement screen in Python, like any Python screen, gets its input from handleInput in CvScreensInterface.py. However, in a quick test that I've just done, that function was not getting called so long as none of the registered screens were up. Just as the comment says: "called when a screen is up". So this is probably no help.

Or try to copycat into this [Ashes of Erebus] which should be similar but probably more difficult to implement (definitely C++?)

I don't want to look it up either; I agree that it sounds similar, maybe simply a Python call in CvInitCore::setOption.

I'm not sure if you're interested in detecting a key press because you think that'll be easier to implement than opening the screen automatically upon entering the Custom Game screen - or because you don't want the screen to appear automatically. In the latter case, I guess some invisible dummy screen could be opened just so that input can be received. Making a Python call upon entering Custom Game is imo not difficult, or at least not much code, mainly just these changes to CvInitCore (or those for the Staging Room screen). Most of my other DLL changes thus far expose CvInitCore to Python so that the replacement screen can display the current settings (e.g. currently selected game speed) and change them according to user input.

 

[P&nny]

I thought about the keyboard shortcut route simply because I am several notches below your ability skills and thought it would be the simplest way. I'm just looking at what I can realistically get to work without necessarily being 100% clean.
Anyway, I did build myself just enough skills to make your code work with the python callback quite easily actually, call my own screen directly, thanks a ton it's definitely the right start.

So right now it's "locking" me because I need to work out the code of my screen to be a bit different so that it would allow me to close (...) and stuff like that,
But one think I note straight as it that the screen is OVER the staging room, so it's like I can put it on the side (if it was a 1x1 screen just for keyboard shortcut) to use the staging room screen normally.
This might be blocking as my option check is just that a CHECKER, that you do to help you mostly VERIFY the options.

I wonder if that's why Ashes of Erebus went the option clicking route.
Should we note move your python call back to CvInitCore::setOption, hardcode the name of an option "GAMEOPTION_CLICKMEFORMENU", call this hardcoded option, and pop it up ? And I put it in a category that doesn't show on my own version of the options screen.

 

[P&nny]

Ahhh so doing this straitght after :

void CvInitCore::setOption(GameOptionTypes eIndex, bool bOption)

is an excellent way for my use, it suits better to be honest with you.

I do realise now, unfortunately, that my screen does'nt do anything, because all my changes rely on CyGame().setOption(...
And that the game doesn't exist yet.
I presume what we see in custom/staging screen are reflection of what's in the .ini of the host player, it seems to be the case because when your proceed with launching the game it will save the options you've clicked; where as ingame menus like mine don't save (it only changes the GAME's value for those options).

Sadly I don't know how to change that (or anything stored in the .ini)

 

[f1rpo]

But one think I note straight as it that the screen is OVER the staging room, so it's like I can[not?] put it on the side (if it was a 1x1 screen just for keyboard shortcut) to use the staging room screen normally.

I imagine that CyGTabCtrl (which your menu seems to use) is always on top. Or maybe there is something in the API to change that? CyGInterfaceScreen behaves differently (but also looks different): if I make my replacement Custom Game screen a little smaller, so that the Launch button of the original screen isn't covered up, then I can click that button without having to close my replacement screen first. However, I think your screen is too large for that anyway, so you'll need some means of opening and closing it. A special game option sounds like a good approach. That's also similar to the Full of Resources map script, which has a special map option that can (repeatedly) bring up the popup menu. I suppose one could try to use a small, undecorated CyGInterfaceScreen to graft a single button onto the Staging Room screen – which will then open the CyGTabCtrl with the options. But that would be polishing once everything works.

I do realise now, unfortunately, that my screen does'nt do anything, because all my changes rely on CyGame().setOption(...
And that the game doesn't exist yet.
I presume what we see in custom/staging screen are reflection of what's in the .ini of the host player, it seems to be the case because when your proceed with launching the game it will save the options you've clicked; where as ingame menus like mine don't save (it only changes the GAME's value for those options).

CvInitCore gets initialized with the data from the INI, and the EXE uses the data in CvInitCore (specifically CvGlobals::m_initCore; there are at least three global instances) to initialize everything else when Launch is pressed. So that's the class to work with - and hence my exposing e.g. CvInitCore::setGameSpeed to Python. But, for multiplayer, a net message will have to be sent (assuming that this can even work before game launch), so I guess your Python screen would have to use the message control interface (already exposed to Python?), and the CvInitCore data would be set when processing a received net message. No need then, I guess, to expose CvInitCore functions to Python.
Edit: CvMessageControl::sendModNetMessage is already exposed to Python. That might suffice. But I guess the processing of incoming ModNetMessages would still happen in the DLL.