Diplo Screen Mod Component

[Leoreth]

Heeding my own advice, I am creating this thread to allow us to discuss/plan a custom diplo screen mod component that would replace the unmoddable Firaxis diplo screen hidden away in the EXE, before we take over the Modding Questions thread.

Here's the discussion that led to this point:

Spoiler :

Does anyone know where the Diplomacy screen is defined?

@ DH: in the exe

This is something, which annoys me quite a lot. It severely limits what you can do to diplomacy. Perhaps the most annoying part is that it isn't bug free (at least in colonization) and being in the exe, bugfixing is not really something we can do. The bug is that under certain conditions, the AI will offer peace, but the request to open the diplomacy window to offer peace vanishes in the exe. Sending the request will make it fetch all sorts of needed data to make the window (leader head and stuff), but then nothing and it just carries on. Setting the two teams to peace and then request the window will make it open, indicating that the bug is a toggled bool check. Why is that check even in the exe?

I was so upset with it at some point that I considered rewriting the diplomacy window from scratch. However I haven't even started doing that and I'm no longer sure that I will. It seems like a big task


The concept is to set up UI in python. There are some parts in the DLL. They are (mainly?) popup windows with buttons and pedia string generation. My guess is that opening a new window would be ugly in python compared to the relatively clean C++ code. The pedia requires lots of data access, which may or may not be available to python. Also C++ is significantly faster than python, meaning CPU heavy UI elements could make the game lag in python, particularly on minimum system requirements computers.

As for why some screens are in the exe? Who knows? Maybe they have such brilliant code that they wanted to keep it a secret. Or maybe they just didn't consider modder access to screens like that. I think the main issue here is that the exe<->dll interface was not planned as a clean interface from the start. I have encountered a number of features where I think "those doesn't belong in the exe", yet they are still there. For instance GameFont ID setup for yields. It's in the DLL in BTS (good), but in the exe in colonization (quite bad as colonization is the one, which needs this to be modded). The exe sets up all the ids at startup, then check if they are what it set them to be and crashes if they are wrong. After that, the modder can set them to whatever he wants as the exe will no longer need the yield GameFont IDs. It just seems to be a silly limitation, which serves no real purpose.

If possible, that would be great. I definitely also share some of your frustrations with it.

That post makes me think. We share a problem and none of us has the solution. However maybe some of us has part of the solution and together we have the complete solution.

There are the following steps:

1. catch the diplo event before it enters the exe.
2. open a python window
3. draw the python window based on diplo event data
4. python code to get which option the player clicked on and send it to the dll
5. dll reading the answer and do the requested action (transmit on network if needed)

My abilities for these tasks:

1. already done it (for a different purpose)
2. should be doable, though I need research to figure out how
3. my biggest issue. No idea how to do this. The leaderhead might be the biggest issue.
4. might be able to figure this one out
5. fully confident that I can handle this one

My biggest problem is python UI. If I can find somebody to do that, this could very well be done. I have experience in fixing network desyncs and know what it takes to make features work in networked multiplayer. I have made new classes in C++ and then added python interface to them, meaning variable communication between the diplo event and the python window would likely be a new class with a bunch of get/set functions. I have experimented with making C++ code where the code is in new files, meaning adding them to other mods is little more than copying in the cpp/h files.

In short, I have a decent idea on what to do to make a replacement window. I just can't make the actual UI itself. Also my current top priority is M:C. There are some people who are stuck modding until I reach a certain milestone and I don't want to keep them idle for longer than I have to. Any other tasks will have to wait.

Are there any python UI skilled people, who could be interested in a joined effort (presumably next year)? I can easily manage a git server to host the source. It should be a modcomp for BTS, but I assume it shouldn't be too difficult to make something, which works for any civ4, including colonization.

2) This is easy. Opening a new screen when a button or command is given can be done via pure python. I done it frequently in my WB, which you can explore for examples. Not at my computer now.

3) Easy as well, perfect sample in pedia.

4) Doable too, although for this case, you should need modnetmessaging to send to all

I'm very interested in this and would like to help where I can. I think my skill set is closer to Nightinggale's (DLL stuff and backend logic), but I'm sure there's something to contribute. I'm less experienced with drawing (good looking) UIs but I'm sure I have something to add there as well.

Maybe there should be some extra brainstorming before we start (I think we're all blocked by other plans for the time being anyway, so let's talk at least). For instance, if we know what information to display in the screen (I'd love optional AI trade values for debug purposes) we can better plan what to expose from the DLL.

Besides solving the technical problems (which sounds more feasible than I thought now that I've read Nightinggale's and Platyping's posts), we should also come up with a plan what to use our newfound power for once we have it. While it is much easier to mod the diplo screen once it has been uncoupled from the EXE, providing the most important information already would help the community get the most out of it right from the start.

I need to do some catching up on how all of this is wired together before I could say anything substantial, but wanted to open the discussion already regardless.

 

[isenchine]

Interesting project: New Diplomacy! ND or better Andy!

First question: how would "Andy" launch the animation of the LeaderHeads and the music accompanying them? Is this easy to do? dll/python?

 

[Dancing Hoskuld]

I have not done music but leaderheads can be done in Python. But really what you do in Python is explicitly ask the exe to do stuff.

 

[Leoreth]

As Platyping pointed out, animated LHs have already been done in the civilopedia.

Music is a more difficult issue, the last time I wanted to mod it I gave up and decided to work around the EXE. In particular, suppressing the usual background music and continuing where we left off after leaving the diplo screen could be complicated.

 

[Nightinggale]

I thought a bit about music since my last post, but before I read this thread. I recall something about the dll being able to turn music on/off, but that's it. The rest is in the exe. I haven't tried using it and don't know if it will remember music position.

I wonder if it would be possible and/or a good idea to disable music in the exe and then add music to the dll. I wrote some code at one point where the dll can figure out where it is and open files relative to its own position. Join that with some mp3 playing library and we could have a dll based music player, which we can control 100%. However I do not consider music to be important compared to the diplo screen itself. If we get a fully modable diplo screen, which is ignored by game music, then I would still be happy.

 

[platyping]

Drawing out the whole Diplo Screen is nothing daunting to me since I pretty much wrote my own pedia and WB.

What matters is the way to transmit data to
1) obtain the right response
2) ensure the trade result is MP compatible and not result in OOS
3) update necessary memory, such as "Favorable Trades"

This is unlike WB which is SP only, so MP was never a concern, or the pedia which is purely UI since it only affects the user.

The result from pressing a button is defined by the WidgetType, iData1 and iData2.
For default Civ Widgets such as adding specialists in City Screens, there are specialized widgets already done.
For all other self made buttons by mods, usually we use WIDGET_PYTHON or WIDGET_GENERAL and manipulate iData1 and iData2 to get our desired results.

 

[Leoreth]

If we have to compromise that's not to be avoided, but personally I think background music is important to the diplo screen. It helps a lot in setting both the mood and establishing the character of the leaders you're dealing with. It's just more fun (or not) to hear Montezuma's theme half a second before his LH is even rendered and already know what to expect. Restarting the background music after the encounter would be acceptable though.

 

[Nightinggale]

I agree that the music is important for creating the right "feeling" while playing. However to make this realistic, I think it would be best to make it a two step process. Step one is to make the screen work. Step two would be non-critical parts, like music. We can get something useful if we get part one only, but getting part two while skipping out on part one is kind of pointless. That's my main reason for thinking music can wait.

OOS safe design
We make a new class in C++. Let's call it diploScreen. We expose it to python.
Opening the window will allocate one instance of diploScreen and give the pointer to python.
Python can read from everything (read only access to data is always OOS safe)
Python can write to diploScreen.
Python will not write anywhere else. That should ensure that python can't cause OOS, only the C++ code. Also no python memory management.

On window close, python should trigger some diploScreen function, possibly freeing the instance.

In C++, diploScreen can read from everywhere.
If diploScreen needs to write to anything, then it should go through the network layer. No exceptions.
diploScreen writing to itself should not go through the network layer.

I think this design should do it. It has a few blocks of code, each with a defined set of requirements and purposes. The blocks would be like one for network communication, one for UI, one for setting up python interface etc. This allows different people to code different blocks. Even better, the level of abstraction mean the skills required to make each block is greatly reduced compared to the skills required to make all blocks. From what I can tell, for each block we will have at least one modder, who says "I know how to do that".

Granted, the plan could do with being more detailed, but as an overall concept, I think this is the way to go.

 

[platyping]

Sample

Drawing the UI is the simplest part.

 

[isenchine]

Way ahead of schedule!...

 

[Nightinggale]

Drawing the UI is the simplest part.

But can you control the drawn attitude? My thinking regarding copying the pedia code is that the LH would not be happy or angry because pedia is more neutral.

Also I need to figure out how to download platyping's python knowledge. It's very useful and I don't have anything that good.

 

[platyping]

VOID addLeaderheadGFC (STRING szName, INT eWho, INT eInitAttitude, INT iX, INT iY, INT iWidth, INT iHeight, WidgetType eWidget, INT iData1, INT iData2)

Attitude towards player can be retrieved easily. The pedia fixed it at Pleased, though there are commands to change it.

Knowledge comes from hands on experience, I guess. I started off making simple wonders and traits based on some works from previous players like GIR and tsentom1.

Then one day stolenrays requested for a better WB, so I did some trials and errors.
I didn't like Sevopedia so I decided to make my own.

You have been modding the dll much, so naturally your expertise is there. With your knowledge of c++, python should be easy to master since the civ specific python commands are defined within the dll, so you should be able to know what each function does exactly.

The only way to get better at something is to keep doing it.

 

[Leoreth]

I agree that the music is important for creating the right "feeling" while playing. However to make this realistic, I think it would be best to make it a two step process. Step one is to make the screen work. Step two would be non-critical parts, like music. We can get something useful if we get part one only, but getting part two while skipping out on part one is kind of pointless. That's my main reason for thinking music can wait.

Agreed.

OOS safe design
We make a new class in C++. Let's call it diploScreen. We expose it to python.
Opening the window will allocate one instance of diploScreen and give the pointer to python.
Python can read from everything (read only access to data is always OOS safe)
Python can write to diploScreen.
Python will not write anywhere else. That should ensure that python can't cause OOS, only the C++ code. Also no python memory management.

On window close, python should trigger some diploScreen function, possibly freeing the instance.

In C++, diploScreen can read from everywhere.
If diploScreen needs to write to anything, then it should go through the network layer. No exceptions.
diploScreen writing to itself should not go through the network layer.

I think this design should do it. It has a few blocks of code, each with a defined set of requirements and purposes. The blocks would be like one for network communication, one for UI, one for setting up python interface etc. This allows different people to code different blocks. Even better, the level of abstraction mean the skills required to make each block is greatly reduced compared to the skills required to make all blocks. From what I can tell, for each block we will have at least one modder, who says "I know how to do that".

Granted, the plan could do with being more detailed, but as an overall concept, I think this is the way to go.

So, I understand you correctly that the diplo screen instance would basically only keep track of what is currently on the table, and once a deal is agreed on actually implement it by writing to the right player/game instances through the network layer? If so, that sounds like a solid concept.

Sample

Drawing the UI is the simplest part.

Great!

 

[Nightinggale]

You have been modding the dll much, so naturally your expertise is there. With your knowledge of c++, python should be easy to master since the civ specific python commands are defined within the dll, so you should be able to know what each function does exactly.

The thing is, my expertise was with the dll prior to even starting modding and I was never a novice modder. Checking the log for RaR (the first mod I committed to), my first commit was on the 13th of May 2013. On the 18th, I fixed a cause of OOS I located by code reading. Not playing, but just reading the code. The thing is, the first time I was involved in fixing an OOS issue was in 2004. On top of years of experience, I'm also an engineer, which mean learning how to mod is mainly learning "how did Firaxis decide to do this" and not as much the technical issues. It's the same with UI. The issue is mainly figuring out what functions Firaxis added and what we can do with them. Python itself is not really an issue and I have done some nice stuff already, like the M:C domestic advisor, which adds columns as the screen resolution gets higher in exchange for less "subpages". The issue really is figuring out the vanilla functions, especially those hidden in the exe.

I know of the python function list and look at it once in a while. However knowing the function isn't enough and it's not like it's really documented beyond providing the name and arguments. The comment about downloading knowledge of the functions is mostly a joke at the cost of my own "memory leak" regarding remembering those python functions.

I only have two issues with python. One is performance, the other is lack of debugger. There are hints of python debugging in a few places, but I haven't been able to get it to work. The DLL on the other hand has full support for both debugger and asserts, making it easier to get stable, reliable and trustworthy.

So, I understand you correctly that the diplo screen instance would basically only keep track of what is currently on the table, and once a deal is agreed on actually implement it by writing to the right player/game instances through the network layer? If so, that sounds like a solid concept.

Yeah, that's pretty much it. Python should just draw what is in the instance and forward clicks. Either it should redraw itself or it should be requested to do so (not sure).

The C++ code can also be divided into sections to greatly simplify the task for each section of code. Also with a well defined interface/API between each section, bug hunting can start with verifying the data in the interface as it will tell which section is causing the wrong data. This mean a well divided design is easier to write and test.

 

[platyping]

Sample 2

 

[oldnooob]

This is a very interesting discussion! I came looking for a modder willing to mod the diplo screen and I have found a whole team!
What I would ask is the creation of a "third option" besides the WHITE "I Will" and the RED " I won't even discuss it" lines.
How about a YELLOW, "I might be willing to consider this if the price is right" line?
I hope this is what you guys are talking about. IF I am incorrect then is my suggestion even possible?

 

[Nightinggale]

What I would ask is the creation of a "third option" besides the WHITE "I Will" and the RED " I won't even discuss it" lines.
How about a YELLOW, "I might be willing to consider this if the price is right" line?

The top priority should be to get the vanilla screen behavior to work in a way where we have 100% of the code. Once we have that as a working code, extra options like this one can be considered.

Adding a new feature like this one would be the logical next step. The UI shouldn't be an issue at all. However it needs some game logic code to function and I can't say offhand how much. Still not knowing the size of a task has never stopped me before.

 

[oldnooob]

Well, for me it will be a dream come true!

I have been playing civ since it was first released way before the turn of the century (I am 72) and I could never figure out why no one , until now, has ever attempted to correct this glaring omission in this fine series of games.

 

[vincentz]

I also wanted to add a third answer, though different: "Never!" with extra negative diplohit, but +1 with 3rd party in mention. I looked through the code, and while it's a huge (in my world) project, I think it's doable even with the vanilla diplo screen.

 

[Dancing Hoskuld]

The three things I wanted to "fix" with the screen were

1. The civics at the top get squashed if here are more than the standard 6 civics, so I thought that having them in a scrollable box like happens in the pedia would be better.

2. It would be nice if you could toggle on and off the trades that they wont do.

3. If you collapse a category of trade it opens again as soon as you do a trade requiring you to collapse it again.​