Nightinggale
Deity
- Joined
- Feb 2, 2009
- Messages
- 5,281
The setup
The game consists of 3 parts: exe, dll and python. Each of those can call functions in the other two, either directly or indirectly.
The problem is not making such calls, but rather mod code, which is called from somewhere else. The biggest issue is modding a dll function, which is called from the exe. If a call goes wrong, the game can crash seemingly randomly or other really weird stuff can occur.
How the exe calls the dll
There are two approaches. One is that the exe has access to anything using the DllExport keyword. This will allow the exe to call using standard C++ calls. The other is calling virtual functions even if they do not have the DllExport keyword.
Very simplified, a function call in C++ (and other compiled languages because it's really about how the CPU works) is done by placing the arguments in registers. A register is a tiny memory inside the CPU and it can hold a single int (32 bit). For simplicity let's say arguments are placed in registers numbered from 1 and forget that it can use the stack too.
Say the function takes one argument. The caller places the argument in register 1. The called function will then assume the variable to be present in register 1. If there is a second argument, the same thing happens with register 2 and so on.
The problem
If a modder change what an argument is or adds an argument, then nobody will tell the exe. If the dll assumes 2 arguments and the exe only adds a variable to register 1, the dll will get the value of the second variable to be whatever was left in register 2 from the last time it was used. This will introduce corrupted data and then all sorts of weird stuff can happen.
Default values will not help because they won't do anything at runtime. If argument 2 is default to 0, then it is a message to the compiler to make the caller place 0 in register 2 while the function itself will not change. Since the exe will not know this, it will not place the default value, hence it will still contain corrupted data.
How to fix the exe from providing garbage arguments
You can't. We are stuck with the function calls the exe makes and we can't change the arguments. We can move the contents of a function into a new function without DllExport and then make the original call the new function. That way we have some sort of "buffer" in the dll where default values can be added.
Which functions will be called from the exe
Ideally it should be any function with DllExport. However vanilla added this to too many functions and over the years modders have added it for new functions as well, apparently because "vanilla works, so copy vanilla even if I don't know what it does".
I attached a list of functions called by the two exe files.
The way to identify the virtual functions called by the exe is.... who knows? Apparently the exe can call say the 4th virtual function in CvPlayer and then it just assumes it's the vanilla function for arguments. No known list of used virtual functions exist.
How to remove the extra DllExport keywords
You can go through the lists, but it would likely be easier to go to the source of the lists. I copy pasted output from dependency walker (DP). You can just download and use that application as well.
The easiest approach would likely to copy the BTS (or Colonization) exe files and dll files into Assets in your mod. Obviously you are using git, meaning you can easily remove those extra files later and there is no need to commit them as they will only bloat your repository.
Open DP, open the exe in Assets. In the left menu is the dll files it can call and the mod file is last in the list. Click it. To the top right will now be a list of functions it can call. First column is a bunch of green icons. Click on the column header once and it will sort according to that list.
Now remove all DllExport from a file and compile. Close the file in DP (file menu) and open the exe again. Select the mod dll again. You will now have red lines on top (because it's sorted). Adding DllExport will fix the red icon and make it go green again. Fix all red icons.
If you do this for all files, then you should end up knowing that you are free to do whatever you want with arguments for any function not using the DllExport keyword while those with the keyword are all hardcoded to "do not touch arguments".
DP will also be able to check if the arguments have been modded, meaning it can help catch already introduced bugs.
The game consists of 3 parts: exe, dll and python. Each of those can call functions in the other two, either directly or indirectly.
The problem is not making such calls, but rather mod code, which is called from somewhere else. The biggest issue is modding a dll function, which is called from the exe. If a call goes wrong, the game can crash seemingly randomly or other really weird stuff can occur.
How the exe calls the dll
There are two approaches. One is that the exe has access to anything using the DllExport keyword. This will allow the exe to call using standard C++ calls. The other is calling virtual functions even if they do not have the DllExport keyword.
Very simplified, a function call in C++ (and other compiled languages because it's really about how the CPU works) is done by placing the arguments in registers. A register is a tiny memory inside the CPU and it can hold a single int (32 bit). For simplicity let's say arguments are placed in registers numbered from 1 and forget that it can use the stack too.
Say the function takes one argument. The caller places the argument in register 1. The called function will then assume the variable to be present in register 1. If there is a second argument, the same thing happens with register 2 and so on.
The problem
If a modder change what an argument is or adds an argument, then nobody will tell the exe. If the dll assumes 2 arguments and the exe only adds a variable to register 1, the dll will get the value of the second variable to be whatever was left in register 2 from the last time it was used. This will introduce corrupted data and then all sorts of weird stuff can happen.
Default values will not help because they won't do anything at runtime. If argument 2 is default to 0, then it is a message to the compiler to make the caller place 0 in register 2 while the function itself will not change. Since the exe will not know this, it will not place the default value, hence it will still contain corrupted data.
How to fix the exe from providing garbage arguments
You can't. We are stuck with the function calls the exe makes and we can't change the arguments. We can move the contents of a function into a new function without DllExport and then make the original call the new function. That way we have some sort of "buffer" in the dll where default values can be added.
Which functions will be called from the exe
Ideally it should be any function with DllExport. However vanilla added this to too many functions and over the years modders have added it for new functions as well, apparently because "vanilla works, so copy vanilla even if I don't know what it does".
I attached a list of functions called by the two exe files.
The way to identify the virtual functions called by the exe is.... who knows? Apparently the exe can call say the 4th virtual function in CvPlayer and then it just assumes it's the vanilla function for arguments. No known list of used virtual functions exist.
How to remove the extra DllExport keywords
You can go through the lists, but it would likely be easier to go to the source of the lists. I copy pasted output from dependency walker (DP). You can just download and use that application as well.
The easiest approach would likely to copy the BTS (or Colonization) exe files and dll files into Assets in your mod. Obviously you are using git, meaning you can easily remove those extra files later and there is no need to commit them as they will only bloat your repository.
Open DP, open the exe in Assets. In the left menu is the dll files it can call and the mod file is last in the list. Click it. To the top right will now be a list of functions it can call. First column is a bunch of green icons. Click on the column header once and it will sort according to that list.
Now remove all DllExport from a file and compile. Close the file in DP (file menu) and open the exe again. Select the mod dll again. You will now have red lines on top (because it's sorted). Adding DllExport will fix the red icon and make it go green again. Fix all red icons.
If you do this for all files, then you should end up knowing that you are free to do whatever you want with arguments for any function not using the DllExport keyword while those with the keyword are all hardcoded to "do not touch arguments".
DP will also be able to check if the arguments have been modded, meaning it can help catch already introduced bugs.
