Unless there is some incompatibility with the old version of Python that Civilization IV uses, I should be able to add python debugging support to MNAI. This would allow us to do step by step debugging (with breakpoints, memory inspection and so on) of everything python related in the game. Would anyone be interested on adding this to debug builds for development?
Python debugging
- Thread starter Terkhen
- Start date
black_imperator
Deity
- Joined
- Dec 4, 2005
- Messages
- 2,014
that's something i'd be interested in for AoE, if possible.
Ah, sure! I will prepare it as a standalone modcomp; it should not be complicated to keep mod-agnostic 
After investigating for a bit, I noticed that Civilization IV should already have support for HAPDebugger (http://hapdebugger.sourceforge.net/). It can be enabled by setting the HAPDebugger option to 1 in CivilizationIV.ini. But this tool does not have support for any Python IDEs besides its own. Also, HAP needs to have a remote debugging host (ConsoleEmbed) into the application to be debugged. Since I have not found any reference to this remote in the code, it is probably inside of the binaries, outdated and out of our control.
Because of these reasons, I will try with PyDev instead. This should work on Eclipse (free) and IDEs from the PyCharm family (not free) and in theory it should not be very complicated to set up. I will post again after I try it.
After investigating for a bit, I noticed that Civilization IV should already have support for HAPDebugger (http://hapdebugger.sourceforge.net/). It can be enabled by setting the HAPDebugger option to 1 in CivilizationIV.ini. But this tool does not have support for any Python IDEs besides its own. Also, HAP needs to have a remote debugging host (ConsoleEmbed) into the application to be debugged. Since I have not found any reference to this remote in the code, it is probably inside of the binaries, outdated and out of our control.
Because of these reasons, I will try with PyDev instead. This should work on Eclipse (free) and IDEs from the PyCharm family (not free) and in theory it should not be very complicated to set up. I will post again after I try it.
Yesterday I managed to find a way to force Civilization IV to load pydevd's modules with the module path that pydevd is expecting them to have. I need to add a manual override for each module so it will take a while to get all of them working, but it is doable.
I stumbled upon a more serious problem, though. For some reason, the standard module select is not present in the list of loaded python modules of the game, so pydevd fails when it tries to import it. I have seen other standard modules like threading, so I wonder why this one is not there. Next time I will try to find either a way to load it, to add it manually, or find a workaround (a mocked module that does the same thing with synchronous, maybe).
I stumbled upon a more serious problem, though. For some reason, the standard module select is not present in the list of loaded python modules of the game, so pydevd fails when it tries to import it. I have seen other standard modules like threading, so I wonder why this one is not there. Next time I will try to find either a way to load it, to add it manually, or find a workaround (a mocked module that does the same thing with synchronous, maybe).
I have not been able to find a workaround for the missing select module. It provides functionality for asynchronous network communication, which we really cannot mock because pydevd will make the game stop completely if the communication between them is not asynchronous.
Does anyone know if it is possible to recompile the python library and make the game accept it, and if so how? That's really the only option remaining for making this work at this moment
Does anyone know if it is possible to recompile the python library and make the game accept it, and if so how? That's really the only option remaining for making this work at this moment
Yes, but my main concern is if the game will accept it or not after I compile it. I do not know the deployment they used, what environment they used to compile it, if the library needs to be compiled in a specific way to make it compatible with the C++ interpreter included in boost, and so on. I do not want to spend a lot of time doing trial and error in this. Do you know more details about this process besides the version? I have been trying to find more about the python version included in Civilization IV but I didn't find anything interesting.
Edit: Looking at your posting history I found this forum thread (http://forums.civfanatics.com/showthread.php?t=484205) which mentions that it is possible to just recompile python. I will give it a try and see if the select method is included
The thread itself is algo quite interesting
I managed to compile the python dll and the select module (which apparently is compiled as a separate pyd file which is just a renamed dll). Sadly, the game is not able to find the select module with that version either. Maybe it is a limitation of boost-python?
Similar threads
