How to debug python for civ4?

[mikezang]

I downloaded and installed Python 2.4, and I modified some civ4's python code, then I run civ4 to check if my modified is correct, this wasted a lot of time, how can I debug civ4 python code without run civ4?

 

[naf4ever]

If your getting errors on startup you can see what line and where they are occurring at in the Logs directory for civ 4. Check the python error text files.

Otherwise if you're not getting errors on startup but your mod isnt working start or load up a game with your mod. Then hit the ~ key to bring up the console. In it type: game.toggledebugmode then hit enter. Then the error messages in your python will pop up if and when they occur.

 

[Locutus]

how can I debug civ4 python code without run civ4?

By and large you can't. Civ4 needs to run because all the stuff from the CvPythonExtensions API can only loaded with Civ4, you can't run it seperately. That said, there are a few options available to you that can make your work easier.

1) Hotswapping. Civ4 allows you to hotswap Python code. This means that you can run Civ4 and your an editor with your Python code simultaneously and Alt-Tab between them. Whenever you change your code in the editor and save it, Civ4 will immediately reload the file, so you can instantly test the changes without having to restart the game.

2) Console. Civ4 has a Python console, this allows you to test and change the values of variables and the like while the game is running, a great debugging tool.

3) Syntax checking. If you have Python installed seperately of Civ4, you can let it check the syntax of your code without having to run the game, this can avoid errors while loading the game. Following is a slightly modified excerpt from a Python manual I'm working on:

To do this, type at the command line (in Windows: Start > Run… > type command and click OK):

C:\Python24\python -c "import py_compile; py_compile.compile('<fileName>')"

Where <fileName> is the name of the file you want to check. When you do this, Python will compile your script to byte code. Now, byte code is not terribly useful when using Python for Civ4, but if the compiler runs into a syntax error in the process of creating this byte code, it will terminate immediately and report the error. This process only takes a fraction of a second, so it's much faster than starting up Civ4 to see if your code is correct.

Of course, now you still have to type in a rather long and complex command every time you want to check the syntax of a file. So, why not let Python do that for you? Create a file called syntaxCheck.py (or whatever you want to call it) and paste the following code in there:

import os
import py_compile

strDir = []
for strFilePath, strDirList, strFileList in os.walk('.'):
	for strFileName in strFileList:
		strDir.append(os.path.join(strFilePath, strFileName))

[py_compile.compile(strEntry) for strEntry in strDir if (os.path.isfile(strEntry) and strEntry[-3:] == '.py')]

Create another file called sc.bat and paste the following in it:

@echo off
C:\Python24\python syntaxCheck.py
pause

Save both files and put a copy of them in every Assets\Python folder that you want to be able to syntax-check quickly. Now when you double-click sc.bat, Python will try to compile all files with the extention .py in the folder in question and in all its subfolders. A shell window will open and list the first syntax error Python encountered. If no errors are encountered, it&#8217;ll just prompt to you press a key to continue and you&#8217;ll know your code is free of syntax errors. Keep in mind that Python will only return the first error it encounters, then it terminates. So after fixing the problem you&#8217;ll have to run sc.bat again to make sure there are no other errors.

Note that this code will leave a .pyc file in your folders for every .py file it encounters and successfully compiles. These .pyc files are of no use to you, you can delete them if you want (though there&#8217;s no harm in keeping them either, apart from cluttering your folders).

If you have an advanced editor, you can probably use the external tools functionality of that editor to simplify this process by doing the compiling directly from within the editor. For example, in my own editor EditPlus, I can go to Tools > Configure User Tools > Add Tool > Program and then set C:\Python24\python as command, -c "import py_compile; py_compile.compile('$(FileName)')" as argument and $(FileDir) as Initial Directory. These details vary per editor though, so if you use something else check the help info of your editor for details. Note that you should use the command line code above as basis for the editor settings (as opposed to the syntaxCheck.py or sc.bat file).

Better yet, if you use a Python-specific editor it may even have the ability to check code built in already. For example if you use IDLE, which by default comes with most Python distributions, you can load the file into IDLE and from the menu Run select Check Module. The program will then basically compile the file and automatically delete the .pyc file when it's done. It will even automatically take you to the line that caused an error if it encounters one.

But remember, these methods detect syntax errors and syntax errors only, so run-time and semantic errors are not detected. Those can only be found by loading Civ4 and running the game.

 

[12monkeys]

1) Hotswapping. Civ4 allows you to hotswap Python code. This means that you can run Civ4 and your an editor with your Python code simultaneously and Alt-Tab between them. Whenever you change your code in the editor and save it, Civ4 will immediately reload the file, so you can instantly test the changes without having to restart the game.

There some restriction. Not all modules can be changed while Civ4 is running. Any changes done with hotswapping in the CvMainInterface.py for example will lead to a cascade of resulting errors.

Syntax checking. If you have Python installed seperately of Civ4, you can let it check the syntax of your code without having to run the game, this can avoid errors while loading the game. Following is a slightly modified excerpt from a Python manual I'm working on:

To do this, type at the command line (in Windows: Start > Run… > type command and click OK):

C:\Python24\python -c "import py_compile; py_compile.compile('<fileName>')"
...

This is useful!! Have to check that at home.

 

[Locutus]

There some restriction. Not all modules can be changed while Civ4 is running. Any changes done with hotswapping in the CvMainInterface.py for example will lead to a cascade of resulting errors.

Well, you can hotswap it, but if you make an error you will indeed get a cascade of errors, because the interface is refreshed like 30 times per second. But if you just alt-tab out real quick and fix the problem the number of errors you get is not so high that you can't click through them. Still, you're not wrong, if you make two mistakes in a row or you wait too long to alt-tab out, you might as well reboot. But that's only CvMainInterface.py in my experience, and as long as you avoid syntax and runtime errors (i.o.w. double-check everything you type and use the above trick to check syntax), it's not really an issue...

 

[The Great Apple]

I found that if you mess up modding CvMainInterface.py, you just have to alt-tab in and out a bit after fixing and it works it out. I usually add a little bit of code in my events manager which sets the interface dirty on window activation, which seems to work sometimes as well.

CvTechChooser.py is a pain though - the way it works means the techs are only placed once, and then the changes recorded (and changed). If you alt-tab out it clears the list of techs, and you almost always get "list index out of range". Simple fix though - just put screen.setPersistent(False) at the start of the code when editting.

 

[mikezang]

Thanks for all your suggestion.
I found there is a method enableGridlines(...) in class CyGInterfaceScreen, how can I use this to draw grid when I load a saved game or start a new game? In other word, can I simulate the minimap button's event to toggle resources, yield, grid on when starting game?

 

[mikezang]

2) Console. Civ4 has a Python console, this allows you to test and change the values of variables and the like while the game is running, a great debugging tool.

How do I use console? may I need to set HAPDebugger = 1?

 

[Locutus]

Set CheatCode = chipotle IIRC (I have a debug build so I don't have to do it myself).

 

[surt]

From my config:

; Move along
CheatCode = chipotle

Make sure there is only one instance, civiv's config writer occassionally screws up and adds multiple copies of things.
Once you have set this, you can get the debugger console with ~ (shift-` on american keyboards) regular ` gets you the game console.

The other one is:

; Establish connection to Python Debugger
HAPDebugger = 1

If you set this to one, civiv will freeze at 'python loading' and wait for you to start the hap debugger and connect to it. With the hapdebugger you can actually do runtime debugging (see & modify variables on the fly, set breakpoints, etc).

learn about it and download from here:
http://hapdebugger.sourceforge.net/

Open question for locutus or any firaxis person: does anybody know if it is possible to get pychecker or some other lint tool to load and work with civiv's python? I made a very hackish workaround for this, but I would love to be able to lint check my work more easily.

 

[mikezang]

learn about it and download from here:
http://hapdebugger.sourceforge.net/

I already downloaded this tool, but there is no any info about how to use HAP Debugger at there homepage, do you know where I can get document about HAP Debugger?

 

[surt]

I already downloaded this tool, but there is no any info about how to use HAP Debugger at there homepage, do you know where I can get document about HAP Debugger?

I guess there's not really a lot. They have a forums
https://sourceforge.net/forum/?group_id=43297

and a very brief faq
http://hapdebugger.sourceforge.net/faq.html

Not much else. It isn't terribly complicated to use though, pretty standard debugger. Just run civiv in window mode if you're using it.