TGA's Python Tutorial

The Great Apple · Jan 15, 2006

Introduction
Python can be used to achieve a suprising amount of things in Civ 4, but only if you know how. Now, while I'm probably not the best person to be writing this tutorial, I feel I am quite familier with the sort of things you can do to the game with it.

If you want to know a little bit about python, I'd check out: http://en.wikipedia.org/wiki/Python_programming_language and www.python.org

Just a brief rundown on what python is capable of in Civ 4 terms:

Python can:

be used to make complex scripts causing events to happen on a trigger. For example, you can make it so that if you take the holy city of a religion, all Civs following that religion will declare war on you.
be used to edit the interface. Pretty much all of the game interface is generated dynamically by python, and can be editted in python. You can make new buttons, alter the layout of the Civelopedia, change the way the tech tree is generated... and even, if you so desire make a small dancing pig in the corner of your screen - with music (though I don't believe anybody has tried this, I'm pretty sure it's doable). There are limitation to this - mouseover texts, for example, are far better suited to the SDK.
be used to generate new map types. I haven't tried this myself, so it might not be covered too well in this tutorial.
be used to edit the AI... I'd do it with the SDK though, as it'll probably save alot of time!

Python can't:

be used exactly how you might want to use it. While the amount of things you can do in it is really quite large, there are several things you just can't do. This is really quite annoying, but it's something you've got to learn to live with. The amount of things you can do is staggering, so don't complain. Hopefully with the release of the SDK more experienced coders will enhance the ability of python to change the way the game works.

Still reading? Good!

Now my suggestion for first steps would be to browse through GBM's python tutorial, which gives a brief outline of how to program in python. Also, if you have not done so, check out the python website which has lots of information about python (unsuprisingly). Finally, I'd check out this python tutorial by Jon (Trip) Shafer, Firaxis, who's tuturial you may find better than mine, after all, he is the guy who coded alot of it in the first place.

Okies, so after all that I can finally start!

Contents:

Page 1: Introduction (you're on it)
Page 2: The basics
Page 3: Using the API
Page 4: Event scripting & Interface mods
Page 5: Debugging
Page 6: Misc
Page 7: Conclusion

I'd very much appriciate comments with other people attempting python in Civ 4. I'm also open to any questions anybody may have.

This tutorial is also on Civ4Wiki.com. If you have anything you don't like about it, feel free to change it and I'll try to update my post here.

The Great Apple · Jan 15, 2006

The basics
As you should know by now, there are several different types of "thing" you can get in python: ints, floats, strings, and bools. You also might be aware of "pointers". I wasn't when I first came to python, so I'll give you a little heads-up about them.

Basically a pointer is a game entity. It can take the form of a unit, a plot (square), or a player. In the default python code, and in many modder's code, a variable is shown to be a pointer by prefacing it with "p". So pUnit will define a single unit in the game, pPlayer a single player.

Likewise, if a variable is prefaced with an "i" it is likely to be an integer (int) variable, and "b" a boolean (bool) variable. iPlayer, for example, will probably be a player's individual ID number, rather then the pointer for the player.

It does sometimes get confusing, as most pointers have integers which are closely related to them. This method of naming things helps to avoid confusion in a language like python where variables need not be declared, however it is only a guideline, and just because a varaible is named as if it were a certain type of variable, it won't neccessarily be one.

I also would like to put a note here about "Types". In most xml entries you will need to enter something into a type field. This field is a very important link to python, though must be converted to an int for it to work properly. There is one function that you'll be finding yourself using over and over again, so I thought I'd put it here. This fucntion will find the int corresponding to the type - and as most functions in the API (soon to come) will want the int value of the type it's very useful. For example:

Code:

gc.getInfoTypeForString("TECH_MYSTICISM")

will return the int corresponding to the tech Mysticism, which, as it is the first entry in the relevant xml file will be zero by default.

Code:

gc.getInfoTypeForString("RELIGION_TAOISM")

will return the int 6 by default, as it is the 7th religion listed.

If you don't get that last bit right now, don't worry - hopefully it'll come clear when you read the next section.

The Great Apple · Jan 15, 2006

The API
The Civ 4 API is a list of all the functions specific to Civ 4 which interface with the actual game. There are two copies of the API. The first, by Locutus can be found Moderator Action: *snip* invalid URL , and the second, by GBM can be found Moderator Action: *snip* invalid URL . Locutus' API is up to date with the latest patch, so I suggest you use that one.
Moderator Action: ATTENTION, current API see here.

Now, in the past month or so answering questions on the forums about python I have noticed that quite a few people have had trouble reading the APIs. Interpreting what the functions do can be quite difficult, however the APIs do give you lots of hints, if you care to read them.

I'll use Locutus' API as an example, so Moderator Action: *snip* invalid URL !

Firstly, we have the classes. These list all the functions that can be done to a specific entity, or pointer. For example, CyUnit lists all the funtions that will work on a unit entity (a unit in the game). The classes are listed in the top-left hand frame. You will notice that there are several classes with a little "+" symbol by them. These classes are used for getting specifc information directly from the xml files on an object.

Now we have the types. These the different types which can be extracted from the xml files. The int values shown are the default values, and while you could use them, and your code would most likely work, I would recommend against it, as if the order of the xml is changed in a future patch, you're code will break. Instead I recommend using gc.getInfoTypeForString("...") as shown above, as this is much more patch-compatable.

Finally we have the functions. Each class has it's own set of fuctions which must be done to that class and that class only. If you try to take a function from a class, and use it on a different class it won't work. The functions are the main bit of the API, the stuff you want to know.

Functions

Now, there are lots of different functions. Most of the functions don't "do" anything in the game, instead they get a value from your pointer which you can use in an equation. If you look on the left hand side of the function, it will show you what this function returns. For example, if you used:

Code:

 BOOL CyPlayer.canChangeReligion()

it would not set the player able to change religion, instead it would return a BOOL, a true or false value depending on whether the player can or cannot change religion. See what I mean about it not "doing" anything? (note: to actually use this function you would have to replace CyPlayer with a player pointer, pPlayer)

Things a function can return are:

Bool - true or false, 1 or 0
Int - an integer value (can be negative) (256 for example)
Float - a floating point number (1.3423 for example)
String - a string ("The cat sat on the mat")
Turple - a list ([1, 4, 2, 8, 2])
Void - see below

The funtions that do "do" things return a "VOID" - meaning that they setting rather than getting data. These functions are the most important ones really, as you can't do much without them, and unfortunetely there aren't really enough of them to do everything you would want. Sorry, but that's just the way it is.

So, you know what a fuction returns, but that's not all you need to know. You also need to know what information to give it to return this data. Some functions don't require any inputs.

Code:

CyUnit.canMove()

for example doesn't need to know anything apart from which unit it's checking to see if can move (the unit pointer should be in the place of the CyUnit). However, most fucntions aren't this simple, and require an imput to create an output.

Lets take, for example:

Code:

CyUnit.setHasPromotion(PromotionType eIndex, BOOL bNewValue)

This function takes two arguments (inputs), the promotion type you want to give to the unit, and a bool value of weather to take, or remove the promotion. To give an example:

Code:

pUnit.setHasPromotion(gc.getInfoTypeForString("PROMOTION_COMBAT1"), 1)

would give the pointer, pUnit the promotion combat 1. Note that it is important to give the int value of the promotion in this case, as when the function asks for a type, it really wants an int... don't ask! As luck would have it, if you put the wrong arguments into a function python will tell you when trying to run the function. More about this in the debugging section!

CyGlobalContext

Commenly abbreivated to gc in files this is probably the most useful class, as it contains all the general Civ 4 related functions. GlobalContext is one of the few classes that doesn't have to be called with a pointer, along with CyGame and CyEngine, to name a few others.

It has a number of useful purposes:

As I have said before you can use it to get integers from types. For example:
Code:
```
gc.getInfoTypeForString("TECH_MYSTICISM")
```
would get the int value for Mysticism.
You can use it to get from an integer ID to a pointer. For example:
Code:
```
gc.getPlayer(0)
```
would return the pPlayer pointer for the the player with an ID of zero.
You can use it to get info from xml files. For example
Code:
```
gc.getPromotionInfo("PROMOTION_COMBAT1")
```
would return the pointer for the combat 1 promotion, which could then be used with CvPromotionInfo to get information about that specific promotion.
A commen usage is to get the active player pointer, which returns the pointer of the player on the current computer:
Code:
```
gc.getActivePlayer()
```
The final major use is to get the number of instances of a certain thing. For example:
Code:
```
gc.getNumPromotionInfos()
```
would return the number of promotions which are available in the game. This is very useful for if you want to cycle through the promotions and check if a unit has them, activating a new function if it does.

While there are other uses for CyGlobalContext, they are a bit to numerous to list here. I've tried to list what I think are the major ones.

The Great Apple · Jan 15, 2006

Events

Events are things which occur due to certain triggers. There is only certain amount of fixed triggers, and this ties down modders slightly, however the triggers will accomedate most scripts. If you open up CvEventManager.py, the event triggers run from line 193 down to line 747 (patch 1.52). There are nearly 60 different triggers that can be used, ranging from onUpdate which runs every update (meant to be 4 times a second), to onGameStart which runs only once, at the start of the game.

Most triggers have arguments associated with them. These arguments are stored in the argsList. Under most event headers these arguments are defined. It's usually pretty obvious what they do.

Perhaps the simplest method of adding events on triggers is to just to add the event into the events manager. This will work for small mods, although it raises certain compatability issues. A better way of doing it is to create your own CvCustomEventManager file. I recommend Dr Elmer Jiggle's event manager (found here). It's a bit tricky to understand at first, but well worth it if you can get it working. For examples of it in action I'd check out any of TheLopez's ModComps.

Things to note:
The onEndPlayerTurn tigger does not occur at the end of a player's turn, instead it occurs at the end of the start of a player's turn, after all the cities have built, etc. If you want to do an event on the end of a player's turn you'll probably have to bodge up some code to do it at the start of the next player's turn. It would appear something similar goes on with onBeginGameTurn, and onEndGameTurn, although both occur at the END of the game turn rather than at the beginning like the player ones. Thanks to Kael for spotting this.

Example:
If, for instance, you wanted a message to come up on every player's screen at the start of every game turn (in other words, at the start of the first player's turn), you could replace this code in the event manager:

Code:

def onBeginGameTurn(self, argsList):
	'Called at the beginning of the end of each turn'
	iGameTurn = argsList[0]
	CvTopCivs.CvTopCivs().turnChecker(iGameTurn)

with the following code:

Code:

def onBeginGameTurn(self, argsList):
	'Called at the beginning of the end of each turn'
	iGameTurn = argsList[0]
	CyInterface().addImmediateMessage("You have just started a new turn", "") # Adds the message "You have just started a new turn" with no sound attached.
	CvTopCivs.CvTopCivs().turnChecker(iGameTurn)

NOTE: When making a formal mod you shouldn't simply add stuff to the event manager like this as it can cause compatability issues. Check out the custom event manager linked to above.

Interface

As I mentioned in the introduction almost all of the GUI is created dynamically either when you enter the relavent screen, or at the start of the game. Just about all of it is moddable. The files for modding the interface are in .../Assets/Python/Screens.

Things to note:

When placing items in the interface that the x and y co-ords that you enter will correspond to the top left hand corner of the item.
Remember that the resolution can change. It may often be a good idea to change how your interface displays depending on the resolution. To find the X and Y resoultions you can use the code:
Code:
```
CyGInterfaceScreen.getXResolution()
```
and
Code:
```
CyGInterfaceScreen.getYResolution()
```
If an item needs a name, that name must be unique to that item, or strange things start to happen.
The way the techchooser script is written can make it quite hard to mod. Instead of being generated everytime you load up the screen, the techchooser is instead generated at the start of the game, and then only the colour changes are done in-game. To stop this happening in game, and to ensure that the techchooser shows the mods you make to it while you're game is running I suggest you comment out line 72 (screen.setPersistent(True)). This will make the techchooser be constantly regenrated while open, and although this may cause your computer to slow slightly, it should work better. Sometimes though, due to the way it is generated for techchooser mods you will have to restart the game to see any changes.

The Great Apple · Jan 15, 2006

Debugging

Everybody has to do debugging when their scripts don't work. It is so easy to make one little mistake, causing the whole code to become one big mess. The first step in debugging is to switch the in-game debugging options on. To do this, open up Civilization4.ini (make sure to back it up), and change the following entries to the following values:

Code:

HidePythonExceptions = 0
ShowPythonDebugMsgs = 1
LoggingEnabled = 1
OverwriteLogs = 1
SynchLog = 1
RandLog = 1
MessageLog = 1

This should enable in-game python popups, and cause all errors and messages to be printed in your My Documents/My Games/Civilization 4/Logs directory. The three important logs to look for when debugging are PythonDbg, PythonErr and PythonErr2.

Note: Civilization4.ini should be located within you My Documents/My Games/Civilization 4 directory. There is a shortcut to it in your main Civilization 4 directory also.

Common error messages and their causes:

<I would appriciate it if somebody could check over this section as I'm not certain as to all of it's accuracy, and cannot test it as I am away from my computer for the next few months>

In the logs usually the errors will have a traceback on them. This traceback lists all the files that are effected by the error, and the lines the error occur on. The lase line is usually the one you are interested in, as this is the one where the error occurs. Sometimes you'll have to go back a few steps as the error won't be on the last line, but usually it is!

If you come up with one I haven't covered - feel free to ask.

Syntax errors:

These are mainly typing errors - missed ":"s, incorrect whitespace, or bad bracketing. The game usually picks up on these when you load it up. The error logs should show you exactly where the error is in the syntax with a little ^ symbol underneath it.

NOTE: Sometimes the in-game error reporting doesn't work too well for syntax errors, and the error is in fact the line before the line it says is wrong. This is often caused by incorrect bracketing, or empty statements.

Argument errors:

If you try to use a function designed for a pointer on an integer value, you will get a type error. Go to the line specified and check that the function you're are using is valid. Usually it will say what type of function it is expecting, and what you've given it. NOTE: Some classes need () after them in functions, some don't. If you are getting a Type Error this may be the cause.

The arguments must also be of the same type (int, bool etc...) as the function is expecting, and in the same order that the function is expecting.

Type Errors:

These either occur when you are passing a function too many, or too few arguments. You have to make sure the amount of arguments you are passing a function matches the amount of arguments it takes, or the function will throw up an error when you try to run it.

Name Error:

This means that you've tried to use something that the game doesn't know what is. For example, if you were to type a = b without first saying what b was, there is no way the computer could set a vale to a. Remember a=b does not do the same as b=a. Also, make sure you haven't any "==" where you want "=", or vice versa.

Sometimes you'll get "Argument referanced before assignment". I don't know if this goes alongside the Name Error, if it needs it's own catagory, but basically it's like a name error, but you define "a" later on in your code.

"List index out of range"

This happens when you try and referance a index in a list, turple, or dictionary which doesn't exist. This one can be tough, and you might have to use methods described in the next part of this section to see exactly what is going wrong.

That's all the error messages I can think of right now. I'm pretty sure there are one or two I've forgotten - if you find any please feel free to ask and I'll include them here. As I said at the start I'm not 100% certain on the accuracy of these, but they should be ok.

Code not behaving:

So you've written your code. It all functions with no errors... but it doesn't actually produce the results you expect it to in the game. There can be several causes of this:

Firstly, if you think a piece of code should be having an effect in the game, make sure that it is by checking the API to see if it is a VOID function. It's quite a common mistake when first starting out to use other functions thinking that they will do something, when in fact they just retrieve information.

Secondly, if everything seems right you need to start putting debug messages in. This section brings you back right to the very start of python coding, when you were telling your computer to print "hello world". What you will need to do is to get the computer to print a message telling you what a certain value is at a certain point in the code, and compare it to what you think it should be. For example, the following code will print "a=b" in the debugging log if a=b, and print the values of a and b otherwise:

Code:

if a == b:
     print "a=b"
else:
     print a
     print b

This can be quite hard to decypher in the debug log (often you will have many many numbers), so you can do more advanced messages:

Code:

if a == b:
     print "a = b, a and b are %d, and %d"%(a,b))
else:
     print("a = %d"%(a))
     print("b = %d"%(b))

Where %d will referance the number outside the "" after the %. If you want to place a string there you have to use %s. This can give you detailed information as to exactly what is going on inside your code, and hopefully from that you should be able to see what is going wrong with it.

The Great Apple · Jan 15, 2006

Misc

This post contains other stuff about Civ 4 python that isn't really covered in the other posts, and is not obvious at first glance. Some of it is fan-made - creating shortcuts to do some quite complicated things.

PyHelpers.py:
Most python files import this file. Basically it adds a few shortcuts to some more complicated functions. If you can't find exaclty what you want in the API there is a chance that this file might be able to help you - though it only really creates shortcuts, and still follows the API.

CvGameUtils.py:
This file is used when deciding certain things, like what can/cannot be built. It can be used to interrupt some basic game functions. Normally each fuction returns "False", but if you were to make it return "True" under some certain conditions on those conditions the unit, for example, could be removed from the list of units available. You could use this to make having the slavery civic make a slave unit be buildable, for example. I'd check it out yourself to see exaclty how much power you have in there.

Scriptdata (pickling):
If you want to add additional information about certain parts of the game world you need scriptdata. For example, if you want a unit to use up "ammo" everytime it fought, and then have to return to a city with an armoury to resupply, or maybe you want plots to become muddy if too many units travel on them without roads, you need scriptdata.

If you want to play around with scriptdata, I recommend getting Stone-D's excellent SD-Toolkit. This toolkit allows you to attach pieces of data to parts of the game. Each part must be identified with a unique mod-name (for compatability).

NOTE: There is an improved version of the toolkit posted on this post by Teg_Navanis. This version is faster than the default version, and fixes seveal minor issues the default had.

Action Buttons:
talchas has released an Action Button's Utility Mod which is a template for adding buttons to the GUI to do custom functions when pressed. The AI will have no clue about it.

Square Selection:
Once again by talchas this mod is a template. With suitable modification this could be used to make artilliary act exaclty as Civ 3 artilliary, with no bodges. Once again, AI shortcomings come in.

.ini file modifications:
While I have not actually tried it myself, there is a mod by Dr Elmer Jiggle which allows you to add variables to a mod's .ini file. Get it here

The Great Apple · Jan 15, 2006

Conclusion

Not much of a post here, only really to say that the best way of learning any new language is to go out and use it. It can be incredably frustrating sometimes, but is quite rewarding when everything works.

Good luck!

IamSid · Jan 15, 2006

This will help me very much! Thank you!

Napo981 · Jan 15, 2006

Thx, Thx, Thx, Thx!!!

:worship:

woodelf · Jan 16, 2006

Great work so far TGA. When I finally do dive headfirst into Python I know where to look for help.

The Great Apple · Jan 17, 2006

Right, I've completed the tutorial. If anybody feels that they have anything to add (or they want to fix my spelling... as seems to be the norm on wikis) please feel free to add it on http://civ4wiki.com/wiki/index.php/Python_Tutorial_-_Introduction . I've left a bit of room for expansion on this thread.

Frostyboy · Jan 26, 2006

Thanks for this Mr Squiggoth :goodjob:

I really hope your Python skills will come good in Warhammer 40K (and hopefully the Fantasy version)

Thanx

Shqype · Feb 8, 2006

Thanks for the tutorial TGA. Could you add a section about creating dictionaries and serializing (pickling)? I'm not sure how to make use of it right now, but I would like to learn because it would really help me out.

The Great Apple · Feb 9, 2006

Shqype said:
Thanks for the tutorial TGA. Could you add a section about creating dictionaries and serializing (pickling)? I'm not sure how to make use of it right now, but I would like to learn because it would really help me out.

I was planning on avoiding general python language. GBM covers dictionaries quite well here. The three forms can be very useful, especially for organising data. I would also suggest the help files at python.org for doing specific things with them. If you have any specific questions feel free to post them/pm me.

Pickling I've covered in the Misc section. Basically, I recommend using a cool little set of python functions made by Stone-D - SD-Toolkit, which allows you to store arrays (turples/lists/dictionaries) using pickle in just one command, giving each array a unique identifier so that it can be retrieved with a different command.

Shqype · Feb 11, 2006

Thanks again TGA, you always do come to the rescue. I still have some learning to do before I do some of the above (I can't seem to make much sense of the SD-Toolkit from glancing at it, then again I haven't really analyzed it). GBM's post 120 was helpful though.

naf4ever · Mar 7, 2006

Oh my god this is so useful... So far ive been learning python by studying other peoples work and mods and simply doing lots of trial and error to see what exactly does what. This helps immensely though. Thanks!

EDIT: If you ever get time you should include an example or two here. Perhaps talk the player through making a complete basic modification to the game. Something simple like having a new message pop up at the end of every turn saying you've completed it or some small interface change. Just something to get their feet wet and to show the whole process of making an addition.

The Great Apple · Mar 8, 2006

naf4ever said:
Oh my god this is so useful... So far ive been learning python by studying other peoples work and mods and simply doing lots of trial and error to see what exactly does what. This helps immensely though. Thanks!

Good to hear that!

naf4ever said:
EDIT: If you ever get time you should include an example or two here. Perhaps talk the player through making a complete basic modification to the game. Something simple like having a new message pop up at the end of every turn saying you've completed it or some small interface change. Just something to get their feet wet and to show the whole process of making an addition.

Good idea - I'll see what I can come up with.

mikezang · Mar 27, 2006

Hi, TGA
You article is great, thanks a lot.
By the way, do you know how to show Chinese characters in civ4? I tried to use XML in utf-8 format, but it didn't work, do I have to modify some python in game?

The Great Apple · Mar 27, 2006

mikezang said:
By the way, do you know how to show Chinese characters in civ4? I tried to use XML in utf-8 format, but it didn't work, do I have to modify some python in game?

No idea. Sorry.

The Great Apple · Apr 28, 2006

Updated API section for links to Locutus' latest API.

TGA's Python Tutorial

Big Cheese

Big Cheese

Big Cheese

Big Cheese

Big Cheese

Big Cheese

Big Cheese

TheTrueIdiot!

***ernEmperor

Bard

Big Cheese

Never Beaten

Shqyptar

Big Cheese

Shqyptar

Dread Lord

Big Cheese

Pegasus

Big Cheese

Big Cheese

Similar threads