Now it has been confirmed that in CIV4 not even you have an editor, you actually can replace some of the scripts written in Python to change the AI behavior.

In CIV3, a mod is basically a _data_ file, and in CIV4 it looks like it's a combanation of both data and _code_.

This does worry me a bit. No offense to the modding community, and I agree having this kind of capability does have much more flexibility. However, this brings more risk as well.

I want to hear what Firaxis thinks about it, or whether it actually thought about it. Technical wise, I am not an expert in Python, but I know a bit of Perl. If someone could explain to me if there is a security sandbox of thing Python has to ensure the local security (as what Java does), it would be even better.

So no one cares about security problem? :)

Everyone is fine with downloading code from web and runing it on their computers?

Makes me scared... I guess a duel install will be the way of the day...

I am no expert at scripting languages, but i really dont think that it would be hard to not include any command that could alter anything besides civIV.

I mean, this is not assembler we are talking about, where code accesses directly CPU/memory. I think that worst that these scripts are capable, is to feed CivIV engine with processes so long that it takes forever to calculate, meaning you just have to close CivIV and lose your game.

But then again, i cannot comprehend how a .jpeg might contain malicious code also.

A good point, something Firaxis should really look into.
Hopefully they already took into account the evilness of people and already thought of it :)

I don't know much about this highlevel programming, but couldn't it be made so that the code only gets read by the game and not be able to go to the 'outside'?
This would mean there are parts of the game which cannot be accessed, the hardcoded part, in which it is programmed that all other stuff cannot control stuff outside the game.
Sounds easy to do to me :p

The other part of this is control on games - there needs to be some way of ensuring that all players in a multiplayer game or single-player tournament are playing with the same ruleset.

As for the security issue, I guess that a register of 'approved' scripts could be maintained. CFC surely won't host any scripts that have any hint of malicious code.

Let me elaborate a bit what the risk is.

Let's imagine the "script system" works this way. There are a certain number of functions predefined which the game would call into if the modder provides it, for example, a CalcScore function would be called to calculate the score of a certain civ which allows the user to change the way score is calculated.

The function is supposed to take a few parameters and return a number which the game then uses to display in the F8 and various other screens, and to determine the histograph result.

Now the problem is that how you enforce the function is doing what it's supposed to do and not acting as a trojan (other than by reading the source code)? In addition to returning the score, the function could read your credit card number and send it to some unknown host in the universe.

We have the same concern with Java applet, but there is a built-in security mechanism to ensure that an applet cannot call any API that might compromise local security. For example, you cannot access any local files. Is there any such security provided in Python? I never heard of it. The game cannot do anything about it - it has to come from the underlying language. Once the game transfers its control to the script, it's all at the script's hands (or its writer's).

Let me elaborate a bit what the risk is.

Let's imagine the "script system" works this way. There are a certain number of functions predefined which the game would call into if the modder provides it, for example, a CalcScore function would be called to calculate the score of a certain civ which allows the user to change the way score is calculated.

The function is supposed to take a few parameters and return a number which the game then uses to display in the F8 and various other screens, and to determine the histograph result.

Now the problem is that how you enforce the function is doing what it's supposed to do and not acting as a trojan? In addition to returning the score, the function could read your credit card number and send it to some unknown host in the universe.
Well, such a function should be fairly straight-forward. For example, it should not have any need to call any APIs. If it does, then either python should prevent them being called (best solution), or the person simply shouldn't use the script.

The good thing about scripting languages is that you can't compile them. As such, any modded scripts are 'plain text', so people can open them, look at them and work out whether what they are doing is malicious or not.

Whilst not everyone would have the skills to do this, I expect that there will be some vetting on sites such as this.


I do agree that some form of security to prevent APIs being called (or limited to game APIs only) is the best solution.

This could help professional or hobby modders but people that want to change the game just a little might have a problem!

I'm a bit worried now... mostly about security, I hope this has been thought through (maybe it has).

Also... no editor?

What about us poor, technically inept folks who would like a nice, simple, WYSIWIG editor? :( (Something like Civ II, only that doesn't screw up the whole scenario I'm trying to make when I import a graphic from another Civ scenario?) I'm grateful to all the Civ II and Civ III scenario makers, but if Civ IV is better all around it will not only give them more tools to make even better scenarios, but also give me the tools I need to make at least simple ones.

I think there still will be an editor for those mods that just change some parameters (the same as what we have today). The script is more advanced but surely optional.

I told Firaxis about this many months ago. Python scripts have the full power of your operating system at their disposal. That is what Python was built for. In addition, it can run any C or C++ routines (or call functions in ddls) that you properly set up for it. This is a very real risk.

On the upside, the source code is there to inspect (unless they call a compiled routine - but the fact that it is doing that is apparent also). Anything malicious should be obvious to a knowledgable Python programmer. CFC inspected and approved mods should be the name of the game.

Python can be compiled into intermediate bytecode. This executes the same as regular Python (actually, a little faster since the first thing a running Python program does is compile to this code - the is a simplification, but close enough). This is not particularly human readable (unless you are the rare human who can read undocumented assmebly and make heads or tails of it). I would not recommend anybody accepting a mod that came in this format or that includes a dll unless you really trust the source.

I imagine changing things like adding units or modding the stats of units will not be done in the Python at all, but rather in the XML data files.

Thanks warpstorm. That confirmed my suspicion. It is important for everyone to realize the risk and only download mods from trusted sources.

One thing the game could provide is an option to enable/disable "advanced scripting". With the feature disabled (which should be the default), any scripts provided by the mod would be ignored or cause the game to exit.

I'm not an expert in python, but surely Firaxis can embed a cut down version of the interperetor that does not allow file IO or the calling of functions from dlls?

Not really since Python uses dll loading as the core way that it allows you to call C/C++ functions. To prevent this would probably make it so that they couldn't use it for their purposes.

Sounds like maybe Thunderfall may need to get "script moderators" to vet any scripts before they are allowed to be posted on the site. I could see some guy get really pissed off about being banned for flaming, writing a malicious script during his ban, and then posting it when he gets back. It would be caught fairly quickly, but if he got even one person to run it, mission accomplished.

I imagine Firaxis needs to license a version of Python to distribute with the game. If so, they might license a special/restricted version that do not allow loading DLL and calling external APIs. Or even just specify some command line options when they do so.

I imagine Firaxis needs to license a version of Python to distribute with the game. If so, they might license a special/restricted version that do not allow loading DLL and calling external APIs. Or even just specify some command line options when they do so.
I believe most of the attraction of python is that its free (open source).

I believe most of the attraction of python is that its free (open source).

Great. So they could modify Python to produce a restricted version.

Great. So they could modify Python to produce a restricted version.
Well, from here (http://www.python.org/doc/Copyright.html):


The gist of it is that Python is absolutely free, even for commercial use (including resale). There is no GNU-like "copyleft" restriction.

A clarification: some very picky lawyers are worried that the notice doesn't explicitly grant permission to distribute modified copies (even though it grants permission to copy, modify, and distribute). This can be considered an oversight in the notice; we have no intention to limit distribution of modified copies (that otherwise play by the rules). Similarly, the words "without fee" are to be read connected to "permission": permission is granted without fee. We are in the process of revising the notice to clarify it, but this is a slow process.


So they are free to modify it.

However - I think that is probably more hassle for the developers that they wouldn't want.

To do this would imply having someone become familiar enough with the inner workings of Python to make the changes. I suspect that they have enough on their plate without having to make a new one-off version of Python.

What about someone modifying their own source code that has malicious code towards other players in MP games? It could be as subtle as planting trojan horses or self-replicating viruses in the other players systems.

I would assume that only data would be passed in MP games rather than code. Now there are ways to exploit this in nearly any MP game if you really wanted (buffer overruns, etc.)

(Warpstorm: ) To do this would imply having someone become familiar enough with the inner workings of Python to make the changes.


FWIW, here's some idle speculation: according to MobyGames, the new Lead Programmer for Civ4, Mustafa Thamer, previously worked on URU: Ages Beyond Myst, which used Python. Although his main responsibility lay on the multiplayer side, it's possible that he has the skills to do this.

On the other hand, perhaps David McKribbin is dealing with it (from the Firaxis website):


McKibbin Trivia: David once designed security systems in a nuclear power plant!

It wasn't the skills I was concerned about, I'm sure many of their programmers could do it given the time. It's the time I was concerned about.

Yes, Moose is very skilled with Python, but I doubt fiddling with the way Python executes isn't on the top of Firaxis' list.