Idea: scenario making with IDLE

Baldyr · Apr 9, 2010

So I'm making a scenario maker of sorts in Python. As it stands now the scenario designer types Class invocations into a module at __main__ level that get stored into a list variable. This variable is stored at setup and used for calls from the Event Handler during game. I just had this idea, though. :rolleyes:

What if I would make a stand-in CivIV game environment that reads the WBS file and stores all the contents into classes and objects. So there would be a game class, a map class, tile class, player class and so on. Then I would create a module that mimics CvPythonExtensions with all the functions used in the my scenario maker code, so that they would be valid and work with the scenario maker code.

What this would in essence give me is an environment where I can use CivIV Python in the IDLE interpreter. So if I wanna test my scenario trigger instances, I don't have to fire up the actual game in order to do so. Also, I could make changes to the stand-in game session by using IDLE just like I would using the Python console in CivIV. (So if I wanna test a trigger condition depending on a city being on a certain plot, I could first use the interpreter to initialize a new city object belonging to that tile. Then the trigger instance would recognize the presence of the city and fire.)

I don't know if this even makes any sense to anyone else, but the hurdle in front of me is that I have little knowledge of XML. Because I would need to interpret the contents of an actual WBS file using Python. I also believe that I would have to load most if not all of the CivIV XML in order to get full functionality. Is there a guide or something in entry-level terms available that explains how this is done?

Lastly, it does sound like a huge undertaking, but I don't think I will at least never be taking the time to learn how to make a graphical interface for my scenario maker. So a purely text based interface would be an option to scripting, which is what I can offer at this point. Like the interface present in IDLE.

The_J · Apr 9, 2010

You want to create a program, which simulates the civ4 python interface?
Waste of time, imho. Just the starting time of civ will never ever be more than the time you need to program this.

Baldyr · Apr 10, 2010

Never mind, I just found it (CvWBDesc.py). Thank you for pointing out the obvious, though. :rolleyes:

EmperorFool · Apr 10, 2010

www.w3schools.com has a great guide for learning XML. XML is really quite simple--much more so than Python.

Baldyr · Apr 10, 2010

What I'm looking for is a Python XML parser, and I'm betting on that CivIV already has one. Does anyone know the specifics of how the game initializes the XML? (Because, its done right after Python.) There really is no need to reinvent the wheel, now is there?

EmperorFool · Apr 10, 2010

Civ4 uses C++ to read and parse the XML into Cv<Type>Info objects--not Python. Python 2.4 does, however, come with xmllib, and I use this in BUG to parse the config files. Sadly, it is SAX only which makes it harder to deal with. A DOM parser tends to be much easier to manage.

If you're going to be doing this in IDLE, you can use the "xml" module from Python 2.0, specifically xml.dom.minidom.

The_J · Apr 10, 2010

I found the SAX parser so hard to deal, that i prefered to write a generic file reading and string cutting for 2 of my modcomp instead of using it.
Not a good sign for the parser.

Baldyr · Apr 11, 2010

The_J said:
I found the SAX parser so hard to deal, that i prefered to write a generic file reading and string cutting for 2 of my modcomp instead of using it.
Not a good sign for the parser.

Would you mind charing your code here? Maybe also give us an example of the XML that it parses.

Because I was thinking along those lines myself. Not just for the idea presented in the OP, but for making it an option to design a scenario in XML. In fact, that might be ideal once I add functionality for in-game messages and pop-ups.

The_J · Apr 11, 2010

I hope nobody here will laugh :blush:

.
This is a (cleaned) version of the code from this mod.

Spoiler :

The XML looks like this:

Spoiler :

I know, that there are many sources for problems in the whole thing.
If somebody puts more than 1 entry in a line, the whole thing will crash, if a line misses the whole thing will crash, etc.
The structure itself is also not read, the CivilizationInfo entries are just there for better user readability.
It's sure nothing which should be used for general things or by anybody who has no experience and doesn't know, what he/she is doing.
It's enough for the things i want to do, and really nothing more.

Baldyr · Apr 11, 2010

Thanks, that seems to be just the stuff I was looking for. Once I wrap my head around this XML thing all I have to do is lookup the stuff I haven't used before (like readlines, strip or xrange). Since I haven't done that yet, the only thing unclear at this point would be this:

Code:

        string = str(string)

Wouldn't CurString already be a string? :confused:

Clearly you're asking it about strings before ever calling on CutString().

Anyway, a XML approach to scenario making could look something like this, then:

Spoiler :

What I'm sensing at this stage is that XML would be less dynamic. The same Trigger would look like this in a script:

PHP:

Trigger(2).turn(65).valid().city(104, 45, "Qufu")

A lot of arguments can be omitted, but I guess the dot notation and all the parenthesis would be confusing to someone new to Python. So there could be two ways of creating triggers.

Also, I plan on including support for Python scripting for greater flexibility. So you could use the CheckTurn class and the City class separately with your own if statements and whatnot. There is no reason to not use all three methods at once, really.

The_J · Apr 11, 2010

Baldyr said:
Thanks, that seems to be just the stuff I was looking for. Once I wrap my head around this XML thing all I have to do is lookup the stuff I haven't used before (like readlines, strip or xrange).

readlines: Reads the lines of the file one by one.
strip(): Cuts of the spaces at the beginning at the end of a string.
xrange: Somebody has told me, that it's just more performant than range, but can't say anything more :dunno:

.

Baldyr said:
Since I haven't done that yet, the only thing unclear at this point would be this:

Code:

string = str(string)

Wouldn't CurString already be a string? Clearly you're asking it about strings before ever calling on CutString().

I guess that is a left over from the beginning :blush:

....er, i mean, that's just to handle every possible exception

.

Baldyr said:
Anyway, a XML approach to scenario making could look something like this, then:

Spoiler :

PHP:

<Trigger> <ePlayer>2</ePlayer> <elements> <turn> <iTurn1>65</iTurn1> <iTurn2>None</iTurn2> <iInterval>1</iInterval> <iModulo>0</iModulo> </turn> <valid> <tCoords>None</tCoords> <bDomestic>False</bDomestic> <bForeign>False</bForeign> <bLand>True</bLand> <bWilderness>False</bWilderness> <bAdjacent>False</bAdjacent> </valid> <city> <iX>104</iX> <iY>45</iY> <name>"Qufu"</name> <iSize>2</iSize> <city> </elements> </Trigger>

What I'm sensing at this stage is that XML would be less dynamic. The same Trigger would look like this in a script:

PHP:

Trigger(2).turn(65).valid().city(104, 45, "Qufu")

A lot of arguments can be omitted, but I guess the dot notation and all the parenthesis would be confusing to someone new to Python. So there could be two ways of creating triggers.

That looks at all logical to me, but also like a good bunch of work.

EmperorFool · Apr 11, 2010

Use xrange() when you are iterating over a very large range since range() will create and return the full range of numbers in a tuple while xrange() will return each number individually as its needed.

How large is very large? Use your best judgment, but probably larger than 40.

Also regular expressions would be very helpful here. While you'd need to learn them and how to use them in Python, I cannot stress enough that regexps are the coolest and most useful tool in my toolbox. You will never, ever regret the time you spent learning them. Most editors have a regexp-based search and replace feature so you can play with them.

Baldyr · Apr 11, 2010

EmperorFool said:
Use xrange() when you are iterating over a very large range since range() will create and return the full range of numbers in a tuple while xrange() will return each number individually as its needed.

xrange for iteration only, not for creating a list (tuple?). Ok, got it. :goodjob:

EmperorFool said:
Also regular expressions would be very helpful here. While you'd need to learn them and how to use them in Python, I cannot stress enough that regexps are the coolest and most useful tool in my toolbox. You will never, ever regret the time you spent learning them. Most editors have a regexp-based search and replace feature so you can play with them.

Oh, boy... When you say stuff like that I know I will be burying my nose in a textbook sometime very soon.

I just took a quick look and kinda understand what regexp is - its useful for handling strings, right?

So that would make it useful for XML also.

Baldyr · Apr 11, 2010

The_J said:
I guess that is a left over from the beginning ....er, i mean, that's just to handle every possible exception .

Yeah, I was actually getting that, but wanted to double-check. I'm new at this, you know.

The_J said:
That looks at all logical to me, but also like a good bunch of work.

Well it is. Or was, as I'm nearing beta. With a good API I've done some 1200 lines of code on this thing. Right now I'm testing the whole thing and the problem is that I get carried away with huge overhauls of the code, so I have to start all over... On the upside its even more flexible than I ever imagined possible, so I don't feel like I'm wasting my time.

This is my biggest project to date so I'm still learning all the time. I'm actually glad I didn't take your advice from the other thread, by the way, because then I probably wouldn't bothered to learn OOP. (That was somewhat much to swallow...) And this project has really opened my eyes to what it is all about and what power it holds.

But you're of course right - I'm taking the long road.

EmperorFool · Apr 11, 2010

Yes, regular expressions are for pattern matching in strings. For example to match an XML element and extract its value you would use a pattern like this:

Code:

<([a-zA-Z-:]+)>(.*)</([a-zA-Z-:]+)>

Breaking some of it down:

[a-zA-Z] - any letter
+ - one or more of the previous pattern
(...) - collect what's inside into a "partial match" element
< / > - literal characters for XML elements

So the above matches <tag> and </tag>, extracting the tags, and (.*) is everything between them. You'd use it in Python a little like this (not correct syntax):

Code:

import re
pattern = re.compile("<([a-zA-Z-:]+)>(.*)</([a-zA-Z-:]+)>")
for line in file.readLines():
    values = pattern.match(line)
    openTag = values[1]
    data = values[2]
    closeTag = values[3]

Baldyr · Apr 11, 2010

I took another look at CvWBDesc.py - the Python parser for the World Builder. This is what the actual parser class looks like, since we're on the topic:

Spoiler :

One could pretty much just import this class and be set for parsing XML!

What I need to do, however, is to parse strings and turn them into classes and functions than are then executed. For example, I might end up with something like this:

Code:

function = [[Trigger, (2)], [turn, (65)], [valid], [city, ((104), (45), ("Qufu")]]

The first problem would be that the length of this things could be any possible number. So this example has four elements, but I could create the actual trigger instance first:

Code:

object = function[0][0](function[0][0][0])

Then loop the methods (which all return the instance of the object, by the way):

Code:

for i in xrange(1, len(function)):
    method = function[i]

Then extract the arguments for the current method:

Code:

    arguments = []
    if len(method) == 1: continue
    for j in xrange(1, len(method)):
        arguments.append(method[j])

And finally call the method with the target object:

Code:

    object.method(arguments)

Would this really work? :eek:

I think I would have to spill the contents of the arguments list into individual arguments upon the method invocation, but how? Maybe with next? Like:

Code:

next(arguments)

One question is also if you really can store function (and class and method) names in lists, or does it have to be variables. I understand that the Target class would have to be imported for them to be valid though.

Also, wouldn't all the values in the function list be strings - if I was parsing them from a XML or a txt file? How to differentiate between what should be what? Would the XML have to actually have a tag for Type also? Or could I just read it from the variable tag, like the "tCoords" value would be equal to tuple(value) and "iNum" would be treated as int(value)?

But what about the function names themselves? What data type would they get? :confused:

Do I use encode/decode to turn the string into Unicode, or something? :crazyeye:

I read there are something called function and method "types", as well as "NotImplemented" and other strange stuff...

EmperorFool · Apr 11, 2010

AFAIK WB scenarios are not XML; they are text files, yes, but with a different format which is line-based. It's similar in structure (hierarchical) but it isn't XML.

As to how to answer your other questions, um, yes all that is possible and you could make it fairly generic. But explaining how to do that is far from trivial. Keep in mind that a function in Python is an object just like a Trigger instance: you can assign it to a variable or stick it in a list. You call a function in a variable just like a normal function: stick parentheses after it with the arguments inside:

Code:

def foo(x):
    print x

bar = foo # assigns the value of foo (a reference to a function) to the variable bar
bar(4) # prints 4

If you have arguments in a list/tuple, you can pass them to a function using *:

Code:

args = [5]
foo(*args) # prints 5

If you have named arguments in a dictionary, you can pass them to a function using **:

Code:

def foo(x, y):
    print x + y

args = {'x': 2, 'y': 4}
foo(**args) # prints 6

BUG does a lot of this kind of work to parse the Config XML files. I can't write up how to do it here as it is a lot of code, but you can check it out. BugConfig and BugOptions are good places to start. The code is rather complex, but I can answer specific questions about it.

Baldyr · Apr 12, 2010

That much I already knew, but I'm not quite sure on every specific. So thanks for the explanation.

The dictionary argument assignment was new to me though, and that would be exactly what I needed for my example. But I still don't know how to turn a string like "turn" into a variable containing a function object turn. Because if I go like:

Code:

string = "turn"
function = parserOutput

...it will still be a string and not a function object. I could of course create a table identifying the function for each string, but it would look like:

Code:

string = "turn"
index = {'turn':turn}
function = index.key(string)

...when it should be possible to simply change the type. Or so it would seem to an amateur like myself. :rolleyes:

Baldyr · Apr 12, 2010

EmperorFool said:
AFAIK WB scenarios are not XML; they are text files, yes, but with a different format which is line-based. It's similar in structure (hierarchical) but it isn't XML.

No, and I only realize this now. :blush:

But that setup would probably be even better for any code-alliterate would-be scenario-designer. So it would be a "text based" approach, then. No reason not to include an XML option also, but simply writing the different elements of a trigger in something like the WBS format would be intuitive for someone just getting into modding and managing to edit the actual scenario file. It would then be a matter of creating a second txt-based file - the Scenario Trigger File (STF) to accompany the WBS. :king:

The Python code would then parse that file and run the different elements of a trigger. Since I might have to use a dictionary in order to turn the string into a method object, I might as well give the conditions and actions and whatever more descriptive names:

Code:

BeginTrigger
        PlayerIndex=2
BeginElement
        Type=Condition_GameTurn
        GameTurn=65
EndElement
BeginElement
        Type=Condition_ValidMapTile
EndElement
BeginElement
        Type=Action_FoundCity
        Xcoord=104
        Ycoord=45
        CityName="Qufu"
EndElement
EndTrigger

Then I would pretty much be able to import the WB parser that is already in the Assets folder. I would have to write code that deciphers what the parsed strings correspond to in order to initialize the triggers. Any default values would be defined in a global dictionary and passed if they are not found by the parser. Or the user would have to enter a value for each argument using this method. (This would probably be clearer also.)

EmperorFool · Apr 12, 2010

You can use a table as you have above or you can use getattr() to look up a variable given its name:

Code:

class foo:
    def __init__(self, x):
        self.x = x
    def show(self):
        print self.x

...

f = foo(5)
func = getattr(foo, 'show')
func(f)

You have to pass f to foo() because you looked up the unbound function foo() so you must provide a value for the self argument. I think we went over this in your other thread.

As for turning other strings into values such as "5" into the number 5 you use functions: int("5"), float("5.5"), and bool("True"). To turn "(5, 2)" into a tuple you could use eval() which takes raw Python code and executes it as if it were in your module directly:

Code:

string = "(5, 2)"
coords = eval("tuple" + string)
x, y = coords

Idea: scenario making with IDLE

"Hit It"

Say No 2 Net Validations

"Hit It"

Deity

"Hit It"

Deity

Say No 2 Net Validations

"Hit It"

Say No 2 Net Validations

"Hit It"

Say No 2 Net Validations

Deity

"Hit It"

"Hit It"

Deity

"Hit It"

Deity

"Hit It"

"Hit It"

Deity

Similar threads