On the official Civ V website, Firaxis lists Unprecedented Modding Tools as one of the features. Specifically:

With unprecedented modding tools, players will have unlimited options for modifying Civ V any way they like.

Considering that the word tools is plural, I am interested to see what they will be. Any thoughts?

(I'm putting some of my thougts and ideas in the below spoiler in case you would like to supply your own thougts first before reading mine)

For me, I would like to see an external map editor for starters. The internal one with Civ IV could be very difficult to use. When I first got the game, it became virtually impossible to use due to the fact that my computer (though above the requirements) was very taxed by it. The other problem was creating new maps with predefined civ locations could be very problematic. Often, I would have to leave the map up and running over several days while I worked on a map. Exiting in the middle could cause problems when trying to go back in later to edit.

I would also like to see the use of a database for doing basic mods (instead of using xml). Something that is user-friendly for the rest of us.

Something that I have mentioned in other threads here would be create-a- tools which are kind of like the tools in Spore/The Sims. The main difference would be that the ones for those games are part of the main gameplay while Civ would be side tools. However, the principal is similar and could be used to make quick plug-and-play elements like new Civs, leaders, units, buildings, etc. For models of the elements, the player would have access to a simple modeler that can allow them to change certain aspects like colors (hair, skin, clothing) and attire (from a basic library of clothing that covers the different ages and locations like Europe, Asia, etc). Facial features of leaders and the like may be minimally editable, but not necessary. Advanced skinning would still be available to those more advanced at doing such, but I am just looking for something that covers the basics.

Well, those are just some things that I am hoping to see. I was a bit dissapointed with the modding aspect of Civ IV because it seemed to cater strictly to those more advanced at modding. They could always include the more advanced things too for those with the ability to do so... I just hope they don't forget about the rest of us this time.

Do I smell marketing hype? Maybe.

But things to improve modding (based of Civ4's tools):
- External map editor
- XML editor in easy to use language
- Less multi-file reliance

There's no point asking for a python or SDK editor, as that would be near impossible. Those languages just don't work that way, and would severely limit the ability to add new code.

I thought the tools of C3C were top notch.

World builder is fine, whatever.

Keep XML, its perfect. Maybe include a guide so people don't have to learn what stuff is by trial and error.

Use another language than Python for scripting. Maybe C. Similarly, include a guide.

Use a more available compiler, maybe Microsoft Visual C++ 2008. And include a guide. And actually write the code to be easy to modify, where its not that much trouble.

Make the art easier to mod. For example, in Civ 4, you can use XML to determine which animation and background goes with which leaderhead model, but you have to use Nifscope to "reskin" a model, ie give Augustus blue hair or something by referencing a diferent dds file, ie flat graphic that is used to "paint" the model. It would be nice if the skins could also be mixed and matched in xml. Maybe even include some kind of basic model tweaking utility.

How about an AI editor? Maybe this would work via being able to define trigger conditions and reactions to them, along with a random selecter or prioritizing integer in case of multiple options being viable. Sort of like random events.

And for goodness sake no more "gamefont.tga." Why could that not be done like the rest of the art?

- External map editor


Amen. The map editor for Civ III was, to me, almost perfect...why did they have to change that?

I still play Civ III because the map editor was just sooo easy to use. I hope they have a similar editor for ciV.

Use a more available compiler, maybe Microsoft Visual C++ 2008. And include a guide. And actually write the code to be easy to modify, where its not that much trouble.

We'll probably find the version we have to use will be the version that was common at the start of dev. Civ4 was a couple years in dev, so that's why it was VS2003 (not 2005). I assume they started in 2008 so it might be in VS2008, otherwise we might have to find 2005 (depending when in 2008 they started coding).

I think what this amounts to is that there may or may not be an external map editor and there may be a database editor. A database editor would be pretty cool though, an actual UI to add new civs, units, techs, etc, without having to sort through multiple files and work out the XML would save a lot of time and be pretty user-friendly.

We don't know if there will be more python scripting or how that stuff will work, but while you can probably expect lots of open code to modify again, don't expect much more than a database editor in terms of actual tools.

I still play Civ III because the map editor was just sooo easy to use. I hope they have a similar editor for ciV.

I don't play Civ III anymore, but I did like the map editor much better than the one with Civ IV. And the Rules Editor was much better than sorting through XML files.

With unprecedented modding tools, players will have unlimited options for modifying Civ V any way they like.
This is what convinces me it is mainly marketing speak.
I can modify Civ 5 any way I like? That is rubbish and I would almost bet my life savings on that.

It would imply releasing the code for the entire game including the engine (now that would be unprecedented!). How can a game that has multiplayer as a component be completely open to be modded in any way? :lol:

Civ IV was pretty flexible and allowed (those with the know how... and time) to create some pretty incredible mods that deviated quite a bit from the core game.

I would definitly rather go back to an external map editor.

Civ 4 was awesome, except for the core graphics engine, there is nothing you can't mod it to do, if you have the time. That's not to say some core tools, also with their source code released so they can be modded, wouldn't be good. A map editor is a must, the whole in-game world builder thing was a horrible idea to begin with.

I'm all for them making whatever tools they want to, but it better in no way mean that they aren't going to also release the source code. I am afraid Civ4 modding may have ruined me and if there is no source code to play with that might severally limit its replayablity for me.

I still play Civ III because the map editor was just sooo easy to use. I hope they have a similar editor for ciV.

+1

Never downloaded a single scenario, or made one. But on a weekly basis, I'm playing and tweaking with the rules and map of my own games in the Civ3 editor.

Visual C++ 2010 anyone?

+1

Never downloaded a single scenario, or made one. But on a weekly basis, I'm playing and tweaking with the rules and map of my own games in the Civ3 editor.
I do excactly the same. To me, being able to modify rules/maps easily is an important part of the core game.

Maybe they could give us a way into the exe? That would let us do a whole bunch more.

Maybe they could give us a way into the exe? That would let us do a whole bunch more.

Probably not. They are probably linking in a lot of libraries statically. Many of these libraries probably aren't free software and probably cost quite a bit of money.

On a side note, I personally hope they compile with GCC and use OpenGL, but that probably won't happen.

Maybe they could give us a way into the exe?

Will never ever happen.
Because then everybody could re-invent the whole game.


On a side note, I personally hope they compile with GCC and use OpenGL, but that probably won't happen.

DirectX11 support is confirmed, so that will sure not happen.

DX11? That out slready? I thought DX10 was the newest one.

What advantage do firms gain by using private software they have to pay for? Is DirecX11 really that much easier to code for then OpenGL? I know GIMP is just as effective as Photoshop, and NSIS is in many ways better then InstallShield, yet most companies use install shield, even though they need to pay for it. Why? I mean I can understand licensing entire engines, like the Gamebryo engine, the cost benifit is there, but why do firms consistently use privately made tools when there are Open Source alternatives that are just as good, if not better?

What advantage do firms gain by using private software they have to pay for? Is DirecX11 really that much easier to code for then OpenGL? I know GIMP is just as effective as Photoshop, and NSIS is in many ways better then InstallShield, yet most companies use install shield, even though they need to pay for it. Why? I mean I can understand licensing entire engines, like the Gamebryo engine, the cost benifit is there, but why do firms consistently use privately made tools when there are Open Source alternatives that are just as good, if not better?

Well, I don't know too much about that subject.... but, if I were to guess, then I would say the PR reason would be security and product protection, but the basic answer comes down to corporate politics.

What advantage do firms gain by using private software they have to pay for? Is DirecX11 really that much easier to code for then OpenGL? I know GIMP is just as effective as Photoshop, and NSIS is in many ways better then InstallShield, yet most companies use install shield, even though they need to pay for it. Why? I mean I can understand licensing entire engines, like the Gamebryo engine, the cost benifit is there, but why do firms consistently use privately made tools when there are Open Source alternatives that are just as good, if not better?

Documentation and Support. If your engine spontaneously crashes, and you have no private firm to call up to get help from; your SOL. ;)

Use another language than Python for scripting. Maybe C. Similarly, include a guide.

-1 on this for sure. While I resisted learning Python for many years due to the whitespace-is-syntax issue, you'll include far more modders with Python than with C. 99% of the time someone says, "you gotta do that in the DLL," modders here turn back. Plus Python is much more expressive than C++ or C in a short amount of code. There's no way I would have done as much as I have with BUG in C++, and even less in C.

Experienced coders have trouble with the STL, and you're going to ask amateurs to understand it? :lol: In C you won't even get that: roll your own linked list buddy! :p

-1 on this for sure. While I resisted learning Python for many years due to the whitespace-is-syntax issue, you'll include far more modders with Python than with C. 99% of the time someone says, "you gotta do that in the DLL," modders here turn back. Plus Python is much more expressive than C++ or C in a short amount of code. There's no way I would have done as much as I have with BUG in C++, and even less in C.

Experienced coders have trouble with the STL, and you're going to ask amateurs to understand it? :lol: In C you won't even get that: roll your own linked list buddy! :p

I agree with EF all the way here. While the whitespace still bothers me to this day in python; and I always forget the stupid colon at the end of if's, :lol: I still can see why it is so attractive. ;)

Will never ever happen.
Because then everybody could re-invent the whole game.
If FfH2 is not reinventing the whole game, what is it, then?

If FfH2 is not reinventing the whole game, what is it, then?

It's a turn-based strategy game set in a fantasy world versus a turn-based strategy game set in a reality-based world. Yes, it's very different, but it's also very much the same.

Experienced coders have trouble with the STL, and you're going to ask amateurs to understand it? :lol: In C you won't even get that: roll your own linked list buddy! :p
STL usage is simple and straightforward, unless the coder wants to do something perverted with STL or dig really deep into it

i bet you that all dynamic data structs where written in C a long time ago. reinventing the wheel just wastes time

about the unprecedented modding toolz, we will just have wait and see. i expect it to be nothing more than a catchy marketing phrase

about the unprecedented modding toolz, we will just have wait and see. i expect it to be nothing more than a catchy marketing phrase

Well here's to hopping we get a good Scenario Editor; the civ4 WB is horrible, and editing WBS manually in a text editor is about the most tedious thing ever. The more I think about it, the more I wouldn't mind a simplified and user friendly unit editor either. I also can't imagine a viable tool for editing the art, you need a graphics artist to do that right, just like you need someone who can code to mess with the actual mechanics of the program. I do worry about tools getting in the way of in depth modding though, the tools may end up reducing the potential mod makers who are willing and able to dig into the code have, as ease of use tools still equates in my mind to locked platforms.

What advantage do firms gain by using private software they have to pay for? Is DirecX11 really that much easier to code for then OpenGL? I know GIMP is just as effective as Photoshop, and NSIS is in many ways better then InstallShield, yet most companies use install shield, even though they need to pay for it. Why? I mean I can understand licensing entire engines, like the Gamebryo engine, the cost benifit is there, but why do firms consistently use privately made tools when there are Open Source alternatives that are just as good, if not better?

The short form answer: Built in development pipeline.

Gamebryo did (still does) offer more than an engine, it also had a smooth pipline for the intergration of assets. Why reinvent the wheel ?

You know what would please me a lot in terms of "unprecedented modding tools"? Having them (partially) available before the release, similar to the toolset for Neverwinter Nights 2 or the Dragon Age character creator.

An early release of the map editor or something akin to that would give modders a head start to get to terms with the overall game and serve as a great preview (even if it's restricted to pre-orders only, because I'm so going to pre-order it anyway!).

And considering that a lot of games give pre-order bonuses nowadays, I can't help but think that that would be perfect! :D

Cheers, LT.

The short form answer: Built in development pipeline.

Gamebryo did (still does) offer more than an engine, it also had a smooth pipline for the intergration of assets. Why reinvent the wheel ?

Oh, I totally understand why a firm would pay to license gamebryo. What I don't understand is why the purchase tools like Photoshop, InstallShield, and DirectX, when GIMP, NSIS, and OpenGL are all out there, and free and are pretty equivalent in quality.

Oh, I totally understand why a firm would pay to license gamebryo. What I don't understand is why the purchase tools like Photoshop, InstallShield, and DirectX, when GIMP, NSIS, and OpenGL are all out there, and free and are pretty equivalent in quality.

First off, GIMP is no where near the level of Photoshop. I love Opensource software, but that's just not true.

As for InstallShield, I suspect what has happened is better known as "Vendor Lockdown". Nobody chooses Installshield anymore, but their existing installers are too complex and would take too long to move over to another system. That and the strange fear management seems to have with opensource software.

Also, DirectX is also light years ahead of OpenGL. Not really OpenGL's fault, as Nvidia and ATI aren't always very helpful...

They need to use Microsoft Visual C++ 2010 for the SDK it would make it much more powerful and up to date

Personally I've got very good hopes about this, mostly because the PC version of Civ haven't got such a young audience (or am I wrong here?) like The Sims. I remember GalCiv, a game like Civ in more than just name, had a pretty fun unit-editor, though it was basically lego. In a 2010-title like Civ 5 I think we can expect something nice.

STL usage is simple and straightforward, unless the coder wants to do something perverted with STL or dig really deep into it

i bet you that all dynamic data structs where written in C a long time ago. reinventing the wheel just wastes time
I'm confused a bit here. Are you just nitpicking EmperorFool's post or are you seriously supporting the idea of dumping Python and forcing all modders to edit the DLL?

I'm confused a bit here. Are you just nitpicking EmperorFool's post or are you seriously supporting the idea of dumping Python and forcing all modders to edit the DLL?
i'm nitpicking EmperorFool's post and i do not support dumping Python as a scripting language, if it will not be replaced by another scripting language: maybe more specialized for civ.

as for STL i have so far not seen any usage of it in the code snippets scattered around on this forum :dunno:

Why invent a new scripting language? All you will do is introduce another possible source for bugs while not gaining anything in functionality that couldn't be done with proper bindings. Keeping with python will benefit the institutional memory of the civ4 modders and make more advanced civ 5 mods out the door quicker.

STL simply isn't a workable solution for amateurs. Seasoned engineers have enough trouble with it. I can use it when necessary, but I'd rather use this


units = []
for eUnit ...
if ...
units.append(eUnit)
for eUnit in units:
# use eUnit


Than the STL equivalent. Manually dealing with iterators is so 1990. :p

As for standardized C data structures, I'm not familiar with them. I'm sure plenty of people have written good libraries, but none are standard as the STL is.

But it doesn't matter, even in the SDK code that could make use of the STL they avoided it almost entirely. They used Vector in a few places and probably used the STL's hashtable for the info type map (string => int). Otherwise it's all arrays and looping where sets and maps could be used to good effect. If paid Firaxis developers aren't going to use the advanced tools available, you can't possibly expect amateurs to do so.

I don't want to imagine trying to explain the STL to someone, especially since I don't consider myself an expert on its use. I was able to become a Python expert after a month of using it (probably less) and can easily explain how to use lists and dictionaries to modders here.

I don't get why they used arrays. Vectors are so much easier!

Why invent a new scripting language? All you will do is introduce another possible source for bugs while not gaining anything in functionality that couldn't be done with proper bindings. Keeping with python will benefit the institutional memory of the civ4 modders and make more advanced civ 5 mods out the door quicker.
:dunno: probably you are right: creating a new scripting language is a waste of time and resources + require modders to learn a new language.

STL simply isn't a workable solution for amateurs. Seasoned engineers have enough trouble with it. I can use it when necessary, but I'd rather use this


units = []
for eUnit ...
if ...
units.append(eUnit)
for eUnit in units:
# use eUnit


Than the STL equivalent. Manually dealing with iterators is so 1990. :p

As for standardized C data structures, I'm not familiar with them. I'm sure plenty of people have written good libraries, but none are standard as the STL is.
STL is part of the C++ Standard for over nine or ten years :banana:


for eUnit in units:
# use eUnit

i suppose the e prefix means that eUnit is an enum?

for the sake of arguement let's suppose it is not:
let's eUnit be of some STL container type unit_t.

iteration:

for(unit_t::iterator it = eUnit.begin(); it != eUnit.end(); ++it)
it->do_something();

there is a catch though :D : adding to or deleting from any STL container invalidates any pointers and iterators that reference the to-be-modified container :crazyeye:

But it doesn't matter, even in the SDK code that could make use of the STL they avoided it almost entirely. They used Vector in a few places and probably used the STL's hashtable for the info type map (string => int). Otherwise it's all arrays and looping where sets and maps could be used to good effect. If paid Firaxis developers aren't going to use the advanced tools available, you can't possibly expect amateurs to do so.
really strange :crazyeye:

actually i expect amateurs to think for themselves :p
STL does simplify your life. i cannot think of any reason to not use STL :dunno:

I don't get why they used arrays. Vectors are so much easier!
:thumbsup:

Direct array access is faster at runtime even though it's more error-prone for developers.

What do you think about keep C++ and python just update them to the most recent version

STL is part of the C++ Standard for over nine or ten years :banana:

That's what I said. In C there is no equivalent standard data structure library.

i suppose the e prefix means that eUnit is an enum?

Yes, that is Firaxis's convention (Hungarian Notation).


for(unit_t::iterator it = eUnit.begin(); it != eUnit.end(); ++it)
it->do_something();


Are you making my point for me? Clearly that is more complicated to read, type, and understand than


for eUnit in units:
# use eUnit


Here units is a standard Python list object. Or it's a Python set. Or it's a tuple. it doesn't matter. :) [Yes, the same is mostly true for the STL]. Let's say you want to loop over a dictionary (map) and print out each key and value.


for k, v in d:
print k, v


What's the equivalent code in C++? Exactly.

STL does simplify your life. i cannot think of any reason to not use STL :dunno:

Of course the STL simplifies makes writing C++ code easier. I couldn't imagine writing my own C++ data structures. But that's not the point. Someone above suggested using C++ in place of Python, and you simply cannot convince me that C++ even with the STL is easier for amateurs to learn than Python.

Someone above suggested using C++ in place of Python, and you simply cannot convince me that C++ even with the STL is easier for amateurs to learn than Python.

It probably isn't, but it is definitely more powerful and more efficient, not to mention no reliance on whitespace!

I prefer C over both though.

What do you think about keep C++ and python just update them to the most recent version

Huge problems with both versions of both of those statements.

Python: No. Python 3.x is not at all standard compared to 2.x. Perhaps update to the latest 2.x, but not to 3.x.

C++: There is no new version of C++. Don't say that there is Visual Studio 2010. This is not a new version of C++. This is a new version of Visual Studio, not C++. They are completely different things and should not be used interchangeably. The fact that the code was not immediately upgradable was an injustice to Firaxis done by Microsoft due to Microsoft not sticking to the ISO C++ Standard.

It probably isn't, but it is definitely more powerful and more efficient, not to mention no reliance on whitespace!


While I too prefer C++ over python, the whole power/efficiency thing is bad reason to argue one language over another. If that's your decision making standard, you might as well write in direct machine code, that's much faster than having a compiler do work for you. ;)

While I too prefer C++ over python, the whole power/efficiency thing is bad reason to argue one language over another. If that's your decision making standard, you might as well write in direct machine code, that's much faster than having a compiler do work for you. ;)

Good point. I think efficiency is a good reason to use a language to a certain point. Mainly, I think its just best to use a compiler-based language rather than in interpreted ones, which tend to be slower.

But part of the main benefits of using python was that it is an interpreter language, and doesn't require setting up a compiler to start messing with. Alot of hobby modders got their start programming in python (like myself), and simply would not have been willing to jump into actual coding if they would have had to have gone through the whole process of setting up the compiler up front; this is a very daunting task for someone who isn't sure if they really understand what they are looking at. For me I needed to fix a python function another modder had set up, and they pointed me toward the code, I wasn't sure what I was looking at, but intuitively the structure and syntax of the language made sense. I was pretty sure I "got it" and was willing to start tinkering with it, but I couldn't be certain, it was just guessing off intuition, but I would not have been confident enough to invest the couple of hours it takes to set up a compiler for the first time just to find out, especially since at that time I didn't even know what a compiler was. Using an interpreter language like python allows for easier access into the whole process of programming for novice modders who otherwise are unwilling or unable to code in the conventional sense.

Get rid of Python. That language drives me nuts... whitespace for delimiters? come on...

Get rid of Python. That language drives me nuts... whitespace for delimiters? come on...

That was my initial aversion as well. But seriously, get an editor instead of Notepad and you'll never have an issue with it again. The fact that someone can use Notepad to mod the game vastly increases the likelihood of getting new modders.

To me, avoiding Python due to its use of whitespace for indentation is like turning down a free Testarosa because it's not red. :crazyeye:

I was pretty sure I "got it" and was willing to start tinkering with it, but I couldn't be certain, it was just guessing off intuition, but I would not have been confident enough to invest the couple of hours it takes to set up a compiler for the first time just to find out, especially since at that time I didn't even know what a compiler was. Using an interpreter language like python allows for easier access into the whole process of programming for novice modders who otherwise are unwilling or unable to code in the conventional sense.

:yup: right, that's the point of a scripting language being in for modders.
But hopefully they'll use one, which is easier to merge and to deal with. First thing to mention here is sure perl (which i'm not capable of at the moment, but re-learning for something you want to do is not hard ;)).

I'll tell you why STL is harder than Python:
You chose to write this:
for(unit_t::iterator it = eUnit.begin(); it != eUnit.end(); ++it)
it->do_something();
I'd have written that:
for(unit_t::const_iterator it = eUnit.begin(); it != eUnit.end(); ++it)
it->do_something();
Until the next version of C++ where the auto keyword might be used to infer the iterator type... Not only is that about twice as verbose as python (count the number of statements or characters), but if someone advocating C++ forgets the const-ness part, how do you expect someone to learn it?
C++, STL are way harder to learn than python. Furthermore, unit_t is also extraordinarily decieving. The type you'll be using is likely something like that:
std::vector< Unit >::const_iterator.

At which point you probably typedef'ed it away into unit_iter, CIT_Unit or whatever.
Although very powerful, the stl sometimes makes your life hard where python doesn't. Try that for instance: std::vector< std::auto_ptr< argh > >. You'll probably end up using boost::shared_ptr instead. A python user will just totally ignore all that and will be finished coding when his fellow newbie C++ coder will still be reading why it was a bad idea to put that kind of pointers in this kind of container.

C++ is a very powerful and efficient tool, but it's not something for everybody to use, and most modders shouldn't ever try to use it.

But part of the main benefits of using python was that it is an interpreter language, and doesn't require setting up a compiler to start messing with. Alot of hobby modders got their start programming in python (like myself), and simply would not have been willing to jump into actual coding if they would have had to have gone through the whole process of setting up the compiler up front; this is a very daunting task for someone who isn't sure if they really understand what they are looking at. For me I needed to fix a python function another modder had set up, and they pointed me toward the code, I wasn't sure what I was looking at, but intuitively the structure and syntax of the language made sense. I was pretty sure I "got it" and was willing to start tinkering with it, but I couldn't be certain, it was just guessing off intuition, but I would not have been confident enough to invest the couple of hours it takes to set up a compiler for the first time just to find out, especially since at that time I didn't even know what a compiler was. Using an interpreter language like python allows for easier access into the whole process of programming for novice modders who otherwise are unwilling or unable to code in the conventional sense.

In what way is it harder to set up a compiler than in interpeter?

To run a Python program:

python prog.py


To compile a C program:

gcc prog.c

Have you ever set up the compiler for Civ4? Try it and tell me a novice could do it. ;)

The way the python is loaded in Civ, there is no setting anything up. You just edit the python file with notepad++, then start up the game to see if your code works. With the SDK it's not that simple, setting up the compiler is a non trivial process the first time, and requires careful attention to about a 20 step process.

Yeah, compiling from the command line is easy. Compiling with a GUI, and the non-standard VS libraries, that's hard. I still don't understand the steps to set up civ4's SDK and have to read from the tutorial every time I need to set it up.

Python doesn't have to be interpreted. Star Trek: Bridge Commander was based entirely on compiled python files; it would simply compile any new/updated files at runtime. Civ4 has a bunch of compiled python files in the Python\System directory.

Have you ever set up the compiler for Civ4? Try it and tell me a novice could do it. ;)

Yes.

I installed Visual Studio 2003. I opened up the project. I compiled using the built-in menu.

Yeah, compiling from the command line is easy. Compiling with a GUI, and the non-standard VS libraries, that's hard. I still don't understand the steps to set up civ4's SDK and have to read from the tutorial every time I need to set it up.

Python doesn't have to be interpreted. Star Trek: Bridge Commander was based entirely on compiled python files; it would simply compile any new/updated files at runtime. Civ4 has a bunch of compiled python files in the Python\System directory.

Those aren't technically machine-code though. Its still "interpreted" in a way. Think of how the java system works. You compile into a class file, but then that has to be JIT-compiled by the java virtual machine, which is similar to an interpreter.

Yes.

I installed Visual Studio 2003. I opened up the project. I compiled using the built-in menu.

It's a lot more complicated for those of us that don't have Visual Studio 2003 (that is, most of us). Even later versions of Visual Studio need to have a special setup so you can compile using VS 2003's libraries.

First thing to mention here is sure perl.

I'm pretty sure I read that wrong. Perl? PERL?! The most cryptic procedural programming language out there, Perl would be horrible as a modding language. I won't elaborate because I'm still pretty sure you didn't just suggest Perl. :D

In what way is it harder to set up a compiler than in interpeter?

As phungus420 answered, the compiler is harder because you don't have to set up the Python interpreter at all to do modding. Heck, you can even modify most of the Python modules on-the-fly and see your changes in Civ4 without restarting it.

I installed Visual Studio 2003. I opened up the project. I compiled using the built-in menu.

Huh? Does the VsProject file even work? I tried opening it before, but it never compiled for me. Getting Civ to compile on VS 2008 was not trivial.

Huh? Does the VsProject file even work? I tried opening it before, but it never compiled for me. Getting Civ to compile on VS 2008 was not trivial.

Civ4 was develop on VS2003. The project file is as pointed out earlier, simply install VS2003, open project file, compile. No errors. :)

VS2005 & VS2008 are different library sets, so of course they're going to be complicated. :rolleyes:

Have you ever set up the compiler for Civ4? Try it and tell me a novice could do it. ;)

1) If you own VS 2003, it's a snap. Like others have said, just open the file... Do YOU know how to use VS 2003?

2) If you don't own VS 2003, there is a long-standing thread on these forums describing in detail how to use the free version of VS 2005 or VS 2008.

To me, avoiding Python due to its use of whitespace for indentation is like turning down a free Testarosa because it's not red. :crazyeye:

Why would you want a Testarossa if it wasn't red? *grin*

1) If you own VS 2003, it's a snap. Like others have said, just open the file... Do YOU know how to use VS 2003?

2) If you don't own VS 2003, there is a long-standing thread on these forums describing in detail how to use the free version of VS 2005 or VS 2008.

If the people requesting modding be easier were willing to read tutorials, there would be a lot less people requesting modding be easier.

I'm pretty sure I read that wrong. Perl? PERL?! The most cryptic procedural programming language out there, Perl would be horrible as a modding language. I won't elaborate because I'm still pretty sure you didn't just suggest Perl. :D


Guess it, i don't really know anything about Perl :D.
I've just mentioned it, because it's mentioned together with python when somebody talkes about scripting languages.

I have a perl script here, it looks not that good readable like python, but it shouldn't be hard for a coder to learn it.
And i just hope, that, if there'll be a scripting language included, it will be one, where the intendation doesn't matter. Would debuging (not for me, debuging other people's merges) make a lot easier.

Guess it, i don't really know anything about Perl :D.
I've just mentioned it, because it's mentioned together with python when somebody talkes about scripting languages.

I have a perl script here, it looks not that good readable like python, but it shouldn't be hard for a coder to learn it.
And i just hope, that, if there'll be a scripting language included, it will be one, where the intendation doesn't matter. Would debuging (not for me, debuging other people's merges) make a lot easier.

I've seen some INSANE programs written in scripting languages. Awesome stuff.

So don't 'write off' script languages :P

Perl? :wallbash: If the idea is to make things unworkable for 99% of the amateur coders , better go to the basics and sugest ForTran :D

for k, v in d:
print k, v


What's the equivalent code in C++? Exactly.

template<typename Container>
void print_container( Container const& container )
{
for (Container::const_iterator it = container.begin(); it != container.end(); ++it)
{
stdout << it->first << ", " << it->second << "\n";
}
}

Note that the above does full type-checking when you compile the code.

The python code compiles and runs ... until it hits the point where something went wrong. It then fails catastrophically, because the type of d wasn't what you thought it was.

Delayed type checking has a cost, especially when you are working in an environment where your test harness is a fully running game of Civ, and the code is called at a nearly arbitrary part within it.
Of course the STL simplifies makes writing C++ code easier. I couldn't imagine writing my own C++ data structures. But that's not the point. Someone above suggested using C++ in place of Python, and you simply cannot convince me that C++ even with the STL is easier for amateurs to learn than Python.
*nod*. But Python keeps the 10 to 100 fold slowdown problem, which is important when your AI relies on a combinatorial examination of options to determine what it wants to do.

AI in civ4 was the slow part of "next turn".

AI in civ4 was the slow part of "next turn".

This is why the AI is in C++ and the interface is in Python (except hover text). Sure, there are Python callbacks and events as a modding extension point, but nothing forces the modder to use that. You are free to do it all in C++.

But that is why python can be extended and/or embedded with c/c++. Python is not very fast with heavy math. For that you either create a C++ module with python bindings or alter the sdk code.

AI in civ4 was the slow part of "next turn".

Technically correct, but not totally true. There are/is 2 major bottlenecks for turn times in Civ4.

1.) Python callbacks. Don't use them. Disable those in the SDK that Firaxis didn't add callbacks for. The python to C++ parser they used was slowww...

2.) Units. Unit pathfinding, unit grouping, unit decision making.... The sheer number of units late game really hurts the time. It's one reason I'm optimistic about Civ5's 1 unit per tile rule, there will be a lot less units, and that basically eliminates this problem entirely. They could use the Civ4 engine again (I'm not saying they are or will), and get 50-75% boosts in speed.

But that is why python can be extended and/or embedded with c/c++. Python is not very fast with heavy math. For that you either create a C++ module with python bindings or alter the sdk code.

None of the python in Civ4 is used for much math, other than simple counting. If you're using python for heavy math, you're doing it wrong. ;)

2.) Units. Unit pathfinding, unit grouping, unit decision making.... The sheer number of units late game really hurts the time. It's one reason I'm optimistic about Civ5's 1 unit per tile rule, there will be a lot less units, and that basically eliminates this problem entirely.

Moving from squares to hexes will make a big difference in pathfinding as well, dropping from 'up to 7' to 'up to 5' choices of direction may not seem like much but over the course of a long path it will be very significant.

I am also hopeful that on top of a reduction in the number of units the long term planning component of the AI should result in far fewer calls to lengthy 'what should I do with this unit' methods (for every unit on every turn) as the AIs and thus their units become committed to a preplanned course of action until some event or trigger causes a reevaluation.

Moving from squares to hexes will make a big difference in pathfinding as well, dropping from 'up to 7' to 'up to 5' choices of direction may not seem like much but over the course of a long path it will be very significant.

I am also hopeful that on top of a reduction in the number of units the long term planning component of the AI should result in far fewer calls to lengthy 'what should I do with this unit' methods (for every unit on every turn) as the AIs and thus their units become committed to a preplanned course of action until some event or trigger causes a reevaluation.
Yes, and hopefully they can make the Unit AI much more map aware in Civ5 because of all this extra time.

If you're using python for heavy math, you're doing it wrong. ;)

If you're still talking about Civ, and don't count algorithms into math, you're right ;).

Naw, dropping from squares to hexes won't help much, especially with infantry move increasing to 2 (and hence every other unit getting faster).

The real thing that will help is if they rework their A* algorithm to (A) cache smarter, and (B) give up when the path gets too long (passed-in parameter).

There is lots of ugly code that basically is about keeping the pathing search space small, and every time the ugly code springs a leak, things get slow again.

Naw, dropping from squares to hexes won't help much, especially with infantry move increasing to 2 (and hence every other unit getting faster).

The real thing that will help is if they rework their A* algorithm to (A) cache smarter, and (B) give up when the path gets too long (passed-in parameter).

There is lots of ugly code that basically is about keeping the pathing search space small, and every time the ugly code springs a leak, things get slow again.

Why would the movement speed of units negatively affect the computational speed of pathing algorithms?

I can see that their A* might be poorly written or inefficient (hopefully the phrase 'an astonishing new engine built from the ground-up' means this has been fixed!) but I just don't see how increasing a units move to 2 tiles from 1 would affect the time it takes to find a path.

Reducing the number of choices at each node on the path on the other hand would clearly have a significant effect since the number of computations for n choices at each node on a path of length x is more closely related to x^n than x*n

Sorry if I am being dense.

Why would the movement speed of units negatively affect the computational speed of pathing algorithms?
In the Civ4 AI, units explore a certain multiple of their speed to find a target.

The faster they move, the larger the area they have to search to find targets they could attack in a round or two.
I can see that their A* might be poorly written or inefficient (hopefully the phrase 'an astonishing new engine built from the ground-up' means this has been fixed!) but I just don't see how increasing a units move to 2 tiles from 1 would affect the time it takes to find a path.
Because they have to search a larger radius of area for valid paths.
Reducing the number of choices at each node on the path on the other hand would clearly have a significant effect since the number of computations for n choices at each node on a path of length x is more closely related to x^n than x*n
What you care about is how many tiles there are in a radius of n away from a location.

Now, as unit velocity goes up, n goes up.

As the branching factor of the game goes up, the base goes up.

On a square map, the number of squares within distance n is (2n+1)^2 = 4n^2 + 4n + 1

On a hex map, the number of squares within distance n is ...
http://en.wikipedia.org/wiki/Hexagonal_number
(2n+1)(2(2n+1)-1)
=(2n+1)(4n+1)
=8n^2+6n+1
That isn't quite right. It should be 1 7 19 37 (+6, +12, +18) or so, as opposed to 1 9 25 49. (+8,+16,+24, etc).

But in any case, it will end up also being an O(n^2) number. Hexagonal ends up being about 3/4 the volume at a given radius, but with unit velocity being faster, the search space blowup from a 50% to 100% boost in speed (for the same number of turn lookahead) is a factor of 2 to 4.

Worst case pathing situations (where you have problems reaching your target) end up having to search a volume whose radius is roughly proportional to n. The fact that one is a hexagonal tiled, and the other is square tiled, won't matter much.

In the Civ4 AI, units explore a certain multiple of their speed to find a target.

The faster they move, the larger the area they have to search to find targets they could attack in a round or two.
...


Yakk, Thank you for the detailed explanation.
I really appreciate your taking the time to explain all that.

I was fixated on using the algorithm to find the fastest route between two known points. Using the same algorithm to find potential destinations is certainly a different use case than I was considering and I can see how that requires additional inputs and would significantly increase the effort involved.

Thanks again.

To be honest, I doubt we're going to see any awesomely efficient path-finding algorithms in Civ5, if only because they appear to be putting strong limits on the number of units you can have so it's a much smaller issue. That doesn't make it any more pleasant for all those modders who already desire to undo the 1-unit-per-tile rule. (Has that actually been confirmed yet? Some posters seem to be speaking like it has been confirmed)

Hypothetically speaking, let us say a Civ4 editor is being developed, say for units. Assuming, as Dale has suggested, the schemas are being used so that the UI widgets can be dynamically generated, how should those schemas and their xml files be parsed?

Could/Should the Civ4 dll be hijacked and modified to remove/ignore the actual game code so that its parsing code can be reused, or would their be some third-party library that could be used to do all the work?

This is a topic that has piqued my interest as my preferred field is GUI development and if the xml parsing with schemas is possible (ie without hand writing a parser) my interest might become more then just piqued. :)

Hypothetically speaking, let us say a Civ4 editor is being developed, say for units. Assuming, as Dale has suggested, the schemas are being used so that the UI widgets can be dynamically generated, how should those schemas and their xml files be parsed?

Could/Should the Civ4 dll be hijacked and modified to remove/ignore the actual game code so that its parsing code can be reused, or would their be some third-party library that could be used to do all the work?

This is a topic that has piqued my interest as my preferred field is GUI development and if the xml parsing with schemas is possible (ie without hand writing a parser) my interest might become more then just piqued. :)

There are quite a few third party libraries for parsing XML.

A tool for creating leader art would be good. If they check out the avatar editor in Cities XL, that is a great example. It allows you to alter size and shape of several facial features, and allows modification of skin/hair colour, haircut, clothing, build and accessories (glasses, moustache etc). YOu can make almost anyone without being an artist.

The ability to import art and civ/lh modules from Civ III or Civ IV would be great too. Gaps in XML properties could be solved by using inbuilt defaults for missing values. Some default art files (eg a default unit or leaderhead that appears only when there is missing art) would be good for preventing crashes to desktop when modding.

Gaps in XML properties could be solved by using inbuilt defaults for missing values.
Yes. Actually, most xml elements don't even need to be there as they are always empty. Making these optional would go a great length in order to simplify modders' work.

If you use a DOM parser for the XML, it will give you a tree of elements and attributes that is much easier to walk generically. Do not try to use the SDK code as it's so brittle and not at all generic. Depending on the language you'll use, I would bet you can even find a library to parse a schema (or massage a DOM) into a schema-specific data structure.

Making these optional would go a great length in order to simplify modders' work.

I don't think that's true. It would seem to be easier at first, because there won't be a zillion tags with the default values making the file intimidating, but it would actually be harder for all but the simplest things. For example, let's say you want to allow a unit to explore enemy territory that couldn't before. Just add the tag, right? WRONG. The tag has to go in the right place (if the tags aren't in the right order, you will break your mod). How do you find the right place? If you're lucky, there will be an element that does use that tag that will have the tags that the element you're modifying which are above and below the tag you're adding. If not, you'll need to look at the schema file to find out where it goes, which is more intimidating than having the default values there to begin with.

I don't think that's true. It would seem to be easier at first, because there won't be a zillion tags with the default values making the file intimidating, but it would actually be harder for all but the simplest things. For example, let's say you want to allow a unit to explore enemy territory that couldn't before. Just add the tag, right? WRONG. The tag has to go in the right place (if the tags aren't in the right order, you will break your mod). How do you find the right place? If you're lucky, there will be an element that does use that tag that will have the tags that the element you're modifying which are above and below the tag you're adding. If not, you'll need to look at the schema file to find out where it goes, which is more intimidating than having the default values there to begin with.
No. There's no necessary order in the tags. If they build a DOM tree, it really doesn't matter. If they use SAX, order may matter, but only if they do it wrong. In general, if elements need an order, they must be nested. If elements at the same level of the xml need to be ordered, then there's probably a flaw in the design of the xml format.
Ordered elements may make sense for some documents, but for game data, I can't see why one would want something that complicated. considering itt's easier not to bother about the order of elements, I can't see why they'd do it.

Hopefully they'll write their parsing code so it doesn't depend on the order of elements. There's simply no reason to require this other than to allow writing brittle parsing code. It's easy enough to read the entire document into a DOM object and walk the tree searching for elements or use a SAX parser for maximum modder hostility. :)

Either way would allow them to remove all the empty XML elements without making life harder for modders. A modder would be able to add a missing tag anywhere in the parent element.

If you use a DOM parser for the XML, it will give you a tree of elements and attributes that is much easier to walk generically. Do not try to use the SDK code as it's so brittle and not at all generic. Depending on the language you'll use, I would bet you can even find a library to parse a schema (or massage a DOM) into a schema-specific data structure.

The language will be C++ and I agree that DOM would be more appropriate then SAX. I've done some work with non-schema xml files using TinyXML library which is DOM, so I know that would work. Could load up two DOM objects, one with the schema and the other of the xml file and then use one to validate the elements in the other. However, that seems like a lot of grunt work to do by hand.

At the moment, I am doing some research towards the idea of using an xml data binding code generator to create the initial classes based off the schema files, but that still has the drawback of the xml elements being hard-coded at compile time, or at least that is how it seemed from what I read.

I am looking for a library that handles a schema as a natural part of parsing an xml file, but the best googling skills don't help when your searching for broad words like xml, schema, parser, etc. I am going to keep looking as this is something I need to handle regardless of the Civ editor issue, but maybe I'll get lucky and some one will know of a good library that can handle the task.

This is getting deep, and you may want to start a new thread specifically about it. In the meantime I would suggest loading the data into your own tree structure (or leave it in the DOM depending on how nice the API is). There's no reason to parse it into a field-level data structure since this is just for editing values and saving back to XML.

You'd use the schema to generate the UI components and the data XML for the values. I can't help with tool selection as I haven't had to do XML in C++ before. God help anyone who has to. Oh, er, I guess that's you. :mischief:

Hehe, yea, I was also thinking this would need its own thread (in the Civ4 forum maybe ;)), but I'm going to do a little experimenting with ways to design the UI and layout a base plan so specific issues can be addressed before I start the thread.

I'm not going to promise that I'll code this, but I love making tools and creating them to access xml data is very high on my list of priorities. And hey, even failing is a success if you learn something from it. :)