1. We have added the ability to collapse/expand forum categories and widgets on forum home.
    Dismiss Notice
  2. All Civ avatars are brought back and available for selection in the Avatar Gallery! There are 945 avatars total.
    Dismiss Notice
  3. To make the site more secure, we have installed SSL certificates and enabled HTTPS for both the main site and forums.
    Dismiss Notice
  4. Civ6 is released! Order now! (Amazon US | Amazon UK | Amazon CA | Amazon DE | Amazon FR)
    Dismiss Notice
  5. Dismiss Notice
  6. Forum account upgrades are available for ad-free browsing.
    Dismiss Notice

The Modder's Guide to Civ4

Discussion in 'Civ4 - Creation & Customization' started by WildWeazel, Oct 24, 2005.

  1. WildWeazel

    WildWeazel -- Turns to Completion

    Joined:
    Jul 14, 2003
    Messages:
    6,779
    Location:
    %CIV3%\Conquests\Scenarios\
    DON'T PANIC!

    "Civ4 is big. You just won't believe how vastly, hugely, mind-bogglingly big it is. I mean, you may think it's a long way down the road to XML, but that's just peanuts to Civ4." -Douglas Adams, and me


    So you pre-ordered the game, you've seen every screenshot, and you've been scouring the forums for months. You finally got your hands on that game, and you just can't get enough of it. And if you're like me, there's only one thing on your mind: how am I going to mod this amazingly deep and complex game?

    If you want to mod Civ4, this thread is for you. I'm digging into the modding capabilities of Civ4, and sharing all of it with you. I will post everything that I know in this very thread. And if I don't know it, I'll link to it.

    Whether you're a conscript newb who has never played Civ before, a regular modder like me who hasn't quite mastered all there is to Civ3 yet, or a veteran programmer who just can't wait for the SDK, this thread is for you.

    QUICK LINKS:
    Post 2: Modding Overview
    Post 3: Graphics
    Post 4: World Builder
    Post 5: XML Modding
    Post 6: Python Scripting
    ---------------
    Graphics
    DDS Converter
    jak08's DDS Skin Tutorial
    RAD Game Tools

    World Builder
    Dale's World Builder Manual
    BMP to WBS Converter
    Dale's In-Depth Look at the WBS

    XML
    Civ4Wiki
    W3Schools.com XML tutorial
    XML.org Beginner's Guide
    BrentRichie's XML Tutorial Part 1: Adding Civs
    Part 2: Civ Info
    Part 3: Leaders
    XML Quick Reference by Max_x_fire
    Adding New Units by sovarn

    Python
    Civ4Wiki
    www.Python.org
    GBM's Python Tutorial and Website
    Locutus' Python Documentation
    Civ4 Python API
     
  2. WildWeazel

    WildWeazel -- Turns to Completion

    Joined:
    Jul 14, 2003
    Messages:
    6,779
    Location:
    %CIV3%\Conquests\Scenarios\
    Modding Overview:


    As you probably know, there will be 4 levels of modding in Civ4:

    World Builder: This is the equivalent of the map editor in Civ3, but much more powerful. Here you can create custom maps and basic scenarios. You can edit terrain, place cities and units, and much more. The world builder will ship with the game.

    XML: This is roughly equivalent to the rules editor in Civ3. This is where you can add units and improvements, edit the civilizations, and tweak just about every value in the game. XML can be written in notepad or an XML editor. XML is similar in appearance to HTML, and is fairly simple to learn once you understand the basic concepts. A great XML tutorial can be found at w3schools.com

    Python: Python is a programming language that is used in Civ4 for the game interface, map generation, scripting events, and more. Gingerbread Man is building an excellent Python tutorial focused on Civ4. Several lessons are already available. Python can also be written in notepad. The Python Integrated Development Environment (IDLE), which allows you to run Python files, can be downloaded for free at www.python.org.

    SDK: The Software Development Kit is where the hard-core modding will be done. Everything but the graphics engine and the core Civ engine will be available to modders. Programming will be done in C++. The SDK will not be available until early 2006.

    Graphics: In addition to these forms of modding, there will also be a demand for graphics. Units, icons, and the user interface will all require new graphics as new mods and scenarios appear. Depending on what graphics are involved, they may require XML, Python, or even the SDK in order to be incorporated into the game.

    Creating a mod
    Once you find something you want to change, you can make a mod for it. When you create a mod, no matter how small, create a new folder for it inside the Civilization 4\Mods folder with the name of your mod. DO NOT overwrite the original files with your own. If you are making a lot of small changes to the game to fit your own preferences, you will probably want to play this mod every time, instead of the standard game.
    Fortunately for you, you can tell Civ4 to run a certain mod every time it loads. There are a few ways to do this. The first is the Custom Assets folder. You can store files in here that you want to override (not onerwrite) the original files every time the game loads. When Civ4 loads, it first takes these files, then follows the designated file path. The Custom Assets folder is a good place for small changes such as textures and text that you want to use for all games.
    If are creating a specific mod that you want to load automatically when Civ4 runs, you can direct the game to it from the main configuration file. In _Civ4Config.ini, in the main Civ4 folder, there is a line "Mod = 0". By setting this to a specific folder, such as "Mods\MyMod", you can make Civ4 load a certain mod automatically.
    It is not necessary to copy all files into your mod folder- only those that you have changed from the originals. Like Civ3's scenario search folders function, Civ4 will look first in Custom Assets and take any files it finds. It then moves on the to the specified mod, if any, and takes whatever files that it doesn't already have, before getting all remaining files from the main folder setup.

    CLEAR YOUR CACHE!!!!!11!!!1!!!1!1!11one1!!!
    If your mod is not loading properly, (ie, the game says it loaded but nothing changes), you may need to clear your cache. Civ4 saves a lot of files in a cache folder and uses them instead of reloading the files, so sometimes changes don't appear as they should. Go to C:\Documents and Settings\USERNAME\Application Data\My Games\Civilization 4, show hidden files and folders, and delete the cache folder. You may need to do this periodically as you make changes.

    Mod Structure
    The Assets folder, which contains all files necessary for a complete mod, contains a massive 559 folders, and over 10,000 files. Fortunately for you, you don't have to duplicate it. Unlike Civ3, when you had to create the folder setup to mirror the original, Civ4 does this for you. When you run the game using a mod, it automatically creates a full folder setup in the mod folder. As you add more files to your mod, simply place them in the same sub-folder that the original was in. The basic unpacked* structure looks like this:

    *If your Civilzation 4\Assets folder DOES NOT contain these subfolders, you will need to decompress the file art0.fpk with the PakBuild utility from Firaxis. art0.fpk is a huge compressed file similar to a RAR archive that contains most of the files and folders from Assets. After you unpack the file, move art0.fpk to a different location or rename it. This is because Civ4 will look for this file before looking in the Assets folder. If you do not move or rename the file, Civ4 will continue to load the default files within the package.

    Basic guidelines
    As with all modding (or anything in life for that matter), there a few guidelines to follow:
    Back up your work. Before you start messing with anything, save a copy of the original (or the latest version of your own work, for that matter) in a place where it won't get deleted, overwritten, lost, or eaten. The only way to recover the original is to reinstall the game. And trust me, your hard drive doesn't want that. And whatever you do, don't come begging for somebody to send you an original file. It's called piracy. And Chieftess will eat you alive. You think I'm kidding, don't you?
    Use the Mod folder setup. In other words, DO NOT replace the original files with your own work. First of all, if you did something wrong you might have to reinstall to fix it. Secondly, you may want to play the game as it originally was, instead of your modded version. And thirdly, you won't get any sympathy when you complain about your game being messed up. ;) The Mod folders work the same way as they did for Civ3- just put whatever files your mod or scenario needs into its own folder, copying the setup of the main folder structure. For example, if you create a new XML file, put it in Mods\MyModName\Assets\XML\
    Ask questions. If you don't know how to do something, or want to learn more, don't be afraid to ask. We're all learning this together (except for those beta-testers :p) OTOH, some of us have been modding Civ3 for a while, and some of the same concepts will carry over. We're happy to help new modders, because that means (a) more people to help us make our mods and (b) more mods for us to play. :mischief: No, seriously, we help because we enjoy modding and want other people to enjot it as well. So jump right in and start learning!
    Have fun. This is the most important thing to remember. It's a game. It's meant to be enjoyed. You can gain a lot of computer-related skills by modding, but that's just a bonus. The real benefit is doing something you enjoy, seeing the product of your hard work and dedication, and working with some really cool people along the way. That's what C&C is all about.
     
  3. WildWeazel

    WildWeazel -- Turns to Completion

    Joined:
    Jul 14, 2003
    Messages:
    6,779
    Location:
    %CIV3%\Conquests\Scenarios\
    GRAPHICS

    Graphics in Civ4 are more complicated than in previous games. Because of the 3D environment, most graphics consist of models and textures instead of single images or animations.

    Units
    We are not currently able to create new models and animations. Until Firaxis releases the SDK or a converter plugin for the Gamebryo engine, we have to use the models that shipped with the game. Gaias explained it to me in another thread:
    While we can't make custom models yet, that doesn't mean we can't have new units. Units can be "reskinned" by editing or replacing the textures. With a DDS Converter, you can export the dds texture files to standard bmp images, edit them, and convert them back. A single model can be used with dozens of different textures to create a variety of custom units. jak08 wrote a short DDS Skin Tutorial.
    Buildings
    Leaderheads
    Terrain
    Buildings, Leaderheads, and terrain all use the same graphics format as units- a combination of NIF, KF, and KFN files, with DDS textures. Terrain is especially complicated. It encompasses a massive 1,217 files distrubuted among 168 subfolders. Terrain modelling in Civ4 is not for the weak.
    Buildings and Leaderheads are somewhat limited at this point because buildings are not culture-specific, and LHs are not era-specific. Until the SDK is released, These graphics will be limited to simple skin replacements only.
    Interface
    The game interface is comprised mainly of DDS textures, making it the most moddable set of graphics at this point. The interface is run by Python.
    Movies
    Polyphemous discovered the program used to make bik movies for Civ3, and it turns out that Civ4 uses the same format. The Bink Tools program is a free download from RAD Game Tools.
     
  4. WildWeazel

    WildWeazel -- Turns to Completion

    Joined:
    Jul 14, 2003
    Messages:
    6,779
    Location:
    %CIV3%\Conquests\Scenarios\
    The World Builder

    The World Builder (WB) is the part of Civ4 that is used to create new maps and scenarios. It can be loaded at any time during the game. Many people find the WB slow and clucnky, but fortunately there are other ways to edit the saves. An interesting thing about the World Builder Saves is that they are saved in ordinary text format. You can open, edit, and save them using the most powerful computer program known to man- NOTEPAD. The file is easy to understand, and for small changes it's probably easier to tweak the text rather than loading it in the World Builder. There is also a converter that creates maps from a series of bmp images.

    Manual: Dale (who turned out to be another one of those elusive beta testers) has made a great World Builder Manual in pdf format.
    Manual (7.1 mb)

    Making Maps: There are 3 ways to build new maps. You can use the WB to make it in-game, you can manually set it up in Notepad, or you can use the BMP to WBS converter to generate maps using bmp images.

    The World Builder Save (WBS): Dale is also working on a complete guide to editing the WBS in text format in this thread.

    Unlock the full capabilities: If you have tried to use the World Builder, you have probably noticed that a lot of the functions have been disabled. This is to keep the World Builder from being a cheat machine, but you can't make good scenarios. Fortunately, Soren Johnson released the "cheat code": In the main ini file, which can be accesed through "Shortcut to _Civ4Config" in the main Civilization 4 folder, replace the line "CheatCode = 0" with "CheatCode = chipotle". This will unlock all the features of the World Builder, allowing complete scenario control.
     
  5. WildWeazel

    WildWeazel -- Turns to Completion

    Joined:
    Jul 14, 2003
    Messages:
    6,779
    Location:
    %CIV3%\Conquests\Scenarios\
    XML Modding

    XML is short for Extensible Markup Language. It appears similar to HTML, but serves a different purpose. While HTML displays information, XML is used to organize information. The unique thing about XML is that there is noset, predefined system. When you use XML, you make up your own tags based on what your program needs to do. XML itself is not a programming language. It's a "metalanguage"- it's used to define new languages.

    So what does this have to do with Civ4? Civ4 uses XML to organize information about the game. Instead of the complicated rules editor from Civ3, with obscure labels and buggy functions, Civ4 uses XML to arrange all of the necessary information so that it is easy to read and edit.

    I have found 2 good online XML tutorials that I would recommend. They are much more in-depth than my simple introduction, and are very easy to follow.
    w3schools.com: XML Tutorial

    XML.org: XML Beginner's Guide

    (If you want to jump right in and see some XML in action, keep reading. If you just want to know what this has to do with Civ4, skip this section.)

    --------------------------------------------------------------------------
    --------------------------------------------------------------------------
    Spoiler XML crash course :

    That's all great, but what does XML look like, and how does it work? If you are comfortable with HTML, or even the VB code used in your posts, then learning XML should be no problem. It consists of tags like <b>...</b> that wrap around pieces of information to describe them. To see a good example of XML in use, look no further than your PMs! You can download PMs in XML format- there is an option in the menu where you delete them.

    As an example, here is a PM that I sent to myself:
    Code:
      <?xml version="1.0" encoding="ISO-8859-1" ?> 
    - <!--  Civilization Fanatics' Forums;http://forums.civfanatics.com 
      --> 
    - <!--  Private Message Dump for User Weasel Op; Oct 25, 2005 09:31 AM 
      --> 
    - <privatemessages>
    - <folder name="Inbox">
    - <privatemessage>
      <datestamp>2005-10-25 09:31</datestamp> 
      <title>XML in Civ4</title> 
      <fromuser>Weasel Op</fromuser> 
      <touser /> 
      <message>Hey, did you know that Civ4 uses XML? :D</message> 
      </privatemessage>
      </folder>
      </privatemessages>
    Now that probably looks awfully complicated, especially if you have never seen HTML, but don't worry. I'll explain how it works.
    Code:
    <?xml version="1.0" encoding="ISO-8859-1" ?> 
    - <!--  Civilization Fanatics' Forums;[url]http://forums.civfanatics.com[/url] 
      --> 
    - <!--  Private Message Dump for User Weasel Op; Oct 25, 2005 09:31 AM 
      --> 
    That part just tells the program reading the file, and you the coder, what this is. The first tag informs the program that this is an XML file, written in version 1.0, and that it uses the Western European character set. The next 2 lines are comments- notice the <!-- and -->. Everything between those is a comment, and is ignored by the program. It's just there for you, so you know what this is.

    Next comes the good stuff:
    Code:
    - <privatemessages>
    - <folder name="Inbox">
    The first tag that really does something is the <privatemessages> tag. This tag tells the computer that everything from this tag through the closing tag (a tag with a / before the name, in this case </privatemessages>) is an element. An element is a piece of information. It can contain other elements, text, or both. In this case, <privatemessages> begins an element that contains all of the private messages in the file. (There is only 1 here, but if I had saved my entire Inbox they would all be in there.)

    Next is the <folder> tag. Notice that this tag does not only have a name, but a description. This description is called an attribute. An attribute is a little piece of information included in an element tag. It gives the element a characteristic, in this case the name "Inbox." This is an example of the flexibility of XML. "Inbox" could have been a separate element within "folder" like this:
    Code:
    - <privatemessages>
    - <folder>
    - <Inbox>
    Here, "Inbox" is an element within "folder." However, "Inbox" is better as an attribute because logically, "Inbox" is a specific folder, not a part of a folder.

    Are you getting the hang of it yet? Let's move on. This is where things start getting interesting:
    Code:
    - <privatemessage>
      <datestamp>2005-10-25 09:31</datestamp> 
      <title>XML in Civ4</title> 
      <fromuser>Weasel Op</fromuser> 
      <touser /> 
      <message>Hey, did you know that Civ4 uses XML? :D</message> 
      </privatemessage>
    That's a big chunk of code, but if you look closely it folloes the same pattern. This is the PM itself, the actual information that I sent myself. Notice the <privatemessage> tag. This is NOT the same as <privatemessages>. That tag defined everything in it to be a group of private messages, but this tag defines itself to be a single PM. Look at the bottom line. See that closing tag? It has the same name, but a slash before it: </privatemessage>. This is the closing tag for <privatemessage> indicating the end of that element. Everything in between, including the tags themselves, are part of the element.

    The second line gives us the date and time the message was sent. The <datestamp> tags wrap around it, indicating that it is also an element. Similarly, the next line is the title of the PM, and the line after that indicates who the message is from. Keep in mind that each of these elements are part of the <privatemessage> element, which is part of the <folder> element, which is in turn part of the <privatemessages> element.

    The next line looks a little different. This tag combines the opening and closing tags into one. Since the line is blank, there is no need for two separate tags. Adding the slash to the end after the slash is like xml shorthand- it opens and closes the element with the same tag. The "touser" element is blank becuase this is my own PM. If it were addressed to someone else, it wouldn't be in my inbox!

    The next element is the heart of it all. This is the actual text of the PM, enclosed in the <message> tags. Depending on the PM, this element could inclde several lines of text, VB code, etc. But the important thing to note is that it's all text. There is nothing magic about XML. There is no compiler, no functions, no special characters. Everything is saved, sent, and edited in text format only.

    As I mentioned, the last line indicates the end of the <privatemessage> element.

    Now let's look at the last piece of code:
    Code:
      </folder>
      </privatemessages>
    By now you can probably guess what these do. </folder> closes the "folder" element. Notice that the attribute "Inbox" is not included here. The attribute is a part of the element, not an element itself. In fact it can't be closed because it was never opened. It's just a characteristic of the "folder" element. If there's no more "folder", it can't be an "Inbox". The final line closes the <privatemessages> element.

    Now you may be wondering, why are there 3 elements inside of each other like that? If you look back at the entire piece of code, you will see that the 3 major elements open and close together:
    Code:
    <privatemessages>
    - <folder name="Inbox">
    - <privatemessage>
    .
    .
    . 
      </privatemessage>
      </folder>
      </privatemessages>
    Why couldn't those all be one element? Actually, the only reason they appear that way is because I only saved 1 PM. If I had downloaded all of my PMs in one file, it would be different. <privatemessages> would still contain everything, because that is the root element. That just means that is the primary, basic element of the file. The whole file is essentially one element, <privatemessages>. Inside <privatemessages> would be two <folder> elements: <folder name-"Inbox"> and <folder name="Sent Items">. After the first </folder> tag, the second folder would then begin:
    Code:
    </folder>
    <folder name="Sent Items">
    The same would apply to <privatemessage>. Each PM would be a separate element, and after each closed the next would begin, until the end of the folder.

    Now that wasn't too hard was it? Once you understand the basic concepts of how XML works, it's a simple process.

    Now I know you're probably wondering, how does that help me mod Civ4? PMs have nothing to do with it. That's right, but you will be surprised at how similar it will be.

    --------------------------------------------------------------------------
    --------------------------------------------------------------------------

    XML in Civ4

    You can find detailed information about the XML used in Civ4 at Civ4Wiki. Civ4Wiki includes pages for each XML file, with an explanation of every tag. Wiki sites allow users to submit their own material, which means that if you find an empty or incomplete page, you can update it yourself.

    Max_x_fire wrote an XML Quick Reference, which covers all 129 files and includes a basic description of what each file is for, in a neat Excel spreadsheet. If you don't know which files you need to modify, check this out.

    BrentRitchie's XML Tutorials provide a more in-depth look at specific tasks dealing with XML:
    Adding new Civs
    Civ Information
    Leaders
     
  6. WildWeazel

    WildWeazel -- Turns to Completion

    Joined:
    Jul 14, 2003
    Messages:
    6,779
    Location:
    %CIV3%\Conquests\Scenarios\
    Python Scripting

    Python is a relatively new object-oriented programming language that is used for various things in Civ4, including map generation, event scripting, diplomacy, and the user interface. Python is available as a free download from www.python.org

    Python is a portable language. It can be written once and run on any system that has a Python development environment installed. Python is also an interpreted language, meaning that it is compiled as it is run. This means that you can write the code and immediately execute it without the need to compile it into binary code like most other languages. The downside to this is that Python does not operate as quickly as other languages, making it unsuitable for the core game code.

    Like any new language, Python takes time and practice to master. If you would like to learn how to use the Python language, Gingerbread Man (another beta-tester) has compiled (no pun intended) an excellent tutorial. The tutorial is simple yet thorough. GBM will have you writing Python code in no time.

    Once you have a basic understanding of how the language works, you will probably want to start digging into Civ4. Locutus (also a beta-tester) explains the terms used in the Civ4 Python code. The Civ4 Python API, which provides additional information for programmers, is also available online.

    The Python files for Civ4 are located in the Assets\Python folder. The files can be opened, edited, and save using Notepad, although Python editors are available. Remember when you edit these files, DO NOT overwrite or change the originals. Create a folder in the Mods folder for your creations.
     
  7. WildWeazel

    WildWeazel -- Turns to Completion

    Joined:
    Jul 14, 2003
    Messages:
    6,779
    Location:
    %CIV3%\Conquests\Scenarios\
    *SDK Programming*
     
  8. WildWeazel

    WildWeazel -- Turns to Completion

    Joined:
    Jul 14, 2003
    Messages:
    6,779
    Location:
    %CIV3%\Conquests\Scenarios\
    *Future home of modding FAQ*
     
  9. WildWeazel

    WildWeazel -- Turns to Completion

    Joined:
    Jul 14, 2003
    Messages:
    6,779
    Location:
    %CIV3%\Conquests\Scenarios\
    Okay, you can post now if you have any specific questions or comments about modding. I will address them if I can.
     
  10. Pascal

    Pascal Delusional idiot

    Joined:
    Feb 15, 2002
    Messages:
    131
    Location:
    Liege
    Will it be possible to improve the World Builder using the SDK or Python?
     
  11. WildWeazel

    WildWeazel -- Turns to Completion

    Joined:
    Jul 14, 2003
    Messages:
    6,779
    Location:
    %CIV3%\Conquests\Scenarios\
    Good question. :) I never thought about that before. I assume that since the world builder will be an integral part of the game this time (you can even use it during a game to change things while you play), it would be part of the game itself, and would fall under the "everything but the graphics engine and the core civ engine" category. So I would say most likely yes, with the SDK.
    I do know that Python is used for things such as map algorithms and game scripting, which would involve the World Builder.
     
  12. MarineCorps

    MarineCorps Explosion!

    Joined:
    Jun 26, 2003
    Messages:
    8,187
    Gender:
    Male
    Location:
    Cape Cod
    I was thinking of a thread like this. As we still have no idea how moddable this game is yet.
     
  13. Civrules

    Civrules We the People

    Joined:
    Apr 6, 2003
    Messages:
    5,621
    Gender:
    Male
    Location:
    US
    I'll see how this thread evolves and it'll probably be stickied later. ;)
     
  14. vbraun

    vbraun Raytracing

    Joined:
    Jul 7, 2003
    Messages:
    3,530
    Location:
    Arizona, USA
    I will help when I can. :)
     
  15. Dale

    Dale Chieftain

    Joined:
    Mar 14, 2002
    Messages:
    6,986
    Location:
    Melbourne
    Civrules:

    If I may suggest waiting till after release, there may be something more substantial to sticky for modding.

    Dale
     
  16. WildWeazel

    WildWeazel -- Turns to Completion

    Joined:
    Jul 14, 2003
    Messages:
    6,779
    Location:
    %CIV3%\Conquests\Scenarios\
    This will be substantial after I get my hands on the game. :)
     
  17. WildWeazel

    WildWeazel -- Turns to Completion

    Joined:
    Jul 14, 2003
    Messages:
    6,779
    Location:
    %CIV3%\Conquests\Scenarios\
    XML modding introduction added. XML in Civ4 will be added when I get the game.
     
  18. Ekmek

    Ekmek on steam: ekmek_e

    Joined:
    Aug 7, 2002
    Messages:
    6,033
    Location:
    San Diego, California
    What about the leaderheads are they going to use the same flcs are civ3 (if you know) or we can discuss later
     
  19. WildWeazel

    WildWeazel -- Turns to Completion

    Joined:
    Jul 14, 2003
    Messages:
    6,779
    Location:
    %CIV3%\Conquests\Scenarios\
    I don't know for sure, but I assume they will use FLC files. They will probably be a different size though.
     
  20. Olorin0222

    Olorin0222 Maia

    Joined:
    Aug 2, 2004
    Messages:
    1,347
    Location:
    Flooded Missouri!!!
    There may be more FLC files as well, since CIV LH's seem to have emotions that Civ3 LH's did not have.
     

Share This Page