Civ3/C3C/PTW Memory Structures

[Antal1987]

Hello, everyone!

I research Civ3 game and it's addons with ArtMoney and IDA.
My general purpose is developing ultimate trainer for this games.

Maybe using Save Game Editors is cool but doing same things in runtime is much more better. For example, I intend to study how city corruption and pollution are calculated to make them reduce to zero.

Here that I've already found:
All map tiles are stored in object array.
In C3C-v1.22 a pointer of that array is stored at 0x009C7474.
Also Map_Width is stored at 0x009C7494; Map_Height - at 009C7480;

The array itself is actualy array of pointers to Tile Class Object. Probably it's being created by command like:
Tile[] Map = new Tile[Map_Width /2 * Map_Height];
for(int i=0;i<Map.Length;i++) Map = new Tile(....);

Tile Object has size = 0xE8.
I've recognized the following Tile data:

+0x00 - Virtual Method Table Pointer for base class: Tile_Base_1 (4 bytes)
+0x08 - Resource ID (4 bytes integer). = -1, if there is no Resource; 0 - Horses; 1 - Iron; ....
+0x28 - Overlays. There are 8 bit flags:

[Flag]
public enum MapOverlays{

Road=0x01,
Railroad = 0x02,
Mine = 0x04,
Irrigation = 0x08,
Fortress = 0x10,
GoodyHuts = 0x20,
Pollution = 0x40,
BarbarianCamp = 0x80
​

}
Description can be found at (link).
​

+0x2D - Tile type = BaseType << 4 | RealType.

Each type by default is in the set: 0 - Desert; 1 - Plains; 2 - Grassland; .... ; 0x0D - Ocean;
So the value 0xDD describes the ocean tile. But it only affects on how the game thinks what this tile is. Rendering algorithm uses that value partially. It affects on view only for 'ground' tiles.
​

+0x3C - Virtual Method Table Pointer for base class: Tile_Base_2 (4 bytes)
+0x44 - 4 bytes string 'TILE'.

So each tile object can be found in artmoney by searching that string. There should be at least Map_Width * Map_Height / 2 found entries.
​

Now I'm looking for information about the other game data structures.
Maybe someone has investigated the game enough to find fixed data pointers and recognized some structures.

 

[vmxa]

Well I think you could just create a new biq with corruption turned off. That is accomplished in CCM mod.

 

[Antal1987]

Well I think you could just create a new biq with corruption turned off. That is accomplished in CCM mod.

The problem is that no one mod can reduce corruption and pollution to zero permanently, because game logic has formulas that give nonzero values in some cases (although it's rare).

I'm going to find out what kind of parameters affect on these formulas and change it to make formulas give zero value regardless on having city improvements or wonders and other factors.

 

[Quintillus]

Heh; I was just about to link to the BIX/BIQ format thread when I saw your link already goes there. I've recognized many of the sections of the BIQ (and some of the less-well-documented SAV) in the memory structure. Unfortunately, I haven't done much in memory beyond fiddle with a couple values while the game was running to test the theory that modifying them could work (such as renaming a unit, modifying its strength, etc.). So I can't tell you a whole lot that isn't already in the BIQ documentation.

Are you using the Pro version of IDA? I've fired up the free one a few times, but wasn't really sure where to start with it. I've never worked with assembly language on anything remotely as big as CivIII, though, and haven't dealt with x86 much, which may well be why. A security course in college that taught the basics of identifying security flaws in (Linux x86) executables, and a class with a small amount of non-x86 assembly, and that's about it. I haven't heard of ArtMoney; what's its specialty?

I'm also curious where you recommend starting out in terms of modifying the memory (programmatically). I know there must be a way to do it, since HxD can do it, and that's what I've used to modify values so far, but it's never been high enough on my priority list to go ahead and figure it out how to do it programmatically.

The Image and Tile bytes in the TILE section are what's really important to the rendering. The tile type only really affects what values it uses for food/shields/gold.

 

[Antal1987]

Heh; I was just about to link to the BIX/BIQ format thread when I saw your link already goes there. I've recognized many of the sections of the BIQ (and some of the less-well-documented SAV) in the memory structure. Unfortunately, I haven't done much in memory beyond fiddle with a couple values while the game was running to test the theory that modifying them could work (such as renaming a unit, modifying its strength, etc.). So I can't tell you a whole lot that isn't already in the BIQ documentation.

The problem is that the game writes its data in save file with short format. But it in the memory data are stored in class objects. Each entity like Unit or City has its own corresponding class (maybe not only one when it comes to inheritance). Objects to stored in memory are instances of that classes. Objects store not only the data but also the internal information such as virtual fuction tables, address pointers, etc. And of course objects have no need to be shrinked. For example, if a value occupies 2 bytes of the file, then it may take the memory as many as 4 bytes.

That's why it's more important to know where to find than to know what to find.

In that case memory exploring software is indispensable. One on this software is ArtMoney (ArtMoney page. It has freeware editions to download).

I haven't heard of ArtMoney; what's its specialty?

ArtMoney allows user to investigate internal program memory with different useful tools. For example, you can search a single value (with different types), then filter results with new value to eventually locate addresses having the value changed.

Are you using the Pro version of IDA?

I'm using IDA Pro 6.1. I guess I've never dealt with demo version. But in Pro there are no simple ways to start too.
To start exploring and find smth you need some info about what to find. Exploring data segments is useless. The point is looking for some data offsets in code segments. When you find a variable being modified with such offset you can trace this variable through function calls and finally locate a function having entire data structure to be processed and place in data segment where the structure or its pointer is stored.

The other effective way is looking for vitrual function table pointers. If you so lucky to recognize data structure in memory having first 4 bytes with some unknown pointer, it can be exactly that pointer. 32-bit windows executables assembled with MS compiler have .rdata segment that stores all virtual function tables for all classes having virtual functions (means that classes inherite some other classes with virtual functions or interfaces or they just have functions to be overloaded in child classes).
Knowing precise location of that tables in .rdata segment allows you to trace all functions having a construction of class of iterest.

I've already researched Civ1 game (for Windows, CivWin) using that way. My results can be found at Link, Link or Link.

I'm also curious where you recommend starting out in terms of modifying the memory (programmatically)

Windows API has wonderful functions ReadProcessMemory and WriteProcessMemory which allow you to read and write (respectively) memory of target program.

I've already had expirience of doing that in C#. So it has taken a little to develop memory core being used in my trainer for CivWin.

You can learn about working with memory in C# at Link.

The Image and Tile bytes in the TILE section are what's really important to the rendering. The tile type only really affects what values it uses for food/shields/gold.

Unfortunately I cannon determine The Image bytes in Tile class objects (it may be transformed into some pointers inside the game), but I know exactly that Tile type byte affects on rendering: if Tile object has "ground" type of terrain (like mountain or plains) changing type type makes the game render Mountain or Plains respectively. But is the Tile object is not "ground" (ocean, sea, coast) then changing tyle type does not affect on rendering at all. But the game perception of tyle type resources depends only on tyle type.

 

[Wolfshade]

So, is there any way to increase number of buildings/resources - to be processable by .exe? It's possible to add more than the limit, but .exe may stop working (in my case with >256 resources).

 

[Antal1987]

I do not rule out this possibility. Technically the capacity of element's array (units, cities, etc) can be limited by constant value (that can be located in exe and then patched), variable type size (16bit means 2^16-1, 32 bits = 2^32-1) and memory (last two are rare).
So the task is to locate appropriate code insructions with limit checks, determine real addresses of limit values and change them.

 

[Quintillus]

Thanks for the information, Antal1987. That's quite helpful in figuring out where to start in looking at the Civ3 memory. I probably won't have any updates for awhile, but now I'll have a starting point if I do get around to looking at enhancing Civ3 with in-memory updates!

 

[Antal1987]

Quintillus, I've found out that there is another, mush more easier, way to find smth interesting in game memory.
As it turns out, game developers have used some class class (about 20 bytes in memory) in many other classes as class to inherit from or it is stored in internal memory of target class.
That small class has one distinctive feature. It contains 4-char string in its memory at offset: 0x08;
In default construction the string is initialized by "XXXX" (capital letters everywhere). But in target classes the string is replaced by distinctive string:
In Unit class the string is "UNIT",
It City class - "CITY"
Tile (map cell) - "TILE"

Also, most data object that relevant to game rules (tile type, terraform types, advances, etc) can be found by Civilipedia_Entry. That structures doesn't contain 4-char string or pointer to virtual function table, but all data structures for one data type are stored in one memory block one by one. Therefore there must be the pointer to structure array that points also on the first data structure of given type.

All pointers are stored fixed in .data segment. For different game versions or mods .data segment can be different in offset and size. But in C3C v1.22 it has [0x680000 to 0xCD1000] data range, and it is mapped in the memory exactly with the same range. By the way, .rdata segments (contains virtual function tables) has [0x6653B0 to 0x680000] range.

Sometimes pointers to different structure arrays are stored near. Once I've located pointer to Tyle Type array. So I've investigated nearest memory and found other interesting array points such as Terraform Type Array or Advance Type Array.