Need a little help with editing Civ4 save files

Arctic Circle

Prince
Joined
Apr 24, 2008
Messages
473
I am trying to decipher and edit existing Civ4 save files, but would be very grateful with some help.

Anyone did any scribbles, any pointers - anyone that done any work in this area. Much obliged with whatever help I can get. References or otherwise.

If this is posted wrongly, I ask any helpful admin to put it in the correct category.

/Thank you very much
 
It would be helpful if you are a bit more specific. In general, a text editor would be enough to edit sav files.
 
Well I've used NotePad++ and IDA PRO - and what I want to be able to do is (in order of importance).

1. Change the Armageddon counter
2. Change gold for a player
3. Change the great peoples counter for a player
4. Add/Remove units (barbarians and otherwise)

Where in the file can I find these things?

Just looking for the representation of even rather odd numbers, I've got gold 1967, tried to find it and got a large numbers of hits that are very hard to seperate.

Guess I am looking for the format/as in 'how do I find stuff, any stuff' in a .sav file?

The file mostly looks like this:
Code:
Ô1ÙK–²Í,ln#OF4+h´MVü•Ê¬¶é1©ëãÔcR£l»ä”dçCXÈÌ;m£ýl¨m•43y4c''îVŒ!-³§cúžÌvzíé¸<\ƒ§Gif&ÊošCûD€V7ãúŸr=»‰æ@LôxïgĐ
hÍq1d$։Í]*žŸJòM‘,ëˆ@!éÀ01m‡œ-ヺCoˆS|Ðîm

And if I handle it otherwise I get lots and lots of:
Code:
000984FB  85 78 6E DF 88 EB 01 5E  3F C6 F8 9F 21 7D 8E E9  àxn¯êÙ.^?㰃!}ÄÚ
0009850B  72 19 AF 85 F4 AD E4 C3  37 A2 FF E5 30 6F B7 6F  r.»à¶¡õ+7óÕ0oÀo
0009851B  44 7D E0 1B 31 F6 B2 99  87 7B DD 08 FE 0F 54 68  D}Ó.1÷¦Öç{¦.¦.Th
0009852B  DE 0A 7D B9 D7 8D C8 FF  FB E4 67 92 7F 94 E1 6F  Ì.}¦Îì+¹õgÆößo
0009853B  22 85 6F C4 58 5D 48 FE  11 E6 FB 46 F2 BF 47 F9  "ào-X]H¦.µ¹F=+G¨
0009854B  16 50 D7 87 19 FE 06 52  F8 46 8C 5D FB 99 F3 43  .PÎç.¦.R°Fî]¹Ö¾C
0009855B  0C 7F 3D E9 77 C8 B7 CF  AB 1D 64 F8 EB 48 FF 82  .=Úw+À¤½.d°ÙHé
0009856B  F9 CF 65 BD 0F B0 FC D7  92 FF 6D F2 E7 90 DF CC  ¨¤e¢.¦³ÎÆm=þɯ¦

Or:
Code:
seg000:00098882                 db 0ADh ; ¡
seg000:00098883                 db  69h ; i
seg000:00098884                 db 0C1h ; -
seg000:00098885                 db 0B7h ; À
seg000:00098886                 db 0FBh ; ¹
seg000:00098887                 db 0A3h ; ú
seg000:00098888                 db 0F0h ; 
seg000:00098889                 db  8Dh ; ì
seg000:0009888A                 db 0B0h ; ¦
seg000:0009888B                 db 0E5h ; Õ

So, what should I look for? How should I process the file data?

Thanks :)
 
If someone needs a reason:

A. I want to be able to do it, its there, so I want to be able to hack it.

B. I want to be able to replay the sort of situations where 'I'd would have won if the armageddon counter hadn't been so high'.
If you have access to the SDK with which the save file is written you are probably better off changing the source code than hacking the save game file.

And the easiest way is to use worldbuilder to change what you need. It might not have an UI for the armageddon counter though so for that you could use the Python console and the respective function to change it (if it is exposed to Python, that is).
 
I played with this a bit: I wrote a certain string as the script data for various classes on write() (city, player, game, unit, plot) but the string did not appear in the save file.
It probably means that the stream written to the save file is either compressed, or encrypted, or both. Probably only compressed, and I'd wager it's done with zlib.
In that case, it might be possible to decompress it (although I'm not sure which parameters it requires. I'm not that familiar with the compression algorithm).

But it'll take some work and testing. Sounds like you're up for it (If you're already using IDA Pro...).

Might be interesting to work on it.

EDIT: BTW, IDA Pro is for code, not for data.

EDIT2: Definitely zlib with the deflate algorithm (starts with 0x789c)
 
I was able to uncompress the data. I'm not completely sure it's the entire data.
But to actually modify it there are other problems - such as the checksum in the end and probably some other header fields.
I might look into it further if there's interest in that.
 
I was able to uncompress the data. I'm not completely sure it's the entire data.
But to actually modify it there are other problems - such as the checksum in the end and probably some other header fields.
I might look into it further if there's interest in that.

I've more or less only worked with re-eng of code. Not these kind of data files.

If I could get details on how to repeat this, I'd be much interested. No experience with zlib but I learn very, very quickly when I need to.

If you can uncompress it, then one needs to be able to modify, and then return it to a format that can be used.
 
If you have access to the SDK with which the save file is written you are probably better off changing the source code than hacking the save game file.

And the easiest way is to use worldbuilder to change what you need. It might not have an UI for the armageddon counter though so for that you could use the Python console and the respective function to change it (if it is exposed to Python, that is).

Thank you, but I think neither of these are the way I can go with this. One I know have tried to make these sort of changes along that path and not made it all the way.

Worldbuilder is fun stuff. :)
 
If I could get details on how to repeat this, I'd be much interested. No experience with zlib but I learn very, very quickly when I need to.

I used the python implementation of zlib, as it's simple to use and easy to write ad-hoc stuff.

I hope you are familiar with the Python interpreter.

An important note: To load the file in the interpreter, you must specify it as binary, like this (note the 'b' in the file mode):
Code:
savedgamefile = open(path, 'r[B]b[/B]')
data = savedgamefile.read()
savedgamefile.close()

I'm not on the same PC in which I did this, but from memory:

First, the compressed stream inside the save file starts with the hex bytes 0x789C, which is the header of the 'deflate' algorithm (see RFC1950).

There are chunks of data before that stream. The exact location could be different in different save files (probably dependent on world size, which mod and other factors). On the specific file I worked on it was @ 0x5f6, and I saw it in another save file.

Then the stream continues almost until the end, I don't know exactly where.

I started by using the following:

Code:
uncomp_data = zlib.decompress(data[0x5f6:])

Which gave an exception on incorrect block lengths or something.

I then tried a shorter stream, something like:
Code:
uncomp_data = zlib.decompress(data[0x5f6:0x5f6+100])

Which gave an error of incomplete stream (again, not exact phrasing).

So I did a binary search on the exact position where it breaks, and found the point in which the error changed from 'incomplete' to 'incorrect'.

It still didn't work properly, so what I did was to use a decompression object which can do this in parts:

Code:
# 'end' is the location I found with the binary search
decomp_obj = zlib.decompressobj()
uncomp_data = decomp_obj.decompress(data[0x5f6:end])

And then uncomp_data contained the data written by the game's DLL.
I'm not completely sure it was complete, and I don't know what else it contained, but I added markers (I hardcoded script data members in the code so I can identify both the positions in the stream and the fact that the stream is indeed decompressed), so it was definitely the raw data.

The decompressed data was something like 500KB (from a 9KB compressed buffer), but I assume that's because it was a game start and most data was zeros.

To save the data to a file, I used:
Code:
uncomp_savedgamefile = open(out_path, 'w[B]b[/B]')
uncomp_savedgamefile.write(uncomp_data)
uncomp_savedgamefile.close()

It requires more research, but it's a start. Let me know if it's working.

If you can uncompress it, then one needs to be able to modify, and then return it to a format that can be used.

Yes, but there appears to be a checksum at the end of the save file. I have yet to check this, but I'm guessing you'll need to update this if you modify the data. In addition, I think there's some kind of a header which contains data length, so this would need to be updated as well.

EDIT: You should add 'import zlib' in the beginning. This package should come with the Python standard installation.
 
I hope you bear with me.

Having trouble finding the 'positions' or rather figure out the positions name in hex. I am using Notepad++ with the Hexeditor plugin.

start - This is where it starts, but what is the hex value for the position?



end1, end2 - I think the compressed part ends at end1, but the file ends at end2. What are the hex positions?





I've done some guesses, but I get the error you've mentioned so I must end up with the wrong values.

/Thanks

---Asaf responded---

Asaf said:
Might I suggest that we can continue this discussion in the thread you've opened so others can benefit from this as well?

As for your question: the first byte (2 hex digits) of each line is in address 0, and the others are 1..f (in hex). You have this displayed at the top of each column.

So the address of the '78' in the starting position is 0x109e.

Note that in Python to mark a hex number (as well as in a number of other languages) you need to add a '0x' prefix. So '0x10 == 16'.

Please post this question and answer in the thread.

Thanks.
 
I have managed to get the information out, using your script. Very nice.

It is clearly in a semi-readable format, I am sure that there exists some default format that would make it more understandable.

However it seems to me mostly a chronological listing of what have happened in the game.

I've started a VS2010 project to edit/work with the result file, but havn't really gotten very far yet.
 
Civ4 BTS uses same compression method as civ5 (or rather its the opposite.. :p )

If you can read perl then you can see how to decompress any file by looking at the perl code I posted in this thread:

http://forums.civfanatics.com/showthread.php?t=388937

The rest of the code is released to early versions of civ5 though (and recompression of the edited savefile wont work the same way due to checksums in the civ4 bts savegame)

When it comes to making sense of the content of the decompressed data, then all the code for reading the data is found in the DLL source code (this is not the case for the data before (part of it is in the DLL), or the data behind the compressed section)
 
Top Bottom