[Col] Colonization DOS Utility to keep track of Indian settlements

[Bladrov]

Yeah, that might be. But there should be a level of alarm kept track of somewhere. They don't go from !!!! to ! (or vice versa) at random.

 

[AyCe]

Yes, maybe 4 bytes in a row, and when they are very angry and you are dutch its 00 00 00 FF or something.

 

[Bladrov]

Could be. Or maybe the "Alarmed by ____" fills in the blank with the nearest colony's owner. I tried to test that idea this morning by changing the flag of who owns a given colony: change one of my Dutch colonies to an English one or something - but I don't see that in the save file. The colony's name is preceded by the coordinates, which are preceded by several 00s. Following the name is data about the colonists working inside and their SoL levels (as http://www.colonization.biz/hack.htm said it would be). So ownership of colonies seems strangely absent from the save file. This is very odd! You'd think that, and that other data I mentioned, would be right here, stored with the colony/settlement, but I don't see it.

 

[AyCe]

Savegame where dutch have a colony and you are spanish. Then use cheats to spawn a soldier next to it and capture it. Then compare the bytes around the colony name. Somewhere 03 should have switched to 02. Just an idea.

 

[Bladrov]

Yes!! Found it. The 25th character after the start of the name is a flag for the owner. Changing it to something other than me (Dutch) activates my soldier inside. I can move it out; I can even move it onto that square again and off again if I want. But I can't open the colony any more.
00 English
01 French
02 Spanish
03 Dutch
04 Nothing (shows a box with an X in place of a flag)
05 Rebel (shows the American Revolutionary flag)
06 Royal (shows a crown)
07 ??? (shows a non-veteran soldier instead of a flag)
08 ??? (shows a cavalry unit instead of a flag)
09 ??? (shows a man-o-war instead of a flag)
10 or above Makes the game crash

Two characters further is a mysterious 40 value. I found that changing that to 02 increases the food and sugar produced on my colony square by one each. Changing it to, say, 01, doesn't seem to do anything.

 

[AyCe]

Food and yield on the colony tile is dependent on difficulty. I noticed that if you look in the AI colonies it will report numbers as they would be for your difficulty, but when you look a turn later what was produced it will be a different value. Maybe it's somehow set there?

 

[Bladrov]

Oh wait, the whole reason I was looking for colony country code was to test how "Alarmed by ____" was determined. I'll have to check that out later.

And yes, maybe that 0A 08 03 FF 00 FF FF FF 00 00 00 00 00 00 00 00 holds the alarm level somewhere. I'll have to play the game, save, watch for when a settlement's level of alarm changes, and compare stats for that village in the save files.

So, it looks like the first Indian settlement starts at 6432 and the last at 7744, and each is 18 characters long.

Of course the tricky part at some point would be writing a program or script that will parse the data, convert values to strings, and interpret it all. I've been poking around with doing this in Ruby, but it's slow going since I don't know what I'm doing.

 

[Bladrov]

Ok, I've created a ruby file that will give you a list of all the Indian settlements. It displays the coordinates followed by all the data characters after them. That should help us work out what it all means.

Unzip this file to your COLONIZE folder, open a command prompt there, and enter ruby ind.rb. Of course you have to have ruby installed first.

 

[Bladrov]

Hum, good news and bad. I might have worked out how Capital is stored: it looks like the first 8 settlements in the file are the Indian capitals. That's possibly why, if a capital is destroyed, another settlement doesn't become the new capital for the tribe.

In less good news, I just ran the file against my save from last night. It's the same game, but late in it - I'm getting ready to declare independence. I'm getting a lot of incongruent data from it now: some normal-looking values, but also things like
(1,1)
1 0 0 0 0 26 0 0 0 191 0 0 0 34 28
(0,0)
0 0 0 1 176 64 0 2 1 13 0 0 0 0 0
(0,0)
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
(0,0)
255 255 255 255 255 255 255 255 255 255 255 255 255 255 255

A lot of this (esp. the (0,0) ones) could be destroyed villages, but what's the rest? I see the file size has increased substantially from 26.6k to 36k, so maybe data gets pushed into it over time and I can't rely on searching from x position to y position in the file.

Edit: looks like my suspicion there is confirmed. I see last night's save file has data for European colonies a bit further down (6848dec) than the first Indian settlement started in the original file (6432dec), and that first settlement's been pushed down to 16160dec. So it looks like new colonies are additional data inserted into the save file rather than filling previously empty "placeholder" space. I don't see anything to distinguish where the first Indian village occurs. Here are the characters preceding the first Indian settlement in the old vs. later save; there's a number of 00s before them:
00 00 00 00 00 00 00 00 00 00 00 38 FF FF FF 38 FF FF FF 06 FF FF FF
00 00 00 00 00 00 00 64 00 00 00 70 FE FF FF D4 FE FF FF A2 FE FF FF

 

[AyCe]

Found a colony, see how many bytes were added to the savegame? You may also need to test this when spawning a unit, since the unit disappeares when it founds the colony.

 

[Bladrov]

Well, but...how does that help us tell the script where to look for the start of the settlement data? There are strings of at least 12 00s in other parts of the file; I don't see a marker we could look for.

 

[Bladrov]

Hey, I found something else! It looks like the number of European colonies is stored in the 46th spot in the file. Here's a screenshot with the Value Coding set to decimal; this save has 32 colonies. I opened the save, founded a colony, and saved again, and the number incremented to 33.

 

[Bladrov]

I've tried changing the place it starts looking for data from 6394 to i = 4778 + colonies.to_i * 202. That works great for that save COLONY00.SAV from earlier, but other saves return jumbled-up data again. I think I have this right, except can't find the value of x for x + (# colonies) * 202. Unless, of course, there's other data pushed in front of the Indian settlements - in which case this is probably a dead end, and I really need to find a marker for where the Indian settlement data starts. But I don't see anything consistent between saves!

Might be that new European units get created in between colony info and Indian settlements. If we could work out some of that data, it might let us determine where the Indian data starts.

 

[carthanc]

Sorry for the necro.

I think I have a solution, but it requires a two-step approach. I am pretty sure the SAV is not structured with fixed offsets. Instead it seems like it is a series of sections, with each section expanding with more data as the game continues. I suspect there is "meta" data in the first sections that would allow you to jump to the exact location for each section, but I haven't cracked those first sections yet. I used the tips in this thread to search for the "village section" instead.

I first scanned the hex data for coordinates followed by one of the tribe flags. I used regex. The "\x" flags made it possible to match a range of hex values like you would use [0-9] to match a digit:

/([\x01-\x38][\x01-\x46][\x04-\x0b])/

This ugly stream of characters searches for three bytes:

• [\x01-\x38] The first byte in the decimal range 1-56 (a valid X coordinate)
• [\x01-\x46] The second byte in the decimal range 1-70 (a valid Y coordinate)
• [\x04-\x0b] The third byte in the decimal range 4-11 (a valid tribe flag)

Secondly, I had to look for the spacing between the matches. There are several sections of the SAV file where these patterns can be found. To find the right section, I looked for a series of matches that were each exactly 18 bytes apart.

This method exactly lists all the villages, but that is just a start. I'm not sure what the other fifteen bytes are yet, nor am I sure what the other section of matches represents.

I'm going to continue working on decoding the full SAV file using another thread: https://forums.civfanatics.com/threads/decoding-colonization-sav-files-1994-old-classic.674707/

If I can decode the full SAV file, then a building a companion utility to keep track of the settlements would be an easier feat.

Thank you for all the discussion in this thread, it really helped me.

 

[Bladrov]

No need to apologize for resurrecting this discussion! I've been hoping someone would.

Your effort to parse the save files for useful information is of great interest to me - I'll be following your other thread. I'm glad my abortive attempt has helped make a difference.