[Col] colsaves - Updates

[henryjfry]

I've recently got back into playing colonization and had been looking on the internet for strategies and other useful info and had stumbled across some of the post here about decoding the save games.
I found ColSaves and having gotten into python in the last few years i have been able to make some modifications and improvements to make it a bit more useful that it already was.

I've forked it on my github and the readme contains an example usage on a windows machine with python:

colsaves/colsaves at master · henryjfry/colsaves

Reading Sid Meier’s Colonization (1994) Save Games - colsaves/colsaves at master · henryjfry/colsaves

github.com

So the attached map.png and savegame.txt are the current output (on lines 1097-1099 it opens the image and txt file which you might want to comment out).
I added additional lookups/textures to the map output so you can see where your colonies are (white text) vs other powers colonies (red text) vs Indian villages (green text)

Using some information found here and in viceroy I was able to decode the indian Scouted/Capital/Learned/Artillery(??) status, which are the S/*/L/A against the indian nation name and population number.
I wasnt able to find the right texture for the colonist orders or how to display that with readable text but tinier letters, so the black on flag is best I could figure. But the flags should be correct for each country (plus the E/F/S/D indicator for English/French/Spanish/Dutch).
And I was also able to figure out how to decode the colonies fortification status, im not convinced the original code is doing it correctly.

The original tool seems to only tile sea/forest/mountain tiles. I dont know enough about decoding the hex to better figure how to get the map to use all the correct textures but I dont think it would take much tweaking to get a game accurate map.

So im hoping this is useful, the map is maybe a bit busy but thats mostly the PIL text which i dont know how to make look better and still be readable so anyone who has an idea on that feel free to make suggestions.

 

[Blake00]

This is some cool work mate!

 

[Bladrov]

That's great! But, when I run it (per the README.txt), I get:

ModuleNotFoundError: No module named 'PIL'

I must be missing something or doing something wrong.

Also, we could save space on the map by not indicating Capital, since that already shows on the game map.

 

[henryjfry]

The PIL module is the python image library and its installed like this:

pip install Pillow

In a python terminal.

The capitals would probably be easily commented out, so comment out this section??

if tribe_bin_state[1] == '1':

#tribe_state = tribe_state + 'C'

tribe_name = i.nation + '*('+ str(i.population) + ')'

However thats the least obstructive text on it so isnt likely to make it look that much better.

Im hopeing someone round here with a better handle on texture might pick it up and integrate the texture tiles better.
If it accurately could portray the terrain that would be awesome.

 

[Bladrov]

Uh, no. That doesn't work for me, either in Windows or Linux. The Python terminal in Linux says

File "<bpython-input-3>", line 1
pip install Pillow
^^^^^^^
SyntaxError: invalid syntax

and in Windows

File "<stdin>", line 1
pip install Pillow
^^^^^^^
SyntaxError: invalid syntax

with the carets (^) under each letter of "install".

 

[bruno.uy]

Try running pun pip from a OS console, not from python itself.

 

[Bladrov]

I've managed to install it in Linux with sudo apt install python3-pip and pip install Pillow. However, when I run reader.py against a save file, I get an odd cursor shaped a bit like an Iron Cross and the blinking terminal cursor disappears, and the computer just sits there. If I Ctrl-C or whatever out of there, I get a bunch of lines like

./reader.py: line 18: trade route,: command not found
./reader.py: line 19: go to,: command not found
./reader.py: line 20: 0x04,: command not found
./reader.py: line 21: 0x05,: command not found
./reader.py: line 22: fortify,: command not found
./reader.py: line 23: 0x07,: command not found
./reader.py: line 24: plow,: command not found
./reader.py: line 25: road: command not found
./reader.py: line 26: ]: command not found
./reader.py: line 28: NATION_FLAG: command not found
./reader.py: line 29: icons.ss_118.png,: command not found
./reader.py: line 30: icons.ss_119.png,: command not found
./reader.py: line 31: icons.ss_120.png,: command not found
./reader.py: line 32: icons.ss_121.png,: command not found
./reader.py: line 33: ]: command not found
./reader.py: line 35: CONTROL: command not found
./reader.py: line 36: player,: command not found
./reader.py: line 37: ai,: command not found
./reader.py: line 38: withdrawn from new world: command not found
./reader.py: line 39: ]: command not found
./reader.py: line 41: DIFFICULTY: command not found
./reader.py: line 42: Discoverer,: command not found
./reader.py: line 43: Explorer,: command not found
./reader.py: line 44: Conquistador,: command not found
and
./reader.py: line 146: 6:: command not found
./reader.py: line 147: 7:: command not found
./reader.py: line 148: 8:: command not found
./reader.py: line 149: 9:: command not found
./reader.py: line 150: 10:: command not found
./reader.py: line 151: 11:: command not found
./reader.py: line 152: 12:: command not found
./reader.py: line 153: 13:: command not found
./reader.py: line 154: 14:: command not found
./reader.py: line 155: 15:: command not found
./reader.py: line 156: 16:: command not found
./reader.py: line 157: 17:: command not found
./reader.py: line 158: 18:: command not found
./reader.py: line 159: 19:: command not found
./reader.py: line 160: 20:: command not found
./reader.py: line 161: 21:: command not found
./reader.py: line 162: syntax error near unexpected token `}'
./reader.py: line 162: `}'

 

[pavelbel]

I've managed to install it in Linux with sudo apt install python3-pip and pip install Pillow. However, when I run reader.py against a save file, I get an odd cursor shaped a bit like an Iron Cross and the blinking terminal cursor disappears, and the computer just sits there. If I Ctrl-C or whatever out of there, I get a bunch of lines like

Great tool! It worked for me, bitmap and savegame.txt produced ok. You should fix your Python installation. Maybe you should try installation of an IDE for working with Python. PyCharm for example.

 

[Bladrov]

Ah, got it working. In Xubuntu Linux at least, you have to also install python3-pip (I used Synaptic). I then ran pip install Pillow in a terminal (not sure if necessary), then I ran it. In Linux the command will be more like:

Then run python3 ~/dos/COLONIZE/colsaves-master/colsaves/reader.y ~/dos/COLONIZE/COLONY##.SAV

I take it that Artillery is for European colonies, showing how many artillery units in town? And the number next to Indian settlements, is that the tribe number?

And the Learned variable - when it shows L, does that indicate the village no longer teaches any skills?

Any chance of being able to see what a given village teaches?

 

[Bladrov]

Huh. When I run it at home I get

SyntaxError: Non-UTF-8 code starting with '\xfe' in file /home/leon/dos/COLONIZE/COLONY00.SAV on line 1, but no encoding declared; see https://peps.python.org/pep-0263/ for details

 

[pavelbel]

Huh. When I run it at home I get

SyntaxError: Non-UTF-8 code starting with '\xfe' in file /home/leon/dos/COLONIZE/COLONY00.SAV on line 1, but no encoding declared; see https://peps.python.org/pep-0263/ for details

It seems that it tries to run not reader.py script with argument "COLONY00.SAV", but COLONY00.SAV itself

 

[Bladrov]

Maybe the script should list the save files in the directory and let you enter 00, 01, etc.

 

[pavelbel]

I'm now working on a utility for enchanced study and modification of Colonization .SAV files - to make it as handy and simple as possible.

There's one interesting thing I already discovered about Warehouse/Warehouse Expansion info. It turns out that it is stored not like other 2-level buildings like carpenters shop, church or printing press. There is only one bit for storing "warehouse" flag inside the 6 bytes buildings struct, right after "schoolhouse". But the next bit is not a flag for Warehouse Expansion! This bit is actually not used and always equals to 0.

The thing is there is a separate BYTE for storing LEVEL of Warehouse Expansion: 0x95 in colony record (0x93 or 147 (dec) steps after the first char of colony's name, right after "building in production" byte), and it is not just a boolean flag but a full level. It equals 0 if there is no warehouse, 1 for warehouse, 2 for warehouse exp. And you can set it to any value, even 0xFF (255)! In this case the storage capacity for any cargo type will be 100 * (LEVEL+1). No more 300 tons limit!

 

[Bladrov]

Oh, cool!! This utility you're working on sounds like it'll be pretty amazing.

 

[AyCe]

I'm now working on a utility for enchanced study and modification of Colonization .SAV files - to make it as handy and simple as possible.

There's one interesting thing I already discovered about Warehouse/Warehouse Expansion info. It turns out that it is stored not like other 2-level buildings like carpenters shop, church or printing press. There is only one bit for storing "warehouse" flag inside the 6 bytes buildings struct, right after "schoolhouse". But the next bit is not a flag for Warehouse Expansion! This bit is actually not used and always equals to 0.

The thing is there is a separate BYTE for storing LEVEL of Warehouse Expansion: 0x95 in colony record (0x93 or 147 (dec) steps after the first char of colony's name, right after "building in production" byte), and it is not just a boolean flag but a full level. It equals 0 if there is no warehouse, 1 for warehouse, 2 for warehouse exp. And you can set it to any value, even 0xFF (255)! In this case the storage capacity for any cargo type will be 100 * (LEVEL+1). No more 300 tons limit!


View attachment 652420

Yep, cheats can also increase the warehouse level up to 255.

 

[pavelbel]

Oh, cool!! This utility you're working on sounds like it'll be pretty amazing.

Hello @Bladrov! Here is the first "release" version of the utility (utilities - there are two): https://forums.civfanatics.com/threads/colonization-1994-sav-files-decoder-and-encoder.681919/
Bug reports are appreciated.

 

[Bladrov]

Hm, it looks as though you can get at least smcol_sav_editor.exe to run in Linux. To do so, open the Wine configuration (), go to the Drives tab, and set a drive letter to match where your COLONIZE directory is. So in my case I set D: to /home/leon/dos/.

Then open smcol_sav_settings.json, and (in my case) set "colonize_path": "D:/COLONIZE",

Then open a terminal in the smcol_saves_utility wherever you have it, and enter wine smcol_sav_editor.exe. I haven't tried changing save files yet, but I did get it to open. You type a number 1-13 to open a save file, and proceed from there.

Edit: I've done some playing around with it this morning, just the plant forest function, and so far it works great. It converts the terrain type of the square at the coordinates you give to its forested state: Prairie to Broadleaf, Desert to Scrub Forest, etc.

 

[Bladrov]

So will this utility eventually create a map like above, showing info on the Indian villages?

 

[pavelbel]

Thank you for testing of the editor utility!

So will this utility eventually create a map like above, showing info on the Indian villages?

On Map. Right now I think there's no need to do it because colsaves functionality cover this task. The first "must have" task should be discovering of how the village's teaching speciality info is stored inside the SAV file structure. Once we discover it I'll surely create some visualization for it.

You can contribute to it! With my decoder/encoder it has become much easier to study the SAV files in SAV.json format. For now we can be sure, that teaching spec info is not stored inside the "tribe" record. Hence it is calculated. But how? No idea by now. The first thing I would like to try is to alter some villages' positions and see how (and whether) their teaching spec has changed.

 

[Bladrov]

Thanks, I'll see what I can do. Incidentally, for any Linux/Mac users out there, this seems to be slightly tricky. I use DBGL as my DOSBox front end, and in there I have the Colonization entry set up to mount my ~/dos/Colonize as C:. But for whatever reason setting the .json to C:/ doesn't work - I have to set it to D:/COLONIZE, and then it works fine. Yes, you use a forward slash, even in Linux.

I wonder if it would be worth noting this for the benefit of Linux and Mac users? Part of the problem is I don't know why it interprets D:/ as ~/dos on my computer.

Edit: Hm, I've saved out a great example, but I'm not sure how to find the village I'm looking for. It's at 52, 42, and my hex editor doesn't see char 52 in the save.