rahorvat
Warlord
And I forgot to mention that the resulting EXE (without old EXE) can be used for normal gaming.
OpenCiv1 Project - Open source rewrite of the original Civilization 1 Game designed by Sid Meier and Bruce Shelley in year 1991
- Thread starter rahorvat
- Start date
Gandhi II
Chieftain
- Joined
- Apr 23, 2020
- Messages
- 2
This is an awesome project! One thing I'm very interested in (it's been a hobby project of mine for a long time) is getting a working online multiplayer for Civ. It was implemented in CivNet, and to my knowledge, this is the only major feature that isn't available on emulated systems (their networking is usually unable to support the necessary protocols). Have you considered working from an implementation of CivNet, instead of CivWin? Their are many quality of life features that make it a great option, but I have no experience with decompiling, so maybe it'd be more complicated. Also, I'd be happy to help with this project, if you had anything you could suggest.
rahorvat
Warlord
Hi, Gandhi,
Yes, the thought passed my mind.
Currently I'm working on getting Civ1 for DOS to work with emulated code/environment. I have successfully reconstructed almost all of the code, but the structure of the program is full of overlays and pointers to a functions which kinda hinders my efficiency and speed. I understand that the memory was these days at a premium, but, boy they complicated things a little
Nowadays almost all machines have at least 8GB of ram, and we are speaking of 640KB - DOS - BIOS stuff.
Got the first part of program to work correctly, and now EGA/VGA (egraphics.exe overlay) stuff comes into play. Hopefully will be able to run the complete game correctly just to be sure I got it all right, and then I can start to translate and optimize code.
Will report when I complete this task as that will be the point when I will need the help the most: What that function exactly does, what are it's parameters and return value(s)?
And then when we translate the code, we can optimize it, fix bugs, and introduce some new features perhaps.
Yes, the thought passed my mind.
Currently I'm working on getting Civ1 for DOS to work with emulated code/environment. I have successfully reconstructed almost all of the code, but the structure of the program is full of overlays and pointers to a functions which kinda hinders my efficiency and speed. I understand that the memory was these days at a premium, but, boy they complicated things a little
Nowadays almost all machines have at least 8GB of ram, and we are speaking of 640KB - DOS - BIOS stuff.Got the first part of program to work correctly, and now EGA/VGA (egraphics.exe overlay) stuff comes into play. Hopefully will be able to run the complete game correctly just to be sure I got it all right, and then I can start to translate and optimize code.
Will report when I complete this task as that will be the point when I will need the help the most: What that function exactly does, what are it's parameters and return value(s)?
And then when we translate the code, we can optimize it, fix bugs, and introduce some new features perhaps.
This is really great. Will read your source to teach myself how overlays work in the game... Big thank you!
rahorvat
Warlord
It's really simple
The Microsoft overlay manager is called by INT 0x3f, after that instruction single byte specifies the overlay number and word specifies the overlay calling address (so the whole instruction is basically CD 3f <overlay number> <address> or 5 bytes long).
The second type of overlay is implemented by Microprose (files mgraphic.exe... also asound.cvl...) the subroutine in the main program basically loads (not executes!) the file by calling DOS 0x21, 0x4b (Load and Execute Program) and adjusts the pointers in the main program to the functions in the overlay.
Hope that helps a little.
Will later publish the source when I get graphics to work correctly.
Last edited:
Yes, darkpanda wrote about this first type, so I know that INT 0x3f <overlay number> <address> is overlay call. What I do not know that's why overlays do not work after unpacking with old dos unpacker (unp.exe). Workaround with 2 exe was used. I only can suppose that exe stores overlay addresses in exe somewhere or something. I have no idea how that Microsoft overlay manager works (or any overlay manager). I'm illiterate. So this source code should be a godsend to me.
rahorvat
Warlord
Yes Tupi,
the old unpacker stripped overlays from the original exe and that is why it didn't work. My unpacker keeps them and that's why it's working.
The overlays are just additional exe's appended to the original exe like so:
This image is taken from The MS-DOS Encyclopedia by Ray Duncan.
There is also the overlay number stored in EXE header at address 0x20.
The image is from 'The MS-DOS Encyclopedia' by Ray Duncan
Hope that clarifies things
Last edited:
rahorvat
Warlord
The Microsoft overlay manager works by loading only one Overlay at a time thus saving the memory needed to load the complete EXE.
The Overlay 0 (Main program) is always loaded and other overlays are loaded as needed.
Last edited:
Ah, so overlays simply were not here! I did not even check that! For some reason I was pretty sure that they ARE still in exe, but game cannot find them. So I tried to understand why it would happen: so I made an assumption that the overlay logic stores their addresses somewhere. But no, all we need is just MZ string, and overlays did not work simply because they were not here! All clear now, thank you.
Last edited:
rahorvat
Warlord
Yes, you are right
that is exactly what happens. Also, notice that only main overlay 0 is packed.
Last edited:
rahorvat
Warlord
Hi Civ community,
I think that I finished the disassembler portion of the code. First I envisioned some kind of C decompiler, but quickly gave out because of heavy use of the assembler alongside C.
The disassembler produces quasi assembler code that directly compiles in C#. The code that disassembles the game is available at https://github.com/rajko-horvat/Disassembler
The other part is where I use produced code and slowly translate it to C#. I finished the Image and palette loading code, the VGA is quick and dirty hack to display the screen, but will be later replaced. For now there is some bug that doesn't display intro and main game screen.
The source code of Civilization 1 rewrite called OpenCiv1 is located at https://github.com/rajko-horvat/OpenCiv1
If anyone is willing to please contribute to the work and translate parts of the code it would be of immense help
.
The final goal is to produce playable game and translate all of the code to C#, after that I think that JavaScript would be best to use as the final engine since every web browser supports it and everyone can play regardless of the underlying platform, that also includes multiplayer games (and the differences between C# and JavaScript are not that great).
Cheers
I think that I finished the disassembler portion of the code. First I envisioned some kind of C decompiler, but quickly gave out because of heavy use of the assembler alongside C.
The disassembler produces quasi assembler code that directly compiles in C#. The code that disassembles the game is available at https://github.com/rajko-horvat/Disassembler
The other part is where I use produced code and slowly translate it to C#. I finished the Image and palette loading code, the VGA is quick and dirty hack to display the screen, but will be later replaced. For now there is some bug that doesn't display intro and main game screen.
The source code of Civilization 1 rewrite called OpenCiv1 is located at https://github.com/rajko-horvat/OpenCiv1
If anyone is willing to please contribute to the work and translate parts of the code it would be of immense help
.The final goal is to produce playable game and translate all of the code to C#, after that I think that JavaScript would be best to use as the final engine since every web browser supports it and everyone can play regardless of the underlying platform, that also includes multiplayer games (and the differences between C# and JavaScript are not that great).
Cheers
Last edited:
Lord_Hill
Prince
Well done for persevering with this. Out of curiosity did you go for DOS or Windows in the end?
rahorvat
Warlord
Well, most of the people said: start with a DOS version. So this code is directly from DOS version
Lord_Hill
Prince
Well I'm no coder, but don't forget plenty of these structures have been reversed in isolation by others already. Could help as a reference. There's a great list of useful threads here.
sourceforge.net
JCivED / Wiki / Welcome to JCivED
sourceforge.net
rahorvat
Warlord
Yes, but, I'm reconstituting original code, so I don't tend to relay on structures yet, maybe later when I will be doing save game logic... Also I found discrepancies between win version structures that code uses and documented structures. I think that the best is to compare reversed structures with actual code.Well I'm no coder, but don't forget plenty of these structures have been reversed in isolation by others already. Could help as a reference. There's a great list of useful threads here.
JCivED / Wiki / Welcome to JCivED
Last edited:
darkpanda
Dark Prince
- Joined
- Oct 28, 2007
- Messages
- 844
Nice effort ! I tried to take a look, but really not familiar with C#, so I don't really know where to start.
Also wondering why you don't just use an "on-the-shelf" disassembler, such as Ghidra ? I personally have been using IDA for a long time, which unfortunately cannot decompile 16-bit assembly.
Re-also
what difference do you make between "original code" and "reversed structures" ? You understand that most reversed structures where actually reversed from original code, tpically through IDA disassembly (or DOSBox debugging), and original disassembled code quite clearly accesses game data as structures (mostly arrays or arrays of arrays).
rahorvat
Warlord
Because I can directly disassemble and then compile code inside C#. I'm also using IDA for comparisons and help, but as you said it can't decompile 16-bit code. Also, a lot of code is directly coded in assembler, so I can't even make some C decompiler since it's known that the code was compiled with Microsoft C 5.1 optimizing compiler.
Yes, but I like to double check just for sure. Since I'm not in a playable stateRe-also
I don't yet need the structures.I'm currently translating graphics functions to make things go faster, and once I start to reconstruct the game logic code off course I will use reversed structures for help.
rahorvat
Warlord
Hi everyone,
I reconstructed fonts.cv file structure (Darkpanda was right almost for all), also a graphics structure (pointer at 0xaa) that holds rectangle, colors and a font ID. Code is still not fully functional, partly because VGA driver is still not functioning properly
One curiosity about compiler medium memory model, the DOS exe shares data segment with stack segment, so they are interchangeable in the code. But allows for easy data corruption in case of increased need for stack space.
Cheers
I reconstructed fonts.cv file structure (Darkpanda was right almost for all
), also a graphics structure (pointer at 0xaa) that holds rectangle, colors and a font ID. Code is still not fully functional, partly because VGA driver is still not functioning properly One curiosity about compiler medium memory model, the DOS exe shares data segment with stack segment, so they are interchangeable in the code. But allows for easy data corruption in case of increased need for stack space.
Cheers
Last edited:
rrreese
Chieftain
This is really impressive work!Hi everyone,
I reconstructed fonts.cv file structure (Darkpanda was right almost for all
One curiosity about compiler medium memory model, the DOS exe shares data segment with stack segment, so they are interchangeable in the code. But allows for easy data corruption in case of increased need for stack space.
Cheers
rahorvat
Warlord
Hi,
If anyone is interested I have published the specification (format) of the overlay (driver) files: EGRAPHIC.EXE, MGRAPHIC.EXE, MISC.EXE, TGRAPHIC.EXE, ASOUND.CVL, ISOUND.CVL, NSOUND.CVL, RSOUND.CVL, TSOUND.CVL.
The specification is at GitHub: CVL and EXE overlays.txt
Cheers
If anyone is interested I have published the specification (format) of the overlay (driver) files: EGRAPHIC.EXE, MGRAPHIC.EXE, MISC.EXE, TGRAPHIC.EXE, ASOUND.CVL, ISOUND.CVL, NSOUND.CVL, RSOUND.CVL, TSOUND.CVL.
The specification is at GitHub: CVL and EXE overlays.txt
Cheers
Similar threads
