Making a Clone of Civ II

[joe@joesblog]

OK. I'll try to set something up on Godot but yeah, I'm far away from implementing civ2 scenario events let alone dealing with Lua integration. I see you've provided a link getting started with Lua, I'll take a look at it.
I'm still mostly doing work on GUI and I've figured this stuff quite well, in fact I'm finishing the map drawing stuff. Anyway the whole thing is still in a state where it can be ported to other environments/engines/frameworks.

Hey have you thought about using SDL? It's pretty powerful and well documented.. I've also found ways to use it in winforms and the ways to handle events and the renderer stuff. It supports the accelerated stuff too and is fast.

In terms of GUI layout stuff I wrote a library to implement the old amiga style GUI called workbench, it may be of some use? I've done quite a few things in winforms and it's always a pain using the GDI system..

https://github.com/joesblog/AmigaControls

I don't seem to get alerts when replies are posted on here.. but will check back every other day..

If I can be of any help I'm around..

 

[axx]

Hey have you thought about using SDL? It's pretty powerful and well documented.. I've also found ways to use it in winforms and the ways to handle events and the renderer stuff. It supports the accelerated stuff too and is fast.

In terms of GUI layout stuff I wrote a library to implement the old amiga style GUI called workbench, it may be of some use? I've done quite a few things in winforms and it's always a pain using the GDI system..

https://github.com/joesblog/AmigaControls

I don't seem to get alerts when replies are posted on here.. but will check back every other day..

If I can be of any help I'm around..

I have thought about SDL but I have little experience with it. Could you achieve the same look as Civ2 has (menus, panels, etc.) with it? Or with your library? I'll take a look at it. I always thought of Winforms as a way to do a mockup of the project then move on to some other environment. Honestly some of the engines we've discussed seem like an overkill for an event driven game such as this.

Anyways made a video on what I've done sofar, don't mind the tile gaps on edges, I need to fill them up but lazy. I've addressed some memory leaks so GC isn't doing much work (seen at end of video).

 

[joe@joesblog]

SDL gives you the ability to do nice things with the renderer, and do it quite fast. SDL does not implement any sort of GUI out of the box, in a CPP environment I would use something like QT to handle the windows etc (QT is also cross platform).

Having a quick look at your code; the library I wrote (AmigaControls) uses a similar pattern to yours I think. The original Civ2.exe uses calls to User32::CreateWindowExA to setup the GUI ( see screenshot).

I don't think a third-party engine is a necessity either. I'm torn as to whether C# or CPP is the way forward, as someone mentioned earlier C# is becoming more popular though I wonder if that in part is due to Unity(i think) having that as its choice for a language.

The map looks to me like a like for like implementation of the original, still really impressive!. That level of memory usage looks now to be stuff on the heap (objects/collections) rather than graphics data.

A few thoughts:

• Do you have a roadmap in mind for how to go forward?
• Maybe taking a modular approach and separate the graphics, controls and logic into separate libraries/projects?
• the SAV file is quite limited, and uses things like bit fields in struct ie unsigned int flatWorld : 1 and stuff like that( i guess done for file/memory availability in 1996), having a more modern file format would be easier to use, can still import/export to the old format where possible.

I'll also include something I quickly knocked up in my test thing for SDL in WinForms.

 

[axx]

That's lots of boing.pngs I've been messing around with SDL yesterday, I'll try to set something up for civ2 now. Your library should be helpful by the looks of things. Well the SDL UI and the core game could be 2 projects, I think CivONE does that, it's also using an SDL C# wrapper. As for the roadmap there is none, I'm just dealing with stuff on the go, but it's pretty obvious the game should include ToT and support Lua. You could use the same SAV format but make something like SAVx which makes more than 7/8 loops for reading tribe data (in that case there should be another byte in the beginning of file which tells you how many civs there are). Then make every byte into short or int so more than 256-whatever can be saved. Or just go to a more modern format like you said.

 

[Blake00]

Anyways made a video on what I've done sofar, don't mind the tile gaps on edges, I need to fill them up but lazy. I've addressed some memory leaks so GC isn't doing much work (seen at end of video).

Wow that video is amazing mate. You totally recapture the feel of the original! I also love how there's a stealth bomber in ancient Rome in your test save haha!

Oh and regarding earlier posts about figuring out differences between MGE & ToT sav and scenario formats there are some old conversion programs out there but I don't know if their source was released, as that seeing what happens in the conversion process would probably be useful! No doubt others here know way more about all that stuff though, but thought I'd mention it.

.

 

[axx]

Oddly I just joined this forum as I was heading in this direction. I've started disassembling and looking at how various things work in ghidra/x64dbg. The isometric grid issue was a right pain in the backside for me too as it doesn't use the normal algorithm for generating an isometric map! grr..

Have you tried using retdec? It attempts to make a .c code from the exe. I tried it on simple .exe I wrote using C and I'm impressed at what it manages to make.
Also tried it on civ2.exe but I get a memory error I'm attempting to solve.

 

[joe@joesblog]

Have you tried using retdec? It attempts to make a .c code from the exe. I tried it on simple .exe I wrote using C and I'm impressed at what it manages to make.
Also tried it on civ2.exe but I get a memory error I'm attempting to solve.

Are you looking at the MGE executable?

I went through quite a few including IDA and I think retdec though they pretty much do a similar jobs and both have their advantages in the stuff they output (ghidra seems to output the best pseudo c code though imho). However when I tried retdec I think it stumbles on the analysis of big files. What's the memory error you're getting? Is it getting stuck on the flow analysis?

Also something cool; the civ2 asm has its own debug methods, they're useful in figuring out bits and pieces. (There are string references to cpp files, a way to find what functions are responsible)

 

[axx]

Are you looking at the MGE executable?

I went through quite a few including IDA and I think retdec though they pretty much do a similar jobs and both have their advantages in the stuff they output (ghidra seems to output the best pseudo c code though imho). However when I tried retdec I think it stumbles on the analysis of big files. What's the memory error you're getting? Is it getting stuck on the flow analysis?

Also something cool; the civ2 asm has its own debug methods, they're useful in figuring out bits and pieces. (There are string references to cpp files, a way to find what functions are responsible)

Yeah I was analysing civ2.exe. Managed to get a .c file with --backend-no-opts parameter I believe (not at my comp right now).
Nice with the asm find. Please do document here everything you find.

 

[FoxAhead]

I was disassembling civ2.exe MGE with IDA. With default settings it gives a lot of incorrect standard library function names. After some analysis I guessed that civ2.exe was build using Visual C++ 4.xx. Then I got LIBCD file and used FLAIR tools to make my own signature library. With some additional fixes now it detects almost all library functions correctly.
Next big step is decompilation. And here is big challenge of decompiling try.. catch code or some vector constructors.. destructors. As it is using some not obvious exception handling structures, and Hex-Rays plugin doesn't generate clean code for them. Most times it fails with stack error. Actually official page says "exception handling is not supported".
There is good topic about it
http://www.openrce.org/articles/full_view/21
But that means too much manual work. Interesting that most game logic part uses simple c/c++ code. And complicated code is found mostly in multiplayer part.
Never used Ghidra. How does it deals with exception handling structures? Does it generates clean pseodo-code for that?

 

[joe@joesblog]

I was disassembling civ2.exe MGE with IDA. With default settings it gives a lot of incorrect standard library function names. After some analysis I guessed that civ2.exe was build using Visual C++ 4.xx. Then I got LIBCD file and used FLAIR tools to make my own signature library. With some additional fixes now it detects almost all library functions correctly.
Next big step is decompilation. And here is big challenge of decompiling try.. catch code or some vector constructors.. destructors. As it is using some not obvious exception handling structures, and Hex-Rays plugin doesn't generate clean code for them. Most times it fails with stack error. Actually official page says "exception handling is not supported".
There is good topic about it
http://www.openrce.org/articles/full_view/21
But that means too much manual work. Interesting that most game logic part uses simple c/c++ code. And complicated code is found mostly in multiplayer part.
Never used Ghidra. How does it deals with exception handling structures? Does it generates clean pseodo-code for that?

It handles it 'reasonably' well. though as well as can be expected without any debugging symbols or source code. Ghidra allows you to create your own structs, and i think import header files.

Whilst not explicitly decompiling to a try..catch it does handle stack unwiding. I've included a pastebin as an example (I think this is from the multiplayer code)
https://pastebin.com/GhXQDAyb

by looking at the flow of the code you can see how it was originally implimented

 

[axx]

Tried with SDL but didn't get further than intro screen.
Anyways if anyone is still interested, I'm re-writing the project with Eto forms so you can compile it not just on Windows, but on Linux and Mac also. Tried it on Ubuntu yesterday and it worked, but I've messed up some references in the meantime. I've also moved the engine part to a separate project, now you interact with it through UI projects.

 

[Rui_TNV]

Tried with SDL but didn't get further than intro screen.
Anyways if anyone is still interested, I'm re-writing the project with Eto forms so you can compile it not just on Windows, but on Linux and Mac also. Tried it on Ubuntu yesterday and it worked, but I've messed up some references in the meantime. I've also moved the engine part to a separate project, now you interact with it through UI projects.

Will this be a game similar with civ 2 test of time? And will it be possible to play on android?

 

[axx]

Will this be a game similar with civ 2 test of time? And will it be possible to play on android?

Since they are similar both Civ2 & ToT can be bundled into a single game (you'd have a switch to chose which one you want). As for Android it's too early to talk about it but Eto currently has an Android platform under development.

 

[CurtSibling]

I salute this project. A real massive goal, but a very worthy endeavour.
Breaking the limitations (like only 7 playable civs) of the original game would be great.

Following this thread with interest. And in a world were getting the original game code
seems impossible, the clone route might be the best way to unlock civ's true potential.

 

[XDXDXDXD]

Hi guys

I'd like to join the project. Not much of a programmer, but I can be a project manager and solve specific problems the project may have.

Languages I know - Python, C++

Other from that I can invest some reasonable resources into the project.

 

[Blake00]

Was anyone here part of the old Clash of Civilizations fan project? How far did it get? I've heard of it in passing but know very little about it due to being absent from the community in the mid 2000s. Had a look at the screenshots and it seems like they were going for a Civ2 like game although it looks more like a FreeCiv reinterpretation. Just found backup of their old website and was curious about it...
https://web.archive.org/web/20100901192522/http://clash.apolyton.net:80/frame

Final demo download still works so I might grab a copy as I'm gonna do a video tributing Civ2 like fan games (finished or not) at some point.

.

 

[LuKo]

How is the project coming along? Is there any changelog/to do list to monitor the progress / offer help?

 

[CRichard]

Just have Civilisation 2 + Test of Time released on GoG 4Head.

 

[axx]

How is the project coming along? Is there any changelog/to do list to monitor the progress / offer help?

Hi, sorry for the late reply, I don't check here nearly as often as I should. Just look at the commits here, it should be self-explanatory.
Another person (reubene on github) joined development and is amongst other things working on ToT support.
You can compile it and try loading a SAV and then report issues on github. Or help contributing if you're familiar with C#.

 

[LuKo]

No problem - I am glad that someone puts effort in creation of something like that
I have some experience in reading and tweaking code but my overall programming knowledge is not great and I have a problem even with the compiling. I have downloaded the whole zip from the GitHub and MS Visual Studio and compiled Civ2clone.sln. Trying to run Civ2engine through VS I receive information that I am unable to do that and than in order to debug I need to add executable project to the solution. Could you please explain me what exactly should I do? As you can tell non-idiotproof methods failed