Mini-engine progress

[AOS9001]

It's 2025, an official remaster would be so poorly optimized, bloated, and stuffed with DRM that it'd run slower than the original two decade old game. See: the original Star Wars Battlefront remasters (not sure if that had DRM or not but I know it was a disastrous launch).

I'll trust the people of CivFanatics any day over a corporation at this point.

 

[Nightinggale]

IIRC that's exactly what VCMI does with Heroes3. VCMI is for 10+ years without any legal issues.

My understanding of copyright law is that it's about distribution. Once you have files on your HD, you can pretty much do whatever you want to them as long as it's for personal use and you don't give them to anybody. Famously there was a court case about that regarding DVD movie copy protection and the first linux driver/player. Court ruled that it's legal to modify the data legally bought in order to use it for personal use even if whoever made the data in the first place said no.

While Firaxis' remaster of Civ4 could save us the hurdles of remaking the civ from scratch on a 64-bit engine, there is a great chance it will go without an SDK. And it would be so over for so many mods.

It's 2025, an official remaster would be so poorly optimized, bloated, and stuffed with DRM that it'd run slower than the original two decade old game. See: the original Star Wars Battlefront remasters (not sure if that had DRM or not but I know it was a disastrous launch).

I'll trust the people of CivFanatics any day over a corporation at this point.

We would not benefit from an official 64 bit remaster. It wouldn't be compatible with anything we have made over the last two decades. Civ4 seems to be a one off regarding releasing the source code for a dll file, so that's unlikely to happen. In fact linking to a dll in the mod is windows only. The mac port should link to a dylib (dynamic library), but the people making the port statically linked all the dll code inside the binary, which is why mac compatible mods can't use any C++ code.

Making our own engine will finally add proper mac support, but ironically I used mac when civ4 came out and windows today so.... wrong order or something for that (mac was completely different before Apple decided to merge MacOS and iOS and put as much as possible in the cloud. The switch started in 2011).

If I'm completely honest, I think we would all be better off with no official remaster. We wouldn't benefit from it and I wouldn't want to compete against one. Also I think Firaxis would be better off making a brand new game rather than cloning the gameplay of a 20 year old game. It will mainly target nostalgia and that will likely fail without supporting existing mods. The vast majority of potential customers have never played civ4, so they aren't particularly interested in such an old game.

It's 2025, an official remaster would be so poorly optimized, bloated, and stuffed with DRM that it'd run slower than the original two decade old game.

Making a new engine means we have the engine source code. This will allow us to optimize everything and that way make the game faster than what we have now. For starters we can optimize the issue where the game starts slowly if a big mod is loaded, through with that complete control, adding support for using more CPU cores could become easier. WTP uses 8 cores for pathfinding and as many as possible for selecting unit professions. True multithreading is partly restricted by exe interactions not supporting multi-threaded execution.

 

[snowern]

This will allow us to optimize everything and that way make the game faster than what we have now.

Because of Linux case sensitivity, I'm enumerating all files in the VFS on startup, so that's slow. Should probably speed this up by doing case-insensitive physical file ops instead, if possible.

In-game though, the engine isn't really a problem. You can multithread all you want in the DLL. Except, when you have to call into the engine through the interfaces.

Python is another problem, but there's nothing you can do about that except stop using Python.

WTP uses 8 cores for pathfinding

I thought about doing that, threading within a single pathing request, but it's difficult to get any benefit out of it, afaik. Process too few nodes, and you have threading overhead. Process too many nodes, and you're doing too much work.

 

[Nightinggale]

In-game though, the engine isn't really a problem. You can multithread all you want in the DLL. Except, when you have to call into the engine through the interfaces.

Be aware that the vanilla code has overloaded new and delete, which means the exe handles memory allocations in a single threaded approach. You need to disable such less obvious interactions before going multithreaded. There is a lot of interaction between the dll and the exe, more than meets the eye at first glance.

I thought about doing that, threading within a single pathing request, but it's difficult to get any benefit out of it, afaik. Process too few nodes, and you have threading overhead. Process too many nodes, and you're doing too much work.

Test shows that WTP benefits from doing this.

 

[Mr Smith]

Civ4 seems to be a one off regarding releasing the source code for a dll file

Pretty sure Civ5 also had source code released for years.

 

[snowern]

Release imminent.

(many bugs were fixed in the making of this footage)

 

[Blake00]

oh boy.. oh boy.. oh boy!

 

[<Nexus>]

Release imminent.

(many bugs were fixed in the making of this footage)

I was just playing EPIC MUSIC in an other tab when starting the video and it was... epic!

But we most certainly need some graphical interface because I only could guess what's going on from the sounds.

 

[Chrischn89]

Release imminent.

(many bugs were fixed in the making of this footage)

It's beautiful and so... fast!! Hearing the original sounds is also amazing! Really outstanding what you achieved @snowern !

 

[Nightinggale]

But we most certainly need some graphical interface because I only could guess what's going on from the sounds.

I completely agree with this. It's impressive in many ways, but without a proper graphical interface the gameplay value is questionable at best.

 

[GreedyManul]

Release imminent.

(many bugs were fixed in the making of this footage)

It feels so eerie seeing it in action. It feels familiar, yet very unfamiliar at the same time. It's like seeing the child of dwarf fortress and civ 4 haha

 

[snowern]

Source code: https://github.com/fortsnek9348/Cv4MiniEngine

Now, will somebody else be able to build it? Who knows. We'll find out. No binary release yet.

I completely agree with this. It's impressive in many ways, but without a proper graphical interface the gameplay value is questionable at best.

Yes, a useful stepping stone to a graphical engine or further DLL changes. General play in a terminal is not the best thing in the world, but the video shows it works!

 

[black_imperator]

Source code: https://github.com/fortsnek9348/Cv4MiniEngine

Now, will somebody else be able to build it? Who knows. We'll find out. No binary release yet.


Yes, a useful stepping stone to a graphical engine or further DLL changes. General play in a terminal is not the best thing in the world, but the video shows it works!

that's very much a key milestone to having an hopefully modern implementation of 4.

Thanks a lot for the work you've put into this.

 

[Chrischn89]

So I tried to open the solution file (Cv4MiniEngine.sln) in VS2022 (Windows 10) and it tells me that it can't find „..\..\HeckModule\PropertySheet.props“. Did I mess up somewhere?

 

[snowern]

So I tried to open the solution file (Cv4MiniEngine.sln) in VS2022 (Windows 10) and it tells me that it can't find „..\..\HeckModule\PropertySheet.props“. Did I mess up somewhere?

Oh, of course that had to happen. I thought I removed external references like that. Just pushed a fix to use "PropertySheet.props" at root instead.

 

[Jerrybi]

The UI (TUI) is separated from other logic?
and would be REALLY great to get some insights on project structure
if UI is separated via good interface it would simplify future graphics development

Move to Python3 and specifically python3.14 with freethreaded mode would be great improvement

 

[snowern]

The UI (TUI) is separated from other logic?
and would be REALLY great to get some insights on project structure
if UI is separated via good interface it would simplify future graphics development

Move to Python3 and specifically python3.14 with freethreaded mode would be great improvement

Because most UI is specified via layouts, it may be feasible to keep some of the same UI code for a graphical implementation. You would probably need plenty of tweaks though. Graphical UI has themes, and text UI has border styles, for example.

The glue code between python and the TUI is mostly CyGInterfaceScreen.

Python 3 is unlikely, afaik. The Firaxis engine used Python 2.4 and Cv4MiniEngine uses the latest Python 2, 2.7, which already needed some engine changes. Python 3 will likely need script changes, because map scripts use the division operator, and that will behave differently. And there may be other differences.

"Free threading" sounds useful, for any parallel loops that would want to call into python. Sounds rather dangerous though. The scripts certainly weren't made for it. If you make BUG logging and some engine functions thread-safe, it may be possible to do it for plot reveals when trading maps.

 

[Jerrybi]

Because most UI is specified via layouts, it may be feasible to keep some of the same UI code for a graphical implementation. You would probably need plenty of tweaks though. Graphical UI has themes, and text UI has border styles, for example.

The glue code between python and the TUI is mostly CyGInterfaceScreen.

Understand, but when you open most of UI toolkits \ engines, the main question is where entry points are , what they expect and what they provide.
so separation via interface or any other abstraction is really needed.

So the only point, for Python3 is ability for existing codebase to work right?
need to check any traverse solutions (AST or py2topy3)

 

[snowern]

Understand, but when you open most of UI toolkits \ engines, the main question is where entry points are , what they expect and what they provide.
so separation via interface or any other abstraction is really needed.

So the only point, for Python3 is ability for existing codebase to work right?
need to check any traverse solutions (AST or py2topy3)

The big entry point is the HeckTextUI project itself. ConsoleDevice, Framebuffer, and UIScene.

At it's simplest, you could port the whole UI system to $3DAPI by emulating a console and putting your 3D world view in the middle.

Python 3 is an interesting idea. I would like to get off old busted python. Even if I would have to change most scripts, they could be provided as diffs, as long as we all have the exact same original files. That includes BUG mod.

 

[Walter Hawkwood]

Python 3 is an interesting idea. I would like to get off old busted python. Even if I would have to change most scripts, they could be provided as diffs, as long as we all have the exact same original files. That includes BUG mod.

That would still pose major potential compatibility challenges to lots of mods with Python components - from what I understand, in the current implementation, they should be compatible with the new engine out of the box.