Mini-engine progress

[1SDANi]

I love compiler memes. Half the conversations on the BFBB decomp server are everyone pulling their hair out over floats.

 

[snowern]

At least Civ4 doesn't do floats... Well, it does a little, in CvPlot::shouldProcessDisplacementPlot. Oh, and getCombatOdds. I hope that's not going to mess up my perfectly deterministic world.

 

[1SDANi]

Just be glad you know you have the compiler the 4K devs used. BFBB's decomp team spent years trying to figure out if the memes were the result of some obscure minor revision of the compiler or if the HI devs genuinely cast all their floats to volatile.

 

[keldath]

so,
is is this project going?
need any help?
are we gonna have civ4 64bit in a few years?

 

[snowern]

so,
is is this project going?
need any help?
are we gonna have civ4 64bit in a few years?

It's playable right now. Just finishing touches. Verification of gigantic map optimisations against "live" data, verification of the regular build against the Firaxis DLL, actually play a gigantic map (hopefully nothing pops up), and will attempt Linux. Linux would be interesting. Probably can't use VS, the compiler probably won't be able to handle MSVC-specifics, you got case-sensitivity, and the terminal will work differently.

Future things post-release could be Python 3/latest pybind (likely to require script changes), get rid of CvString/CvWString, complete and total refactoring to remove almost all globals, fix all clang warnings, GRAPHICS, multiprocess mapfinding, new Civ4 machine-learning environment (like Freeciv) for ML devs. Plenty of things for people to do if they're crazy enough. I'll probably try some refactoring. C++ modules sound fun.

 

[1SDANi]

It's playable right now. Just finishing touches. Verification of gigantic map optimisations against "live" data, verification of the regular build against the Firaxis DLL, actually play a gigantic map (hopefully nothing pops up), and will attempt Linux. Linux would be interesting. Probably can't use VS, the compiler probably won't be able to handle MSVC-specifics, you got case-sensitivity, and the terminal will work differently.

Future things post-release could be Python 3/latest pybind (likely to require script changes), get rid of CvString/CvWString, complete and total refactoring to remove almost all globals, fix all clang warnings, GRAPHICS, multiprocess mapfinding, new Civ4 machine-learning environment (like Freeciv) for ML devs. Plenty of things for people to do if they're crazy enough. I'll probably try some refactoring. C++ modules sound fun.

Can't wait, we'll finally be able to mod the code for rivers and the Great Wall!!!

 

[keldath]

Is there something i can contribute with?

If this project will be what it aims to be,
It could revolutionize civ fans and brong glory to civ4 and its perfection.

 

[Git-384]

Python 3/latest pybind

If you are already moving away from Boost Python skip Pybind and use Nanobind instead.

C++ modules sound fun.

Only if you like to deal with Compiler bugs because sadly no C++ Compiler has complete and stable module support.

 

[black_imperator]

if i could suggest one thing that may need to be murdered, dissected and rebuilt, it's the plotgroup system (which is the system handling which tiles are connected for resource access and trade purposes).

From what i've had to interact with, it seems very innefficient, with groups being completely deleted and rebuilt tile by tile each time there's an effect that impacts it ( pillaged road or improvement, discovered resources, ....)

(storage of the network is the key part to make that work, as of now, it's just a list of all the tiles in the group, without better understanding of the links between them)

 

[Git-384]

if i could suggest one thing that may need to be murdered, dissected and rebuilt, it's the plotgroup system (which is the system handling which tiles are connected for resource access and trade purposes).

The CvPlotGroup code is included in the DLL.
Various mods made changes to that system and if you want to rebuild it completely today you could.

 

[black_imperator]

The CvPlotGroup code is included in the DLL.
Various mods made changes to that system and if you want to rebuild it completely today you could.

i know, i know, but since we're discussing potential upgrades, i'm mentioning it ^^

 

[snowern]

Can't wait, we'll finally be able to mod the code for rivers and the Great Wall!!!

That's a mystery for me. How is the great wall encoded in the save file, and how do you build the wall out of NIFs. It's probably some list of coordinates.

Is there something i can contribute with?

If this project will be what it aims to be,
It could revolutionize civ fans and brong glory to civ4 and its perfection.

A "fun" thing to do would be a total refactoring of the DLL to reduce globals. But there will be a point of friction because a python interpreter is naturally a big global entity, and the scripts expect globals. Something that would be nice is if the game itself and other in-game-only classes were no longer globals. That way, initialising and destroying a game would be more elegant in code.

There's also the unicode refactoring. I'd probably just use wchar_t as-is for the Linux build, and do conversion in serialisation. That's the quick way. The more portable way is to do away with wchar_t and switch to char8/16/32_t. char8_t won't work well with C++ IO, but I'd like the type safety that separates it from non-"text" strings. Check that XML parsing handles encoding correctly too.

And another thing is missed optimisations. No doubt some slow paths remain. Find them in a profiler.

And ultimately, the graphical engine implementation. It would be easier in some ways as the DLL is designed for a realtime engine. But much more difficult in other ways. One random thing I've seen is that some NIFs have spline animations encoded in some format I don't know of.

If you are already moving away from Boost Python skip Pybind and use Nanobind instead.

Only if you like to deal with Compiler bugs because sadly no C++ Compiler has complete and stable module support.

Pybind has python embedding. Ie, starting up the python interpreter and running code. That's nice to have. But if nanobind were to support implicitly convertible int enums, that would be worth switching over for. Because that's how boost python did enums, and how scripts expect it. Pybind enums are strongly typed and I have to hack them to make them work with scripts. It is possible to make something custom, but you have to use class_ or enum_ to get the runtime registration behaviour.

I have actually got a modules branch working. Everything modularised except the engine and gigantic map optimisations. Just about. You only need to do a number of silly workarounds, and Intellisense barely works, and it's basically one big giant module. But compared to includes, the debug rebuild is twice as fast. But compared to the PCH, it's not much faster. I would like to use modules over PCH though, if the dev experience wasn't so terrible.

if i could suggest one thing that may need to be murdered, dissected and rebuilt, it's the plotgroup system (which is the system handling which tiles are connected for resource access and trade purposes).

From what i've had to interact with, it seems very innefficient, with groups being completely deleted and rebuilt tile by tile each time there's an effect that impacts it ( pillaged road or improvement, discovered resources, ....)

(storage of the network is the key part to make that work, as of now, it's just a list of all the tiles in the group, without better understanding of the links between them)

Plot groups were one of the slow paths I sped up. Haven't seen them on the profiler since. Yet. I need to get a fully populated gigantic map to stress test everything.

Anyway, uhh... the barbs have been making too many animals.

#define FLTA_ID_SHIFT                (13)
#define FLTA_MAX_BUCKETS        (1 << FLTA_ID_SHIFT)

╭───────────────────────────────────────────────────────────────────────
│                                                  MILITARY ADVISOR    
├┬─────────────────────┬─────────────────────────────────────╥──────────
││[ ] Julius Caesar    │░░ ╭──────────────────────────────┬─╮║        
╡╰─────────────────────╯░░ │Location                      │▼│║        
┤╭─────────────────────╮░░ ╰──────────────────────────────┴─╯║        
││[ ] Hammurabi        │░░ ╭──────────────────────────────┬─╮║        
│╰─────────────────────╯░░ │Unit Type                     │▼│║        
│╭─────────────────────╮░░ ╰──────────────────────────────┴─╯║        
││[ ] Zara Yaqob       │░░ [ ] Show individual units         ║        
│╰─────────────────────╯░░                                   ║        
│╭─────────────────────╮░░ [ ] ALL UNITS (8190)              ║        
││[ ] Tokugawa         │░░    [ ] Neutral Territory (8190)   ║        
│╰─────────────────────╯░░       [ ] Lion (4465)             ║        
│╭─────────────────────╮░░       [ ] Bear (774)              ║        
││[ ] Ramesses II      │░░       [ ] Panther (1241)          ║        
│╰─────────────────────╯██       [ ] Wolf (1710)             ║        
│╭─────────────────────╮██                                   ║        
╡│[ ] Frederick        │██                                   ║        
│╰─────────────────────╯██                                   ║        
│╭─────────────────────╮██                                   ║        
││[●] Barbarian        │██                                   ║        
│╰─────────────────────╯██                                   ║        
│

(it's a TUI, so I can just paste in the text, right?)

Maybe I'll need to upgrade the ID allocation system to 64-bit or put a hard cap on barbarians. Capping barbs would be the simplest option if possible. Surprised I didn't catch this earlier, it's only turn 32. Is this supposed to happen?

*Ahh... I think it's because I left the difficulty on deity, which has a lower getUnownedTilesPerGameAnimal.

 

[Git-384]

Can't wait for Caveman2Cosmos being ported to your engine. Finally playing it without running out of memory and dreadful turn times almost sounds too good to be true.

 

[snowern]

Can't wait for Caveman2Cosmos being ported to your engine. Finally playing it without running out of memory and dreadful turn times almost sounds too good to be true.

That would take some effort.The TUI world view currently hard codes a lot of things, not very expandable right now. Or rather, this is what a graphical engine is ideal for, then you can just use the mod's assets as-is. After you merge in the DLL changes.

*Okay... so I do need to accelerate multi-unit pathfinding...

Path requests: 344947, acc=129.035s, max=0.10659s
Path requests (FAStar): 111694, acc=111.462s, max=0.106584s
Path requests (lite verify reachability): 87227, acc=0.173779s, max=0.0001732s
Path requests (multi-unit): 22886, acc=91.6555s, max=0.10659s
Path requests (other flags): 4467, acc=0.105721s, max=0.0034276s
Path requests (other unsupported): 84341, acc=19.8126s, max=0.052788s
Path requests (siege move through enemy): 0, acc=0s, max=0s
Path step distance: 101841, acc=0.436463s, max=0.0008209s
Turns: 32