Hello again! I'm sorry that I didn't give an update any sooner (as usual lots of IRL stuff going on
), nonetheless I made quite some progress as I still invest any free minute to get things off the ground:
No, it doesn't work that way.
One of the main goals is to get rid of the 32 bit engine's limitations.
And about the UI:
@Chrischn89 has already started working on the graphics side of things. Even if he won't do it all single handedly, the work HAS BEEN STARTED on it
That's exacty right! My goal is to get a basic prototype going so that more talented and experienced coders can jump straight in and start playing around immediately.
Do we have a status update on this? Last I can find is a month old and about unpacking packed graphics. I'm way more interested in knowing any progress regarding converting the 3D files into an open format modern engines can read. My point of view is that a full engine replacement is "if it can be done", but it will change to "when it will be done" once we have 100% 3D graphics compatibility because then it becomes more of a long list of tasks to do without having tasks where it's not known if they can be done.
I absolutely agree with all of you that's it is a thing of "when" not "if". I still think that with Civ IVs 20th anniversary next month, people might think back of how great Civ IV actually was (and still is!) and might also start wishing for a modern remaster. It would be great if people stumbled upon what's going on here and offered to share their expertise in different fields.
So, having discussed this with the people that do art for AoE, the process of converting nif/kf to modern formats is possible, but they're not aware of a mass convert process, so it'll possibly be tedious.
my personal concern is an issue of chicken and egg. no art people will want to commit to that process (if no mass convert in sight) without a proof of concept of the engine ( if only to decide what the new format will be), so we need that first, with minimal graphic people's support.
Converting the nif/kf files isn't really the problem in my opinion. There's quite a few plugins and converters out there (thanks to the Elder Scrolls still using nifs) so that it's trivial to take those and convert the files to just about anything else. The underlying problem for an open source reimplementation is the copyright though. We can't distribute the original nor converted files that are based on the original files. That leaves us with two options: 1.) convert the files on the user's device and call them up from there (which could be either quite the complex task to automate or cumbersome for the user if he has to do it by himself) or 2.) we let the user point to a valid Civ IV installation on his system and read the original files raw and create our own data structures (or rather structures that the underlying engine can work with) inside the program.
The chicken and egg problem is exactly right, which is why I'm working on a prototype atm, to pull more people in.
Yes, NIFs aren't just meshes. If you want to put them in a new format, that format would have to support animations and particle systems, and whatever else is in there. Maybe you could do it if you use your engine's general scene format, but there may still be points of friction.
Using NIFs as-is shouldn't be too difficult. Especially for a proof-of-concept where you just need meshes. If you convert them, you would need something to load them anyway.
True!
It's understandable. With all the concept ideas being proposed for pretty much everything online, odds are that most of them will never go anywhere. People join in when something starts to work and be useful. It feels like a waste of time to spend possibly months on something, which relies on somebody else doing something and then no somebody else shows up to do their part.
It would be nice to get a guide for how to convert something. On top of teaching more people how to do it, hence spread the workload on more people, knowing what needs to be done could be a help if trying to automate the process. If possible, I would strongly prefer to not include vanilla graphics and instead convert on load, which is for reasons mentioned ages ago about an engine replacement not being a replacement for buying the game. However automating the process is a concern for after having figured out how to even convert at all.
Also I'm not a huge fan of having to manually convert every single modded unit/building/whatever. That could take a while.
100% agree!
------
Current status:
In my last post I hinted at the niftools project as my best bet for getting into those pesky nif files and indeed I've made quite good progress because of it. I've spent quite some time getting the niflib (C++ library) into the Godot project (as a GDExtension) and after fighting with linking and compilation I've finally managed to get my development pipe stable so that I could then start work on the translation layer code.
I think using the niflib library was/is absolutely worth it because it saves so much time by not having to take nif files apart on the byte level ourselves and potentially spent countless weeks of development for troubleshooting errors.
Niflib provides functions for checking a valid nif version, reading out the header section, returning an array/vector of all the "NiObjects" that an entire nif file is composed of, it can return the root node of the nif tree hierarchy and finally it can return the direct children of a given NiObject. These functions are basically all we need to read an entire file and do whatever with it.
So after I've succesfully attached the niflib library to Godot I could feed a file path string into it and with some crude code I could at least draw the most basic meshes of a nif model in Godot:
Which looks pretty sweet if I can say so myself (the ironclad is an homage to your profile pic
@snowern 
)
Initially I thought that that would be a good point to just release it and maybe more talented people could take a look at it but then I realized that the way I'm drawing those meshes is not really useful by itself since it doesn't take into account that A) the nif file hierarchy (I think the proper term is a 3D scene graph?) is composed of a multitude of different types of Nodes that can be attached to each other to create that tree, which means it is super important to keep the hierarchy intact when importing nif files
and B) if I want anyone to understand what I've coded here (and I freely admit that I'm using AI to help me figure things out), that I need to properly comment it and take into account that this all happens inside the Godot engine context, which brings another layer of complexity to it, which is probably a lot to take in at once.
So what I'm doing right now is, rewrite the crude function that draws only the meshes, so that it can read the hierarchy of a given nif file, iterate through the entire file recursively (starting at the root object), determine what type of node it is and then convert it to the corresponding Godot type node (just as a dummy for now since this will be a big undertaking for all the different types of NiNodes) and then attach each converted node to the correct parent Godot node to recreate the original tree structure.
Unfortunately I've run into a CTD while writing it, which I'm debugging right now but for optimal troubleshooting I need to figure out how to work with the debuggers of Visual Studio and Godot (which also required me to recompile the entire Godot engine code with the debug flag set, otherwise I wouldn't even get debug info from it...)
So my immediate goals are:
- comlete the rewrite of the nif load function (create a working debug pipe)
- comment the code
and once completed:
- release the GitHub repo to public
I hope to be done before the anniversary next month, as I think it would be quite fitting to release just around the the 20 year mark for our beloved game
