Plan to get the source code for the exe

A question came up in the quick modding questions thread and since the code in question happens to be in the exe, this line showed up:

This sparked multiple posts about the topic, which in turn resulted in this thread.

Why do we want the source code?

Being able to mode everything
There are some code in the exe, which we would like to mod. Most noteworthy is the Main Menu, including all the "submenus" (new game, custom game etc) and the diplomacy screen.

Bugfree and stable mods
Accessing everything will not only allow modding, but also bugfixing. This is not only bugs in the exe itself, but also bugs in the DLL caused by assuming something about the exe, which seems correct at the time, but isn't. Also debugging anything where the exe is involved will also be a whole lot easier.

There are also some undocumented limitations, such as some enum values are hardcoded in the exe. More than once have modders added something, which results in an offset of one, hence broken code.

Better performance
The exe has inefficient code when communicating with the DLL, like it loops and calls a DLL function to get a bool to tell if it's a certain enum of not, meaning we often run into unneeded loops.
Compiling the exe and dll into one file will not only allow just using the enums, it will also allow some compiler/linker optimization, which is not possible today because the dll can't optimize for something in the exe.

Modern tools
Right now we are using a compiler from 2003 with boost and python from 2004. This is problematic because there is no support for those. For instance I looked at boost threads and it turns out it was changed significantly in 2007 (for the much better), meaning getting info on how to do stuff prior to 2007 is near impossible today. Also the dll is unavailable today meaning you have to compile yourself, which turned into a nightmare.

Using a newer compiler also means more advanced algorithms for optimization and ability to optimize for modern hardware. Modern CPUs have more registers and more instructions, which sits idle when running old code. Just compiling something, which is intended precisely for the CPU in question will make the code execute faster.

Modern C++
When looking up how to do stuff in C++, what you find online today is C++14 or perhaps C++11. We use C++03, which doesn't support all the cool stuff people talk about today. Using a modern compiler will allow usage of C++14.

64 bit application
32 bit applications are limited to 4 GB of memory. 64 bit has no realistically reachable limit in address space. This allows bigger mods.

IPv6
The game uses IPv4 for multiplayer, which is in itself ok. The problem is that we have run out of IPv4 addresses and more and more people can't get a public IPv4. IPv6 has more addresses than there are grains of sand on the planet, meaning everybody should be able to get one and will eventually. Adding support for this will ensure that people will not lose multiplayer. IPv4 will not be dropped, though mixing IPv4 and IPv6 in one game will likely never be supported

Speaking of multiplayer, all players act sort of like a server. Say A hosts a game and B and C joins, C can then be refused because A can be contacted, but C can't connect directly to B. With firewalls and stuff, which can become problematic. Access to the source of the exe can allow the host to be relay between B and C if that is the case.

Modern graphics engine

This seems to be the biggest concern and rightfully so. Graphics engines are big and complex to develop, meaning companies who have done so keep them close to themselves. We might not be able to get the graphics engine at all.

Luckily there is a plan B. Rewriting the graphics interface to use a freely available engine. Unity has great features for this and AFAIK should be free to use until you sell a certain amount of copies. Since we won't be selling copies at all, it should be free to use for us (must verify to be 100% sure. Didn't read everything in their terms). Using a different engine allows modern graphical engine with modern features.

A new graphics engine will also get rid of engine limitations, such as limited to 128 (or was it 127?) rows of plots being drawn when zoomed out.

Cross platform support
With a switch to a cross platform engine like unity, it might be possible to compile for different platforms. We might finally be able to make mac/linux mods, which mods the DLL. This isn't top priority though, more like a possible bonus.

Now what?
The question is who is interested in doing anything for this and how do we do it?

 

Some of the points you make are interesting. I have learned programming purely for the modding here and therefore have NO advanced knowledge beyond what I'm limited to understanding by having examples in the codes here and left from other modders who have worked on the C2C code set. So a lot of what you're bringing up is flying way over my head but it sounds good.

It's the limitations that drive us nuts.

To me, if you were to:

then you might as well just initiate a whole new game.

I found a free to use graphic engine (both 2d and 3d platforms provided) at Garage Games and it looks pretty good. I've also heard of MOgre or something along those lines. I'm terrifically hindered by the lack of experience with setting up a game from scratch and no formal programming education.

Koshling was passively working on a project to setup a new game engine but I'm not sure it's going to be the game I wanted to work on. But if he had the ability to work directly on every layer of the game's architecture, he'd never have had to consider taking this step. My point is that Firaxis needs to realize that there are efforts that WILL eventually pop up to circumvent the brick wall that having no EXE access is keeping us behind.

As far as design and game theory goes, if I can work from a platform of furthering development anywhere near what CivIV provided, I can work on continual innovations for this game to my heart's content... but then if it's going to be of completely original design then we should have every (and I mean EVERY) right to make some money from it when we decide to monetize from the project.

Never being able to have this potential in C2C is a cruelty because I just KNOW that if we ever reach the point where the full vision for the game is reached, it would become one of the most played games in history. So yeah, I see why the .exe is limited and why they'll never release it. But maybe we could write a contract with Firaxis to give a small team of us access for a shared money making opportunity in it. If we create the game Civ IV could have become and really develop out the other branch of the Civilization family tree that they stole from us by moving in the design directions chosen for Civ V+, then it would only be fair that they both Firaxis and the team that develops the mod, both share from any potential profit that might arise, and I don't see why we wouldn't cut in Civ Fanatics as well for the opportunity to have an exception granted to the 'nobody gets to make money here' policy they have as well. Just seems like it would be more beneficial for all to have that eventual end goal potential working from a complete code access platform.

Who to approach and how to beseech them and what lawyers would be needed I have no idea.

 

Actually I was thinking something like releasing the source code under conditions like Call to Power 2.

• just the source code, not commercial libraries
• it's not allowed to distribute anything, which can make the modded game work without a vanilla game installed
• no commercial releases

Getting the exe source, but not the graphics engine means it will fail to compile due to missing headers. We would then have to rewrite each call to a new engine. Precisely which one is an open question at this point.

Sure we could make a brand new game, but leaving the civ4 license behind, we would have to make something, which is clean of civ4 code and that includes the vanilla DLL code, which is the basis for all mods. In other words all mods would have to be rewritten from scratch and then what's the point? Also writing some sort of wrapper for some libraries is way less work than writing a new game engine.

I'm not sure I support the idea of making C2C commercial. I get that you would like it to be on steam or whatever, but it's a legal nightmare. You would need every contributor to sign that they agree (good luck even figuring out who those people are) and you would need some sort of contract for distribution, which is likely much harder to get because their lawyers will need to investigate their original contracts and stuff for their distribution. I suspect this will only make the legal issue so complex that we can fail everything.

 

You're probably right. One cannot help but dream, when they find true opportunities scarce in this world, that the things they've put thousands of hours of effort into that many others have spent thousands of hours enjoying, that there could be SOME means of providing a manner in which one can be a little financially supported by it all. But you make very good points about how much added complexity it presents to reach for that. I believe we'd have to admit that C2C has been built on platform after platform after platform of modding work that preceded us so there could be hundreds of modding contributors that would need to be tracked down. We have a list on our SVN access but that's for just C2C alone. Counting AND, RoM, Rev DCM, Better BtS AI, all the hundreds of individual art contributions and animations and then contributions from other games and even some copywritten material we may not even realize IS copywritten... it would only open the door to a legal nightmare.

If we could obtain some deeper access as you suggest, with the limitations you considered, would we be answering to all the benefits you mentioned in the first portion of your proposal?

 

You can scratch probably. It's not the first time I have dealt with issues like this. If you want to make something for profit, you start with a small group who agrees on it in writing and closed source. I'm not aware of any open source projects, which managed to close the source and sell for profit.

Yes, but I'm quite concerned that it will never happen. If it was to be done, we would need a strong community pressure as well as a team of skilled people to take on the task. Looks like we lack both.

 

The community pressure would have been stronger a little bit ago before many modders just accepted things and moved on to Civ V and VI. I'm sure if they were still paying attention to posts in the general Civ IV forums as much, they would be chiming in. But truly the frustration of the status quo has already made the best and brightest move on to other goals so far as I've seen. This discussion topic has been brought up many many times before in the C2C threads.

I've used modding as a means to gain some coding understanding and I've done pretty well with that I think, but I've seen the added depth professionals bring. And I think it ends up being too demotivating to work on the mods when there are so many ways one can trigger problems in the EXE and have little clue how those problems were triggered. In many ways, it becomes easier to start thinking of starting from scratch. And there's so little, outside of love for the game, to motivate.

The best programmer I saw work on C2C gave up in irritation with the blind spots lost to the EXE when trying to implement Multi-maps.

You said you've worked in the industry and have seen a lot... did you have a hand in the original CivIV project or know anyone that did?

 

Very interesting topic.
Looking forward to see how it all develops...

BTW: Is there the slightest chance of rewrite the EXE at least partially to get som of those benefits mentioned in the opening post?

 

If I had that kind of connections, I would have use those rather than writing on a public forum.

It could be done and we would get all the benefits. The problem is that it would be significantly more time consuming to follow that path. Also I can't say anything about the legal issues from doing so, at least not offhand.

 

As much as I support this idea, I still remember the last time we tried petitioning releaseing Civ3's source code...

 

I would be all for this!
Having the exe open sourced seems very unlikely to me, but this might be achieved with Firaxis...
This alone would already do wonders to the longevity of Civ IV.

 

See if they would sell the required sources

then try and crowd-fund . start a gofundme campaign !

 

why do you think that ? While putting together a deal, never let fear of rejection stop the pitch..

I think we should ask them and let them respond for themselves. The worst thing that could happen is they say NO/

 

Good luck with that.
I doubt that you have ever prospected and initiated a proposal to a large client before. I always start the process with a courtesy call.

 

They are a company and easy to reach by phone, fax, or flight.

now what ? I look at your project as a B2B recycle of an old product.

turn that old clutter into new revenue. They would still retain the ownership

I would like to list information points that support an open source CIV4
and why they should trust you with their legacy.

 

Leoreth. You don't think Firaxis should be contacted. That there is no way to get their support.

I get it. I just don't agree.