Civ IV source code?

[Fish Man]

I'm interested in investigating further the mechanics and properties of RNG in this game, as well as just how the seed affects combat results, mining resource pops, forest growth, and the like. I've done some digging in the early days and realized that people such as Manco Capac and DanF5771 (if they're still around anymore) have done some code-diving into the mechanics of the game. It seems that the .cpp files shipped with the CD version of BTS, which unfortunately is no longer for sale, given how long it's been. Does anyone still have access to the code that the original code-divers used for 3.19 final patch? Specifically, functions like getSorenRand(), which are still a mystery to me.

 

[Anysense]

In GOG version there is this folder: Civilization IV Complete\Civ4\Beyond the Sword\CvGameCoreDLL full of C++ of source files. And there is also CvGameCoreDLL\Boost-1.32.0\include\boost\random folder which apparently contains what you are looking for.

 

[Fish Man]

In GOG version there is this folder: Civilization IV Complete\Civ4\Beyond the Sword\CvGameCoreDLL full of C++ of source files. And there is also CvGameCoreDLL\Boost-1.32.0\include\boost\random folder which apparently contains what you are looking for.

Is the "CvGameCoreDLL\Boost-1.32.0\include\boost\random folder" only in the GOG version? I have Steam.

 

[Anysense]

I don't know about steam, haven't used it for ages.

 

[Fish Man]

I don't know about steam, haven't used it for ages.

Ah, OK. I suppose the GOG version is still dirt-cheap at this point, so I'll look into it.

 

[Gumbolt]

I am still using the CD version. Still have all the CD's not sure what this might provide you.Never got a replacement code from Firaxis. I could of offered an amazing review for Civ 4.

 

[coanda]

Try here?

https://forums.civfanatics.com/resources/cvgamecodedll-c-source-code-files-bts-3-19.16274/

 

[f1rpo]

The DLL source (including the Boost and Python24 libraries) can also be browsed here on GitHub – and can be downloaded through "Code" -> "Download Zip". Here's another GitHub repository (created by me), which tries to reconstruct the game's patch history (but doesn't include the external libraries). Various mods also make their DLL code and change history publically available, in particular BULL.

getSorenRand is declared in CvGame.h. The RNG class is CvRandom (CvRandom.h, CvRandom.cpp). Boost has no part in that; the game uses Boost (almost?) exclusively for the DLL-Python interface. So I don't think Boost is of any interest for code divers.

 

[jmas]

> mining resource pops

This might be the best place to mention what happened in my most recent game. I already had one gold, but another one popped for me (in the same city BFC) and just needed roaded. Later in the game I popped Silver in another city (on a plains hill, no less) and Gems in a third city.

As I recall all these pops occurred shortly after the mines were built, though that might not be much more unlikely than getting all 3 of those to pop in the first place.

When my empire took over cities, people didn’t seem to think it was so bad.

 

[Pangaea]

For people familiar with the source code, would it be possible to fix some of the memory leaks and whatever else causes lag with the UI and such in longer sessions? Seem to recall a modder fixing some of that, but it was with that gameplay mod that changed a thousands things, so it wasn't up my alley.

 

[f1rpo]

@Pangaea: You probably mean K-Mod. I don't think K-Mod fixes those issues; I assume they're caused by the EXE. I doubt that there is any major memory leak in the BtS DLL. (But, then, I don't recall personally encountering degraded performance in long sessions, so I'm not the best person to comment on that. Maybe I just don't play large enough maps for a long enough time.) K-Mod does speed up AI turns substantially, but those tweaks wouldn't necessarily be easy to port to BULL. If a new version of BULL (or BAT) is ever released, one very simple performance tweak would be to enable whole-program optimization when compiling the DLL. Should reduce AI turn times by at least 10%. Well, not a huge difference, but that would be just a one-line change in the makefile.

 

[Pangaea]

That was the name, K-Mod. It's so many years ago now, but I seem to recall the author saying some of the memory leaks and UI misbehaviour had been fixed in his mod, but maybe that wasn't correct after all. In any case, I'm very surprised you haven't experienced these memory leaks (I assume that is what causes it), lags, and UI lag or jerkiness. You need to play longer sessions! It's possible it's more likely to crop up when one checks cities a lot, like going through each one every turn like I do, as there is surely much more data read. When I play in longer sessions, or with big empires, it's essential to restart the game from time to time to remove the lag. Otherwise it will take 1-2 seconds to go from one city to the next (with right arrow), and there are increasing lags when going from one unit to the next (to issue commands).

Excellent that people have posted the entire source code of the game btw. That way we don't have to uncook the game or whatever is required to get at those files. Some is available from the install directory, but not .cpp files and such from what I recall.

Is the .exe available too for those who want to look at it, or is that a black box?

 

[Anysense]

It's possible it's more likely to crop up when one checks cities a lot

Yes, after every ~50 visits to a city the game slows down notably. Sometimes I had to go to task manager 2 or 3 times a turn in my BC Space game. The lag is most annoying when you manage specialists/tiles; every change takes about a second, and it is getting slower after every city. Imagine Slavery/Caste switch with 185 cities. Curiously, hardware has absolutely no effect on these lags. Apparenly the whole thing is botched in a way that does not allow the game to use computer resources.

May be, running the game from RAM can help.

 

[Qactus]

...
May be, running the game from RAM can help.

That should definitely help a lot. I remember Seraiel recommending Radeon RAMDisk which has a free version up to 4GB, so enough for running civ4. He used it for his HoF Huge Marathon Corporation games which otherwise would've been impossible to play out.

 

[aieeegrunt]

Yes, after every ~50 visits to a city the game slows down notably. Sometimes I had to go to task manager 2 or 3 times a turn in my BC Space game. The lag is most annoying when you manage specialists/tiles; every change takes about a second, and it is getting slower after every city. Imagine Slavery/Caste switch with 185 cities. Curiously, hardware has absolutely no effect on these lags. Apparenly the whole thing is botched in a way that does not allow the game to use computer resources.

May be, running the game from RAM can help.

What does that mean?

 

[Anysense]

What does that mean?

Third part is about using RAM disk
http://forums.civfanatics.com/showthread.php?t=560653&highlight=ramdisk

 

[aieeegrunt]

Third part is about using RAM disk
http://forums.civfanatics.com/showthread.php?t=560653&highlight=ramdisk

ok thank you

 

[f1rpo]

Yes, after every ~50 visits to a city the game slows down notably. Sometimes I had to go to task manager 2 or 3 times a turn in my BC Space game.

So I gather it should be possible for me or anyone to reproduce the issue by loading one of your savegames and cycling through all cities once (and maybe ending the turn and then doing it again). If I don't experience lag, it would have to come down to OS, hardware or INI settings and player options somehow. I get no lag by just running AI Auto Play (AIAutoPlay mod or Python console in the original game – same result for me either way) for 250 turns on a large map and then taking control of the biggest civ and cycling through its cities.
Most YouTubers don't seem open the city screens much. 5 hours into this video (4:55:40 to be exact)
www.youtube.com/watch?v=_B9Gv20h8uk&t=17740s
on what looks like a Huge map, the player cycles through his cities. Seems a little bit slow, but maybe not quite the 1-2 seconds that Pangaea mentioned. Well, it seems that he had barely ever opened a city screen before in that game.

That was the name, K-Mod. It's so many years ago now, but I seem to recall the author saying some of the memory leaks and UI misbehaviour had been fixed in his mod, but maybe that wasn't correct after all.

It does fix a few memory leaks in the DLL, but I don't think those were very consequential; a few MB perhaps. The changelog is extensive, but a search for "memory", "leak" or even just "performance" doesn't turn up much that could be relevant here. The download page (correctly) states that the mod features "many technical improvements, including shorter waiting-time for AI turns, fewer OOS errors in multiplayer games, and many user-interface improvements and bug-fixes."

For memory issues with the Realism Invictus mod (which is based on K-Mod fwiw), I've seen an external tool named "Process Lasso" suggested a while ago. Whatever that tool does, a mod could presumably also do. But it might be a different issue than what you two are describing.

Is the .exe available too for those who want to look at it, or is that a black box?

A black box pretty much. One can (somewhat) track the interactions with the gamecore DLL and with some third-party DLLs (e.g. DirectX I guess). Debugging the disassembly of the EXE itself (as described in this guide for the Caveman2Cosmos mod) is difficult and tedious. For example, I've been trying to figure out where the EXE sets the size of the resource bubbles, but this seems like a pretty hopeless quest. Caveman2Cosmos is also where I'd start if I wanted to get the EXE to allocate less or free more memory; this thread in particular comes to mind.