C3X: EXE Mod including Bug Fixes, Stack Bombard, and Much More Release 23

[Flintlock]

Another approach might be to add another City Size, not for the game, per se, but "simply" to the City Graphics Display, such that "Size -1" are Culture Group specific Barbarians, with (of course) a necessity, however implemented, that these Size -1 Cities be limited in how much they can grow. As you know, I'm decades removed from coding, but perhaps a Variable of "0", for City Size might be used to implement this.

Interesting idea, but using city size for this wouldn't work since the size is lumped into a category and the categories are fixed to be one of 0, 1, 2, or 3, corresponding to town, city, metropolis, and town with walls. Editing instead the barb player's era index might work, but I think it would be easier to edit the city graphic sprite index directly. The exact formula for that index is:
5 * (EraID + 4 * SizeAndWallsCategory) + CultureGroupID
The index gets passed to a Sprite::draw_on_map method. I could intercept that method call and substitute in a pointer to a different Sprite. Barbarians have a CultureGroupID of -1 so the city drawing code will read at index -1 of the city graphics array for ancient era barb towns without walls. That must be why that angry leader pic appears.

I don't know if this has been discussed previously, but one annoying aspect when playing with my mod, which I had forgotten during the years, is that the obsolete improvements will continue to cost maintenance after their obsolescence... Would it be possible to have a setting where upon an improvement going obsolete, it would also stop costing maintenance?

It would be possible and in fact relatively easy. What's interesting, though, is that it looks like the game was intended to work this way and it only doesn't because of a bug. There is a game function that recomputes building maintenance from scratch and it *does* skip obsolete buildings, effectively making them maintenance-free. However, that function is not normally called. The way it normally works is each city stores a total maintenance cost that gets incremented or decremented whenever a building is added or removed. Maintenance is only recalculated from scratch when a building granting free improvements is gained or lost. Maintenance is not adjusted when techs are unlocked so the cost of obsolete buildings is never removed from the city's total. However, if you gain or lose a wonder that grants free improvements, all those maintenance costs should disappear.

Anyway, it's straightforward to insert some code that updates the maintenance costs for a player's cities when they unlock a tech that potentially obsoletes some of their buildings. I would probably categorize that as a bug fix not a rule change.

Not so long ago, in this branch, one of the users complained about the bug in the game associated with the collapse of the trade network. Ironically, I had an outwardly similar error. The reason why I've a bug with the collapse of the trade network is probably connected with the same damn limit of 512 cities.

I don't really know what to make of this. As far as I know, no C3X features should break when you exceed the city limit, and the only features that even could break the trade network would be the phantom resource bug patch or city-generated resources. It's possible this is an issue with the base no city limit patch, just one that occurs rarely for some reason (?). I've read conflicting reports of how well the NCL patch works and what problems it can cause. I've been thinking of adding NCL to C3X for some time, except if I'm going to do that, I'm going to do a proper job of it, not something that causes random bugs and crashes. Doing a proper job would require quite a bit of effort unfortunately.

 

[Alekseyev_]

Very interesting! Never noticed that the free improvement wonders fix obsolete improvement costs, but that does sound like a fix indeed then. The the recalculation also affect/consider government-requiring improvements' costs?

 

[Kokuryutei]

I don't really know what to make of this. As far as I know, no C3X features should break when you exceed the city limit, and the only features that even could break the trade network would be the phantom resource bug patch or city-generated resources. It's possible this is an issue with the base no city limit patch, just one that occurs rarely for some reason (?). I've read conflicting reports of how well the NCL patch works and what problems it can cause. I've been thinking of adding NCL to C3X for some time, except if I'm going to do that, I'm going to do a proper job of it, not something that causes random bugs and crashes. Doing a proper job would require quite a bit of effort unfortunately.

It is unfortunate that so far it isn't possible to get closer to the reason for the causes of such errors in the game. It looks like some kind of enchanted witch circle. I play for Spain. I make a deal with the Persians: copper in exchange for amber. After that, I founded my 31th city (513th according to the general list). And about 6 turns after that, the Domestic Advisor shouts: "We lost our resources of amber!" After that, Xerxes are angry with me, they say, it was as if on my own initiative I broke the agreement ahead of schedule and deceived a trading partner. And no longer wants to change amber to copper, because requires an additional payment of +15 gold/turn for safety.

Here is a post of Flintlock about the collapsing of the trade network: https://forums.civfanatics.com/thre...rd-and-much-more.666881/page-64#post-16401798

Thank you! Perhaps, during my absence, I could miss something important among the latest updates.

 

[Flintlock]

Very interesting! Never noticed that the free improvement wonders fix obsolete improvement costs, but that does sound like a fix indeed then. The the recalculation also affect/consider government-requiring improvements' costs?

It's hard to notice since the city screen is also bugged here. It will always show obsolete improvements as costing money even after a maintenance recalculation has eliminated that cost. The domestic advisor screen works correctly. The same problem does not apply to buildings tied to different government types since the game recalculates all maintenance costs for a player when they change governments.

 

[Flintlock]

The domestic advisor screen works correctly.

To clarify, I meant the domestic advisor screen reports how much maintenance you're actually paying. It does not see through the obsolete maintenance bug.

I've fixed that bug for R16, and while I was at it I also fixed the maintenance costs on the city screen and an exploit I thought of. The exploit is this: Say you have some obsolete improvements you're paying maintenance costs on. You can get those costs removed by triggering a from-scratch recalculation of maintenance by switching governments, acquiring a wonder that grants free buildings, etc. Then you can sell the obsolete improvements to have their maintenance removed again, so instead of paying the costs you'd be getting credited for them. That free income will last until you trigger another maintenance recalculation.

 

[Alekseyev_]

That's funny! I wonder how many people unknowingly used this exploit when playing mods that offer the environment for it... probably not terribly many, but certainly some.

 

[Lord_Hill]

I see those barb related commits popping up on github @Flintlock It'll be fascinating to see what the game tolerates and what it doesn't. There must have been a reason why the Civ 3 devs didn't want barbarians acting like an extra civilization and went with the pillaging instead.

 

[Civinator]

I see those barb related commits popping up on github @Flintlock It'll be fascinating to see what the game tolerates and what it doesn't. There must have been a reason why the Civ 3 devs didn't want barbarians acting like an extra civilization and went with the pillaging instead.

Yes, this will become fantastic news for all kinds of making maps in Civ 3 (especially with the no raze option in the Flintlock mod). And with the option of enabling production, may be here the Flintlock mod is on the path to form "city states". Nevertheless the option that the Barbarians are in war with all other civs (except Barbarians) in my eyes should stay valid.

 

[timerover51]

Yes, this will become fantastic news for all kinds of making maps in Civ 3 (especially with the no raze option in the Flintlock mod). And with the option of enabling production, may be here the Flintlock mod is on the path to form "city states". Nevertheless the option that the Barbarians are in war with all other civs (except Barbarians) in my eyes should stay valid.

I am not sure that the Barbarians are at war with all other civilizations. There has been too many times when the Barbarians have ignored a Spearman-Settler combination of a computer civilization to attack me, sometimes attacking an army.

 

[Alekseyev_]

That could have been because of the NW/SE axis bug.

They are definitely at war with everyone.

 

[Flintlock]

I see those barb related commits popping up on github @Flintlock It'll be fascinating to see what the game tolerates and what it doesn't. There must have been a reason why the Civ 3 devs didn't want barbarians acting like an extra civilization and went with the pillaging instead.

I've only done limited testing, but so far so good. I modified the game to run the full economy code for the barbs. So not only do they get city production, they should also be able to do research, etc. After clicking through 20 or so turns, it appears to work, at least there weren't any crashes. I let the barbs capture a city and they were able to build warriors and grow pop. However, that's all they did, and actually they whipped away the pop to make more warriors. Settlers and workers are not available to them. The warriors they built ran the braindead barb AI, so all they did was fortify in place until a unit appeared SW or NE of them, which they would all then beeline toward.

And with the option of enabling production, may be here the Flintlock mod is on the path to form "city states". Nevertheless the option that the Barbarians are in war with all other civs (except Barbarians) in my eyes should stay valid.

I'm not saying this would ever happen, but hypothetically if I were ever to try to work around the 32 player limit, I would probably do something like start with barbarian city states and slowly build them up into being real players. Like first, just let the barbs own & use cities. Then, instead of having one barbarian player, divide them up into tribes, then let each tribe have its own culture, then let them sign peace individually with other players, then calculate trade networks for them, then try to give them full diplomacy. Again, I doubt I'll ever get around to that but it's fun to think about.

 

[Civinator]

Like first, just let the barbs own & use cities.

For my purposes in creating a new form of a world map, that in the past was blockaded by the 256 resources limit, this would be a great help. Flintlock, what an amazing progress you are doing here.

Important is, that the Barbarian cities can be placed on the map by the editor.

 

[timerover51]

That could have been because of the NW/SE axis bug.

They are definitely at war with everyone.

The only Barbarians that I have seen that are truly at war with everyone are the Dinosaurs in the Play the World scenario The Test of Time. Those are just placed on the map with no spawning. As for the Barbarians spawned from Barbarian camps, I still have my doubts.

 

[Alekseyev_]

Check this out: https://forums.civfanatics.com/threads/hunting-down-the-barbarian-nw-se-axis-bug.674365/

 

[Flintlock]

Important is, that the Barbarian cities can be placed on the map by the editor.

Can you use Quintillus' editor for this? I consider modifying the official editor to be outside the scope of C3X since there are already third party editors that bypass its limits.

Check this out: https://forums.civfanatics.com/threads/hunting-down-the-barbarian-nw-se-axis-bug.674365/

If you guys keep distracting me like this, R16 will never be done. But now feels like a good time so I had another look at this bug, I'd looked into it before but only very briefly. As it turns out, it's due to a small, sloppy error in the original code, and is easy to fix. Here's the relevant code, straight out of the decompiler:

// The main barb unit AI method
void __thiscall Unit::ai_move_barbarian(Unit *this)
{
      // A bunch of barb logic omitted, including checking its immediate surroundings for targets

      iStack28 = 1;
      do {
        neighbor_index_to_diff(iStack28,&iStack24,&iStack20);
        uVar5 = Map::wrap_horiz(&bic_data.Map,(this->Body).X + iStack24);
        uStack16 = Map::wrap_vert(&bic_data.Map,(this->Body).Y + iStack24);
        uVar9 = uStack16 & 0xffff;
        uVar7 = uVar5 & 0xffff;
        if (((int)uVar7 < bic_data.Map.Width) && ((int)uVar9 < bic_data.Map.Height)) {
          uVar5 = uVar5 >> 1 & 0x7fff;
          pTVar3 = Map::get_tile(&bic_data.Map,(bic_data.Map.Width >> 1) * uStack16 + uVar5 & 0xffff);
          /*
          Check for targets on tile pTVar3
          */
        }
        iStack28 = iStack28 + 1;
      } while (iStack28 < 0xa9);

      // More barb logic
}

A "neighbor index" is an index specifying a neighboring tile. An index of 1 specifies the NE tile, 2 the east tile, 3 the SE tile, etc., all around the first ring and then a similar pattern around the second ring, then the third, and so forth. neighbor_index_to_diff converts those indices into (dx, dy) tile coordinate distances. For example, an index of 2 (east) would be converted to (+2, 0) and an index of 5 (SW) would become (-1, +1). In the code above, dx and dy are stored in iStack24 and iStack20. The loop variable, iStack28, goes over neighbor indices. For each neighbor index, it converts that into coord distances then tries to get the tile at those distances from the unit's position, and finally checks that tile for targets. It tries to do this for the 168 (=0xA8) tiles surrounding the barb unit. Can you see the error? It's a little bit subtle, but a simple mistake probably due to someone copy & pasting code and forgetting to modify it.

The error is that it's adding the dx distance to both the unit's x & y coordinates, instead of adding dx to x and dy to y. In that call to Map::wrap_vert, it should be adding iStack20, not iStack24. So, in that thread, @BlueSox14 was on the right track when he noticed that the game's buggy logic only checks tiles whose x and y distances are the same from the starting unit. By the way, the reason the bugged logic searches outward by 12 tiles instead of 6 is that each step east or west changes the x coordinate by +/- 2. Six steps east corresponds to a coord diff of (+12, 0), and the buggy logic instead checks (+12, +12).


Lastly, a peek behind the curtain: The way I tracked down this bug was to set up a scenario with a barb unit and a worker of mine a few tiles SE. I then ran the game in a debugger, loaded up that scenario, and set a breakpoint on the game's pathfinder method. I ended the turn and waited for the breakpoint to trigger on the barb unit pathing to attack my worker. That lead me to the loop above (the call to the pathfinder is part of "checking for targets on pTVar3") which I inspected to figure out why it wouldn't work for targets off the NW-SE diagonal.

Here is the actual mod code that fixes the bug (this goes in injected_code.c):

void __cdecl
patch_neighbor_index_to_diff_for_barb_ai (int neighbor_index, int * out_x, int * out_y)
{
    neighbor_index_to_diff (neighbor_index, out_x, out_y);
    is->barb_diag_patch_dy_fix = *out_y - *out_x;
}

int __fastcall
patch_Map_wrap_vert_for_barb_ai (Map * this, int edx, int y)
{
    return Map_wrap_vert (this, __, is->current_config.patch_barbarian_diagonal_bug ? (y + is->barb_diag_patch_dy_fix) : y);
}

This relies on these definitions & patcher instructions (this goes in civ_prog_objects.csv, but note neighbor_index_to_diff is already there):

define,     0x5E6E50,  0x5F66A0,  0x5E6D80,  "neighbor_index_to_diff",              "void (__cdecl *) (int neighbor_index, int * out_x, int * out_y)"
define,     0x426C40,  0x428380,  0x426CC0,  "Map_wrap_vert",                       "int (__fastcall *) (Map * this, int edx, int y)"
repl call,  0x44FEC3,  0x451F64,  0x44FF43,  "neighbor_index_to_diff_for_barb_ai",  ""
repl call,  0x44FEF0,  0x451F91,  0x44FF70,  "Map_wrap_vert_for_barb_ai",           ""

(I hope those addresses are right for the PGGames.de and Steam EXEs. I haven't tested them.)
barb_diag_patch_dy_fix is a member of the injected_state struct defined in C3X.h. The way the fix works is simple. It intercepts the call to neighbor_index_to_diff shown in the buggy code, runs the function, and stores the quantity dy - dx based on what it computes. Then it also intercepts the call to Map::wrap_vert, where the game wraps the y coordinate of the tile it's about to check, and adds dy - dx to the y value, to subtract off dx and add in dy instead. The game will then go on to look up the actual tile at the neighbor index that it intends instead of one along the NW-SE diagonal.

So much for the barbarian diagonal bug. I hope you guys found this little writeup interesting. I often feel tempted to do things like this but don't because of the time required, and because I'm not sure anyone really cares.

 

[Lord_Hill]

You've fixed one of the oldest and most annoying bugs in C3C, that's amazing! I can only speak as a non dev, but a modding enthusiast, I enjoy the writeups. I like to try to understand how it works under the hood.

 

[tjs282]

So we can now have Barbs again acting as they did in the final 1.29f vanilla patch?! Out-standing!

Does that also mean that the NoAIPatrol=0 .ini-file line (which makes the Barb-units move around a bit more -- but also the AI-Civ units) will no longer be needed as a 'fix' for the Barbs' hunting-behaviour?

 

[Civinator]

Can you use Quintillus' editor for this? I consider modifying the official editor to be outside the scope of C3X since there are already third party editors that bypass its limits.

Unfortunately both the Firaxis editors and the Quintillus editor are graying out the ownership of cities for Barbarians.

Flintlock, to give the Barbarians full mobility again, is also great news!

 

[Flintlock]

Does that also mean that the NoAIPatrol=0 .ini-file line (which makes the Barb-units move around a bit more -- but also the AI-Civ units) will no longer be needed as a 'fix' for the Barbs' hunting-behaviour?

With the bug fixed, barbs can target any unit within 6 tiles of their location. Here's what that looks like:

However if there are no units within that area, they will fortify in place forever AFAIK. It might still be good to have them wander around randomly so they're less predictable but it is less important with the bug fix.

Unfortunately both the Firaxis editors and the Quintillus editor are graying out the ownership of cities for Barbarians.

That seems like a missed opportunity. I expect it would be easy to allow for barb cities in scenarios given how the scenario data is laid out. The only catch should be the need to give them proper city graphics since the "none" culture group doesn't have any. Given the choice between modifying the Quintillus or Firaxis editors, I'd rather do the first one since its source code is available. We could even ask Quintillus for help.

 

[Lord_Hill]

I agree that @Quintillus editor would be the place for such a feature, it's already used by modders to get around some of the internal Firaxis limits in the official editor, not to mention that an existing C3X feature (free improvements from small wonders) already requires this editor to use. For such an exotic feature, I don't think it's unreasonable to use a third party editor, plenty of other games can only be changed that way. In that thread I linked which started all this barbarian goodness, Quintillus attempted to extract Barbarian cities from a save file into a biq, but the game crashed when you tried to start the scenario. Maybe the 'none' culture group issue is the reason why.

It's all very exciting, I was only thinking about messing around with playing as the barbarians when I suggested that feature, but making it possible to place barbarian cities in scenarios would be a terrific addition. Good luck!