All new promotions added to every unit?

[Thalassicus]

I encountered a strange problem when I copied over an XML part of a mod from vanilla to the G&K: every unit starts with all the promotions created in the mod. It worked in the vanilla version, but not the G&K version, even though it's the same xml. I've carefully looked through places where I add to the Unit_FreePromotions table without spotting any mistakes. Does anyone know what might cause such bizzare behavior?

 

[Putmalk]

Doesn't this happen after the ID of the promotion goes over 200 or something?

 

[Spatzimaus]

Doesn't this happen after the ID of the promotion goes over 200 or something?

Correct. Specifically, the Lua function Units:IsHasPromotion() returns True if the ID is 200 or higher, regardless of whether the unit ACTUALLY has the promotion. So in theory this is just a UI bug, except that any mod that uses Lua-based custom promotions (like mine) will obviously break completely.

The problem is that the core game has ~160 promotions, plus another 12 for G&K, plus 5 for the DLCs... as you can see, you'll have very little room for custom promotions, especially with G&K. Just one of MANY things Firaxis screwed up with that expansion...

 

[Putmalk]

Well someone tell me WHY the function returns true after 200? What's the reason for it?

 

[Thalassicus]

I suspect they have an array with a fixed size of 200, so going over that gives an out of bounds error and breaks the code. It also runs into the problem of how they did promotion prerequisites. Instead of using a table of variable size, they set a fixed limit of 6 prerequisites, forcing them to split up the extra-attack effect into multiple identical promotions, which brings us closer to the limit.

 

[Spatzimaus]

I suspect they have an array with a fixed size of 200, so going over that gives an out of bounds error and breaks the code.

Short version: "Peoples is stoopid." But yeah, you're almost definitely right.

It's just a lot easier, in some languages, to declare an array once with a fixed size (hopefully way bigger than you think you'll need) and then partially fill it and keep one variable to track how many array elements to use, instead of dynamically adjusting the size of the whole array as you go and checking constantly to see if you've overflowed it. Granted, they're SUPPOSED to be using something like C++, where this shouldn't be hard to get around, but all it takes is one sloppy programmer to screw everything up. If the limit was because of some memory issue, it wouldn't be a nice round number like 200, so it's pretty clear this was set by a human being.

The stupid part is, if 200 is just an arbitrary value that some programmer put in way back when, it should have been TRIVIAL to increase it to, say, 400 or 500 in some patch; it's not like the promotions are some pervasive element that'd cause the size of EVERY array to increase by a corresponding amount (like the max number of players would); this only affects a tiny part of the game. At 400, we'd never even have known the limit existed, because even the most content-heavy mods only add a few dozen promotions and other incompatibilities would crop up long before you combined enough mods to come close to the limit. But no, they set it BARELY ABOVE the core game amount, in a game that was supposed to be really mod-friendly. (No, I'm not still bitter about what they did to my mod, why do you ask?)

 

[Thalassicus]

Does anyone happen to have a list of promotions that can be merged or deleted to free up IDs?

And yeah, I hear you. I wish I'd gotten involved in modding Civ 4 back in the day. It seems like it was more mod friendly than Civ 5. It was especially frustrating when I gave someone code that fixed a major problem, they say it's great, then silence falls and nothing gets done. Why?! The work was already complete.

 

[Spatzimaus]

Does anyone happen to have a list of promotions that can be merged or deleted to free up IDs?

Well, in my mod I'd merged the Drill and Shock lines with Accuracy and Barrage. I deliberately mixed them (that is, combined the open melee with the rough ranged, and vice versa) to keep them equally valuable, since I'd nearly always pick the "rough" option when I played as it just wasn't hard to kill any unit caught in the open. It worked very well, especially since there was no longer an issue for things like the Longbow->Rifle upgrade, or combat classes possessing both ranged attacks and melee ones (my Titans and Myth units). I'd also merged Logistics with Blitz, since Blitz was just way too strong once you had really high-movement units (like I did in my future eras). And, if I'd bought G&K, I'm sure I'd have merged the melee naval lines with the ranged ones. The 200 limit was a real concern of mine, given how many promotions I'd added, and I'd been in the process of trying to pare down a few others when the 674 patch hit.

The problem with deleting promotions is that you just don't know all of the places they're used. What do you do if you delete a certain promotion (say, Accuracy I), and then someone adds a unit that starts with that promotion? Sure, you can add Update lines to the appropriate tables to try and fix that, but if you can't control what order the mods load in, there's still plenty of ways it can fail. For instance, consider Sentry (+1 Visibility). There's already a +1 visibility promotion (EXTRA_SIGHT_I), so it'd be easy to merge the two, right? Well, Sentry is given by traits, policies, some units start with it, and so on. So that's a lot of things to patch, and what happens when someone has Lua code that awards that promotion (which I do)? There's no XML Update command that can possibly intercept a GameInfoTypes.PROMOTION_SENTRY call and turn it into the value for the other promotion, so things'll still break.

Basically, this game's structure is NOT conducive to deleting "primary" elements (anything with an ID). Too many things will break if you delete something that other people are depending on.

 

[Thalassicus]

Yeah, I just disabled promotions in the past to try and maximize compatibility. It looks like we don't have that option anymore... we can basically have 1 mod that adds promotions, and it has to delete some to free up space. I don't see any other choice.

Civ 2 was the first game I ever modded, and I finally start modding the series... right when it starts losing mod support. I've been trying to stay positive over the past two years, but things like the promotion limit, or lack of reliable combat detection, are very discouraging. It probably would take just a few hours to add some of these capabilities.

 

[Spatzimaus]

and it has to delete some to free up space.

And don't forget: the game will crash if there are any non-sequential IDs in the Promotions table, so you can't just do a Delete. You either have to delete it and then explicitly set the ID of your new promotion to fill the hole left by the old one, or use a separate mod that re-indexes everything.

 

[Thalassicus]

Thanks for warning me!

Thinking about how much of the game has poorly-written code, I suspect they also made the mistake of hardcoding a promotion ID somewhere in the files. This would mess up any attempt to re-index the whole table... so I'll try just explicitly setting IDs. Thankfully there appears to be several dozen unused or duplicate promotions, so at least it's possible to replace some, if difficult.

Imagine a world where one of us actually got hired to improve the core game for mod support... heck I'd do it without pay, I'm already working for free!

Does anything bad happen if we change the "type" field of a promotion?

 

[Thalassicus]

(double post)

 

[Putmalk]

What boggles my mind is that I thought IDs were supposed to be incremental. They should just update if a user mod changes the original file (alternatively, allow a "reindex" command to reorder the IDs). Why do they make us jump through hoops and bounds to do something relatively simple?

To that effect - isn't there an SQL command that easily reorders IDs for you?

 

[Thalassicus]

DELETE FROM UnitPromotions
WHERE Type = 'PROMOTION_CARGO_IV';

REINDEX UnitPromotions;

I tested deleting the unused promotion above.

• The game appeared to work without crashing.
• Sqlitespy shows the table is missing row 125. It deleted the promotion okay, but the reindex command did not appear to do anything.
• A new promotion I inserted got ID 177, instead of the empty spot at 125, since the autoincriment still pointed to 177.

What's an alternate way we could reindex the table?

 

[trystero49]

Why don't you try manually updating the ID number of the promotion immediately after the promotion you delete? Wouldn't the auto increment counter start from that number?

 

[Horem]

Can you not just drop the whole table and restructure it to your prefrence? I know its a bit of a pain to redo all the ID's, but it works, atleast for other Tables I have done this way(Gamespeed_Turns, Technologies - which I restructured to list by era, for example).

 

[ww2commander]

At the top of your promotions SQL file put the following:

(EDIT: This line should be placed in the very first SQL file that tries to delete table entries. In my case it happens to be in my buildings SQL. The subsequent table drop entry makes sure any previously used entries are deleted and the table is recreated again. Without the first CREATE line in the first code window the DROP table entry in the second code window would cause an SQL error. Hope I make sense!)

CREATE TABLE IDRemapper ( id INTEGER PRIMARY KEY AUTOINCREMENT, Type TEXT );

At the very end of your promotions SQL file add the following entry:

DROP TABLE IDRemapper;
CREATE TABLE IDRemapper ( id INTEGER PRIMARY KEY AUTOINCREMENT, Type TEXT );
INSERT INTO IDRemapper (Type) SELECT Type FROM UnitPromotions;
UPDATE UnitPromotions SET ID =	( SELECT IDRemapper.id-1 FROM IDRemapper WHERE UnitPromotions.Type = IDRemapper.Type);

This should reindex table IDs. This code can be applied to other tables as well by simple edit of relevant table names. Works perfectly for me and I have deleted almost all vanilla promotions from my scenario.

I got the code from somewhere on this forum but can't remember who it was to credit them

 

[Barathor]

A very interesting and useful thread here. I've never really paid much attention to ID numbers; I see now that they're something to consider. Especially, since an upcoming mod of mine will focus on the game's promotions. I would've been going crazy if I happened to go over the 200 limit!

 

[Putmalk]

I think the credit is lemmy for that one (the guy who was doing the 3D leaderhead art) but he's long gone so it's not an issue now.


And ID numbers are vvvvvveeeeeeeerrrrrryyyyyyyyy important, especially when completely editing the social policies! If you're off by one ID number, the whole thing doesn't work!

 

[Thalassicus]

Thank you ww2commander!