Quick Modding Questions Thread

[hankinsohl]

Quick question to the modders...

Would a program to remove Lock Modified Assets (LMA) from a savegame be useful, transforming the savefile to a normal save without LMA? Or would such a program be subject to abuse and thus undesirable?

 

[<Nexus>]

Quick question to the modders...

Would a program to remove Lock Modified Assets (LMA) from a savegame be useful, transforming the savefile to a normal save without LMA? Or would such a program be subject to abuse and undesirable?

Yes, if you have have unintentionally started the game with LMA but need to edit it it later.
Yes, if you have started a multiplayer game with LMA and the other players discontinued playing but you want to finish the game anyway, maybe with some editing (give a little cheat to the former human players).
Two scenarios I can think of. But it's also a matter ot how difficult it is to accomplish.

 

[f1rpo]

The Hall of Fame has an extra layer of security through encryption:

With the release of the SDK virtually the whole of the file format is now exposed to the public, which would make it pretty easy to write a tool to edit files. The HOF mod encrypts the save file, so making it harder for this to happen. Also some extra CRCs are stored for things we think were missed.

So this is hopefully not an issue. I'm not sure what else LMA is used for – multiplayer, I guess, now that I read Nexus's post. Or I suppose for any minor Game-of-the-Month-type competition not using the HoF mod – all participants starting from the same save with locket assets. I don't think opening the save multiple times to scout out the map can be prevented that way (I guess GotM relies on honesty here?). So, again, it shouldn't make a difference if a tool can strip away the LMA option.

My mod "Taurus" can already do this: Import a BtS, Taurus or BUG/BULL/BAT savegame despite Locked Assets (this is accomplished by altering the EXE at runtime) and save it with an arbitrary mod name (or none) and -necessarily- without Locked Assets. I figured, so long as I don't allow a savegame to be modified while keeping the LMA option in place, it would be fine. If serialized data can be changed while keeping the LMA option and its checksums in place, I suppose this would be exploitable in the non-HOF GotM scenario (but is LMA really used this way?). Not sure about multiplayer. I think a password can be required from each player, and that should also stop any save editor. And, even without such passwords, I guess only the host needs to be trusted (only the host's savegame gets loaded afaik). Maybe a bigger issue for play-by-email where all players save and load all the time.

Unforunately, I really have no experience with competitive/ nontrusted play. So I also wonder whether there's a problem. (Taurus is only 2 years old and hasn't garnered much attention; so no one having complained doesn't prove much.)

On a very few occasions, players of my mods have reported bugs and weren't able to provide a usable savegame because of LMA. So that's another way how removing LMA could be useful. Although this will require the modder first of all to make the save editor compatible with any changes that their mod has made to the save format. For large mods, this is going to be a major undertaking.

 

[hankinsohl]

Thanks for the comments so far.

If I were to write the utility to remove LMA, I'd probably include the ability to remove password protection as well (useful if someone forgot their password; exploitable if, for some reason, password protection is intended to keep others from the savegame).

And even if I didn't include the abiliity to remove passwords... we'll the algorithm for the savegame checksum is now known, so basically anyone could write such a program.

@<Nexus>
"But it's also a matter ot how difficult it is to accomplish."
I'm pretty sure that this would be easy to accomplish. I've already written an application that can generate checksums, so it should be easy to modify that codebase to remove LMA/passwords.

@f1rpo
"Although this will require the modder first of all to make the save editor compatible with any changes that their mod has made to the save format. For large mods, this is going to be a major undertaking"
Very true. I don't plan to support savegames for mods that have altered the savegame layout. The savegame editor project I'm working on faces the same issue - that is, no support for savegames that are non-standard in layout. I've been thinking about making the savegame editor support layout alterations through the use of a "schema" file that it would use. The schema would describe the layout of the savegame. If I end up doing this, then non-standard layouts could be supported by altering the schema file.

RE: Hall of Fame Encryption - interesting. Good to know that a tool to edit savegames was anticipated and protected against.

 

[f1rpo]

Oh, right, the passwords don't encrypt anything, so if they're stripped from a save their protection is just gone. I see. I guess the intention of password-protecting multiplayer saves is that one player alone can't open them by connecting multiple instances of BtS to localhost und look at the opponents' game state. But all that was really necessary to bypass such protection – or any game state secrecy – was uncompressing the serialized game data, basically what that earlier project (savegame parser) you had linked to already did. Although they hadn't figured out how to uncompress larger saves ...

Well, not for me to say whether an option to remove passwords and LMA would create any real problem by making it very easy to cheat. I've looked around a bit on Realms Beyond and didn't get the impression that they rely on passwords. And LMA is actually singleplayer-only, as I realized today. It says so in the tooltip on the Custom Game screen. However, this post claims that PBEM always causes assets to be locked, but this one seems to say that this only applies when an Admin Password has been set. Although some of this info might also be specific to PitBoss being involved. If nothing else, this reinforces that I don't really know what I'm talking about.

I don't plan to support savegames for mods that have altered the savegame layout.

All the widely used mods that I can think of have either not changed the savegame layout at all (BUG. BULL) or have changed it so much that supporting them would be an unreasonable effort. Well, BAT is the one counterexample I can think of with only a handful of alterations. The schema idea sounds nice, but I wonder who would ultimately go through the trouble – even with such an aid. For my part, I might still want to adopt the more structured savegame format of the We the People mod for Civ 4 Colonization (description on their GitHub page), and then efforts I spent on the BtS binary format would be in vain.

 

[kristopherb]

In a rough proof of concept I want to add city states and use the current leaderheads, traits behaviours . So that there is two distinct england-american-zulu-whatever CIVs. e.g. Zulu (normal playable) and Zulu-City-State (AI only). In the Civinfos.xml ive copied and pasted the list of civilizations and added CITY_STATE_XYZ in the type line. I had thought this would work but alas it hasn't, what have i missed?.

The intent with these new city state civs make them unplayable and restrict them to 1 city by making the settlers a UU, and power up the UBs , free resources ect, ect.


Second dumb question, cities add X food, Y production and Z commerce, where do I modify this?

 

[<Nexus>]

I have three bugs that I seek help with:

1) Enslavement

With CIVIC_SLAVERY I have

<iEnslavementChance>25</iEnslavementChance>

which means 25% chance to create a Slave (worker unit) after a successful combat WHILE running Slavery.

However, after some time the chance becomes 100% and won't turn off even when switching out of Slavery.

I guess that iEnslavementChance value gets added under some circumstance, maybe recalc or when ANY civ adopts the civic instead of just the player.

This bug is a little hard to reproduce in a test game but it always shows up in a longer game.

2) City vicinity boni

In CIV4BuildingInfos.xml there is VicinityBonusYieldChanges that gives yield bonus with resources in city vicinity.

The bug happens either on the next turn after building is constructed or when you recalc and it doesn't come back on reload.

3) Agrarian trait

There's also food from Agrarian trait that disappears.

Source code:

https://github.com/Sogroon/CvGameCoreDLL-1.3/archive/refs/heads/main.zip

@keldath already tried to help me:

Spoiler :

hey,


so i studied the Enslavement.


the code seems good. the update of the chance is done in one place, where the civics gets updated so thats good.


i narrow your issue to 3 sources:

1. the civics xml has entries that you might have left for other civics? (low chance)

2. the code in cvPlayer -?

line 32724 this:



could be that you can remove the if (iChange != 0)

and do this :

if (iChange == 0)

m_iEnslavementChance == iChange;


will probably assure that if it gets a 0 it will allow a full reset.


3. in cvInfos.cpp

i suggest add the value of 0 in the end. it sets as a default value. i have done the same in my code for several.

pXML->GetChildXmlValByName(&m_iEnslavementChance, "iEnslavementChance", 0);


--

now i dont know the afforess code there . there are lots of code concept changes but its the same around the edges.


compile with my suggested changes and report back.

--

ill get to the other two.

And

ok vicinity,

this is somewhat complicated and will take time to learn.


there could be some issue with the order of executions. im not certain.

do you know who or when the code came from? or its straight from the c2c and such?


i would try moving this:

//Does vicinity bonus checks

doVicinityBonus();


after the calc of this:

/************************************************************************************************/

/* Afforess Start 04/30/10 */

/* */

/* */

/************************************************************************************************/

for (int iI = 0; iI < GC.getNumBonusInfos(); iI++)

{

m_pabHadVicinityBonus[iI] = hasVicinityBonus((BonusTypes)iI);

}

/************************************************************************************************/

/* Afforess END */

/************************************************************************************************/

in cvCity.


maybe theres a reset there that takes place.


try it.


edit:


weird code:


if (kBuilding.getFreeBonus() != NO_BONUS)

{

changeFreeBonus(((BonusTypes)(kBuilding.getFreeBonus())), (GC.getGameINLINE().getNumFreeBonuses(eBuilding) * iChange));

clearVicinityBonusCache((BonusTypes)(kBuilding.getFreeBonus()));

}


for (int iI = kBuilding.getNumExtraFreeBonuses() - 1; iI > -1; iI--)

{

changeFreeBonus(kBuilding.getExtraFreeBonus(iI), kBuilding.getExtraFreeBonusNum(iI) * iChange);

clearVicinityBonusCache(kBuilding.getExtraFreeBonus(iI));

}


tells that if a building provides the bonus - it will cancel out the vicinity.

i would use this logic. cause having a bonus from a building, is by definition, in the vicinity,...

I tried his suggestions but they didn't work.

After doing some tests the problem seems to be the recalculation function (Ctr-Shift-T).
Recalc piles up enslavement chance, and it also erases the bonus from city vicinity resources.

 

[f1rpo]

For a start, without really understanding how recalculation is supposed to work, you could try adding m_iEnslavementChance=0 to CvPlayer::clearModifierTotals (maybe simply at the start of the function, seeing that m_iEnslavementChance is also the first int member declared in CvPlayer.h). Just from looking at similar variables, this one seems to be missing from that function. I feel that this could explain an increase of the chance at every recalc. Similarly, the contents of m_pabHadVicinityBonus aren't being set to false in CvCity::clearModifierTotals, whereas e.g. m_paiBonusDefenseChanges does get zero'd by that function. I guess this latter one will be easier to test, but I'm less hopeful that the lack of clearing is indeed the problem in that case. As keldath wrote, it's "somewhat complicated"

Spoiler :

Humorous understatement? What a nightmare ...

... and things being updated in the wrong order sounds like a good intuition. Ah, but you write that Ctr-Shift-T does allow the enslave issue to be reproduced quickly? Then that could be worth a try.

 

[f1rpo]

cities add X food, Y production and Z commerce, where do I modify this?

Cities raise the yield rates to at least 2 food, 1 production, 1 commerce. That's iMinCity in Civ4YieldInfos.xml.

In the Civinfos.xml ive copied and pasted the list of civilizations and added CITY_STATE_XYZ in the type line. I had thought this would work but alas it hasn't, what have i missed?.

Not working how? Reminds me of this recent thread – where I've assumed, like you, that copying and changing the type line should suffice or at least should have some effect. I didn't put it to a test.

 

[hankinsohl]

Just a heads up for anyone interested in the savegame editor I've been working on...

I've created a thread in the Civ4 - Project & Mod Development for discussion of the editor.

I've decided to create a static library to enable programmers to use the editor API for other projects. A Boost property tree-style interface will be provided to facilitate interaction with the API.

The library implementation will be parser-based; that is, a parser will be written which will parse a savegame file and generate a corresponding property tree. Parsing will be performed based on schemas which describe the layout of a savegame. Use of schemas will allow the editor to support modded savegames provided that a schema is written to describe the modded layout. Hopefully this will be fairly easy to accomplish for most mods since all that need be done is to edit the schema for unmodded savegames to reflect the layout changes made in the mod. The parser will use the Civ4 XML files to define enumerations and certain constants in a manner consistent with that used by the Civilization 4 exe.

I've written a draft schema for unmodded Beyond the Sword savegames and have attached it to a post in the thread.

Feedback, comments, questions, etc. are welcome.

 

[<Nexus>]

For a start, without really understanding how recalculation is supposed to work, you could try adding m_iEnslavementChance=0 to CvPlayer::clearModifierTotals (maybe simply at the start of the function, seeing that m_iEnslavementChance is also the first int member declared in CvPlayer.h). Just from looking at similar variables, this one seems to be missing from that function. I feel that this could explain an increase of the chance at every recalc. Similarly, the contents of m_pabHadVicinityBonus aren't being set to false in CvCity::clearModifierTotals, whereas e.g. m_paiBonusDefenseChanges does get zero'd by that function. I guess this latter one will be easier to test, but I'm less hopeful that the lack of clearing is indeed the problem in that case. As keldath wrote, it's "somewhat complicated"

Spoiler :

Humorous understatement? What a nightmare ...View attachment 704299

... and things being updated in the wrong order sounds like a good intuition. Ah, but you write that Ctr-Shift-T does allow the enslave issue to be reproduced quickly? Then that could be worth a try.

Thank you so much!
This seems to have solved iEnslavementChance bug.
Well... mostly... Without a recalc the chance is always 0% but a recalc will happen anyway sooner or later and the player can trigger it manually too, so I can live with that.

But what about vicinity? What should I write and where?
Well, I have a guess about the where from what you wrote:

    for (iI = 0; iI < NUM_DOMAIN_TYPES; iI++)
    {
        m_aiDomainFreeExperience[iI] = 0;
        m_aiDomainProductionModifier[iI] = 0;
    }

    m_aBuildingYieldChange.clear();
    m_aBuildingCommerceChange.clear();
    m_aBuildingHappyChange.clear();
    m_aBuildingHealthChange.clear();

    for (iI = 0; iI < GC.getNumUnitClassInfos(); iI++)
    {
        m_paiUnitClassProductionModifier[iI] = 0;
    }

    for (iI = 0; iI < GC.getNumBuildingClassInfos(); iI++)
    {
        m_paiBuildingClassProductionModifier[iI] = 0;
    }

    for (iI = 0; iI < GC.getNumBonusInfos(); iI++)
    {
        m_paiBonusDefenseChanges[iI] = 0;
    }

 

[f1rpo]

Well, maybe that's a hopeful sign for the vicinity issue.

Without a recalc the chance is always 0% but a recalc will happen anyway sooner or later and the player can trigger it manually too, so I can live with that.

That's strange. processCivics looks fine (and that's also what recalc ultimately calls, I think): changeEnslavementChance(kCivic.getEnslavementChance() * iChange);
Then there's the resync/ clearModifierTotals, which I assume only affects the recalc. Getter/ setter, Read/ write, tooltip all look good. Not much to it, really. The chance taking effect is probably more complicated, but the percentage that gets displayed is just a simple stat affected by civics like many others in the game. Setting a breakpoint for the debugger in the recalc-related functions would be my best bet – to rule out that they don't get called in unexpected ways. If keldath is still on the task (and already has the IDE set up for debugging), I feel optimistic that he'll figure it out quickly. Or maybe it could be an issue with testing from an old savegame. Or other code added that was supposed to fix the problem. (I've only looked at the source of Inthegrave's DLL).

But what about vicinity? What should I write and where? Well, I have a guess about the where from what you wrote:

Yes, after bonus defense changes seems like a tidy place. It shouldn't matter because these resets don't depend on each other. I'd try:

Spoiler :

	for (iI = 0; iI < GC.getNumBonusInfos(); iI++)
	{
		m_paiBonusDefenseChanges[iI] = 0;
	}

	for (iI = 0; iI < GC.getNumBonusInfos(); iI++)
	{
		m_pabHadVicinityBonus[iI] = false;
	}

(And thus, I discover the code=cpp forum tag. Suppose I never noticed other posters having syntax highlighting in their code snippets. Or probably most others aren't aware either. Glad I've spotted it now when seeing your reply quoted as BBCode. Obliged!)

 

[<Nexus>]

Or maybe it could be an issue with testing from an old savegame.

Hmmm... I dunno. I had set up a "testing scenario" and didn't start a new game with the new dll. I may give it a try but it works in an acceptable manner now.

Yes, after bonus defense changes seems like a tidy place. It shouldn't matter because these resets don't depend on each other. I'd try: Spoiler :

for (iI = 0; iI < GC.getNumBonusInfos(); iI++)
{
m_paiBonusDefenseChanges[iI] = 0;
}

for (iI = 0; iI < GC.getNumBonusInfos(); iI++)
{
m_pabHadVicinityBonus[iI] = false;
}

Thx! I'll try that

(And thus, I discover the code=cpp forum tag. Suppose I never noticed other posters having syntax highlighting in their code snippets. Or probably most others aren't aware either. Glad I've spotted it now when seeing your reply quoted as BBCode. Obliged!)

I could (accidentally) show you something new? Wow!
"Vak tyúk is talál szemet." ("Even a blind hen may find some seed" - Hungarian proverb)

 

[<Nexus>]

Yes, after bonus defense changes seems like a tidy place. It shouldn't matter because these resets don't depend on each other. I'd try: Spoiler :

for (iI = 0; iI < GC.getNumBonusInfos(); iI++)
{
m_paiBonusDefenseChanges[iI] = 0;
}

for (iI = 0; iI < GC.getNumBonusInfos(); iI++)
{
m_pabHadVicinityBonus[iI] = false;
}

IT'S WORKING! IT'S WORKING!
Thank you so much!

Or maybe it could be an issue with testing from an old savegame.

You may have been right about that. Starting a new game, slaves are generated immediately.
...but I say you may be right, since there is no telling if there was a forced recalc in the background or not.
(I think it's an AND2 feature set up by @45°38'N-13°47'E that the game forces a recalc under certain conditions but I don't know what those are)

Thank you again!

 

[Schmiddie]

Hi Folks,

I have an issue - I assume it has something to do with the heightmaps we use in out mod WTP.

Mines / Improvements on peaks look like this since the last update:

They are covered by the map graphic respectively sink into them. They should look more like this:

Changing the improvement files (e.g. in order to use country geometry) did not solve the issue, because in this case the improvement graphic itself is distorted and does not look good.

Any ideas how to solve this issue would be highly appreciated.

 

[Mattygerst]

Hello all,
.
I am wanting to take a pre-existing Earth Map that comes with BTS as a scenario [Earth1000AD] and be able to use it as a blank map to create my own scenario and also to just start a new game with randomly placed civs. Can someone guide me how I can do that? Also, I have a 48civ DLL that I'd like to use with it. Is there anything special I have to do to be able to use it to create scenarios? Thanks in advance for any guidance.
.
If that sounds confusing, please let me know how I can clarify.
.
Basically, I want to take an existing map that is used in a scenario that comes w BTS, clear it, and have the cleared map available as a public map available for a game start and also have it usable w my 48civ DLL

 

[45°38'N-13°47'E]

IT'S WORKING! IT'S WORKING!
Thank you so much!


You may have been right about that. Starting a new game, slaves are generated immediately.
...but I say you may be right, since there is no telling if there was a forced recalc in the background or not.
(I think it's an AND2 feature set up by @45°38'N-13°47'E that the game forces a recalc under certain conditions but I don't know what those are)

Thank you again!

I recall coding a forced recalc when switching out of a civic, maybe Caste? I remember there was a problem with resources multiplying and forcing a recalc was an easy way out of this. But I'm not 100% sure.

 

[hankinsohl]

Does anyone know how XML loading works for Civilization 4?

My understanding is that XML precedence works as follows:
.\ASSETS\XML\*.xml
..\WARLORDS\.\ASSETS\XML\ *.xml
..\ .\ASSETS\XML\*.xml

That is: BTS > Warlords > Vanilla

I also thought that CustomAssets > BTS

However, when watched under a debugger it looks like CustomAssets are loaded last. Maybe they override previous definitions?

Also, what XML files load in the CustomAssets folder? I put some fake XML files in the CustomAssets folder and they don't load. (They do however load when put in one of the BTS XML folders). Does the modded DLL have to directly specify what's loaded in CustomAssets? Or maybe the filename must match some expected value? It's not clear.

I can continue debugging this of course, but if someone already knows how this works it will save me the trouble.

Thanks in advance.

 

[JDCP]

Hello yall. Heres a question I got, with the world builder how do I remove all cities and units already placed??? I don't want to delete all the damn thing one by one.

 

[Walter Hawkwood]

Hello yall. Heres a question I got, with the world builder how do I remove all cities and units already placed??? I don't want to delete all the damn thing one by one.

With this handy tool: https://forums.civfanatics.com/resources/wbcleaner.9004/