I Believe It's Critical To Integrate .Exe "Patches" With Quintillus' Editor

Thinking/babbling out loud here:

I had been thinking strictly in terms of BIQ and SAV data, but "player preferences" may not necessarily need to be per-SAV.

So, C3X config data is currently in a single .ini . Oz seems to be suggesting scenario-level configs, so to me that's BIQ-based config. And, of course, players may just want a no-raze/sub-fix/etc epic game or even existing scenario, and that could be from the .ini file, or in the SAV.

For UIs to each of these, well .ini is manual for now. For BIQ we're talking Q's editor and/or a utility program to inject config into custom BIQs. In the future, an in-game UI may exist to either add prefs to the SAV or to the .ini. And a utility program could be used if we want to inject config into a SAV.

The big question I'm stumbling on is priority. If the scenario specifies no-raze but the player wants razing, for example: which wins? Which should win? Should the scenario editor allow/disallow C3X player config overrides?

For the nearer term, I had been imagining per-game config being in the SAV, but that means the user has to start the game, run a utility, then reload the game. And how likely is it that a user is going to have different configurations for different games? Probably little; if they want no-raze they probably want it on all games.

The potential downside is in sharing the SAV file, the C3X config is lost. But that is probably less annoying than having to run an external utility and reload every time you start a game.

The conclusions I'm coming to:
  • For now, the .ini config makes the most sense (or JSON or whatever file)
  • Once an in-game UI for C3X settings is available, then maybe in-SAV config with in-.ini defaults makes sense
  • Scenario C3X config vs player C3X config needs to be considered
    • Which one overrides the other?
    • Should the modder be able to prevent config overrides?
  • In the near-term, SAV and BIQ sharing for C3X may be a bit janky as there is no transfer of C3X config, but this is less important than single-player satisfaction for increased adoption of C3X
 
Thanks for the information, especially all that stuff about the compression.

Those 4-char section headers also appear in the run time data, but only once per object. It's strange that they're repeated so many times in saves. The game does use inheritance, frequently for UI objects but rarely for game objects. For example you might guess there would be a base unit class with child classes for land, sea, and air units, but no, there's only one unit class. Similarly there is only one city class and one leader class. The tile class has a child but I haven't seen what it's used for. I would have guessed it would be for LM tiles but it doesn't look like it is. If the game object classes were extended from vanilla to PTW to Conquests, I haven't noticed that. But I also haven't looked into the inheritance chains in detail because it's never been relevant to anything I was working on.

There are a few problems I can think of with packing config data into unused areas:
1. The available space is inherently limited. Packing the data into a string might give unlimited space but that's only possible if the program works a particular way, it can't rely on strlen when serializing the string or the extra data won't be saved.
2. It's not always easy to prove that certain areas are really unused. Again this depends on the particulars, if there's an entire redundant PTW map I wouldn't worry about overwriting that but if it's a 4-byte field in each tile object I would worry that that field is used in some rare circumstances.
3. It makes it difficult to create an external configuration program b/c the program must understand the BIC and save formats in detail. This is especially bad w/r/t compression, it means the configurator must include a decompressor.

The alternative is to tack the configuration data onto the end of the file. If that works it is a preferable solution since it avoids all the above issues. The problem with tacking data on the end is the additional data won't be preserved when a save is loaded and written back out by an unmodded EXE. One blunt workaround is to deliberately break compatibility in cases where the tacked on config contains important gameplay changes. I.e. the modded EXE would write out a deliberately broken save that only it could load. That may annoy players but it means the config data won't be silently lost. If the config data only contains player preferences then just let it be lost, that's only a small nuisance not worth major effort to prevent.

One last remark about storing data in strings: can you think of any string that might be a good candidate? The city and unit names won't work because those are stored in fixed size fields in the run-time data structures. The scenario description might work except that would be part of the BIC which is not present in all save files.
Puppeteer said:
Oh. I finally actually downloaded and looked at your patch/code. It's in C, so I don't have to worry about object references to retrieve any successfully hidden data.
Click to expand...
Not only is the mod in C, so is the decompilation. C++ concepts like references vs pointers and private vs public members don't apply anymore.
 
I know string fields are fixed-length, but they are 0-terminated. I had some hand-wavy idea on an algorithm to hunt down all the e.g. unit type (PRTO) string fields and pack config data after the null to the end of the field, assuming that not all characters would be used for every field in the BIQ.

Also in the BIQ are pretty long path fields and I think a descriptor or note field.

And I just now had a new thought: City Names. At the cost of one of the civs having a goofy city name or three, we could just add as many city names as needed in the BIQ for config if nothing else is more sensible. Or maybe Leader names could be used instead. Each of them are lists of arbitrary length. Well, each string is limited, but you can add more names.

For SAV files I'm thinking far less of string fields as there may be absolutely none at the start of a game unless you want to name the starting units with config info. But there is always a map, and I'm currently enamored with the idea of taking one or more bytes from TILE and iterating tiles for data. Seeking to the map should be simple enough.

And I'm already auto-decompressing the file data in Go and C#, and we have the C code in the zlib contribs, so for me decompression is trivial. Perhaps even recompression, although I haven't done that yet.

However, the way you describe TILE in-memory concerns me a little. There are definitely four TILE char[4]s in the save for each tile. Maybe that's an artifact of serialization and not all of that is in memory...and actually that's starting to make sense. I'll have to do a few tests to see if data persists.

Possible tests for me (although I offer no promises to when or even if):
  • Experiment with concating data to the end of BIQ and SAVs, both compressed and uncompressed...this may be easier than I think based on observations so far
    • See if DCL/blast can handle concating a new compressed stream to the end of the main one
  • Experiment with adding city names and/or leader names to BIQ
  • Experiment with data persistence and retrievability when hidden in TILE PTW terrain byte
Actually, tacking data onto the end of BIQs and SAVs may be easier than I originally thought. You'd have to go back and grab the add-on data from patch code instead of memory, but maybe that's less of a pain and more reliable than I've been imagining.
 
Puppeteer said:
And, of course, players may just want a no-raze/sub-fix/etc epic game or even existing scenario, and that could be from the .ini file, or in the SAV... The big question I'm stumbling on is priority. If the scenario specifies no-raze but the player wants razing, for example: which wins? Which should win? Should the scenario editor allow/disallow C3X player config overrides?

For the nearer term, I had been imagining per-game config being in the SAV, but that means the user has to start the game, run a utility, then reload the game. And how likely is it that a user is going to have different configurations for different games? Probably little; if they want no-raze they probably want it on all games.
Click to expand...

I think here some additional thoughts are needed. The theory, that if players want no-raze they probably want it on all games in my eyes is wrong. The reason behind the Noraze patch done by skyer2 was to provide the needed help for many historical Civ 3 scenarios. In these historical scenarios it was terrible wrong, when the AI razed cities, that were not razed in reality. This was especially bad in the SOE scenario, where land bridges between the UK and France to allow big "amphibious" assaults were covered by destroyed city graphics, that were looking like sea. Now when the AI razed a city (what was bad itself), the terrain around this city looked like a big lake, what made the situation even worse.
Spoiler :

razed-city-jpg.459645


landbridges-britain-france-jpg.459650


landbridges2-jpg.459652


So the players need both versions for playing C3C: A normal version allowing to raze cities (especially for the C3C epic game) and a no raze version especially for many historical C3C scenarios. Here it was helpful to have several shortcuts to start C3C with a no-raze exe and another one to start it without a no-raze exe. This worked very well.

With the Flintlock patch now my configuration for C3C is somewhat different:

Different C3C exes.jpg


From the right side: With the first shortcut I start my upcoming improved German C3C translation on my GOG C3C installation with the Flintlock patch, with the second exe I start the Civ Chronicles C3C installation with the Antal1987 exe, with the third exe I start the Civ Chronicles C3C installation with the Flintlock patch, with the fourth exe I start the GOG C3C installation with the Antal1987 exe and with the fifth shortcut I start the normal GOG Civ 3 Complete launcher.

All these shortcuts can be used separately without any problems for starting a game.

In the past I had additional shortcuts for exes with the no-raze patch, but with the Flintlock patch I use a different handling.

Now it is enough to have an additional shortcut for the default.c3x_config file (for me at present only for the GOG installation needed) for the different adjustments that can be done before triggering one of the Flintlock exes. Per example if I want to start a game with the no-raze settings, I simply first change the entries in the default.c3x_config file by the shortcut to noraze true and than I start the game with the Flintlock patch for the GOG C3C installation.

This allows a quick and comfortable start of C3C even with the configuration file of the Flintlock patch.
 
My plan to integrate multiple multiple sets of config options is to layer them on top of one another, from the most general to most specific, and allow configs to include only the values they care to change. For example, if a save file has config settings that don't include NoRaze, the scenario NoRaze setting will apply. And if the scenario has no NoRaze setting, like every epic game, the default config value will apply. Technically even the default config can be missing settings, in which case the values will be determined by the "base" config that's hard coded into the mod. BTW, you can find that struct by searching for "base_config" in ep.c, its values match those in the unedited default config file.

The disadvantage of this approach is that it makes editing the config more complex since every setting effectively has an additional possible value of "don't care". There's also the matter of if it's better to write scenario settings into saves (as if they were save settings) of if it's better to try to reload the scenario settings from the BIQ each time the save is loaded. I prefer the first approach since it matches the base game behavior and is probably easier to implement.
Puppeteer said:
And I'm already auto-decompressing the file data in Go and C#, and we have the C code in the zlib contribs, so for me decompression is trivial. Perhaps even recompression, although I haven't done that yet.
Click to expand...
Yeah, I just looked at blast.c and it should be easy to slip into the mod. I'm so used to bloatware I just assume that even simple libraries will be tens of thousands of lines and pull in a mess of dependencies.
 
You must log in or register to reply here.
Back
Top Bottom