Breaking Save Game Compatibility for v37

Some issues have come back up regarding maximum selection groups being exceeded on very large maps by the Barbarian player. This is an old problem resurfacing again and one big way to begin the process of resolving it (or at least what should probably be a first step to resolution) would be to split Barbarians and Animals into their own special player assignments. (At the moment they are all barbarians.)

Doing this would break compatibility.

A few other things have been recently discussed as to causes to break compatibility or things we'd like to clean up that can only be done when compatibility is broken.

As it looks like we should now take action to split Barbs and Animals (and reserve a few more potential player slots for other NPC player definitions to be possible to insert down the road without breaking compatibility again) I'm initializing this thread so we can discuss and collect our thoughts as to what could and should be done during this breakage period.

If we do this right it may be the one and only time since the SVN began that we will truly break save compatibility. Of course, Alberts makes a valid point that many times we do things that will corrupt previously saved games if they are loaded into the new code environment but usually that can be overlooked. THIS type of compatibility breakage would mean old saves can't be loaded at all past this point.

So we should do it once and do it right.

Therefore, let's generate a list of tasks to do while we go about this.

So far we have:

• Split Barbs into Animals and Barbs and reserve 5 player slots for these kinds of NPC groups.
• Clean out unused and unnecessary Game Options.
• Possible further cleanups in the enums - deleting an established enum value disrupts saves.
• Rename some xml types to match their text labels.

I know DH said he had a few things to do during a breakage as well. What else would we need to add to this list?

Changing the names in the XML files to be what they should be will break saves, but fixing it will make life easier for all modders. Joseph has been the latest modder to have to deal with this sort of thing This is in all files bonuses, technologies etc. Eg the tech Controlled Fire is actually "TECH_THE_FIRE" in the XML.

As to extra NPC slots we may need one for the Civ V independent city nations (or what ever they called it) and for the Indigenous Peoples also. Both of those are not nations, the only thing they share between members is the same technology level. Current implementation of IP also share the same relationship with each player.

If WE go for this and turn times dont go waaaaay down, and i mean waaaay down, i say No.

If its a main problem with splitting, then for game sakes, lets just SCRAP splitting. To me "personally" dont really see a need for it anyways, WE have a TON of units, but that is ME.
If we do remain with the splitting, i say lets figure out what the HECK is making these turn times waaay longer lately. Because i am again not even in Classical Era (Large map, only 7 civs), and turn times are outrageous. If on a map like this, turn times should NOT be more than 15 seconds AT MOST, at least up to the Modern Era. Again this is just me, no offense to anyone or who has made changes thus far.

Now if we are adding items ie Black OPs, etc, and no change to turn times, then i really dont care about saves . . that is what the SVN should be all about anyways . . .

But again this a a modmod of democracy and opinions are what it is run on, and i will settle for what MOST will want to do, (as usual).

Dancing Hoskuld said:

Changing the names in the XML files to be what they should be will break saves, but fixing it will make life easier for all modders. Joseph has been the latest modder to have to deal with this sort of thing This is in all files bonuses, technologies etc. Eg the tech Controlled Fire is actually "TECH_THE_FIRE" in the XML.

As to extra NPC slots we may need one for the Civ V independent city nations (or what ever they called it) and for the Indigenous Peoples also. Both of those are not nations, the only thing they share between members is the same technology level. Current implementation of IP also share the same relationship with each player.

Click to expand...

1) Good call... lots of work there but would be very nice to align that stuff now.

2) Maybe we need to reserve 10 (including barbs and animals). I'm thinking it would be good for aliens too if they aren't going to be a set of potential civilizations of their own.

strategyonly said:

If WE go for this and turn times dont go waaaaay down, and i mean waaaay down, i say No.

If its a main problem with splitting, then for game sakes, lets just SCRAP splitting. To me "personally" dont really see a need for it anyways, WE have a TON of units, but that is ME.
If we do remain with the splitting, i say lets figure out what the HECK is making these turn times waaay longer lately. Because i am again not even in Classical Era (Large map, only 7 civs), and turn times are outrageous. If on a map like this, turn times should NOT be more than 15 seconds AT MOST, at least up to the Modern Era. Again this is just me, no offense to anyone or who has made changes thus far.

But again this a a modmod of democracy and opinions are what it is run on, and i will settle for what MOST will want to do, (as usual).

Click to expand...

1) Turn times have nothing to do with the need for this and would be completely unaffected either way.

2) Splitting from size matters has nothing to do with this. The Selection Groups are command shells for units to operate within. Each unit has its own selection group unless it has been added to another selection group. When released from a selection group, it must initialize a new selection group for itself.

As groups combine and split (not units as in Size Matters though that can add to this process) they tend to cycle through selection group definitions. And apparently there's a cap on how many there can ever be defined for a player.

This limit is being exceeded by barbs in the Ancient era on extremely large maps so you can imagine that even on smaller maps, later in the game it WILL be eventually exceeded, if not by barbs then by any player (and barbs cannot merge or split even with Size Matters on).

We may need to address it in other ways for that reason but for now at least separating barbs and animals will help tremendously. Will also make barbs and wild animals capable of fighting each other so barbs become a source of competition for your hunts.

3) (I believe) Your turn time complaints are almost entirely a matter of problems in the most complex portions of the AI. I may also be able to remove a few delaying factors now but I doubt it would have much impact. IMO, AI needs a complete overhaul. But that may be an overreaction.

Thunderbrd said:

1) 1) Turn times have nothing to do with the need for this and would be completely unaffected either way.

3) (I believe) Your turn time complaints are almost entirely a matter of problems in the most complex portions of the AI. I may also be able to remove a few delaying factors now but I doubt it would have much impact. IMO, AI needs a complete overhaul. But that may be an overreaction.

Click to expand...

Then nevermind what i said above . . I think you might be correct on the AI overhaul. yikes . . .

Game Options - A base for discussion

To be:

A - Made non-optional

1. GAMEOPTION_ASSIMILATION
2. GAMEOPTION_MOUNTAINS
3. GAMEOPTION_BARBARIAN_GENERALS (animal-barb splitting will make this option viable again.)
4. GAMEOPTION_RELIGION_DECAY

B - Completely removed (never to return)

1. GAMEOPTION_AGGRESSIVE_AI
2. GAMEOPTION_RUTHLESS_AI
3. GAMEOPTION_MORE_RIVERS (confusing and unnecessary) (Some mapscripts gain a lot from this option)
4. GAMEOPTION_MORE_RESOURCES ( --||-- ) ( --||-- )
5. GAMEOPTION_MIN_CITY_BORDER (I think this breaks city flipping from culture, although I'm not too sure)
6. GAMEOPTION_SCALE_CITY_LIMITS

C - Hidden and off until reworked (temp. hidden)

1. GAMEOPTION_RAGING_BARBARIANS
2. GAMEOPTION_FLIPPING_AFTER_CONQUEST (A, IMO, poorly designed feature that seems out of place in C2C)

I will edit the list depending on opinions voiced in the following discussions.

Toffer90 said:

Yeah, there are many of these; although not very important to change it would be nice in some of the worst examples.
Fire Making is TECH_THE_FIRE, controlled fire is TECH_CONTROLLED_FIRE

A few examples:

Spoiler :

TECH_PATHS → TECH_TRACKING (!)
TECH_GRAIN_GATHERING → TECH_GATHERING
TECH_COOKED_FOOD → TECH_COOKING
TECH_STONE_TOOLS → TECH_HARD_HAMMER_PERCUSSION (!)
TECH_PARIETAL_ART → TECH_PETROGLYPHS
TECH_DOG_BREEDING → TECH_CANINE_DOMESTICATION
TECH_CAT_BREEDING → TECH_FELINE_DOMESTICATION
TECH_HORSEBACK_RIDING → TECH_ANIMAL_RIDING (!)
TECH_ELEPHANT_RIDING → TECH_PACHYDERM_DOMESTICATION
TECH_CAMEL_RIDING → TECH_CAMELID_DOMESTICATION

UNIT_WARRIOR → UNIT_CLUBMAN

BUILDING_TAR_PROCESSING → BUILDING_TAR_REFINERY

BONUS_IVORY → BONUS_ELEPHANT
BONUS_IVORY2 → BONUS_IVORY
BONUS_CLAY2 → BONUS_CLAY
BONUS_CLAY → BONUS_FINE_CLAY

ROUTE/BUILD_TRACK1 → ROUTE/BUILD_PATH
ROUTE/BUILD_TRACK → ROUTE/BUILD_MUD_PATH

Click to expand...

"When" we do change things like this then ie: Controlled Fire, then WE need to make sure its someone who is very dedicated to the XML or python changes needed, ie pic1 is just an example of the files needed to do one tech change like this for Controlled fire for instance . . .

In past I was thinking about this in ART_UNIT xml files but lack of time and other projects stop me.
IMO Its just cosmetic changes, only for modders. And I think C2C team has better things to do

Sparth said:

In past I was thinking about this in ART_UNIT xml files but lack of time and other projects stop me.
IMO Its just cosmetic changes, only for modders. And I think C2C team has better things to do

Click to expand...

So you want us to break saves more than once and don't want us to make changes that will make life easier for new modders.

Dancing Hoskuld said:

So you want us to break saves more than once and don't want us to make changes that will make life easier for new modders.

Click to expand...

Finding what is what is a first exam for every new modder

Sparth said:

In past I was thinking about this in ART_UNIT xml files but lack of time and other projects stop me.
IMO Its just cosmetic changes, only for modders. And I think C2C team has better things to do

Click to expand...

IMO it is enough for modders to do what was done in the Building XML. A comment over every entry that lists the name of the building. And I think whoever did it said that it wasn't hard to set up. I think it's less work than changing the tags and is equallly effective. Just my 2 cents

Sparth said:

Finding what is what is a first exam for every new modder

Click to expand...

I think I got a D on my report card.

JosEPh

Using something like Notepad++ gives you the ability find and replaces within files and even directories. Changing TECH_COOKED_FOOD to TECH_COOKING everywhere at once should be relatively straight forward.

Arakhor said:

Using something like Notepad++ gives you the ability find and replaces within files and even directories. Changing TECH_COOKED_FOOD to TECH_COOKING everywhere at once should be relatively straight forward.

Click to expand...

True... between that and grepwin it wouldn't be hard on a type by type basis. What I would think would be the tough part would be going through every info type definition in the mod to ensure that it's named according to its text.

However, I suppose I will be doing something similar in checking every tag in the mod to see if and how its being used and if it can be safely deleted or restructured into something more memory efficient. And that's certainly a part of such a project.

Alberts already imported my tool to generate a full list of type vs description

I've seen several comments about things needing to be done but aren't able to be done due to Save game Compatibility, which is a noble thing but sometimes you need to crack some shells to make an omelet,

Make sure that when you break saves you implement everything you otherwise couldn't, it saves on problems further down the pipeline, It's not the first time i just restarted because of changes, so I'm not one to start complaining about my saves

there is a movement in this topic? last svn does not work

Should I go ahead and commit the tag changes shown in post #6?
I could do the two routes too, but they can completely break old saves; so I thought that might wait some more.

Toffer90 said:

Should I go ahead and commit the tag changes shown in post #6?
I could do the two routes too, but they can completely break old saves; so I thought that might wait some more.

Click to expand...

Breaking saves includes loosing all your units and buildings you know, it is just not as catastrophic, but could lead to you loosing the game. Mind you, Joe and others may like it if the ai lost all its log rams in current games.

Dancing Hoskuld said:

Breaking saves includes loosing all your units and buildings you know, it is just not as catastrophic, but could lead to you loosing the game. Mind you, Joe and others may like it if the ai lost all its log rams in current games.

Click to expand...

Hehe, true that.
I can hold it off until someone else also got something that will break saves ready.