Using the strategy layer to store extra data

[Yakk]

So I did a bit of strategy layer research.

And it looks like it won't work, unless I missed something.

You can see in CvGameTextMgr::getGlobeLayerName it uses GLOBE_LAYER_STRATEGY to place the text for the buttons when you are in strategy mode. Elsewhere in the code, the 'other layers' at the global level have cases that let you change the color of each square at the global level -- but GLOBE_LAYER_STRATEGY does not seem to 'dip into' the DLL anymore.

In particular, you can see in CvPlayer::getGlobeLayerColors, each of the other layer options gets written up. Each then procedurally builds the colour display for that option and that layer (so as far as I can tell, you cannot stuff data into there, either).

However, I did come across something ... interesting.

The replay message code is a bunch of strings with some extra meta-data. We could stuff AI thoughts into the replay message code, and filter out the AI thoughts when the replay is done. A BTS better AI game that is then switched over to a normal game would then display all of the 'AI thoughts' when the game ends on replay -- but it would remain binary compatible.

 

[MadmanAtW]

Fascinating!
That's actually a really cool idea. (Stupid unschooled-in-Civ/mod-coding question: 'replay messages' are which? Is that the "So-and-so has built the Oracle! Your archer has defeated a barbarian schoolmarm!" etc? In which case we'd be adding fake messages but then filtering them out before they were entered into the chat log?)

 

[Yakk]

From what I can tell, this is the messages that appear at the end of the game, after you retire, when the game is being replayed.

If I'm right, then the nice thing is that it won't impact non-BAI games that much. I do not know how long the strings are allowed to be: If the strings have no limit on their length, I could see encoding arbitrary data into a replay event, and replacing it on each save (and loading it on each load).

 

[MadmanAtW]

Ah, that makes sense. And sounds really promising, if the mod can indeed add and remove items from the replay queue!

Be kinda funny to see the replay thoughts by ending without BBAI... I would find myself inclined to make it a code of sorts, where "I want ice cream" means player 2's 2nd city is a GP farm or something, etc etc. Just for the surreality.

 

[Roland Johansen]

Good idea Yakk. I hope this can developed into an AI that can plan for certain specialised cities or an AI that can remember where it was planning to attack the previous turn.

 

[BobTheTerrible]

Good idea Yakk. I hope this can developed into an AI that can plan for certain specialised cities or an AI that can remember where it was planning to attack the previous turn.

I think you missed the point. We can't store stuff like that in the strategy layer - but on an unrelated note, it is possible to store ai "thoughts" into a reply message.

 

[Roland Johansen]

I think you missed the point. We can't store stuff like that in the strategy layer

I know. I reacted to the second part of Yakk's post and his second post which suggest there could be possibilities using the replay message to store data. I congratulated him on that finding and suggested some things to store in the replay message.

However, I did come across something ... interesting.

The replay message code is a bunch of strings with some extra meta-data. We could stuff AI thoughts into the replay message code, and filter out the AI thoughts when the replay is done. A BTS better AI game that is then switched over to a normal game would then display all of the 'AI thoughts' when the game ends on replay -- but it would remain binary compatible.

From what I can tell, this is the messages that appear at the end of the game, after you retire, when the game is being replayed.

If I'm right, then the nice thing is that it won't impact non-BAI games that much. I do not know how long the strings are allowed to be: If the strings have no limit on their length, I could see encoding arbitrary data into a replay event, and replacing it on each save (and loading it on each load).

 

[PieceOfMind]

I also wish to congratulate Yakk on this discovery. I've long held the position that some of the most substantial AI improvements would require at least a basic form of memory. Really I'm surprised it was not done properly by Firaxis in the first place.

Does this mean we can start working on AI-suggestions on the assumption that some memory is allowed? Or is there a low limit to how much can be stored?

Finally we might be able to give AIs long-term and mid-term strategies, like beelining techs! (remember how complicated the discussion about that became? )

 

[Yakk]

I need to test this, but the thought is...

Assuming we can fit strings of arbitrary length into the replay stream, and it doesn't cause crashes, we stick a string of the format:
AI_MEMORY_INTERNAL: $1
where $1 is some arbitrary formatted blob of data. Possibly we append a bunch of newlines to the end of it so in standard BTS, it scrolls out of view.

When loading a file, we check for AI_MEMORY_INTERNAL: at the front of the replay list, and if we find it, we stuff it into a run-time ai memory structure. We do not save it to the run-time list of replay items. If we don't see AI_MEMORY_INTERNAL, we simply tell the run-time ai memory structure to clear it's state.

When saving a file, we go talk to the run-time ai memory structure, ask it for a string, and stuff it at the start of the replay information before writing the rest of the replay information.

We could also stick it at the _end_ of the replay queue instead of front -- roughly equivalent.

Standard BTS games end up with an AI_MEMORY_INTERNAL spew when they replay the game.

Possible issues: This could easily cause the application to crash, if any code assumes that the length of a given replay element is bounded.

The spew you would get in a BTS normal game.

A better AI game that is then loaded in BTS, then played for a bit, then loaded in better AI again needs to be able to deal with an out of date ai memory internal.

Meh properties: The AI memory global item ends up existing relatively orthogonally with the rest of the code. So we aren't attaching this memory to a particular unit or plot. Without a unique way to identify units/etc, this could cause problems, or be fragile.

Nice properties: No limits on the kind of data we can store there -- so long as it can be saved out in a string, and extracted from that string.

As noted, I need to test if arbitrary length strings work.

 

[MadmanAtW]

Assuming we can fit strings of arbitrary length into the replay stream, and it doesn't cause crashes, we stick a string of the format:
AI_MEMORY_INTERNAL: $1
where $1 is some arbitrary formatted blob of data. Possibly we append a bunch of newlines to the end of it so in standard BTS, it scrolls out of view.

Is there a reason not to put multiple such lines in the stream? Have BBAI read until it reaches a line that isn't AI_MEM_INT. Fewer worries about string length if you can use multiple strings.

We could also stick it at the _end_ of the replay queue instead of front -- roughly equivalent.

Except possibly in the case where a game goes BBAI -> BTS -> BBAI- this based on the assumption that the game is adding things to the end of the queue as it goes. Stuff at the beginning will at least stay in the correct position when you go through different versions.

Meh properties: The AI memory global item ends up existing relatively orthogonally with the rest of the code. So we aren't attaching this memory to a particular unit or plot. Without a unique way to identify units/etc, this could cause problems, or be fragile.

For BBAI/BULL mergers we could use the BUG routine that names units after their cities of origin and numbers them. Not sure that that is guaranteed to be unique enough, though. Plots aren't too bad- City(x,y) can uniquely identify a plot assuming the city is at the origin. Busts if the city gets razed, though. But that's not quite what you're talking about anyway- yes, there is an inherent fragility to needing to "attach" these memories to the plots and units, even if we can uniquely identify them, rather than making them part of the plot or unit's data.

I think it's all extremely promising!

 

[r_rolo1]

This has definitely potential.....

I think this will definitely work well with the tech paths, especially if BBAI gets a routine for tech choice that looks atleast 2 levels in front of the current techs Other areas could prove more problematic, like city specialization ( can this be done easily ? I mean, the AI deciding that wants HE in a spot and get only the hammer multipliers and more skeleton buildings ( granary courthouse, and enough .... ) and not spam cottages in the city BFC ) or position of enemy units.....

 

[BobTheTerrible]

I know. I reacted to the second part of Yakk's post and his second post which suggest there could be possibilities using the replay message to store data. I congratulated him on that finding and suggested some things to store in the replay message.

I'm sorry, I didn't realize that the replay code could be read by the AI, I just thought it would be written.

 

[Yakk]

Naw, it the replay data is stored, saved and loaded in the DLL. The application talks to the DLL to get the replay strings.

Was gardening last night, so still haven't checked if this solution is viable.