Persisting custom data with saves

greyTiger

greyTiger

Warlord
Joined
Oct 7, 2010
Messages
198
Location
Australia
Has anyone been able to persist custom data when a game is saved, and then have that same information loaded when the game is reloaded?

I was trying this using the SetValue and GetValue of the GameConfiguration object but it appears that the data is cleared when the game is reloaded. I use a key generated from the player starting positions and the current turn. When I serialize my custom data to a string and call SetValue() I don't get any error. If I immediately read back the saved data I get what I expect from GetValue() for the same key.

If I call GetValue() after a reload for the same key I get nil :(

* Does anyone know where I can view the data stored by SetValue()? Is it a file or db somewhere?
* Is there an alternative mechanism we currently have available to us for saving\loading custom data?
 
You can set GameConfiguration values during the setup of a new game, i.e. by creating custom Paramaters to select on the Advanced Setup screen, but so far as I know they cannot be changed during a game. Only when starting a new game.

So any persisting of data would have to be done some other way. I have no idea how. :(
 
Last edited:
Strange, GameConfiguration.SetValue / GetValue works for me. It's stored in the savegame (from the size of it)
 
Here is the output from Firetuner when I save my data, then load an earlier save. I tried using a static key this time but seems to have the same issue of not finding anything in the GameConfiguration.GetValue.

The code I currently use to save my custom data.
Code: 
    local key:string = GOLDEN_AGE_DB_KEY;
   WriteToLog("SaveCustomData: key=" .. key);
 
   local data:string = Serialize(m_PlayerGoldenAges);
   WriteToLog("SaveCustomData: data=" .. data);
 
   GameConfiguration.SetValue(key, data);
   WriteToLog("SaveCustomData: Serialized data stored using GameConfiguration.SetValue");

The code I currently use to load my custom data.
Code: 
    local key:string = GOLDEN_AGE_DB_KEY;
   WriteToLog("LoadCustomData: key=" .. key);
 
   local data:string = GameConfiguration.GetValue(key);
 
   if (data ~= nil) then
       WriteToLog("LoadCustomData: Deserializing data from GameConfiguration.GetValue");
       m_PlayerGoldenAges = Deserialize(data);
   else
       WriteToLog("Nothing to load for key {" .. key .. "}");
   end

Output after save on turn 4 of the game.
Code: 
 GoldenAges: SaveCustomData: key=GT_CiVIGoldenAges
 GoldenAges: SaveCustomData: data=0,25,1.2234375,10,0|1,25,0.439453125,10,0|
 GoldenAges: SaveCustomData: Serialized data stored using GameConfiguration.SetValue

 GoldenAges: SaveCustomData: reading data back using GameConfiguration.GetValue
 GoldenAges: SaveCustomData: data=0,25,1.2234375,10,0|1,25,0.439453125,10,0|

Above we can see that calling GameConfiguration.GetValue immediately after SaveValue returns the same data that was saved.

Output after save on turn 8 of the game.
Code: 
 GoldenAges: SaveCustomData: key=GT_CiVIGoldenAges
 GoldenAges: SaveCustomData: data=0,25,3.139453125,10,0|1,25,2.197265625,10,0|
 GoldenAges: SaveCustomData: Serialized data stored using GameConfiguration.SetValue

 GoldenAges: SaveCustomData: reading data back from GameConfiguration.GetValue
 GoldenAges: SaveCustomData: data=0,25,3.139453125,10,0|1,25,2.197265625,10,0|

Output after loading the game saved on turn 4.
Code: 
 GoldenAgeSupport: ExposedMembers.GoldenAgeSupport functions initialized...
 TradeOverview: Initializing BTS Trade Overview
 TradeOverview: Initializing BTS Trade Support Tracker
 TradeOverview: Retrieving previous routes PlayerConfig database
 TradeOverview: Saving running routes info in PlayerConfig database
 TradeRouteChooser: Initializing BTS Trade Route Chooser
 TradeRouteChooser: Initializing BTS Trade Support Automater
 TradeRouteChooser: No running route data was found, on load.
 TradeOriginChooser: Initializing BTS Trade Origin Chooser
 InGame: Loading InGame UI - C:/Users/Craig/Documents/My Games/Sid Meier's Civilization VI/Mods/CiVI Golden Ages v1.0/UI/GoldenAges
 GoldenAges: GoldenAge.lua Initialized!
 InGame: Loading InGame UI - C:/Users/Craig/Documents/My Games/Sid Meier's Civilization VI/Mods/CiVI Reformation Rankings v1.2/UI/Popups/InGameCivAndCityRankings
 InGameCivAndCityRankings: InGameCivAndCityRankings.lua Initialized!
 EndGameMenu: Initialize: LuaEvents.CustomVictoryTriggered(player, victoryType) added!
 TutorialUIRoot: Tutorial: Firaxis in game tutorial prompts.
 TutorialUIRoot: Version: 1
 TutorialUIRoot: Loading bank of items for tutorial scenario: 'BASE'
 LoadScreen: OnLoadGameViewStateDone
 GoldenAges: LoadCustomData: key=GT_CiVIGoldenAges
 GoldenAges: Nothing to load for key GT_CiVIGoldenAges

Above we can see that the GameConfiguration.GetValue did not find any data for the same key.


I'm not sure what the issue could be at this point. Quite frustrating.
 
Last edited:
Last edited:
It seems if I only try to save a single string value then the data gets loaded when the game is loaded. e.g

On Turn 4 I saved the text 'miles' using GameConfiguration.SetValue(). I then immediately read it back using GameConfiguration.GetValue() and got back the same value.
Code: 
 GoldenAges: SaveCustomData: Saved data 'miles' to key 'GATEST'
 GoldenAges: SaveCustomData: Data check for key 'GATEST' equals 'miles'

I then closed the game and restarted Civ6. Chose the save file and loaded it.
Code: 
 LoadScreen: OnLoadGameViewStateDone
 GoldenAges: LoadCustomData: Data found for key 'GATEST' equals 'miles'

The correct data was loaded. I am guessing that there is a limit to the amount of information that can be saved\loaded in the GameConfiguration. I'll have to continue with some tests to see what the limit is.
 
OK. After more testing I am even more confused with the GameConfiguration.SetValue\GetValue results.

I created a new game and played up to turn 4. In Fireturner I get the following output for my custom data at the start of each turn.
Code: 
TURN1   PLAYER0=10,0,10,0   PLAYER1=10,0,10,0
TURN2   PLAYER0=10,1,10,0   PLAYER1=10,0,10,0
TURN3   PLAYER0=10,2,10,0   PLAYER1=10,0.4,10,0
TURN4   PLAYER0=10,3,10,0   PLAYER1=10,0.8,10,0

The information being stored for each player is the 'PointsToTriggerNextGoldenAge, CurrentGoldenAgePoints, GoldenAgeDuratuion, TurnsRemainingForCurrentGoldenAge'.
In the first 3 turns player0 gained 3 points toward their next golden age, while player1 gained 0.8 toward theirs. This is all as I expect at this point.

I then saved the game on turn 4 with the following debug output in Firetuner.
Code: 
GoldenAges: SaveCustomData: Saving data in m_PlayerGoldenAges collection...
GoldenAges: SaveCustomData: Saved '10,3,10,0' to key 'GAP0'
GoldenAges: SaveCustomData: Saved '10,0.8,10,0' to key 'GAP1'
GoldenAges: SaveCustomData: Save completed

I then continued up to the start of turn 7.
Code: 
TURN5   PLAYER0=10,4,10,0   PLAYER1=10,1.2,10,0
TURN6   PLAYER0=10,5,10,0   PLAYER1=10,1.6,10,0
TURN7   PLAYER0=10,6.1,10,0   PLAYER1=10,2,10,0

At turn 5 an auto save kicked in and the following was logged.
Code: 
 GoldenAges: SaveCustomData: Saving data in m_PlayerGoldenAges collection...
 GoldenAges: SaveCustomData: Saved '10,4,10,0' to key 'GAP0'
 GoldenAges: SaveCustomData: Saved '10,1.2,10,0' to key 'GAP1'
 GoldenAges: SaveCustomData: Save completed

I then saved to a new file on turn 7 and the following was logged.
Code: 
GoldenAges: SaveCustomData: Saving data in m_PlayerGoldenAges collection...
GoldenAges: SaveCustomData: Saved '10,6.1,10,0' to key 'GAP0'
GoldenAges: SaveCustomData: Saved '10,2,10,0' to key 'GAP1'
GoldenAges: SaveCustomData: Save completed

All seems ok so far.
I then shutdown the game and restarted. Then loaded my save for turn 4.
Code: 
 LoadScreen: OnLoadGameViewStateDone
 GoldenAges: LoadCustomData: Loading data from GameConfiguration...
 GoldenAges: LoadCustomData: Nothing to load for key 'GAP0'
 GoldenAges: LoadCustomData: Nothing to load for key 'GAP1'

It seems the data for the save no longer exists. I then tried loading the last save on turn 7.
Code: 
 LoadScreen: OnLoadGameViewStateDone
 GoldenAges: LoadCustomData: Loading data from GameConfiguration...
 GoldenAges: LoadCustomData: Loaded '10,4,10,0' for key 'GAP0'
 GoldenAges: LoadCustomData: Loaded '10,1.2,10,0' for key 'GAP1'

Only to get the data from the auto save that took place on turn 5. What the @#&%?

I think I'll try including the turn number in the key in my next set of tests and see what happens, but this is very confusing.
Anyone have any thoughts?
 
After changing my code to include the turn number in the key used with SetValue\GetValue I got the following results.

Code: 
GoldenAges: SaveCustomData: Saving data in m_PlayerGoldenAges collection...
GoldenAges: SaveCustomData: Saved '10,0.82,10,0' to key 'GAP0T4'
GoldenAges: SaveCustomData: Data check for key 'GAP0T4' equals '10,0.82,10,0'
GoldenAges: SaveCustomData: Saved '10,0.88,10,0' to key 'GAP1T4'
GoldenAges: SaveCustomData: Data check for key 'GAP1T4' equals '10,0.88,10,0'
GoldenAges: SaveCustomData: Save completed


AUTO SAVE GETS WRONG TURN
GoldenAges: SaveCustomData: Saving data in m_PlayerGoldenAges collection...
GoldenAges: SaveCustomData: Saved '10,1.23,10,0' to key 'GAP0T4'
GoldenAges: SaveCustomData: Data check for key 'GAP0T4' equals '10,1.23,10,0'
GoldenAges: SaveCustomData: Saved '10,1.32,10,0' to key 'GAP1T4'
GoldenAges: SaveCustomData: Data check for key 'GAP1T4' equals '10,1.32,10,0'
GoldenAges: SaveCustomData: Save completed


GoldenAges: SaveCustomData: Saving data in m_PlayerGoldenAges collection...
GoldenAges: SaveCustomData: Saved '10,2.05,10,0' to key 'GAP0T7'
GoldenAges: SaveCustomData: Data check for key 'GAP0T7' equals '10,2.05,10,0'
GoldenAges: SaveCustomData: Saved '10,2.2,10,0' to key 'GAP1T7'
GoldenAges: SaveCustomData: Data check for key 'GAP1T7' equals '10,2.2,10,0'
GoldenAges: SaveCustomData: Save completed


LOADING TURN 4
LoadScreen: OnLoadGameViewStateDone
GoldenAges: LoadCustomData: Loading data from GameConfiguration...
GoldenAges: LoadCustomData: Nothing to load for key 'GAP0T4'
GoldenAges: LoadCustomData: Nothing to load for key 'GAP1T4'


LOADING TURN 7
LoadScreen: OnLoadGameViewStateDone
GoldenAges: LoadCustomData: Loading data from GameConfiguration...
GoldenAges: LoadCustomData: Nothing to load for key 'GAP0T7'
GoldenAges: LoadCustomData: Nothing to load for key 'GAP1T7'


LOADING TURN 7 AFTER RESTART
LoadScreen: OnLoadGameViewStateDone
GoldenAges: LoadCustomData: Loading data from GameConfiguration...
GoldenAges: LoadCustomData: Nothing to load for key 'GAP0T7'
GoldenAges: LoadCustomData: Nothing to load for key 'GAP1T7'

As you can see nothing is being loaded when the save game is loaded now. It seems that SetValue\GetValue is very unreliable for persistence.
Will have to find an alternative, perhaps creating an additional save file for the game using the io object in lua.
 
greyTiger said:
It seems if I only try to save a single string value then the data gets loaded when the game is loaded. e.g

On Turn 4 I saved the text 'miles' using GameConfiguration.SetValue(). I then immediately read it back using GameConfiguration.GetValue() and got back the same value.
Code: 
 GoldenAges: SaveCustomData: Saved data 'miles' to key 'GATEST'
 GoldenAges: SaveCustomData: Data check for key 'GATEST' equals 'miles'

I then closed the game and restarted Civ6. Chose the save file and loaded it.
Code: 
 LoadScreen: OnLoadGameViewStateDone
 GoldenAges: LoadCustomData: Data found for key 'GATEST' equals 'miles'

The correct data was loaded. I am guessing that there is a limit to the amount of information that can be saved\loaded in the GameConfiguration. I'll have to continue with some tests to see what the limit is.
Click to expand...
The amount is not a problem, I had serialized strings of more than 1 million chars in my test (the serialization time beginning to be an issue at this point)
 
Have you tried saving on multiple turns with your serialization routine? If so what are your results?
 
I had a problem with the serializer I use and table using playerID as key, like {[0] = 1, [1] = 2} which were reloaded as {[1] = 1, [2] = 2}, so now I always use string for key ({["0"] = 1, ["1"] = 2}) but no other issues detected so far, saved data is reloaded.
 
Okay. Thanks for the reply. Just curious, at what point do you persist your data? I just had a thought that perhaps my data is being persisted too late and thus getting lost.
 
greyTiger said:
Okay. Thanks for the reply. Just curious, at what point do you persist your data? I just had a thought that perhaps my data is being persisted too late and thus getting lost.
Click to expand...

https://github.com/Gedemon/Civ6-GCO/blob/master/Scripts/SaveLoad.lua
Code: 
----------------------------------------------
-- Events for saving
----------------------------------------------
-- This Lua event is called when listing files on the save/load menu
function SaveMyTables()
    print("Calling LuaEvents.SaveTables() on FileListQueryComplete...")
    LuaEvents.SaveTables()
end
LuaEvents.FileListQueryComplete.Add( SaveMyTables )

-- This should happen just before the autosave
function SaveOnBarbarianTurnEnd(playerID)
    local player = Players[playerID]
    if player:IsBarbarian() then
        print("Calling LuaEvents.SaveTables() on Barbarian Turn End...")
        LuaEvents.SaveTables()
    end
end
Events.RemotePlayerTurnEnd.Add( SaveOnBarbarianTurnEnd )


-- This event to handle quick saving
function OnInputAction( actionID )
    if actionID == Input.GetActionId("QuickSave") then
        print("Calling LuaEvents.SaveTables() on QuickSave action...")
        LuaEvents.SaveTables()
    end
end
Events.InputActionTriggered.Add( OnInputAction )

and for example https://github.com/Gedemon/Civ6-GCO/blob/master/Scripts/CityScript.lua
Code: 
local GCO = {}
function InitializeUtilityFunctions()
   GCO = ExposedMembers.GCO -- contains functions from other contexts
end
LuaEvents.InitializeGCO.Add( InitializeUtilityFunctions )

function PostInitialize() -- everything that may require other context to be loaded first
   ExposedMembers.CityData = GCO.LoadTableFromSlot("CityData") or {}
end

function SaveTables()
    GCO.SaveTableToSlot(ExposedMembers.CityData, "CityData")
end
LuaEvents.SaveTables.Add(SaveTables)
 
I think that the timing of my use of SetValue may indeed be the issue. Have changed my code to update the GameConfiguration using SetValue just after I have calculated the player custom data at the end of the turn.
Will see what effect that has. Thanks for the help Gedemon. Very much appreciated.
 
The issue was indeed my timing of my saves. Currently able to use GameConfiguration SetValue\GetValue to persist my custom data.
Once again thanks for the help. Just need to figure out another two issues then I may have my new Golden Age mechanism functional :)
 
Where are these function available: Serialize and Deserialize? Is it Serialize.lua or some other script?
 
Infixo said:
Where are these function available: Serialize and Deserialize? Is it Serialize.lua or some other script?
Click to expand...
yes, it's in the Serialize.lua, I'm finally using this one after some tests:
https://github.com/fab13n/metalua
https://github.com/fab13n/metalua/blob/no-dll/src/lib/serialize.lua

The game fail to save the string some others (maybe faster) were producing.

There are also some serialize functions (serialize.persist, serialize.unpersist, ...) available in the civ6 lua (from havokscript ?) but I have no idea on how to use them (they may also require write permission to an external file, which we don't have in our scripts AFAIK)
 
Thanks!
A question: AFAIK there also SetValue/GetValue for PlayerConfiguration and MapConfiguration. I suppose Serialize/Deserialize will work with them too?
 
I imagine that would be the case though I have not tested it.
 
You must log in or register to reply here.

Similar threads

D
Custom properties on objects
Replies
2
Views
433
DeckerdJames
D
B
How to persist values between "game" and "shell" contexts
Replies
8
Views
585
LilBudyWizer
LilBudyWizer
W
How to persist values across saves
2 3
Replies
59
Views
2K
DeckerdJames
D
B
Using python to interface with Civ 5 using the Firetuner port
Replies
0
Views
210
Battary
B
B
Using python to interface with civ 5 using the FireTuner Port?
Replies
0
Views
198
Battary
B
Back
Top Bottom