pickling (loading/unloading) game data

vicawoo

Chieftain
Joined
Feb 12, 2007
Messages
3,226
pickle.loads(loadDataStr)

I'm trying to unpickle Civ 4 BTS save files, that is, extract it from a data file into the game object.

I'm assuming you can use pickle.loads to load save files, because in CvAppInterface.py, it uses

Code:
def onLoad(argsList):
	'Called when a file is loaded'
	import pickle	
	import CvEventInterface
	loadDataStr=argsList[0]	
	if len(loadDataStr):
		CvEventInterface.onEvent( ('OnLoad',pickle.loads(loadDataStr),0,0,0,0,0 ) )

and correspondingly it stores data using

Code:
def onSave():
	'Here is your chance to save data.  This function should return a string'
	import CvWBDesc
	import pickle	
	import CvEventInterface
	# if the tutorial is active, it will save out the Shown Messages list
	saveDataStr = pickle.dumps( CvEventInterface.onEvent( ('OnSave',0,0,0,0,0 ) ) )
	return saveDataStr

Since I have no idea what I'm doing, I copied this pickling code from docs.python.org

Code:
import pprint, pickle

pkl_file = open('datasave', 'r')

data1 = pickle.loads(pkl_file)
pprint.pprint(data1)

data2 = pickle.load(pkl_file)
pprint.pprint(data2)

pkl_file.close()

where I renamed a BTS save file datasave

and I get an error

Code:
Traceback (most recent call last):
  File "C:\Python24\pickle_test.py", line 5, in -toplevel-
    data1 = pickle.loads(pkl_file)
  File "C:\Python24\lib\pickle.py", line 1393, in loads
    file = StringIO(str)
TypeError: expected read buffer, file found

Certainly there's a lot that could have gone wrong. For now, I want to load the save in the correct format.
pickle.loads(loadDataStr), it seems so close to unravelling the BTS save format. Help?
 
You must read the contents of the file into a string and pass that string to pickle.loads(). However, before you go too far down this road understand that Civ4 doesn't pickle the game itself--only a data string that the preLoad() event produces. This pickled string gets written into the save game file along with the game itself.

All of the loading/saving of games is controlled by the DLL. Most objects have a pair of read()/write() functions that work on byte streams attached to files by the EXE.
 
I have a SDK/pickle question and this looks like a good place to ask it. I store scriptdata for various items in mod in Python. The data is organized as a dictionary then pickled or unpickled as it is set to and from scriptdata. Can I access the dictionary (via scriptdata) from within the SDK? If so what do I need to do to unpickle it then read a Python dictionary in C++?
 
I have a SDK/pickle question and this looks like a good place to ask it. I store scriptdata for various items in mod in Python. The data is organized as a dictionary then pickled or unpickled as it is set to and from scriptdata. Can I access the dictionary (via scriptdata) from within the SDK? If so what do I need to do to unpickle it then read a Python dictionary in C++?

Well, you can access script data from the SDK. But I don't know if you can unpickle it.

But why use script data if you're working in the SDK? Just create new variables for whatever it is you want to store (so if you're storing an array of integers for cities, create a new array of integers in CvCity), expose it to python, and access it the normal way.
 
Yea, I guess your right. I should spend the time and move it to the SDK.
 
I agree with TC01. Once you get the hang of it you'll be adding stuff to the SDK quickly. There are probably pickle libraries for C++ to get at the data, but I'd still go the route described above.

The other option is to expose a Python function for the DLL to call to get the value. This is fine for things that control hover text (BULL uses this to get user settings from BUG), but it's definitely not recommended for anything happening in game logic.
 
cpickle>pickle
 
AFAIK cpickle is the pickle library for Python implemented in C so that it runs faster. It still produces Python objects for Python to consume. Are you saying that it also provides a way for C++ to access those objects (e.g. Python's dictionary)?
 
Thanks for all the info. Just out of curiosity is it possible to access pickled data from C++. I don't need to do it at the present. I was just wondering if it's possibe, so it's no big deal. I understand what cPickle is and how it works in Python, so there is no need to expand on that, unless it can be used in C++ somehow.
 
Top Bottom