1. We have added a Gift Upgrades feature that allows you to gift an account upgrade to another member, just in time for the holiday season. You can see the gift option when going to the Account Upgrades screen, or on any user profile screen.
    Dismiss Notice

What can store ScriptData?

Discussion in 'Civ4 - SDK/Python' started by deanej, Aug 22, 2010.

  1. deanej

    deanej Deity

    Joined:
    Apr 8, 2006
    Messages:
    4,859
    Location:
    New York State
    For a couple Star Trek scenarios I need to store data across save games concerning scripted events. I was hoping to use ScriptData, but I can't find anywhere to put the data on. I can't use units for obvious reasons. Plots are out of the question as well because solar system data is stored on them. Some AI info is on players, which sucks because they seem to be the only suitable object to store my data on. The game object is used as well, but even if it wasn't, I don't know anything about pickling so I wouldn't be able to use it. I tried using teams but they don't take ScriptData. Help would really be appreciated!
     
  2. General Tso

    General Tso Panzer General

    Joined:
    Oct 12, 2007
    Messages:
    1,543
    Location:
    U. S. of A.
    As far as I know there are 5 objects that use script data. The Game, Player, Plot, City, and Unit objects. The best way (in my opinion) to store script data is using a dictionary. That way you can store multiple items in one object and the only two constraints are memory usage and giving each saved item a different name (key).

    If you are interested here is the code that I created for my mod.

    Code:
    # This sets the value of an variable within an objects scriptData.
    def SetScriptVar(obj, name, value):
    	data = GetScriptData(obj)
    	data[ name ] = value
    	SetScriptData(obj, data)
    
    # This returns a variables value from the objects' scriptData. If the variable doesn't exist then default is returned.
    def GetScriptVar(obj, name, default = None):
    	data = GetScriptData(obj)
    	return data.get(name, default)
    
    def ClearScriptData(obj):
    	SetScriptData(obj, { })
    
    # This stores the scriptData dictionary for this object.
    def SetScriptData(obj, data):
    	obj.setScriptData(pickle.dumps(data, PG_PICKLE_PROTOCOL))
    
    # This returns the scriptData dictionary for this object - if it doesn't exist then an empty dictionary is returned.
    def GetScriptData(obj):
    	pickleData = obj.getScriptData()
    
    	if pickleData:
    		return pickle.loads(pickleData)
    	else:
    		return { }
    You just need to use SetScriptVar and GetScriptVar and give every variable a different name.

    PG_PICKLE_PROTOCOL is currently set to 0 because I was having problems with the other protocol values. 0 is the safest (but lowest performance) value to use.
     
  3. deanej

    deanej Deity

    Joined:
    Apr 8, 2006
    Messages:
    4,859
    Location:
    New York State
    Too bad I know nothing about dictionaries. Any idea on what I'd store it on anyways? Everything is either used already and/or shouldn't be used. I didn't think of cities, but they're kinda inconvenient to access, and I don't want to take the risk of "what happens if the city is razed?". Ditto for units, but worse. Everything else is already used by the Final Frontier code.
     
  4. Baldyr

    Baldyr "Hit It"

    Joined:
    Dec 5, 2009
    Messages:
    5,530
    Location:
    Sweden
    What is this whining about not knowing stuff? "I don't know anything about pickling!" "I don't know anything about dictionaries!" Do you know anything about programming? Why don't you?

    Geez, you would do yourself a big favor if you just looked into these things. I bet you waste that amount of time every single time you try your hand at programming, because you haven't bothered to learn any of the basics.

    And this is someone else's problem? The General has already solved your "problem" IMHO - why not use his solution then? He even supplied you with the actual code you need! So you don't really even have to learn anything - because that would suck big time, right? :rolleyes:

    If this offends you, or anyone else, I guess the moderators can just delete this post. Man, I'm actually offended by this thread!
     
  5. davidlallen

    davidlallen Deity

    Joined:
    Apr 28, 2008
    Messages:
    4,743
    Location:
    California
    Somebody seems cranky today.

    The game object is a good place to store this. In general, if an object is already used for something, it may take more effort to understand, but it is still possible. A dictionary is an associative array, which you can probably learn about from a python source if you choose to. Pickling is a way of automatically storing a dictionary as a string. But, if you want, you can also store your own data as a string. This is simpler.
     
  6. Afforess

    Afforess The White Wizard

    Joined:
    Jul 31, 2007
    Messages:
    12,239
    Location:
    Austin, Texas
    Made my day. *High Fives* :lol:
     
  7. davidlallen

    davidlallen Deity

    Joined:
    Apr 28, 2008
    Messages:
    4,743
    Location:
    California
    Somebody else seems cranky today.

    The problem that deanej is relating is not trivial to solve even for experienced programmers. He points out that inside the huge Final Frontiers code, other people have already put data onto the objects he might use. So it is not just a matter of slapping a dictionary onto the game scriptdata; he has to puzzle out how the other parts of the code are using that, and then mix something new into it without breaking the old functionality.

    Granted, some learning about dictionaries is probably needed to puzzle out how the game scriptdata object is already used; but I think this can be done in a constructive way.

    If you are interested to help, you can also find the Final Frontiers code and help to understand how the game scriptdata object is being used.
     
  8. Afforess

    Afforess The White Wizard

    Joined:
    Jul 31, 2007
    Messages:
    12,239
    Location:
    Austin, Texas
    I wasn't condoning Baldyr's attitude, I just thought it was funny. ;)
     
  9. primordial stew

    primordial stew Emperor

    Joined:
    Sep 1, 2005
    Messages:
    1,219
    Location:
    a puddle
    How about just expand the number of civ slots and use them only for storing the new data?
     
  10. deanej

    deanej Deity

    Joined:
    Apr 8, 2006
    Messages:
    4,859
    Location:
    New York State
    That is one thing I haven't checked. Neither of the scenarios where I'm storing data goes that close to the max civ amount. I could also try plots, though in that case I need to get the game to not attempt to load solar systems from them.

    @Baldyr: I'm not sure how you can consider dictionaries and pickling "basics", given that I've been doing civ modding in python for years now and never went anywhere near those topics. All this time I've only been using strings with ScriptData; this was my first attempt to use anything else (integers) and only by attempting to convert ScriptData into an integer did I find out that the player ScriptData was already used in the first place!
     
  11. TC01

    TC01 Deity

    Joined:
    Jun 28, 2009
    Messages:
    2,216
    Location:
    Irregularly Online
    You're going to have to use an array, or a dictionary (and therefore pickling) for this unless you want to use a unit or a city object. Which you could do if you had code in onUnitKilled that transferred it's script data object to another unit... but that's a little ridiculous. It would be much easier to add data to the existing script data.

    The easiest object to add additional stuff to would be the game script data. That contains the FF tutorial (an array of booleans as to whether or not a tutorial event has already fired).

    Without using dictionaries, I'd split the game script data into pieces

    Code:
    	def onPreSave(self, argsList):
    		"called before a game is actually saved"
    		self.parent.onPreSave(self, argsList)
    		
    		printd("Calling onPreSave")
    		
    		self.saveSystemsToPlots()
    		AI.doSavePlayerAIInfos()
    		
    		[COLOR="red"]gameData = []
    		gameData.append(Tutorial.saveData())
    		gameData.append(self.getDS9Data())
    		CyGame().setScriptData(pickle.dumps(gameData))[/COLOR]
    
    	#	CyGame().setScriptData(pickle.dumps(Tutorial.saveData()))
    			
    	def onLoadGame(self, argsList):
    		self.parent.onLoadGame(self, argsList)
    		
    		self.iWinningTeam = -1
    		self.iTimeLeft = 0
    		
    		self.initValues()
    		
    		CyGame().makeNukesValid(true)
    		
    		self.loadSystemsFromPlots()
    		AI.doLoadPlayerAIInfos()
    		
    		[COLOR="Red"]gameData = pickle.loads(CyGame().getScriptData())
    		Tutorial.loadData(gameData[0])
    		self.setDS9Data(gameData[1][/COLOR]
    
    	#	Tutorial.loadData(pickle.loads(CyGame().getScriptData()))
    		
    		printd("Loading game, initing score, updating it, then setting it dirty")
    		self.initScoreStuff()
    		CyGame().updateScore(true)
    		CyInterface().setDirty(InterfaceDirtyBits.Score_DIRTY_BIT, True)
    
    I made up the getDS9Data() and setDS9Data() functions. getDS9Data() might look like this:

    Code:
    	def getDS9Data(self):
    		data = []
    		data.append(self.DS9Data1)
    		data.append(self.DS9Data2)
    		data.append(self.DS9Data3)
    		#...
    		return data
    
    And setDS9Data() might look like this:

    Code:
    	def setDS9Data(self, data):
    		self.DS9Data1 = data[0]
    		self.DS9Data2 = data[1]
    		self.DS9Data3 = data[2]
    		#...
    
     
  12. Baldyr

    Baldyr "Hit It"

    Joined:
    Dec 5, 2009
    Messages:
    5,530
    Location:
    Sweden
    deanej, firstly I feel like I want to apologize for my terrible morning mood. Perhaps I shouldn't turn on the computer at all before work? Sorry about the outburst, anyway. :rolleyes: I actually felt bad afterwards, so I'm disappointed with myself. Hopefully this counts for something.

    But I do see where you're coming from; You've basically learned to do Python by looking at sample code and doing it, and you probably feel like you're pretty successful with it. But still, pickling is like not even a chapter of its own in a entry-level textbook on Python. And not knowing about dictionaries is... like not knowing what the parking brake handle is on your car. You might not be using it, much, but surely you should know what its good for? Dictionaries are also covered in any entry level textbook and its just as easy as anything else with Python. (You probably just need to use them once to get the feel for these data structures.)

    What you are basically saying is that you've run out of places to store script data. And you probably already knew that pickling a dictionary is the way to get more data into the same object? But you can't be bothered with it - just because you never had to before? - so you're posting here requesting new places to put stuff. Sorry mate, you're out of room. (I presume that you already searched the API for "setScriptData".) Its time to start economizing your storage locations.

    If you ever wondered whether or not you can get away with never having to learn about these "advanced" topics (for someone with years of practical experience - are you kidding me?) - then now you know. Because you've reached the point where you can't manage without these tools.

    The good news is that you can pretty much learn the basics in an hour or so. Are you up for it, are you just gonna rely on all that experience you have instead? Any option is fine with me, of course.
     
  13. Baldyr

    Baldyr "Hit It"

    Joined:
    Dec 5, 2009
    Messages:
    5,530
    Location:
    Sweden
    davidlallen has a valid point, and that is the fact that the mod itself is suboptimal from a custom value storage point-of-view. I doubt anyone is prepared to fix that mess, even if it probably should be done once-and-for-all. Even though I can't help out with that (it does sound like a monumental task) I can share some of the "mysteries" of dictionaries and pickling.

    Ok, dictionaries: Its a data structure like a list - or even a string. So it stores data, but not in a sequence (like a list or a string) but it rather works with keys. So this is a empty dictionary:
    Code:
    scriptDict = {}
    You add a key (some value) and its associated content (another value) with:
    Code:
    scriptDict['customValue'] = 3
    Now this is what the dictionary looks like:
    Code:
    { 'customValue': 3 }
    You can add any number of key-values pairs to a dictionary:
    Code:
    scriptDict = { 'customValue1': True, 'customValue2': None, 'customValue3': 42, [COLOR="Red"](0, 1)[/COLOR]: 99 }
    The last key is actually a tuple data structure!

    You access the data from the dictionary by indexing it, just as with lists and strings (which would surely count as "basic" Python, right?):
    Code:
    scriptDict['custom value']
    There is of course a whole lot more to know about how to use dictionaries, but this would be the bare basics. Advanced stuff, eh? If you feel up to it there is more to read on dictionaries here (first five sections).

    And pickling is just a way to covert some type or data structure into a valid string value. Its not more "advanced" than that. What you need to do however is import the pickle module:
    Code:
    import pickle
    Or you could just go for the faster C++ version called cPickle. The choice is yours.

    So, to turn any value into a string (to "pickle" it) you use the dumps() function:
    Code:
    pickledScriptDict = pickle.dumps(scriptDict)
    Then you use setScriptData() to store the string value. And to retrieve the dictionary you of course use getScriptData() and use the loads() function:
    Code:
    scriptDict = pickle.loads(pickledScriptDict)
    Thats about it. Any questions?
     
  14. General Tso

    General Tso Panzer General

    Joined:
    Oct 12, 2007
    Messages:
    1,543
    Location:
    U. S. of A.
    If you add the code I listed above to your mod then replace all setScriptData calls with calls to setScriptVar and all getScriptData calls with getScriptVar you can do what you want without dealing with pickling and dictionaries.

    For example, if you have a variable called iMyData and you want to save it as script data to the game object just do this.
    setScriptVar(gc.getGame(), "MyData", iMyData)

    And to retrieve it do this.
    iMyData = getScriptVar(gc.getGame(), "MyData")

    The name "MyData" can be anything you want it to be as long as it's Python legal and you use the same value when saving and retrieving a given variable. You must also use different names for different variables. As mentioned above, if you use this system you need to use it for all script data or else the dictionary used by this code will erase the other data. That's all there is to it - it should be pretty simple. Just use an individual name for each variable and use this code for all script data.
     
  15. Baldyr

    Baldyr "Hit It"

    Joined:
    Dec 5, 2009
    Messages:
    5,530
    Location:
    Sweden
    General Tso, we have to consider the possibility that deanej doesn't even know how to use functions... Because otherwise he would already be using your solution, right? (That or he isn't even reading the replies he's getting. I believe I just developed a twitch. :huh:)
     
  16. davidlallen

    davidlallen Deity

    Joined:
    Apr 28, 2008
    Messages:
    4,743
    Location:
    California
    None of us posting on this thread except TC01 and deanej are familiar with the FF code. I just know it is really complex. Deanej has stated that he has used scriptdata as strings before, so it is safe to assume he can use functions! The key point is to analyze how FF already uses game scriptdata, and find a way to add new stuff around that. This is what TC01 has posted about.
     
  17. Baldyr

    Baldyr "Hit It"

    Joined:
    Dec 5, 2009
    Messages:
    5,530
    Location:
    Sweden
    Then I guess deanej needs to read what TC01 posted. I'd say that is the complex option, but to each their own. All the options are pretty much on the table, sans using sdToolkit i suppose. But that would really only make sense if all the other stored values are converted to this setup, right?
     
  18. TC01

    TC01 Deity

    Joined:
    Jun 28, 2009
    Messages:
    2,216
    Location:
    Irregularly Online
    Here is a brief summary of the mess that is Final Frontier's script data system, if it helps:

    Final Frontier Plus stores three different kinds of script data. The game tutorial data, player AI data, and solar system data. It is all actually pickled in onPreSave and onLoadGame.

    Each data type has a class associated with it. And each class has a "getData()" function that creates an array and returns it, and a "setData()" function that accepts an array and then continually assigns data from it to a class variable and increments it.

    Since there is only one CyGame object, the game tutorial class's getData and setData functions are called directly from onPreSave and onLoadGame. (Therefore, I used it in my example).

    Solar system data and player AI data, since it is per plot and per player respectively, is assigned in helper functions called from onPreSave and onLoadGame.


    The best solution, from my point of view, would be to add additional script-data style variables (strings that do nothing yet can be accessed and set from Python) to the DLL, and replace Final Frontier's current usage of script data with them. This would allow any modder, including ones who have learned what they know of programming through modding only, to use script data without needing to implement a solution. I am now planning to do this for Final Frontier Plus.
     
  19. Baldyr

    Baldyr "Hit It"

    Joined:
    Dec 5, 2009
    Messages:
    5,530
    Location:
    Sweden
    TC01 for president! :king:
     
  20. deanej

    deanej Deity

    Joined:
    Apr 8, 2006
    Messages:
    4,859
    Location:
    New York State
    Actually I've kinda been away from my computer from 10-3.

    I think I'll go with TC01's code or stick a dictionary on plot (0,0); I already know none of the scenarios use that for a solar system and I can probably make the game skip loading systems from it.

    As for not knowing about dictionaries (other than being aware that they exist), that has to do with the fact that even though I've done stuff in python for civ for years now, most of my programming is in C++, so even when programming in python, I still approaching from the perspective of a C++ programmer.
     

Share This Page