Assistance with Pickle

[smeagolheart]

Question, anyone know stuff about pickling?

I don't

I'm trying to get the great doctor mod merged into my main mod and got this error:

Traceback (most recent call last):
  File "BugEventManager", line 361, in _handleDefaultEvent
  File "DestinyCvEventManager", line 1142, in onLoadGame
  File "DestinyCvEventManager", line 1772, in getGodsVariables
  File "e:/main/civilization4/warlords/assets/python/system\pickle.py", line 1394, in loads
  File "e:/main/civilization4/warlords/assets/python/system\pickle.py", line 872, in load
  File "e:/main/civilization4/warlords/assets/python/system\pickle.py", line 894, in load_eof
EOFError

My e drive is my cd drive, I'm not sure where it's getting that stuff (and mod is based off BTS) though I guess the great doc is based of godsofold which was in warlords maybe?

The lines that are being to referred to are this:

	def onLoadGame(self, argsList):
		CvAdvisorUtils.resetNoLiberateCities()
###########  Great Doctor Mod by Tsentom1 updated by Smeagolheart Start  ###########  						
		self.getGodsVariables()    ## 1142
###########  Great Doctor Mod by Tsentom1 updated by Smeagolheart End  ###########  						
		return 0

and

	def getGodsVariables(self):
		
		pPlayer = gc.getPlayer(0)
		lPlagueCityCoord = [ ]
		
		# Load Script Data - Score
		szScriptData = pickle.loads(pPlayer.getScriptData())  ## 1772
		
		self.iReligiousVictoryPossible = szScriptData[0]
		self.iStarAlignment = szScriptData[1]
		self.lPlagueCities = szScriptData[2]
		
		return 
###########  Great Doctor Mod by Tsentom1 updated by Smeagolheart End ###########

 

[TC01]

Well, what appears to be happening is that the pickle module is failing to convert the player's script data from a pickled string into an array.

Are you sure you have pickled the array in the first place? If you're not, I would suggest you print out the player's script data, like so:

	def getGodsVariables(self):
		
		pPlayer = gc.getPlayer(0)
		lPlagueCityCoord = [ ]
		
		# Load Script Data - Score
		[COLOR="Red"]print ("Player 0 Script Data = %s" % (pPlayer.getScriptData()))[/COLOR]
		szScriptData = pickle.loads(pPlayer.getScriptData())  ## 1772
		
		self.iReligiousVictoryPossible = szScriptData[0]
		self.iStarAlignment = szScriptData[1]
		self.lPlagueCities = szScriptData[2]
		
		return 
###########  Great Doctor Mod by Tsentom1 updated by Smeagolheart End ###########

It should be a lot of nonsense.

 

[smeagolheart]

I didn't pickle anything myself, I don't just copied the code from the great doctor mod. Hmmm, well you make it sound like I missed something then.

 

[Baldyr]

I don't have all the answers for you, but I have learned some pickling recently. What is happening here is that each player or Civ seems to have a string value attached to them as "scriptData". Its similar to giving each player a name or a custom value of some sort. But if you pickle any data (even whole dictionaries or class instances) first it is converted into a string of letters. And can thus be stored as "scriptData" - in this case the string value you attach to the player.

So, the mod is storing "God Variables" as picked data in each player instance. But you wouldn't be able to fetch (pickle.loads) any data that hasn't been stored in the first place, right?

I realize that your merging code without understanding everything that it does... So good luck with it!

You would do yourself a big favor by just learning some Python, not?

 

[Baldyr]

I didn't pickle anything myself, I don't just copied the code from the great doctor mod. Hmmm, well you make it sound like I missed something then.

Yeah, the code responsible for pickling would be something along the lines of:

pickle.dumps()

This takes any value and turns it into a string.

 

[smeagolheart]

Any idea why the path (from error msg) is pointing to something that doesn't exist? And from warlords as well, even though I'm running the mod in BTS?

 

[Baldyr]

Any idea why the path (from error msg) is pointing to something that doesn't exist? And from warlords as well, even though I'm running the mod in BTS?

No, no idea. Do you have the game disc in the CD drive?

I do however think that CivIV has become quite modular over time, where some stuff was added with Warlords and yet more stuff was added with BTS. All of this isn't duplicated everywhere so the game will look for stuff in all these versions.

Perhaps the game thinks it needs to get something from your game disc that it isn't finding in your game install folder? It does look like it thinks that the pickled data is in drive E...

 

[smeagolheart]

How do it tell it where to pickle then?

	def clearGodsVariables( self ):
		
		self.iReligiousVictoryPossible = 0
		self.iStarAlignment = 0
		self.lPlagueCities = [ ]
		#self.lTsunamiCoor = [ ]


	def setGodsVariables(self):
		pPlayer = gc.getPlayer(0)
		
		szScriptData = []

		szScriptData.append(self.iReligiousVictoryPossible)
		szScriptData.append(self.iStarAlignment)
		szScriptData.append( self.lPlagueCities )	
		
		print szScriptData

		# Save Script Data - Score
		pPlayer.setScriptData(pickle.dumps(szScriptData))

I commented out the tsunami bit since it doesn't seem to do anything and maybe it was causing problems since it's not read back. Anyway, that's where it writes it, it seems.

 

[Baldyr]

The code you posted does exactly what I expected it to.

	def clearGodsVariables( self ):

Somewhere in the mod there is an instance of

self.clearGodsVariables()

This is a method invocation and executes the assignment statements following the def-inition line.

		self.iReligiousVictoryPossible = 0
		self.iStarAlignment = 0
		self.lPlagueCities = [ ]
		#self.lTsunamiCoor = [ ]

So, you're learning Python already!

	def setGodsVariables(self):

This other method would be invoked by the code:

self.setGodVariables()

And the line-by-line breakdown of the method's content:

		pPlayer = gc.getPlayer(0)

This is another assignment statement and it assigns whatever the CyGlobalContext.getPlayer() method returns to the variable pPlayer. (Because functions/methods always return something, and this particular method returns a reference to a CyPlayer instance.) The value 0 (zero) refers to the first player in the game and this means that the pPlayer variable now points to the first CyPlayer instance in the game.

I interpret this as the mod is using the first player as a place to store data - so that it gets saved with the game - and loaded again with the game. This is a handy trick, and I'm assuming that the game instance itself is already being used for some other scriptData.

		szScriptData = []

Another assignment statement, and this time szScriptData is an empty list variable. (The lonesome brackets indicate that there is no actual content, yet.)

		szScriptData.append(self.iReligiousVictoryPossible)
		szScriptData.append(self.iStarAlignment)
		szScriptData.append( self.lPlagueCities )

Now the plot thickens, as we're revising the variables that were assigned with values in the first method, remember? Now these values are being appended as list entries to the list created in the previous line (by setting it to "empty"). So the szScriptData will contain three list entries, then.

I guess that the first time this happens its the default values that end up in the list. If the clearGodsVariables() method hasn't been invoked this wouldn't be possible though. As the Python interpreter wouldn't know what those variables were.

		print szScriptData

This line prints the content - all the three values - into the Python Debug Log. It has no bearing on the game though.

		pPlayer.setScriptData(pickle.dumps(szScriptData))

And finally, the pickling. The method setScriptData() is used to store scriptData in objects, like an instance of the CyPlayer class. And you do remember that pPlayer is pointing to one such object, right?

So, some scriptData will be stored in the first player object then, but which scriptData? Well, the scriptData that is pickled with the pickle.dumps() method, of course, turning it into a valid string that can pass as scriptData. And as you've already deducted, the scriptData in question is none-other than the hero of this tale - the list also known as szScriptData.

End of story. School is out.

 

[Baldyr]

What ever happened with this particular issue, by the way? I gather you managed to overcome the pickling issue - what turned out to be the problem then?

 

[smeagolheart]

I'm not getting this exception anymore, I don't know why I was getting it then either, there is nothing I can see that even refers to an E: drive in any of my python and I have nothing in my e: drive (a dvd drive)

Since I had the error, I started over from scratch and am up to the point from the other mod.

  File "e:/main/civilization4/warlords/assets/python/system\pickle.py", line 1394, in loads
  File "e:/main/civilization4/warlords/assets/python/system\pickle.py", line 872, in load
  File "e:/main/civilization4/warlords/assets/python/system\pickle.py", line 894, in load_eof
EOFError

 

[EmperorFool]

Those most likely refer to compiled Python modules from when the developers built their Python integration with Civ4. The modules had their paths on the developer's machine compiled into them, and you can ignore them.

 

[Merkava120]

I know this is bumping a super old thread. But just in case anyone else gets the weird "pickle.py" errors, here's how I fixed it.

If you notice before the "pickle.py" errors show up, there are other python pop-ups (and in my case XML too). My guess is that one of those python errors is messing up the scriptdata for your mod and causing the pickle.py errors (which then appear on the screen 100,000 times and you have to close the game to get rid of them).

To catch those original errors, grab your phone and get the slow-mo camera ready. Hit "record" then immediately hit "launch game" (really quickly because slow-mo might have a timer). Film until the 100,000 "pickle.py" errors show up. Then exit the game.

Go back through your film and find the pop-ups that appear before the pickle.py pop-ups; that's where the real problem is. For me, it was a misnamed list in a python file (lSubtropicalWorlds referring to lSubTropicalWorlds). I'm not sure how, but that error (or possibly several errors) is having a cascading effect giving you tons of pickle.py nonsense.

You may have to do that several times; for me the pickle.py errors went away as soon as I fixed that list error.