[PYTHON] SD ToolKit

Stone-D

Chieftain
Joined
Dec 4, 2005
Messages
68
Code:
SD-ToolKit 1.22
by Stone-D ( Laga Mahesa )
For Civilization IV
Compatibility : 1.00+
Data Source   : 1.09
New Game Req  : N/A

Well, for a start, it isn't something for a regular player to use, so if you're not a python modder, stop now.

During my python escapades, I've had cause to make a small number of global functions that I believe others will benefit from, especially end users. This is NOT a standalone mod. It is a python module that can be incorporated into other projects. The major function of this library allows modders to store data without conflicting with anything else.

I felt that this was something that needed to be addressed sooner or later.

The following is a brief API :


Basic Utilities
===============
sdEcho( string )
Outputs 'string' to the python debug log AND onto the screen.
Either output mode can be toggled using two boolean variables inside the function definition.

sdGetTimeInt( turn_int )
Converts a year string of the form "1234 AD" or "1234 BC" to an integer. BC is negative.

sdGameYearsInt()
Returns the total span of years a given game should last, irrespective of victory conditions.
Standard Firaxis gamespeeds cover exactly 6,050 total years, but others do not - for example, my SD-Glacial mod covers 6,200 years.


Data Storage
============
Internal Functions :
sdModInit( 'MyModName_string' )
Initializes a central reservoir of custom variables for your mod's use.
For internal use. You should not need to actively use this function.

sdModFixCase( 'MyModName_string', Mod_dictionary )
For internal use. You should not use this function.

sdModLoad( 'MyModName_string' )
Loads previously initialized data from the central reservoir.
For internal use. You should not need to actively use this function.

sdModSave( 'MyModName_string', Mod_dictionary )
Saves a mod's entire variable data to the central reservoir.
For internal use. You should not need to actively use this function.

MOD Functions :
sdEntityInit( 'MyModName_string', 'UniqueName_string', Entity_dictionary )
Initializes a unique data entity (city, unit, plot).
UniqueName_string = Any string used to indentify the entity, such as 'Alcatraz' or '5435' or 'Warrior55'
Entity_dictionary = A python dictionary containing the data set for the 'Entity', such as {'var1' : 56, 'var2' : 'My Rectum', 'var3' : [1, 2, 3]}

sdEntityWipe( 'MyModName_string', 'UniqueName_string' )
Removes an entity that has been previously initialized by sdEntityInit.
An entity can be anything that contains a set of data... a city, a unit or your dog's name. Use your imagination.

sdEntityExists( 'MyModName', 'UniqueName_string' )
Checks whether or not an entity has been initialized by sdEntityInit.
Returns bool False on failure, bool True on success.

sdGetVal( 'MyModName_string', 'UniqueName_string', 'VariableName_string' )
Fetches a specific variable's value from the entity's data set.
Will raise an exception on failure.

sdSetVal( 'MyModName_string', 'UniqueName_string', 'VariableName_string', any_value )
Stores a specific variable's value within the entity's data set.
Returns bool False on failure, bool True on success.

sdDelVal( 'MyModName_string', 'UniqueName_string', 'VariableName_string' )
Removes a specific variable from the entity's data set.
Returns bool False on failure, bool True on success.

sdGetGlobal( 'MyModName_string', 'GlobalVariableName_string' )
Fetches a specific variable's value from the mod's global data set, which is automatically initialized.
Will raise an exception on failure.

sdSetGlobal( 'MyModName_string', 'GlobalVariableName_string', any_value )
Stores a specific variable's value within the mod's global data set, which is automatically initialized.

sdDelGlobal( 'MyModName_string', 'GlobalVariableName_string' )
Removes a specific variable from the mod's global data set.
Returns bool False on failure, bool True on success.


To clarify, the saved data is stored with saved games and will be restored when a game is reloaded by the user. Saved data will not clash with other mods using this toolkit.


History
=-=-=-=

v1.00 :
Initial Release.

v1.10 :
BASIC-UTILITIES Addition : sdGetTimeInt( turn )
BASIC-UTILITIES Addition : sdGameYearsInt()
SD-DATA-STORAGE Addition : sdModFixCase( 'MyModName', Mod_dictionary )
SD-DATA-STORAGE Addition : sdEntityWipe( 'MyModName', 'UniqueName_string' )
Now checks for changed case in mod names.
Changed all defs so the leading characters are lower case.

v1.20 :
Some minor cleanups and optimizations.
Added comments to the functions with examples and descriptions.
SD-DATA-STORAGE Addition : sdEntityExists( 'MyModName', 'UniqueName_string' )
SD-DATA-STORAGE Addition : sdDelVal( 'MyModName', 'UniqueName_string', 'VariableName_string' )
SD-DATA-STORAGE Addition : sdGetGlobal( 'MyModName', 'GlobalVariableName' )
SD-DATA-STORAGE Addition : sdSetGlobal( 'MyModName', 'GlobalVariableName', any_value )
SD-DATA-STORAGE Addition : sdDelGlobal( 'MyModName', 'GlobalVariableName' )
Boolean return values added to some functions. See API section.

v1.21 :
Fixed a major bug in sdDelGlobal.

v1.22 :
Fixed another bug in sdDelGlobal.
 

Attachments

  • SD-ToolKit.v1.00.zip
    2.7 KB · Views: 298
  • SD-ToolKit.v1.10.zip
    3.5 KB · Views: 287
  • SD-ToolKit.v1.22.zip
    4.8 KB · Views: 1,691
Awesome. I was looking for a way to spit out debug text to the debug log, and I was wondering how I was going to save my variables in the game save file. This seems to solve both those problems - thank you!
 
Zurai said:
Awesome. I was looking for a way to spit out debug text to the debug log, and I was wondering how I was going to save my variables in the game save file. This seems to solve both those problems - thank you!

Wait does it really save it to the game file? From the looks of the functions, it doesn't look like it does (though it may). I guess the question is what is the "central depository."

Also, spitting out debug text to the debug log is easy. Just use CvUtil.pyPrint.

If you had asked, I'm sure someone would have told you.

These tools look interesting, though.

Req
 
Zurai: Hehe, glad someone's using it!

Yes, it does save it to the game file. Some entities/methods in Civ4 have a variable called szScriptData - cvGameInterface, cvUnit, cvCity for example. I use the cvGameinterface one, the top level 'node'.

Although the code used to save the variables is very simple, I am anticipating a time when such mods will collide - pick one, you cannot mix because they overwrite that vital variable.

This module allows for multiple mods to use the same variable to store their data in a hierarchy.

Save(YourModName, YourEntityName, YourVariableName, the_value)

Save ('Mylon', 'Washington', 'TimeToLive', 558)
Save ('Requies', 'Warrior_Elite_34', 'KillCount', 12)

etc.
 
Stone-D said:
Zurai: Hehe, glad someone's using it!

Yes, it does save it to the game file. Some entities/methods in Civ4 have a variable called szScriptData - cvGameInterface, cvUnit, cvCity for example. I use the cvGameinterface one, the top level 'node'.

Although the code used to save the variables is very simple, I am anticipating a time when such mods will collide - pick one, you cannot mix because they overwrite that vital variable.

This module allows for multiple mods to use the same variable to store their data in a hierarchy.

Save(YourModName, YourEntityName, YourVariableName, the_value)

Save ('Mylon', 'Washington', 'TimeToLive', 558)
Save ('Requies', 'Warrior_Elite_34', 'KillCount', 12)

etc.

I guess I'm still kind of confused as to what this does. Do you mean you store PYTHON variables onto a CIV 4 SAVE file? Or does it save some different variable?

Can you give an example of HOW someone would use this IRL?

Req
 
Yes, python... for an example, download "SD Culture Conquest". For a quicky, here ya go :

Code:
	def sdCultureConquestCityAcquired(self, argsList):
		# Called at city acquired event.
		loser,playerType,city,bConquest,bTrade = argsList
		sdSetVal('sdCultureConquest', city.getName(), 'CaptureState', 1)
		sdSetVal('sdCultureConquest', city.getName(), 'loser', loser)
		sdSetVal('sdCultureConquest', city.getName(), 'oCulture', city.getCulture(loser))
		# Decrease this to 1 in order to avoid losing another population point due to starvation
		city.setOccupationTimer(1)
		return 0

.... blah blah....

	def sdCultureConquestEndGameTurn(self):
		# Called at the end of the turn. Odd, that, isn't it?
		for i in range(gc.getMAX_PLAYERS()):
			iPlayer = PyPlayer(i)
			if ((iPlayer.isAlive()) or (iPlayer.isBarbarian())):
				cityList = iPlayer.getCityList()
				for c in range(len(cityList)):
					city = cityList[c]
					self.sdCultureConquestValidateCity(city)
					if (sdGetVal('sdCultureConquest', city.getName(), 'CaptureState') > 0):
						CaptureCount  = sdGetVal('sdCultureConquest', city.getName(), 'CaptureCount')
						CaptureState  = sdGetVal('sdCultureConquest', city.getName(), 'CaptureState')

....blah blah....

						CaptureCount += 1
						CaptureState  = 0
						sdSetVal('sdCultureConquest', city.getName(), 'CaptureCount', CaptureCount)
						sdSetVal('sdCultureConquest', city.getName(), 'CaptureState', CaptureState)
 
I'm bumping into errors...

Code:
  File "CvEventInterface", line 25, in onEvent

  File "CvEventManager", line 164, in handleEvent

  File "CvDynExp", line 236, in onUnitCreated

  File "CvDynExp", line 383, in setupScriptData

  File "SdToolKit", line 62, in SdEntityInit

  File "SdToolKit", line 55, in SdModSave

PicklingError: Can't pickle <type 'Boost.Python.instance'>: import of module Boost.Python failed

It happens in this code:
Code:
		scriptDict =	{	'iTERRAIN_COAST': 0,
					'iTERRAIN_HILL': 0,
					'iFEATURE_FOREST': 0,
					'iFEATURE_RIVER': 0,
					'iFEATURE_JUNGLE': 0,
					'iUNITCOMBAT_ARCHER' : 0,
					'iUNITCOMBAT_MOUNTED' : 0,
					'iUNITCOMBAT_MELEE' : 0,
					'iUNITCOMBAT_SIEGE' : 0,
					'iUNITCOMBAT_GUN' : 0,
					'iUNITCOMBAT_ARMOR' : 0,
					'iCITY_ATTACK' : 0,
					'iCITY_DEFENCE' : 0,
					'iCOLLAT_DAMAGE': 0,
					'iCOMBAT' : 0,
					'iMOVEMENT' : 0,
					'iSTATIC' : 0,
					'iHEAL': 0,
					'iWITHDRAW': 0,		
					'iBLITZ': 0,
					'iRIVTER': 0			
				}
		SdEntityInit('DynExp', pUnit, scriptDict)

Am I doing something wrong? I've looked at your file, and apart from the try: and except: thingies (which I have no idea what are...) it's the same format.

EDIT: And does it work if your entity is not a dictionary? I have something which only needs one piece of information!
 
Stone-D said:
Yes, python... for an example, download "SD Culture Conquest". For a quicky, here ya go :

Code:
	def sdCultureConquestCityAcquired(self, argsList):
		# Called at city acquired event.
		loser,playerType,city,bConquest,bTrade = argsList
		sdSetVal('sdCultureConquest', city.getName(), 'CaptureState', 1)
		sdSetVal('sdCultureConquest', city.getName(), 'loser', loser)
		sdSetVal('sdCultureConquest', city.getName(), 'oCulture', city.getCulture(loser))
		# Decrease this to 1 in order to avoid losing another population point due to starvation
		city.setOccupationTimer(1)
		return 0

.... blah blah....

	def sdCultureConquestEndGameTurn(self):
		# Called at the end of the turn. Odd, that, isn't it?
		for i in range(gc.getMAX_PLAYERS()):
			iPlayer = PyPlayer(i)
			if ((iPlayer.isAlive()) or (iPlayer.isBarbarian())):
				cityList = iPlayer.getCityList()
				for c in range(len(cityList)):
					city = cityList[c]
					self.sdCultureConquestValidateCity(city)
					if (sdGetVal('sdCultureConquest', city.getName(), 'CaptureState') > 0):
						CaptureCount  = sdGetVal('sdCultureConquest', city.getName(), 'CaptureCount')
						CaptureState  = sdGetVal('sdCultureConquest', city.getName(), 'CaptureState')

....blah blah....

						CaptureCount += 1
						CaptureState  = 0
						sdSetVal('sdCultureConquest', city.getName(), 'CaptureCount', CaptureCount)
						sdSetVal('sdCultureConquest', city.getName(), 'CaptureState', CaptureState)

Hmmmm, ok. I guess it doesn't really apply to me since I don't really create mods....

Thanks for the explanation

Req
 
The Great Apple said:
I'm bumping into errors...

Code:
PicklingError: Can't pickle <type 'Boost.Python.instance'>: import of module Boost.Python failed

Hrrm. I'll play around with it shortly. Gonna upload 1.2 after posting this, but there's nothing changed which would fix that... err, I think. :)

Am I doing something wrong? I've looked at your file, and apart from the try: and except: thingies (which I have no idea what are...) it's the same format.


Try: Attempt the following code. If an error occurs, then...
Except: ... do this.

They're kind of a kludge in that code, haven't got around to putting proper checks in... but they work and don't result in annoying python errors mid-game. Good for the users.

EDIT: And does it work if your entity is not a dictionary? I have something which only needs one piece of information!

Forever? ;)

I'll add such functionality if the answer is yes, although the space saved versus the annoyance of multiple saving defs might be worth thinking about. :D
 
The Great Apple said:
I'm bumping into errors...

Code:
  File "CvDynExp", line 236, in onUnitCreated

  File "CvDynExp", line 383, in setupScriptData

  File "SdToolKit", line 62, in SdEntityInit

  File "SdToolKit", line 55, in SdModSave

PicklingError: Can't pickle <type 'Boost.Python.instance'>: import of module Boost.Python failed

It happens in this code:
Code:
		scriptDict =	{	'iTERRAIN_COAST': 0,
					'iRIVTER': 0			
				}
		SdEntityInit('DynExp', pUnit, scriptDict)

Is scriptdict being declared elsewhere, perhaps as a global?

Check out this google search, there're four results. Seems to be a pickling limitation - can't pickle base objects.

Google Results
 
Here it is:

Stone-D said:
Is scriptdict being declared elsewhere, perhaps as a global?
Nope - I did a search, and the only two instances of it were in that quote...

As for the base objects thing - I have absolutely no idea what a base object is!

EDIT: File removed
 
Got it. You were passing a CyUnit object to SdEntityInit, it has to be a string. I've fixed it, added a function that generates a unique name for every unit, inserted above setupscriptdata.

Code:
PY:Mod Data Initialized : DynExp True
PY:Cumae has grown
PY:Initializing Galley_0000352286
PY:Entity Initialized : Galley_0000352286
PY:Initializing Worker_0000360479
PY:Entity Initialized : Worker_0000360479
PY:Initializing Euclid (Great Scientist)_0000466948
PY:Entity Initialized : Euclid (Great Scientist)_0000466948
PY:Initializing Galley_0001105965
PY:Entity Initialized : Galley_0001105965
PY:Athens has grown
PY:City Athens's culture has expanded
PY:Initializing Galley_0001056799
PY:Entity Initialized : Galley_0001056799
PY:Corinth has grown
PY:Initializing Archer_0001064978
PY:Entity Initialized : Archer_0001064978
PY:Initializing Spearman_0001040406
PY:Entity Initialized : Spearman_0001040406
PY:Seattle has grown
PY:Initializing Archer_0001048615
PY:Entity Initialized : Archer_0001048615
PY:Initializing Galley_0000786507
PY:Entity Initialized : Galley_0000786507
PY:Initializing Archer_0000311314
PY:Entity Initialized : Archer_0000311314
PY:City Minoan's culture has expanded
PY:City Acquired Event: Minoan
iTurn: 253
 

Attachments

  • CvDynExp012-Fixed.zip
    12 KB · Views: 281
Stone-D said:
Just uploaded version 1.1. See the first post for details.

Nice. Maybe you could add: current version, release date, and change log to your first post :).
 
Stone-D said:
BTW, I got a crapload of other errors, but they were due to me running this from an existing savegame. You could avoid that by using the Try: / Except: method, though admittedly this mod would be best run from a fresh game.
Ta very much!

I think I will shove the Try: / Except: thing in. I also need to add a "sdEntityWipe" fuction in.

Thanks for the great work!
 
Stone-D said:
TGA: No problemo. Heh, I actually got carried away and started adding a try/except bit before reminding myself I wasn't working on my own code. Such things happen when you're coding at 4:30 am. :D
Hehe.

I am having ever so slight problems with the game printing "Entity Initialized : Wolf_0000024577" and so forth.

I've poked around, and it seems to be coming from the toolkit. As I'm not really sure I want it to print that message every time a unit is made I've commented that bit out here. Is this just a testing relic?

EDIT: And what happens if two units are initialised with the same number? (I know chances are very very very slim)
 
Top Bottom