Technical documentation

[embryodead]

After a major code overhaul in versions 0.4-0.5, I had to update this documentation, so here it is. Feel free to ask questions about RFC-SoI engine here (but for general RFC modding questions please use the The Modding Q&A Thread). The old version of this documentation (for SoI 0.3.x) is archived at the end of the post.

Introduction

SoI 0.5 removed a lot of the data hard-coded in the DLL to Python and XML. Moreover, the data is stored in the savegame. This only takes a few kilobytes and allows editing of previously hard-coded civ and map data even when the game is running. It is possible, for instance, to modify the civ balance modifiers, settler maps or war maps, with a Python event. It is also possible to have maps of different sizes in the same mod, with the same DLL.

All the relevant data from CvRhyes is now kept in Consts.py and Maps.py. It is loaded into the DLL through DataLoader.py module. CvRhyes.cpp known from RFC is no longer needed and has been removed from the project.

Below is the detailed list of files you'd typically want to edit to make your own mod.

Python

Consts.py - nearly all the data is stored here, with appriopriate descriptions; other modules use con.* constants from Consts.py, so you'll have to modify pretty much everything accordingly, file by file. Fortunately with Python exceptions and debug messages turned ON, Python will report everything in-game.

Maps.py - region maps, art style maps, AI settler maps; note that unlike the old maps in CvRhyes, these don't have to be the same size as the actual map - well, they should, but the game won't crash if they don't, which means that you don't have to prepare all the data for a new or extended map just to launch the game.

CityNameManager.py - city name map and a dictionary for city name changes.

XML

GlobalDefinesAlt.xml - this extra file contains a number of defines used by the game; comments should be sufficient

CIV4CivilizationInfos.xml - note that in RFC, SoI and other modmods, the order of civs in this file should be the same as the order in Const.py and WBSave!

WBSave

Map & player data is stored in WBSaves. Two things to remember:
- the number of teams & players must be the same as MAX_CIV_PLAYERS in CvDefines.h
- the order of teams & players should be the same as in Consts.py and CIV4CivilizationInfos.xml

CvGameCoreDLL

Here's a list of stuff that needs to be modified in the CvGameCoreDLL code. The list doesn't include all the changes I did, but ALL changes are commented usually by "edead" or "Rhye" (or modcomp name in case of Better BTS AI, Unofficial patch, BTS on Speed etc.)

CvDefines.h - MAX_CIV_PLAYERS determines the max. number of players the DLL can support; note that changing this has serious consequences, as you have to modify the following accordingly: all lists/tuples in Consts.py and other Python files, players & teams in WBSave; NUM_MAJOR_PLAYERS is the number of players excluding independents / minor civs (assuming they are placed at the end).

CvEnums.h - enums for Players, Religions, Wonders etc. are placed here, they have to be modified in accordance with XML data

CvCityAI::AI_bestUnit - civ-specific UNITAI weights
CvCityAI::AI_bestBuildingThreshold - building and location specific AI weights
CvCityAI::AI_chooseProduction - code related to air units is commented ("speed / unused features"); uncomment if you need air units

CvCity::getArtStyleType - this function replaces the default city style with another; replace style types with your own
CvCity::getReligionPercentAnger, CvCity::getReligionBadHappiness, CvCity::getRevoltTestProbability - Fatimid UP is hard-coded here
CvCity::getRealPopulation - real population formula is modified from pop^2.8 to pop^2.0
CvCity::updateCultureLevel - barbarian & minor civs are limited to culture level 2 here
CvCity::getProductionToCommerceModifier - Sindhi UP is hardcoded here
CvCity::isActiveCorporation - the code is modified so that the corporation is always active, whether the player has needed resources or not
CvCity::getGreatPeopleUnitRate, CvCity::getGreatPeopleUnitProgress - barbarian & minor civs have Great People Unit Rate reduced to 0
CvCity::doReligion - declining spread of Buddhism is hard-coded here

CvDLLTranslator.cpp - here you have some extra icons defined for use in game / civilopedia - they use enums from CvEnums.h so remove or modify this accordingly

CvDLLWidgetData.cpp - here you have the widget data for RFC/SoI leader tooltip etc.; no enums here, but if you want to modify stability, piety and title display, it's the place to look

CvMap::calculateAreas - this contains the code that splits solid continents into subcontinents for AI logic; all of this code should be removed for a new map; you can modify it as well, but I don't recommend using this feature unless you know what you're doing - splitting continents with this method makes the AI think it's actually two continents/islands split by water, so it will not link them with roads and try to use transport ships to carry units across; in SoI it's used for Oman, Bahrain and Mahra, as those areas are seperated from the rest of Arabia by unworkable desert.

CvPlayer::canConstruct - first, there's a hard-coded check that makes it impossible for Portugal to build any wonders; near the end, there's another hard-coded check that prevents the AI from building certain UHV-related wonders too early
CvPlayer::canConvert - code that disables converting to minor religions is here, uses religion enums from CvEnums.h
CvPlayer::getAnarchyModifier - code for Prophet's Mosque is hard-coded here, delete it
CvPlayer::getProductionModifier(UnitTypes eUnit) - Turkish UP is hard-coded here
CvPlayer::getProductionModifier(BuildingTypes eBuilding) - Armenian UP is hard-coded here
CvPlayer::getGreatPeopleRateModifier - Abbasid UP is hard-coded here
CvPlayer::getMaxConscript - Ottoman UP is hard-coded here
CvPlayer::verifyAlive - code that makes Portugal immortal is here

CvPlayerAI::AI_bestTech - civ-specific tech modifier (what the AI likes)
CvPlayerAI::AI_civicValue - civ-specific civic modifier (what the AI likes)
CvPlayerAI::AI_religionValue - civ-specific religion modifier (what the AI likes)
CvPlayerAI::AI_conquerCity - some years are hard-coded here, in the part that deals with AI city razing; remove it
CvPlayerAI::AI_targetCityValue - Franco-Mongol alliance is hard-coded here, uses fixed years & enums etc.; delete it all
CvPlayerAI::AI_getSameReligionAttitude and CvPlayerAI::AI_getDifferentReligionAttitude - religion relations are hard-coded here with usual enums; you don't need this unless you want something similar to SoI's Catholicism-Orthodoxy and Sunni-Shia relations
CvPlayerAI::AI_bonusTradeVal - the "Muslims not liking pigs or wine" feature is hard-code here... remove it

Extras

Attached to the post is my ODS file that contains the map data in spreadsheet format. ODS is the native format of OpenOffice/LibreOffice calc, but I'm sure MS Excel can import it somehow. This is mainly to show you how I (and Rhye) worked: all modifications are done in the spreadsheet, then I select the contents of a tab, copy it and paste to a Python or C++ file. Tab legend:
CityNames - single city name map (CityNameManager.py)
Provinces - province map (Maps.py)
ArtStyles - city art style map (Maps.py)
everything else - settler maps for all civs (also used for stability and AI wars in Maps.py)

Also check this post for a python script that will print WB map data into a CSV format that you can copy & paste to make your own spreadsheets.

 

[corovanrobber]

There is no other modder that has contributed to post-modern RFC more than you.

 

[Opera]

Thanks for this! I won't have to skim through all the dll files then

As for your .ods file, Excel has a handy feature: you can run Visual Basic scripts that make exporting data as file-ready text easy enough. At least I found it great to generate the provinces textkeys which can be numerous.

 

[Opera]

Something I don't understand: why are uniquePower, uniqueGoals and rating tridimensional arrays? For instance, there is uniquePower[NUM_MAJOR_PLAYERS][2][16]; I get the [NUM_MAJOR_PLAYERS] and the [2] but why the [16]?

Edit: after some research it seems related to the maximum number of characters in the string?

Also, why do rating have [6] when there's only 5 ratings?

 

[embryodead]

@Opera

Your research is correct, char array = string, so "char uniquePower[NUM_MAJOR_PLAYERS][2][16]" is NUM_MAJOR_PLAYERS sets of 2 strings of up to 16 characters each.

Why the size is 6 with ratings - no idea - this was set up by Rhye, maybe he originally had 6 ratings. You can change it to 5 but it's not like it's going to save you any substantial amount of memory

 

[Opera]

Thanks

But aren't the unique powers way more than 16 characters long? Or am I confusing characters and something else?

 

[embryodead]

Thanks

But aren't the unique powers way more than 16 characters long? Or am I confusing characters and something else?

Note what data is actually stored in the arrays. It's not the text that appears in game, but text keys which are 14-15 chars long, e.g. TXT_KEY_UP_BYZ2.

 

[Opera]

Dammit, that makes sense. I counted everything but these...

Thanks!

 

[Opera]

In Consts.py, the comment for tFall says "no resurrection". However I searched for it in all the files and it doesn't seem to hinder the resurrection in any way since it always check for tFallRespawned.

So with tFall I can set a stability malus for civs surviving too long and with tFallRespawned I can set at which point a fallen civ cannot respawn anymore, maybe set it different than tFall so that it can respawn after it has theoretically fallen?

My idea would for example be: player A (which is Dynasty 1 for now) is supposed to have fallen by -1000. However I use player A's slot for a respawn after -1000, in -850. In this case, it would be Dynasty 2, which didn't fall until -500. So the values should be:

tBirth = -2000
tFall = -1000
tRespawn = -850
tFallRespawned = -500

If I understand this correctly, if the player collapses before -500, it will have a random chance to respawn but it will have a set (historical) respawn in -850, and then keep respawning until -500 is past, if it ever collapses. If it never collapsed and thus never respawned but kicked around past -1000, it will receive a stability penalty (and may have some of its provinces' status be overriden).

Is all of this correct?

 

[embryodead]

In Consts.py, the comment for tFall says "no resurrection". However I searched for it in all the files and it doesn't seem to hinder the resurrection in any way since it always check for tFallRespawned.

So with tFall I can set a stability malus for civs surviving too long and with tFallRespawned I can set at which point a fallen civ cannot respawn anymore, maybe set it different than tFall so that it can respawn after it has theoretically fallen?

My idea would for example be: player A (which is Dynasty 1 for now) is supposed to have fallen by -1000. However I use player A's slot for a respawn after -1000, in -850. In this case, it would be Dynasty 2, which didn't fall until -500. So the values should be:

tBirth = -2000
tFall = -1000
tRespawn = -850
tFallRespawned = -500

If I understand this correctly, if the player collapses before -500, it will have a random chance to respawn but it will have a set (historical) respawn in -850, and then keep respawning until -500 is past, if it ever collapses. If it never collapsed and thus never respawned but kicked around past -1000, it will receive a stability penalty (and may have some of its provinces' status be overriden).

Is all of this correct?

EDIT: I wrote an elaborate explanation of how it really works, only to find out that it actually works like you described So, yeah. While checking this I found out that there's an issue with province status being displayed wrong for Respawned civs - i.e. their status is not taken into account, though it's visual thing only (i.e. color map).

NB, I'm planning to randomize the dates in the next version, so everything except civ births doesn't happen exactly at year X, but X +/- 5-10 years.

 

[Opera]

Sounds interesting, how will you do that? By actually changing the value in tRespawn and all or by changing when it checks for the dates?

 

[embryodead]

Sounds interesting, how will you do that? By actually changing the value in tRespawn and all or by changing when it checks for the dates?

That or through a getter function. Either way the tRespawn date is going to be modified by a random number based on seed value that is generated when the game starts (some Religion.py events already work more or less this way).

 

[AbsintheRed]

So, about adding the Persecution popup from SoI
(I hope you don't mind if I post my questions in this thread, it seemed perfect for it)
After a little messing around this morning, these are the files I modified:

CvDllTranslator.cpp:

	aIconMap[L"[ICON_JUDAISM]"] = std::wstring(1, (wchar)GC.getReligionInfo((ReligionTypes)JUDAISM).getChar());
	aIconMap[L"[ICON_CATHOLICISM]"] = std::wstring(1, (wchar)GC.getReligionInfo((ReligionTypes)CATHOLICISM).getChar());
	aIconMap[L"[ICON_ORTHODOXY]"] = std::wstring(1, (wchar)GC.getReligionInfo((ReligionTypes)ORTHODOXY).getChar());
	aIconMap[L"[ICON_PROTESTANTISM]"] = std::wstring(1, (wchar)GC.getReligionInfo((ReligionTypes)PROTESTANTISM).getChar());
	aIconMap[L"[ICON_ISLAM]"] = std::wstring(1, (wchar)GC.getReligionInfo((ReligionTypes)ISLAM).getChar());

CvRFCEventManager.py:

			7628 : ('Religious Persecution', self.relEventApply7628, self.relEventBegin7628),

	def relEventBegin7628(self):
		pass

	def relEventApply7628(self, playerID, netUserData, popupReturn):
		self.rel.eventApply7628(popupReturn)

I probably didn't make any mistakes in the above two

In the Religions.py I added the modified event (while I see the storedata is used here):

		def eventApply7628(self, popupReturn):
				"""Persecution popup event."""
				iPlotX, iPlotY, iUnitID = sd.getPersecutionData()
				religionList = sd.getPersecutionReligions()
				utils.prosecute(iPlotX, iPlotY, iUnitID, religionList[popupReturn.getButtonClicked()])

Thus the StoreData.py:

	def getPersecutionData(self):
		return self.scriptDict['lPersecutionData'][0], self.scriptDict['lPersecutionData'][1], self.scriptDict['lPersecutionData'][2]

	def setPersecutionData(self, iX, iY, iID):
		self.scriptDict['lPersecutionData'] = [iX, iY, iID]

	def getPersecutionReligions(self):
		return self.scriptDict['lPersecutionReligions']

	def setPersecutionReligions(self, val):
		self.scriptDict['lPersecutionReligions'] = val

I just added these to the end of the file, and I'm not sure here if I should change anything or not. In RFCE no similar functions were defined in this file

RFCUtils.py:

	def showPersecutionPopup(self):
		"""Asks the human player to select a religion to persecute."""
		
		popup = Popup.PyPopup(7628, EventContextTypes.EVENTCONTEXT_ALL)
		popup.setHeaderString("Religious Persecution")
		popup.setBodyString("Choose a religious minority to deal with...")
		religionList = sd.getPersecutionReligions()
		for iReligion in religionList:
			strIcon = gc.getReligionInfo(iReligion).getType()
			strIcon = "[%s]" %(strIcon.replace("RELIGION_", "ICON_"))
			strButtonText = "%s %s" %(localText.getText(strIcon, ()), gc.getReligionInfo(iReligion).getText())
			popup.addButton(strButtonText)
		popup.launch(False)

This is also fairly clear, but I'm not sure what else to add to the part where I define the persecution itself in this file (def prosecute in RFCE)

And the CvMainInterface.py:

		if (inputClass.getNotifyCode() == 11 and inputClass.getData1() == 666 and inputClass.getData2() == 666):
			if gc.getGame().getActivePlayer() == utils.getHumanID():
				pCity = gc.getMap().plot(g_pSelectedUnit.getX(), g_pSelectedUnit.getY()).getPlotCity()
				if not pCity.isNone():
					religionList = []
					pPlayer = gc.getPlayer(gc.getGame().getActivePlayer())
					for iReligion in range(gc.getNumReligionInfos()):
						if pCity.isHasReligion(iReligion) and not pCity.isHolyCityByType(iReligion) and iReligion != pPlayer.getStateReligion():
							religionList.append(iReligion)
					if len(religionList) == 1:
						utils.prosecute(g_pSelectedUnit.getX(), g_pSelectedUnit.getY(), g_pSelectedUnit.getID())
					elif len(religionList) > 1:
						sd.setPersecutionReligions(religionList)
						sd.setPersecutionData(g_pSelectedUnit.getX(), g_pSelectedUnit.getY(), g_pSelectedUnit.getID())
						utils.showPersecutionPopup()
			else:
				utils.prosecute(g_pSelectedUnit.getX(), g_pSelectedUnit.getY(), g_pSelectedUnit.getID())

I'm sure I made a lot of mistakes. Especially wasn't sure about the diff where to use iPlotX, iPlotY or iX, iY

 

[embryodead]

1. In CvDLLTranslator you used consts like JUDAISM and PROTESTANTISM - make sure they are actually defined as numbers corresponding to XML indexes in CvRhyes.h

2. As I said, SoI / RFCM handles StoredData differently. It has all the scriptData functions moved to StoredData module which is imported by other modules, while RFC/E has them scattered over several files. So instead of copying these functions to StoredData, copy them to RFCUtils.py, remove the self. prefix from their code, and then change all parts of my code that refer to these functions to refer to the proper module (utils.* instead of sd.*)

3. In StoredData.py itself, make sure to add the actual data fields:

				'lPersecutionData': [-1, -1, -1],
				'lPersecutionReligions': [],

4. Finally you have to modify the RFCE persecution function (prosecute in RFCUtils) to work with the arguments supplied by my code.

 

[J. pride]

Embroydead,
what steps do i have to take in order to add tiles to the map. I am trying to add Greece to the map; what i did was add 20 tiles in the beginning and then changed the rest accordingly so what was previously x=0 became x=20 and so on. All of those 20 new x times were copies of the previous x=0 (this way i wouldn't have to add the Sahara and the Mediterranean since the they are all desert and ocean and only have to work with the coast of Africa and Greece).

That was long and confusing but anyway heres the problem. I can't open the game, everytime i pick a civ and start the game it crashes when its loading. i was under the impression that u dont have to edit anything else in order to add tiles (even though it screws up all the locations cities etc). I thought i could just open the map see how everything looks, edit things in the world builder and if i like how it is then i just open Middle East 750 map in notepad and change things.

I uploaded what i did incase u wanted to see it.

 

[embryodead]

J.pride,
In RFC & all modmods, map size is fixed and set in the DLL because all kinds of arrays depend on it. In order to use a map of different size you need to modify a lot:
- CvRhyes.h: EARTH_X, EARTH_Y, CATAPULT_X, CATAPULT_Y
- CvRhyes.cpp: make new settler maps for all civs, new province map, new artstyle map
- Consts.py: iMapWidth, iMapHeight, iFlipX, iFlipY, city coords
- CityNameManager.py: new city name map

Changes to CvRhyes.h/cpp, rebuilding the DLL and new map width/height in Consts.py are absolute minimum to launch the game at all. Mind you, SoI is very much streamlined when it comes to maps, with RFC* it's even more work. The opencalc spreadsheet document I attached to the first post is pretty much a must if you want to modify the map & related arrays.

 

[J. pride]

Thanks Embrydead but a couple more questions:

I realize that consts.py and CvRhyes.h are required but how do i add Settler map and provinces when i havent even been able to make Greece. All I have done is add the tiles and i cant start work on Greece without going to word builder and seeing what to do with what tiles. Is it possible to start the game with adding settler map and provinces; if not what do u suggest i do or am i just approaching this all wrong?

PS: Whats the artstyle map?

 

[embryodead]

I realize that consts.py and CvRhyes.h are required but how do i add Settler map and provinces when i havent even been able to make Greece. All I have done is add the tiles and i cant start work on Greece without going to word builder and seeing what to do with what tiles. Is it possible to start the game with adding settler map and provinces; if not what do u suggest i do or am i just approaching this all wrong?

As I said, to make settler/province/artstyle/cityname maps, use the spreadsheet I provided. Also, you don't have to make "proper" maps yet, but they have to be there with the correct dimensions, otherwise the game will crash.

There's also an external editor called MapView which you may find useful. It's much more convenient than WorldBuilder, but crashes a lot. It's somewhere in C&C utitilies forum.

PS: Whats the artstyle map?

Map of city art styles, also available on the spreadsheet.

 

[Opera]

I was wondering why some civs are minor? Zengids and Buyids for instance, I understand, being rather shortlived. Special respawns I understand as well. But my concern mostly is about Malwa. They stick around forever yet aren't playable. Why?

 

[2phunkey4u]

Yep, they can be a force, but I remember edead stating that he found their history and map position too uninspiring to come up with good UHVs for them.