[BTS] Modding help request: making/changing mapscripts.

high tyrol

Chieftain
Joined
Sep 25, 2022
Messages
26
I want to learn how to modify and make mapscripts. I am a proficient python coder. I can also do C++ in a pinch. I am pretty new to Civ 4 modding though.

If you know of any tutorials, or references on the topic I would love to have links. Or if someone was willing to teach me how to work with them I would like that. I would be very grateful in that case.
I have a couple ideas for map gen algorithms I am interested in maybe making into maps.
And once I become proficient enough, I am hoping to add large river map gen to col 3 Religions and Revolutions as well as port that to work with beyond the sword.
 

f1rpo

plastics
Joined
May 22, 2014
Messages
1,419
Location
Germany
I'm not a fount of knowledge about tutorials, unfortunately, but I can improvise a quick overview of the control flow:

We've got CvMapGenerator::generateRandomMap and CvMapGenerator::addGameElements in the (GameCore) DLL, which, I think, get called by the EXE – unless the map is re-generated, which happens via CvGame::regenerateMap in the DLL. generateRandomMap calls generatePlotTypes (ocean, flat, hills, peak) and generateTerrain (Grassland etc.). addGameElements calls addRivers, addLakes, addFeatures, addBonuses, addGoodies, afterGeneration (for misc. final touches). Those are all CvMapGenerator functions. Lastly, CvGame::setInitialItems causes starting plots to be chosen and to be beefed up ("normalized"); that's all handled by CvGame. All those CvMapGenerator and CvGame subroutines first attempt a Python call to the map script module (CvDLLPythonIFaceBase::getMapScriptModule - this getter is implemented in the EXE). The map script may perform work in addition to the DLL (signaled by calling CyPythonMgr().allowDefaultImpl()) or may fully replace the DLL default behavior. For generatePlotTypes and generateTerrain, the DLL has no (useful) default behavior, and CvMapGenerator::addFeatures doesn't place Forest, Jungle and Ice, i.e. map scripts have to implement those functions, but they can (and most do) make use of CvMapGeneratorUtil.py. In other words, default behavior for land shapes, relief and vegetation, to use non-technical terms, is implemented in Python. For generating shapes (of continents, deserts etc.), CvMapGeneratorUtil.py uses "height maps" produced via the CvFractal class in the DLL (exposed to Python through the EXE).

For large (navigable) rivers, my approach would probably be to "widen" some of the small rivers into large rivers. Could add a function CvMapGenerator::addLargeRivers for this that gets called after addRivers by addGameElements.

It seems that the latest approach taken by the "We the People" (WtP; successor of RaR) developers has been to add the large rivers in each map script separately:
So ok, after a constructive internal "concept" discussion, we figured out that there is probably a better technical solution. @FlaviusBelisarius is going to try to implement this directly in the FaireWeatherTweakEx.py Mapscript. :thumbsup: The reason for that decision is, that this MapScript already creates pretty natural looking landscapes and also good looking "small rivers", so it should theoretically also be able to create good looking "large rivers". [...]
Doesn't look like MagisterBelsarius published any such code: GitHub contributions in 2021

For FaireWeather, or any script of the PerfectWorld family, it may make sense to generate the large rivers before the small ones based on rainfall data, and I guess it would be nice to generate the large-river terrain along with the other terrain types, i.e. well before small rivers, but, at least in a (BtS) mod that needs to work with many different map scripts, a single algorithm for all map scripts is imo a necessity, and I don't think an additional custom algorithm for e.g. PerfectWorld is going to be worth the extra effort. The small rivers generated by PerfectWorld will be based on rainfall, so large rivers placed based on small rivers will also, indirectly, be based on rainfall.

As for Python being your preferred language, that should not be difficult to accommodate. The DLL can call (through an unwieldy syntax) any function in a Python (v2.4 :() module, and all CvPlot and CvMap functions relevant for placing large rivers should already be exposed to Python. The approach consistent with generatePlotTypes would be to let the map script forward a call received from the DLL to CvMapGeneratorUtil, but that would require every map script to be changed. Better to call the map script from the DLL to allow an override and, then, if not overridden, to call CvMapGeneratorUtil (or whichever module) directly from the DLL for the default behavior.

edit: clarity
 
Last edited:

high tyrol

Chieftain
Joined
Sep 25, 2022
Messages
26
I'm not a fount of knowledge about tutorials, unfortunately, but I can improvise a quick overview of the control flow:

We've got CvMapGenerator::generateRandomMap and CvMapGenerator::addGameElements in the (GameCore) DLL, which, I think, get called by the EXE – unless the map is re-generated, which happens via CvGame::regenerateMap in the DLL. generateRandomMap calls generatePlotTypes (ocean, flat, hills, peak) and generateTerrain (Grassland etc.). addGameElements calls addRivers, addLakes, addFeatures, addBonuses, addGoodies, afterGeneration (for misc. final touches). Those are all CvMapGenerator functions. Lastly, CvGame::setInitialItems causes starting plots to be chosen and to be beefed up ("normalized"); that's all handled by CvGame. All those CvMapGenerator and CvGame subroutines first attempt a Python call to the map script module (CvDLLPythonIFaceBase::getMapScriptModule - this getter is implemented in the EXE). The map script may perform work in addition to the DLL (signaled by calling CyPythonMgr().allowDefaultImpl()) or may fully replace the DLL default behavior. For generatePlotTypes and generateTerrain, the DLL has no (useful) default behavior, and CvMapGenerator::addFeatures doesn't place Forest, Jungle and Ice, i.e. map scripts have to implement those functions, but they can (and most do) make use of CvMapGeneratorUtil.py. In other words, default behavior for land shapes, relief and vegetation, to use non-technical terms, is implemented in Python. For generating shapes (of continents, deserts etc.), CvMapGeneratorUtil.py uses "height maps" produced via the CvFractal class in the DLL (exposed to Python through the EXE).

For large (navigable) rivers, my approach would probably be to "widen" some of the small rivers into large rivers. Could add a function CvMapGenerator::addLargeRivers for this that gets called after addRivers by addGameElements.

It seems that the latest approach taken by the "We the People" (WtP; successor of RaR) developers has been to add the large rivers in each map script separately:Doesn't look like MagisterBelsarius published any such code: GitHub contributions in 2021

For FaireWeather, or any script of the PerfectWorld family, it may make sense to generate the large rivers before the small ones based on rainfall data, and I guess it would be nice to generate the large-river terrain along with the other terrain types, i.e. well before small rivers, but, at least in a (BtS) mod that needs to work with many different map scripts, a single algorithm for all map scripts is imo a necessity, and I don't think an additional custom algorithm for e.g. PerfectWorld is going to be worth the extra effort. The small rivers generated by PerfectWorld will be based on rainfall, so large rivers placed based on small rivers will also, indirectly, be based on rainfall.

As for Python being your preferred language, that should not be difficult to accommodate. The DLL can call (through an unwieldy syntax) any function in a Python (v2.4 :() module, and all CvPlot and CvMap functions relevant for placing large rivers should already be exposed to Python. The approach consistent with generatePlotTypes would be to let the map script forward a call received from the DLL to CvMapGeneratorUtil, but that would require every map script to be changed. Better to call the map script from the DLL to allow an override and, then, if not overridden, to call CvMapGeneratorUtil (or whichever module) directly from the DLL for the default behavior.

edit: clarity
Thanks. That is a lot of detail. It gives me a pretty good idea of how to tackle that.
I didn't know RaR had a successor. I will have to check it out. I guess they got the feature implemented for their map scripts.
I was planning to make a base module that could be easily put into most map scripts. But, If I saw an interesting way to integrate it to a particular one to try it.
The basic idea is about what I had thought. But, knowing what parts of the CIV 4 code I need to hook into will be super useful.
Having to be python 2.4v is kind of meh. But, it shouldn't be too hard. I started with python 2 anyways. It is slightly concerning from a security standpoint. Since 2.4 is a lot less secure than the later 2.x versions. 2.8 is a lot better than 2.4v if you have to run python 2.
It was probably whatever version was current when they made CIV 4 though.
 

f1rpo

plastics
Joined
May 22, 2014
Messages
1,419
Location
Germany
I didn't know RaR had a successor. I will have to check it out. I guess they got the feature implemented for their map scripts.
I don't believe so:
[...] Summary: It is in the game and playable, just not in randomly generated maps by MapScripts.
That's from Sept 2021; the quote in my previous post is from Oct 2021, and that's the latest post on that matter that I'm aware of.
I was planning to make a base module that could be easily put into most map scripts.
I see. If you're OK with having to touch every map script that is supposed to be affected by your changes, then there's probably no need to change anything in the DLL. (And you could still, at a later point, decide to have the DLL call some functions in your base module.)

Edit - I just remembered that there is already a community-created module with utility functions for map scripts: MapScriptTools (MST)
Rise of Mankind (RoM) includes MST; could be that the RoM modders have made some tweaks, or maybe the Caveman2Cosmos modders. In other words, I'm not sure what the latest and greatest version of MST is.
 
Last edited:

high tyrol

Chieftain
Joined
Sep 25, 2022
Messages
26
I don't believe so:That's from Sept 2021; the quote in my previous post is from Oct 2021, and that's the latest post on that matter that I'm aware of.I see. If you're OK with having to touch every map script that is supposed to be affected by your changes, then there's probably no need to change anything in the DLL. (And you could still, at a later point, decide to have the DLL call some functions in your base module.)

Edit - I just remembered that there is already a community-created module with utility functions for map scripts: MapScriptTools (MST)
Rise of Mankind (RoM) includes MST; could be that the RoM modders have made some tweaks, or maybe the Caveman2Cosmos modders. In other words, I'm not sure what the latest and greatest version of that script is.
Cool, one of the mods I was hoping to integrate them into was a RoM derivative. Although, if the RaR series doesn't have it I might be better off not being dependent on it. I will check it out though.
You said We the People was a Successor or based on RaR though in your post. But then in your next post you said you don't believe so? Which is slightly confusing?
 

f1rpo

plastics
Joined
May 22, 2014
Messages
1,419
Location
Germany
Cool, one of the mods I was hoping to integrate them into was a RoM derivative. Although, if the RaR series doesn't have it I might be better off not being dependent on it. I will check it out though.
You said We the People was a Successor or based on RaR though in your post. But then in your next post you said you don't believe so? Which is slightly confusing?
I only meant to say that WtP didn't get large rivers implemented for any of its map scripts. Actually, my understanding, just from reading the Civ4Col forums, is that RaR didn't get large rivers implemented at all, that it was only a planned feature, and that WtP got it done for scenarios:
[...] one of my oldest, most wanted but never realized concepts is: Large Rivers
(The link is to my original RaR concept, but please anwer here in WTP forum.)
But why was it not implemented then, you might ask? :think: [...]
Therefore, this post of yours confused me:
I saw that navigable rivers where added to Religion and revolution. I really enjoyed them there and wished they were in normal Civ 4. [...]
- and I had kind of assumed that you (and @<Nexus>) were really talking about WtP.
 

high tyrol

Chieftain
Joined
Sep 25, 2022
Messages
26
I only meant to say that WtP didn't get large rivers implemented for any of its map scripts. Actually, my understanding, just from reading the Civ4Col forums, is that RaR didn't get large rivers implemented at all, that it was only a planned feature, and that WtP got it done for scenarios:Therefore, this post of yours confused me: - and I had kind of assumed that you (and @<Nexus>) were really talking about WtP.
RaR had large rivers in the world map scenarios when I played it a year or so ago. As well as when I played it a couple weeks ago.

To my knowledge they never added it to the mapscripts for generation.

But, yeah I guess WTP developed it further?
 

f1rpo

plastics
Joined
May 22, 2014
Messages
1,419
Location
Germany
WtP represents large rivers as a special terrain type. Maybe navigable rivers in RaR work much like in the Strategic Rivers Mod, i.e. still run in between tiles. And perhaps that more low-key approach is what you and Nexus would prefer; so WtP won't necessarily be of interest to you.
 

high tyrol

Chieftain
Joined
Sep 25, 2022
Messages
26
I was pretty sure it was a special terrain type. Although, maybe I just ran we the people and thought it was RaR and didn't realize the difference.
I didn't realize the ones in the strategic rivers mod still run between tiles. I am curious how that works with making them navigable. As, I don't see how that would work. I haven't actually had a chance to play it yet. Although, I have installed it.
 

raystuttgart

Civ4Col Modder
Joined
Jan 24, 2011
Messages
8,852
Location
Stuttgart, Germany
Hi guys,

I read some posts about RaR / WTP.

----

1) WTP is the successor mod of RaR.
(Basically it just continued the project under a new name.)

2) In WTP I implemented Large Rivers as a new Water Terrain and our Maps contain it.
(There are also e.g. "Lakes", "Ice Lakes" and "Shallow Coasts" as new Water Terrains.)

3) There is currently still no MapScript yet published that creates Large Rivers.
(But there is one in work actually - it is simply not done.)

4) All in all WTP is still in development.
(Progress is slow however.)

5) In RaR I had never started "Large Rivers", because I was too busy with other stuff.
(It was simply one of the many todos on the list we never got finished.)

----

Hope that answers some questions. :)
 
Last edited:

high tyrol

Chieftain
Joined
Sep 25, 2022
Messages
26
Hi guys,

I read some posts about RaR / WTP.

----

1) WTP is the successor mod of RaR.
(Basically it just continued the project under a new name.)

2) In WTP I implemented Large Rivers as a new Water Terrain and our Maps contain it.
(There are also e.g. "Lakes" and "Shallow Coasts" as new Water Terrains.)

3) There is currently still no MapScript yet published that creates Large Rivers.
(But there is one in work actually - it is simply not done.)

4) All in all WTP is still in development.
(Progress is slow however.)

5) In RaR I had never started "Large Rivers", because I was too busy other stuff.
(It was simply one of the many todos on the list we never got finished.)

----

Hope that answers some questions. :)
Well, if you have one in progress then perhaps maybe I could help with that rather than starting mostly from scratch. (Other than the actual implementation of the rivers in gameplay that can be placed on maps.)
 

raystuttgart

Civ4Col Modder
Joined
Jan 24, 2011
Messages
8,852
Location
Stuttgart, Germany
Well, if you have one in progress then perhaps maybe I could help with that rather than starting mostly from scratch.
@devolution is the modder working on the new MapScript, not me.
But sure, go ahead and contact him, maybe you two can cooperate. :thumbsup:

However, the MapScript is adding many many more Terrain Types, since we have about 20+ Terrains in our mod.
(It is not necessary compatible to "Civ4 BTS" actually, since our MapScripts call custom DLL functions for MapGeneration which Civ4BTS does not have.)

This screenshot is showing some Terrains of WTP.
 
Last edited:

high tyrol

Chieftain
Joined
Sep 25, 2022
Messages
26
@devolution is the modder working on the new MapScript, not me.
But sure, go ahead and contact him, maybe you two can cooperate. :thumbsup:

However, the MapScript is adding many many more Terrain Types, since we have about 20+ Terrains in our mod.
(It is not necessary compatible to "Civ4 BTS" actually, since our MapScripts call custom DLL functions for MapGeneration which Civ4BTS does not have.)

Well, that would be important to look into.
So any BTS adaption would also need to port the new terrains?
I know of several mods with expanded terrain sets. Have the WTP terrain sets ever been exported to BTS before?

Worst case, I might be able to have some of the logic transfer. But, libraries used may end up completely different.
Presumably similar errors might occur if I were to try to make map scripts that only added the large rivers if I took that element out of WTP?
 

raystuttgart

Civ4Col Modder
Joined
Jan 24, 2011
Messages
8,852
Location
Stuttgart, Germany
So any BTS adaption would also need to port the new terrains?
No, of course not. You just need to check which changes of XML, Python and DLL you want to adapt.
You just cannot simply "copy&paste" a Python MapScript from WTP into a Civ4BTS mod, without also adjusting XML and DLL.

Have the WTP terrain sets ever been exported to BTS before?
Some befriended modders used some of my new textures.
But otherwise, no it has never been "exported" to BTS before.

... added the large rivers if I took that element out of WTP?
If you want to add Large Rivers into your mod, you will have to diff / adapt / merge quite a lot of files.
(There are massive amount of changes in XML, DLL, graphics, ...)

----

"Large Rivers" is the biggest and most complex feature I ever implemented. :dunno:
(I actually kind of introduced a complete set of many small features for it.)

It took me about 2 months to implement it and get all the bugs fixed.
Merging it properly to Civ4BTS will probably take a couple of days.

By the way:
In the last public release there was still a CTD related to "Large Rivers".
(It is however already fixed in internal development branches.)
 
Last edited:

high tyrol

Chieftain
Joined
Sep 25, 2022
Messages
26
No, of course not. You just need to check which changes of the XML, the Python and DLL you want to adapt.
You just cannot simply "copy&paste" a Python MapScript from WTP into a Civ4BTS mod, without also adjusting XML and DLL.



Some befriended modders however used some of my Textures.
But otherwise, no it has never been "exported" to BTS before.


If you want to add Large Rivers into your mod, you will have to diff/ adapt / merge quite a lot of files.
(There are massive amount of changes in XML, DLL, graphics, ...)

----

"Large Rivers" is the biggest and complex feature I ever implemented. :dunno:
(I actually kind of introduced a complete new feature set for it.)
Yeah, I expected the XML to have to be changed at least.
Is the DLL structure for Mapscripts significantly different from BTS though?
If so that would make sense.
If I had to mess with those for the adaptation I would. Once I became familiar enough with them to work with them well.
I just thought I recognized a lot of(but not all) those new terrain textures from other mods. Most notably the savanna, dessert and arid textures.
The main thing that stood out was the mountains with different base terrain colors. I am guessing that those are aridity + temperature levels maybe?

But, I will contact devolution. See if maybe I can help.

Also, thanks for the info, advice, and direction.
 

raystuttgart

Civ4Col Modder
Joined
Jan 24, 2011
Messages
8,852
Location
Stuttgart, Germany
@high tyrol
Large Rivers is a DLL based feature. Without the code adapations it does not work
All the special logic for its sub-features and mechanics are implemented in C++.

Civ4BTS and Civ4Col DLL share many common functions, since Civ4Col code bas was branched from Civ4BTS code base more or less.
But still, it is a different game and our mod is also a custom mod that comes with its own custom DLL.

----

Also the DLL feature "Large Rivers" and the MapScript are not one and the same.
(The MapScript alone would not actually add any new game mechanics.)

DLL feature "Large Rivers" --> game mechanics
MapScript --> just "Terrain Placement"

----

I just thought I recognized a lot of(but not all) those new terrain textures from other mods. Most notably the savanna, dessert and arid textures.
Correct, many of the WTP textures are also from other mods.
Some of the WTP textures however I created myself.
 
Last edited:
Top Bottom