MapScriptTools (.. and Associated Maps)

[Temudjin]

Oops! Mostly doing Civ5 now, I didn't realize my thread became active again.
Many thanks EmperorFool for answering questions and resolving errors in my absence.

The 'key' keyword parameter in the sort() method for lists was also new in Python 2.4.
The old-fashioned way to write:

vList.sort( key = lambda test: len(test[1]) )

would be:

vList.sort( cmp=lambda x,y: cmp( len(x[1]), len(y[1]) ) )

I've used this only in makeMapLegend() and showContinents(). The changes should be fairly straightforward.

Thanks Xyth for trying out the MapScriptTools and not giving up after finding some errors. I am still interested in getting my MapScriptTools right for the MAC, so I'll probably post a version 1.02, if I can get some confirmation on the solution.

 

[Temudjin]

@dommain:
I'm not quite certain if I understand what you mean, but the landmarks in FFH are the names of special improvements on a single tile, while the 'landmarks' in MST are the names of the special regions (several tiles) generated by MST. They tend to be near the middle of that region.
How many of those regions exist, depends on the map and it's size.

@LunarMongoose:
Thanks. I'm not altogether sure I'd always want Atlantis and the Elemental Quarter myself; that's one reason I introduced some uncertainty (adjustable by the map-script) for these features to actually exist on a particular map.

 

[Baldyr]

vList = [(len(x[1]), x) for x in dict.values()]
vList.sort()
vList.reverse()
vList = [x for _, x in vList]

Man, this is one of those cases where I just can't make heads or tails of a piece of code. Could you, perhaps, run this script through line by line? Probably start with what the names x and dict are referring to. And what the resulting vList array would look like. Please?

 

[EmperorFool]

vList = [(len(x[1]), x) for x in dict.values()]
vList.sort()
vList.reverse()
vList = [x for _, x in vList]

Putting a loop inside a pair of square brackets as on the first line is called a list comprehension and builds a list from the values from the loop. "dict" is from the original code and I assume the name of a local variable which is actually a very bad idea because it hides the name of the dict() global function. "dict.values()" pulls all of the values without their keys from said dictionary. So "for x in dict.values()" loops over each value in "dict" and puts each one into a variable named "x" which you can use.

This value "x" is used to pull the second element x[1] from it, assuming it's a tuple, and calculates its length. It packs the length and the tuple x into a new tuple as the element to place into the new list. Now vList is a list of two-element tuples where the first element is the length of one of the elements contained in the second element.

(2, (4, "hi"))
(3, (21, "bye"))
...

That's just an example of what could be contained in there.

The sort() and reverse() lines should be self-explanatory. If not, use Google.

The final line uses another list comprehension to extract the original dictionary values from the vList elements. It loops over the list and assigns the elements one-by-one to _ and x. In Python _ is the universal "I don't care about this value" variable. It's a normal variable, but people use it by convention to signify to the reader that they aren't going to use it. Assigning a list to multiple variables with one assignment "_, x = ..." breaks up the tuple/list so each variable holds a single element.

So this code loops over the two-element tuples, grabs the second element into x, and puts those into a new list.

 

[Xyth]

Looks the Mac implementation of Python won't accept anything inside the brackets of .sort():

TypeError: sort() takes no keyword arguments

That's the same error for both the original code and 'the old-fashioned way'. EmperorFool's code seems to work though, at least no errors were thrown. I replaced the code around lines 8468, 8474, 8482, 8801 and 8803 (along with EmperorFool's fix for line 2126) and the main file seems to be working without problem.

I haven't tried every mapscript yet but the included Earth3, Inland Sea, Sea Highlands, and Tectonics works fine, as does using the template in the standard Archipelago.py. I know that Fractured World and a few others were throwing the same error as above but I'll try EmperorFool's fix on them and see if that solves it. I'll let you know how that goes when I get a chance.

 

[Xyth]

Actually I've just realized some of those lines weren't the same as the one I posted earlier:

		if not (addLine == None):
			aList = addLine
			aList.sort( key = lambda test: len(test) )
			aTab = len( aList[0] )

		aList.sort( key = lambda test: test[18:26] )
		aList.reverse()
		wList.sort( key = lambda test: test[18:26] )
		wList.reverse()

I confess that EmperorFool's explanation made my brain melt, how do I convert these two given they don't have any mention of dict.values()?

 

[EmperorFool]

I'll need more context around those lines to be sure. If aList from the first isn't used after that code, you can use this:

aList = [(len(x), x) for x in aList]
aList.sort()
aList = [x for _, x in aList]
aTab = len( aList[0] )

For the second block you can use the same pattern as the first: build a list combining the key and the value into elements, sort it, and break it apart again.

aList = [(len(x[18:26]), x) for x in aList]
aList.sort()
aList = [x for _, x in aList]

wList = [(len(x[18:26]), x) for x in wList]
wList.sort()
wList = [x for _, x in wList]

You might first try using the fix that Temudjin proposed but removing the name of the parameter. The error states that sort() doesn't take any namedparameters, but IIRC that is the case if the parameters don't have default values. Remove the name so it becomes a positional parameter.

vList.sort( lambda x,y: cmp( len(x[1]), len(y[1]) ) )

 

[Xyth]

You might first try using the fix that Temudjin proposed but removing the name of the parameter. The error states that sort() doesn't take any namedparameters, but IIRC that is the case if the parameters don't have default values. Remove the name so it becomes a positional parameter.

vList.sort( lambda x,y: cmp( len(x[1]), len(y[1]) ) )

Ah this seemed to work for each of the lines in question (I changed the bit in square brackets to match the original version of each). Thanks to you both!

 

[Temudjin]

Thanks again Xyth & EmperorFool.
The new scheme is now incorporated into MapScriptTools 1.02.
I've also changed FracturedWorld, Ringworld and SmartMap accordingly. They and the other maps should now also work on the MAC - I hope.

Publication will be soon. (Yeah, I've heard that one before. )

 

[HardRocker]

would this work with RFC when making a modcomp???

 

[Xyth]

Temudjin, I was wondering if I could make a request. I'm experimenting with reefs and was thinking I'd like it if reefs never appeared as single tiles but rather as 2-5 tiles in a rough line.

If you're too busy, I might have a go at trying to write it myself. Any tips on useful functions I could use to achieve this?

 

[Temudjin]

MapScriptTools 1.02
This version should fix the MAC issues - I hope. I've also had a look and tried to fix those same issues within the map-scripts.
Mana-Types are now automatically recognized and reported in all FFH modmods.
In SmartMap I've rewritten the addGoodies() function to produce fewer of them. They are also not clumped anymore.
Reefs are now likely to expand into a chain. (Some balancing may still be needed here.)
It also comes with a new map: Planetfall_101_mst.py.

 

[Temudjin]

@HardRocker: I think RFC works only with it's very own .CivBeyondSwordWBSave scenario maps.

@Xyth: You inspired me.
I've had a closer look at reefs and expanded a bit on them, with somewhat longer chains though. The place to look at is the FeaturePlacer class: there are two functions - placeReefs() and expandReef() - which handle reefs.
I haven't play-tested them much - RifE seems to be the only mod that handles them yet - so there may be some issues.

 

[Xyth]

Thanks for the update Temudjin!

Just did a quick test of each mapscript (default settings) and everything seems to be working great on Mac except the Smartmap and Planetfall scripts. The latter isn't a problem given Planetfall itself doesn't work on Mac. Smartmap never worked on Mac either though it's probably fixable. The errors thrown are:

Traceback (most recent call last):

File "SmartMap_922_mst", line 278, in addBonuses

File "SmartMap_922_mst", line 2717, in addBonuses2

NameError: global name 'sorted' is not defined
ERR: Python function addBonuses failed, module SmartMap_922_mst
Traceback (most recent call last):

File "SmartMap_922_mst", line 3380, in assignStartingPlots

NameError: global name 'sorted' is not defined
ERR: Python function assignStartingPlots failed, module SmartMap_922_mst

I'm guessing sorted() is another 2.4 feature. Not a problem though, I'm more than happy with everything else included. Thanks too for the expanded reefs, I'll let you know how I get on with them

 

[EmperorFool]

Can you please post the code for those broken lines with a few lines on either side for context? It's probably a simple matter cloning the list and sorting it, but I don't know without the code.

 

[Xyth]

Can you please post the code for those broken lines with a few lines on either side for context? It's probably a simple matter cloning the list and sorting it, but I don't know without the code.

Sure thing, here they are:

	bonusIdsAndOrder = []
	for bonusId in range(cgc.getNumBonusInfos()):
		bonusInfoXML = cgc.getBonusInfo(bonusId)
		if bonusInfoXML.getPlacementOrder() >= 0:
			bonusIdsAndOrder.append((bonusId,bonusInfoXML.getPlacementOrder()))

	bonusIdsAndOrder = sorted(bonusIdsAndOrder, key=operator.itemgetter(1))
#	mst.printList( bonusIdsAndOrder, "bonusIdsAndOrder:", rows = 1, prefix = "[SMap] " )

		#pair continent lists with their scores, and sort by the scores
		listsScoresPlayers = []
		for index in range(len(postContinentLists)):
			listsScoresPlayers.append((postContinentLists[index], postContinentScores[index], 0))

		listsScoresPlayers = sorted(listsScoresPlayers, key=operator.itemgetter(1))
		listsScoresPlayers.reverse()
		mst.printList( listsScoresPlayers, "listsScoresPlayers:", rows=1, prefix = "[SMap] " )

 

[Temudjin]

I think I have the fix, could you check the file? Thanks for pointing this out.

What seems to be the problem with Planetfall_101_mst ? The whole point of including it is that the map should work with any mod ?

 

[Xyth]

I think I have the fix, could you check the file? Thanks for pointing this out.

Seems to work perfectly now, thank you.

What seems to be the problem with Planetfall_101_mst ? The whole point of including it is that It should work with any mod ?

This also seems to be working without issue now, I'm not sure why it wasn't the first time I tried it. Was probably something unrelated that I was tinkering with at the time.

The new reef functionality is great, I'm still experimenting to find the ideal numbers but I've seen some very fun looking maps generated with it. Thanks so much for this.

I have a couple suggestions for a future version that I was able to implement myself (roughly) but I imagine could be useful to others. Firstly, it seems that some terrain graphics set (Blue Marble in this instance) don't look right when ocean tiles are directly next to certain land tiles. I added a boolean parameter to expandifyCoast() to enable/disable the deep water near cliffs functionality.

However, I've noticed that even with this switched off there is still the occasional instance of land and ocean tiles blending. It doesn't seem to happen when the coast isn't expanded and I switched off bulkifyIslands() to make sure it wasn't that (it only seems to occur on islands). I've been testing this on Firaxis' Archipelago and Pangaea maps. Is there some method of preventing this completely or checking for the existence of ocean next to land and converting it to coast?

Secondly, I 'converted' (quotes to indicate that I wouldn't trust it to work well beyond HR) the MarshMaker class to work with FEATURE_SWAMP if TERRAIN_MARSH isn't found. This might be something useful to others too and I'm sure you could code it far better than I.

 

[Temudjin]

That's good news . Thanks

Cliffs: Wasn't aware of that, but I'm not very graphical minded. Ok, I will put an extra parameter 'bCliffs' into expandifyCoast().
bulkifyIslands() deals with plots and not with terrain, so it shouldn't matter as long as it's called before the terrain is generated. mst.mapPrettifier.bulkifyIslands() should be called somewhere at the start of generateTerrainTypes().
A first parsing of my special regions code (which changes plots after terrain is generated) brought no results either (yet).
To kill that problem you could put

for inx in range( map.numPlots ):
	x,y = mst.coordByIndex(inx)
	if mst.isCoastalWater(x,y): 
		map.plot(x,y).setTerrainType( mst.etCoast, False, False )

somewhere late into normalizeAddExtras(). I would certainly like to know where that problem comes from though.

Swamps: Actually I thought that feature swamp (or feature marsh) was the old solution and modern mods use terrain marsh instead. In Civ5 it's feature marsh though, so I might as well reconsider my approach. Which mods do use feature swamp?

 

[Xyth]

I would certainly like to know where that problem comes from though.

It seems to occur only on islands, so far anyway, for example offshore islands on a Pangaea map or in Archipelagos. This is what lead me to suspect bulkifyIslands() but it definitely still occurs with that disabled. It doesn't occur in connection with BigBogs or BigDents. It's a mystery, though fortunately rare. I'll let you know if I gain any more insight into it.

Thanks for that fix too, I won't apply it yet in case I can help figure out the cause.

Swamps: Actually I thought that feature swamp (or feature marsh) was the old solution and modern mods use terrain marsh instead. In Civ5 it's feature marsh though, so I might as well reconsider my approach. Which mods do use feature swamp?

I think Realism Invictus does, I'm using their graphic for the next version of HR. I'm not sure what other mods use a Swamp/Marsh feature. I think most go for the terrain version as it's readily available from Colonization. I didn't like the way it looked though and having it as a feature gives me a few more options in the XML.