[MOD] Colonization: 2071

[Nightinggale]

I spotted the same issue. I modified traderoutes locally and it starts correctly. However the ship sails to the spice route and the new units arrive at the docks in europe (a different screen). Kailric once said the AI goes to Europe nomatter what meaning somewhere in the code there is a switch to check for human/AI players and I need to figure out how it works to be able to send the ship to the same dock as the units arrive.

One little funny thing is that when the ship starts at sea the colonists aren't in the ship. They are standing on water on the same plot. On new turn event they transport to the nearest discovered land, but they can't board the ship before they have been on land first

I think it would be better if the ship use a different placeholder. It's confusing to have the same graphics for colonists and the ship. Basically it's colonists boarding a colonist and then the colonist walks across water... with footprints and everything.

 

[orlanth]

ok, that makes sense. yeah it's disorienting atm to have a placeholder Colonist-shaped ship not to mention the default sound is presently that of a trebuchet It's tough to assign individual art currently since placeholders are set in a separate script; I'll likely have to merge place.pl into xmlgen then refine the units part to break non-colonist unitclasses down further into several special cases for ship and nonship, etc. It'll take a little doing with the script but is a good example of something that could take even longer with setting paths for all individual unitclasses by hand.

I've been thinking about how best to use the Trade Screens feature as well as the different "wormhole types" in gameinfo/Civ4Europeinfo .. A particularly interesting strategy would be to have the Planets mapscript make larger maps of two to four rectangular "quadrants" divided by a line of impassable Nebula terrain. If each quadrant could contain one of the different "wormhole types" from Civ4Europeinfo, you could gradually unlock access to these different areas by earning Progenitor Secrets (analogous to existing Trade Points in M:C). That would open up all kinds of interesting strategic dimensions, including having some Aliens that start in quadrants which only become accessible to Humans later on. However, unlike M:C most of the players will want and need to regularly access and trade with Earth like in the vanilla game, especially early on. So I wouldn't want each wormhole type to lead only to one specific non-Earth trade screen, or have the wormholes in different areas be inaccessible from Earth itself.

Replacing Spice Route and Silk Road, it might be interesting to let players gain the ability to access one or two non-Earth trade screens such as The Mars Combine (a Martian mining colony now ruled by lethal separatist androids) or Luna, a heavily fortified penal colony . But again, most players will still need & want to be able to access Earth through most of the travel tiles available. And coordinating AI travel to different Trade Screens through separate travel plots for each screen seems to be quite problematic for the existing M:C AI anyway.

It looks like Civ4Europeinfo allows assigning several <TradeScreenTypes> to each entry; so I'm thinking it could make things simpler and more mod-appropriate to let all the "Travel to Europe" types work fairly equally (ie on entering any Wormhole tile, you could choose to travel to Earth or to any other Trade Screen you have access to); while access to each "wormhole type" in Civ4Europeinfo can be unlocked individually by Civicinfos. Would something like this be feasible using the existing system, or would this require extensive changes or losing compatibility?

 

[Nightinggale]

I gave up on getting the ship to return to Europe. Instead I modified to the DLL. Now whenever a unit is placed in Europe and the new unit's owner is a human player, that unit is placed in the spice route. I figure that can't hurt considering Europe has been rendered inaccessible. Now planets map is playable. I still haven't figured out why the units aren't loaded when the game starts. Just make sure you reach land in the first round or you lose your colonists.

You need to pull both 2071 and the source as my update updated both.

yeah it's disorienting atm to have a placeholder Colonist-shaped ship not to mention the default sound is presently that of a trebuchet

I didn't notice as I had played with muted speakers. The the debugger triggers a breakpoint and freezes the game, the sound doesn't die, it repeats a really short part over and over, like half a second. Once you expect that to happen you mute the sound.

I've been thinking about how best to use the Trade Screens feature as well as the different "wormhole types" in gameinfo/Civ4Europeinfo .. A particularly interesting strategy would be to have the Planets mapscript make larger maps of two to four rectangular "quadrants" divided by a line of impassable Nebula terrain. If each quadrant could contain one of the different "wormhole types" from Civ4Europeinfo, you could gradually unlock access to these different areas by earning Progenitor Secrets (analogous to existing Trade Points in M:C). That would open up all kinds of interesting strategic dimensions, including having some Aliens that start in quadrants which only become accessible to Humans later on. However, unlike M:C most of the players will want and need to regularly access and trade with Earth like in the vanilla game, especially early on. So I wouldn't want each wormhole type to lead only to one specific non-Earth trade screen, or have the wormholes in different areas be inaccessible from Earth itself.

Sounds a bit like the two planes in Master of Magic. That could be interesting.

It looks like Civ4Europeinfo allows assigning several <TradeScreenTypes> to each entry; so I'm thinking it could make things simpler and more mod-appropriate to let all the "Travel to Europe" types work fairly equally (ie on entering any Wormhole tile, you could choose to travel to Earth or to any other Trade Screen you have access to); while access to each "wormhole type" in Civ4Europeinfo can be unlocked individually by Civicinfos. Would something like this be feasible using the existing system, or would this require extensive changes or losing compatibility?

I wonder about the same thing. However I don't know the answer to the question.

 

[Nightinggale]

I just had an idea about trade routes. We could place some in the middle of planets. This would give access to something special to that planet, like being able to buy say 5 units cheaper than anywhere else, but you can't buy any other units there. We can brainstorm for ideas on how to fully exploit this. Maybe we could even include randomness.

I must say that the current implementation is horrible from a modding point of view. More or less everything is hardcoded in the DLL. It would be nice if I could get it to be controlled by XML and ideally if it could generate random trade routes for the planets.

I noticed how the DLL generates the defines and I'm starting to remove unneeded data from GlobalDefineALT.xml. All of it is hardcoded in the DLL anyway and quite a number of the settings are known in the DLL if you know where to look. Removing un-modable settings should make it easier to mod as it only leaves the "ok to touch" settings in the file.

 

[Nightinggale]

I just noticed that github rejected my push yesterday. Now I tried again and it pushed without any problems
Whatever happened, the fact is that now the new (old) Europe is back and the planets map works.

I looked at the DLL code for shrines and it appears that they are only used for pilgrims. The AI uses that building class to find targets for pilgrim movement and similar stuff. Nothing about +3 everything in the DLL.

I looked at the building data in python. It consists of lines like

BUILDING_DATA_2071[0] = [40, 40, BuildingDistance, BuildingIconSize]

Only the blue numbers change. The first is the index, which has to fit the index in XML. The other two are multiplied with BUILDING_AREA_WIDTH and BUILDING_AREA_HEIGHT and my guess is it is the pixel placement of the top left corner of the building. The two fixed sized numbers appears to be X,Y of the bottom right of the building relative to the point made by the two previous numbers.

This made me wonder: why do we have this info in python? Why don't we add it to XML instead?
It's just two numbers and then the total count is in XML too. If we add that, then xml and python can't get out of sync.

 

[raystuttgart]

This made me wonder: why do we have this info in python?

Because this allows easy adjustment of the position "on the fly".
You can simply change the numbers in the python file while the game is still running.

If those numbers would be in XML it would be very tedious to reposition buildings in CityScreen.
You would need to end and restart the game each time to test your new postioning.

It is much better to have that directly in Python.

 

[Nightinggale]

My experience is whenever I edit anything in python while the game is running the game becomes unstable and get odd bugs. For instance when I made changes to domestic advisor the game stated that the file was reloaded (ok), but after I got that message the game would go haywire if I entered the city screen.

Still it might be good to keep the building numbers in python as moving the buildings on the fly could work much better than with XML.

 

[raystuttgart]

My experience is whenever I edit anything in python while the game is running the game becomes unstable and get odd bugs.

Considering (re)postioning of buildings in CityScreen, it has always worked fine for me to do that "on the fly" (while the game was still running).

 

[orlanth]

I agree with you both it should be fine to adjust in python (though I won't look forward to the chore of doing that )

There shouldn't be much visible changes with my latest commit but it lays the groundwork for generating Terrain Artdefs to work with selections added from the Infinite Terrain Pack while maintaining ability to reuse vanilla terrains as well (which actually work by a totally different system). According to tesb, the originator of this system of using a single Blend and Grids dds with a variety of Details dds textures, this new technique also saves a ton of memory which should help allow more terrain diversity without impacting performance.

I'm still puzzled about why the Colonists don't start out on board the Corvette I went over the unitinfos xml entries for both and they seem appropriate..

edit: puzzle no longer! the entry for the ship in civilizationinfos apparently must come before passengers in order to be boarded. an easy fix and a welcome one for the asphyxiated Colonists

 

[Nightinggale]

I fixed the "pBestYieldCity != NULL". It turned out to be a DLL bug in how the AI uses YieldGroup_AI_Sell_To_Europe (forgot checking it at a single location). However M:C appears to be unaffected by it by chance due to the combined setup of all yieldgroups.
I also added some text to the assert meaning now it tells which yields is triggering the assert.

It turns out that it was asserting when the king wanted to figure out which yields to transport between his cities and which to export. We can question if it is wise that the kings do that, but it looks like they do that in vanilla and it just stayed that way. Perhaps the traderoute setup should be skipped for players without cities. Then again looping the king's cities is done fairly quickly and the yieldchecks are also done fast now that it just looks up yieldgroups.

EDIT:
It looks like the AI is doing precisely what I was doing in the past, which caused transport problems, which resulted in coming up with feeder service. It might be a wise move to change the AI to use feeder service. It can then export anything not set to import in any city. This will make it fairly simple to extend transports to supply domestic demands and it would be possible to add imports to supply what is needed to build something, like stone. We can make this generic enough to look for any yield, not just the common stone and such.

According to tesb, the originator of this system of using a single Blend and Grids dds with a variety of Details dds textures, this new technique also saves a ton of memory which should help allow more terrain diversity without impacting performance.

If different terrain can point to the same Blend and Grids dds then it sounds right. If you end up including a Blend and Grids dds for each terrain, then it will not help even if they are identical.

edit: puzzle no longer! the entry for the ship in civilizationinfos apparently must come before passengers in order to be boarded. an easy fix and a welcome one for the asphyxiated Colonists

That makes sense. I assume the units are added to the game in the order they are entered in the XML. Once a new unit is added during game start, it checks all units already present in the plot to see if it can board any of them. If the ship is added last, then the colonists will have nothing to board when they are added to the game.

 

[orlanth]

Yes all the Terrain Pack content uses a single common Blend and Grids, so hopefully the savings should be substantial. I added terrain textures for Aquatic Planets; Planets mapscript doesn't yet generate them but they're working well when tested with the editor. I added the Space terrain textures for Coast and Ocean, but they're obscured by waves since they're still water terrains.

It turns out that it was asserting when the king wanted to figure out which yields to transport between his cities and which to export. We can question if it is wise that the kings do that, but it looks like they do that in vanilla and it just stayed that way. Perhaps the traderoute setup should be skipped for players without cities. Then again looping the king's cities is done fairly quickly and the yieldchecks are also done fast now that it just looks up yieldgroups.

I would never have guessed that it even tries that In vanilla the King civs don't seem to have much function other than fighting the final battle. I've often wondered if the AI could handle it if I gave a "King" civ direct ownership a few units on the map pre-revolution, or even let it capture a city or two from disobedient players. Could be cool if a government (or Progenitor Exarch ) naval presence can help fight off pirates or other enemies from time to time if you keep them happy via diplomacy/taxes; but has a downside in creating an immediate threat during Revolution and can occasionally launch punitive raids on your ships or colonies if you don't pay up when asked. It's probably not worth the complication of coding that though if the current King AI wouldn't handle it well.

 

[Nightinggale]

The reason why the king tries to import/export is because it happens to all AI controlled players. Looking closely at it, it turns out that natives will do this too. It could be a good idea to skip this for natives as they can have quote a lot of cities and no transports meaning they spend time setting up something, which will never be used and it will do that every turn. 2071 and possibly M:C can have native transports in the future. However RaR might benefit from this discovery. I will have to look into that to tell if it boosts enough to make it worth the needed testing and stuff like that.

As the king is basically just another AI it will likely handle units similar to other AI controlled players and the coding part of your idea isn't king related as such, but more when and where to place extra units and when to remove them. Assuming he isn't at war with anybody he will attack pirates only, though it's likely he will attack your own pirate ships as well as they are foreign to the king. We can argue if that is a feature or a bug.

I'm still wondering how to set up AI traderoutes. Export to Europe appears to be ok. The issue is the yields consumed locally.
Right now for each non-sold yieldtype it loops all cities to find the best target for imports. The best one is the one where a single unit can produce the most yields with the selected yield as input. It doesn't consider how many units can have that profession and it doesn't look like rebel production bonus is applied either. If more than one city is the best in this case, then it picks the one with the highest population. If more than one has the highest population of those, then it picks the one the civ gained first (time from founding or conquering). All cities then export that yield to that single city.

I'm thinking of setting export and feeder on all yields (except special ones like YIELD_FOOD). It will then turn import on and off automatically according to how much is stored and threshold. The threshold should then be:

• Yield needed by building being constructed
• Yield consumed by units in city * X (possibly potential consuming)
• Yields sold domestically * X

Now the question is what X should be in each line and how well it would work. It would likely be a good idea to keep some "best city" code to prefer blacksmith's shop over blacksmith's house.

From what I can tell this would scale to many cities much better than the vanilla code. For instance right now if the best city is on planet A, then all cities on planet B will end up stockpiling that yield. With this modification both planets will have domestic transports and domestic consumption.

Also it allows more yields to be transported with fewer special rules. Lumber can easily be moved to cities without forest even without special rules. I don't think it's even transported right now. This would make non-forest cities much better for the AI.

Certain yields should be stored even without a need. Food should be maintained at 50% of growth threshold to allow cities, which survives on imported food, tools should always be at least 50 to allow pioneers etc.

Ideas are welcome on how this should work. Right now I'm investigating what the AI should do. Figuring out how to code it and get proper performance is postponed until I have a clear goal of what it should do.

At some point I will also have to look at how the AI picks professions in a city. Cities should start producing more of yields needed by other cities to make a better overall production/consume for all cities combined.

I added terrain textures for Aquatic Planets; Planets mapscript doesn't yet generate them but they're working well when tested with the editor. I added the Space terrain textures for Coast and Ocean, but they're obscured by waves since they're still water terrains.

I don't think we should worry about stuff like waves in space at this point. It might be a trivial thing to solve later on and it's just a graphical thing. The focus should really be to make the game playable, then such glitches can be dealt with. Maybe the solution is to make a new space domain and not remove waves at all from water domain. Time will tell once we get that far.

 

[orlanth]

I don't think we should worry about stuff like waves in space at this point. It might be a trivial thing to solve later on and it's just a graphical thing. The focus should really be to make the game playable, then such glitches can be dealt with. Maybe the solution is to make a new space domain and not remove waves at all from water domain. Time will tell once we get that far.

I can't exactly recall anymore how I solved this for the old 2071, where Deep Space=TERRAIN_OCEAN and Low Orbit=TERRAIN_COAST.. IIRC after a lot of hunting through vanilla XML I changed some height setting to totally eliminate water/waves from the game. For this new version the solution will be to add a non-water Space Domain like Androrc's space domain modcomp to allow water terrains on the planets themselves, though it's not a pressing issue and I can start initial work on the new Planets climate types and terrains without it.

Certain yields should be stored even without a need. Food should be maintained at 50% of growth threshold to allow cities, which survives on imported food, tools should always be at least 50 to allow pioneers etc.

Ideas are welcome on how this should work. Right now I'm investigating what the AI should do. Figuring out how to code it and get proper performance is postponed until I have a clear goal of what it should do.

At some point I will also have to look at how the AI picks professions in a city. Cities should start producing more of yields needed by other cities to make a better overall production/consume for all cities combined.

The details of how the AI tries to plan transport and production have always been pretty unclear to me; though it was obvious the vanilla AI behavior was far from optimal and had been attempted to be improved in RaR. Particularly when there are a large number of individual Yields, and there can be local demand in multiple cities for small amounts of several yield types, the sheer number of individual transports and trips that needs to be made can get excessive even if individual trips can be automated. Since even small amounts of a yield type take up a full cargo bay, the vanilla unit-based transports are not set up to cope with moving small amounts of many yield types locally. So because of this it's worth considering some ability to have local demands not always require physical transport of everything using units, eg my National Domestic Market thread here or Kailric's posts about enabling some sharing of yields locally using plotgroups here

As far as "normal" transport using units, there's still definitely a role for it using the export and feeder system for larger scale intercontinental/interplanetary journeys. I'm thinking you could probably use your existing Yield Groups feature to let the AI use a relatively small number of standard behaviors. YieldGroup_AI_Raw_Material (including Stone, Lumber, and maybe Tools and Weapons/Munitions could use the same system) could first transport yields needed by buildings/units under construction, and then try to keep a "buffer" of x=50 stored in each city if there is overall excess of that yield. For Food/Nutrients (including Luxury Food in M:C) I'd think it would only transport if there was another city running at a deficit which couldn't produce locally; otherwise just leave the yield to stack up locally for population growth.

 

[Nightinggale]

I profiled RaR on a gigantic map to see the impact of disallowing natives to make import/export. It turned out to reduce the wait by 1.148 seconds on average and I don't think this number increases as time passes as time consumed in this appears to be somewhat linear depending on the number of native cities and not really anything else. Doesn't look like we have performance issues in this part of the code and I will not change anything to try to save way less than a second.

The details of how the AI tries to plan transport and production have always been pretty unclear to me

I didn't ask how it works in vanilla, but rather how should it work for us

Particularly when there are a large number of individual Yields, and there can be local demand in multiple cities for small amounts of several yield types, the sheer number of individual transports and trips that needs to be made can get excessive even if individual trips can be automated. Since even small amounts of a yield type take up a full cargo bay, the vanilla unit-based transports are not set up to cope with moving small amounts of many yield types locally. So because of this it's worth considering some ability to have local demands not always require physical transport of everything using units, eg my National Domestic Market thread here or Kailric's posts about enabling some sharing of yields locally using plotgroups here

That gave me an idea and I wrote about it in the medieval economy thread.

 

[orlanth]

Those economic & AI transport upgrades sounds like they will be a great improvement, IMO at least

Added Terrains and Leaderheadinfos XML we're continuing to make progress..

I think maybe I will try adding the Alien civs next, it shouldn't be too bad but I wonder if the M:C DLL will have some problem with them being fully playable. I would think not since the "Green" player in the earliest skeleton version actually already seemed to work. I can't quite remember all the DLL tweaks I had made regarding them in the old version, but they weren't that extensive and are probably not worth salvaging (& might even have contributed somewhat to its earlier slowness).

 

[Nightinggale]

I don't think playing aliens should be a problem for the DLL at all and I think playing natives in M:C would actually work ok. It's completely untested though.

I think the slowdown is mainly due to the performance issues I fixed in M:C. Take for instance yield gain from a plot. For every plot each turn it looped though all civics to find those, which were invented and affected the current plot. I changed that into a cache meaning it just looks up the combined bonus of all invented inventions. This mean the delay to get invented bonuses is faster, it also only happen once for each plot instead of once for each invention. M:C had several of those performance pitfalls (mainly regarding inventions) and I fixed a number of those. I assume the old Colonization 2071 was no better than M:C in this regard. 2071 makes it even worse by the high number of inventions, inventable professions, units etc.

I don't think performance is the main reason for not importing the old changes. However instead of spending ages searching for them we could recode the changes. Besides a recode mean rethink the changes and it could easily end up being better.

 

[orlanth]

Ok, Natives/Aliens civilizationinfos and leaderheadinfos XML is in. Do you notice anything about the Alien Emperor names?

The Aliens themselves are now playable and working, but there's a CTD introduced only in cases where the AI controls an Alien player. When I play a custom game with lots of Colonial players (using Planets mapscript on Small), it runs stably and they all land on planets and set up cities appropriately. (I noticed it can award a Conquest Victory even when there are other players still active, but I think that's just a result of having no Alien players owning cities to prevent that.)

When I play as an Alien civ, with multiple Colonial players as opponents, this also goes smoothly; the Advanced Start works well with either auto-setup or manual setup, and things look to be stable and playable in-game.

However when I play a custom game with even one AI-controlled Alien player, it starts normally and has set up their cities appropriately when I look in Worldbuilder, but gives a CTD without an assert at end of turn 1.

edit: because of seeing a Yield Storage assert I've tried adding <iYieldStorage>1000</iYieldStorage> to the free tier-1 Industry building for everyone, but it didn't affect the above.

 

[Nightinggale]

Ok, Natives/Aliens civilizationinfos and leaderheadinfos XML is in. Do you notice anything about the Alien Emperor names?

Easter eggs

However when I play a custom game with even one AI-controlled Alien player, it starts normally and has set up their cities appropriately when I look in Worldbuilder, but gives a CTD without an assert at end of turn 1.

Fixed... or rather workaround. The native AI code divides with iStoredMax too. If it is 0, then I change it to 100 and give an assert. This makes the game playable, but clearly the AI have problems handling something, which works in M:C.
This change is in the DLL only.

edit: because of seeing a Yield Storage assert I've tried adding <iYieldStorage>1000</iYieldStorage> to the free tier-1 Industry building for everyone, but it didn't affect the above.

Nice try, but not nice enough as we still have the problem. Is it related to something in XML as M:C doesn't trigger those asserts. The assert is a result of a bug elsewhere, but it doesn't tell where the bug is, which makes the assert less helpful than the average assert.

I managed to trigger the assert about no free building. I think I placed my city poorly, but the aliens had taken all the good places. Maybe that problem will go away when buildings, terrain and features are set up correctly in XML. I fear inventions will cause this problem to reappear.

 

[orlanth]

Is it related to something in XML as M:C doesn't trigger those asserts.

I think it may be related to the new M:C tags in leaderheadinfos xml. The "colonial" leaders in M:C get tags like:
<AttackForceKey>Holy Expeditionary Force</AttackForceKey>
<DeclareKey>Declare Emperor</DeclareKey>
<DawnKey>None</DawnKey>
<!--ECONOMY: 1 = HomeCity; 2 = Three Routes to Unlock Land Start; 3 = Inflation-->
<iVictoryType>1</iVictoryType>
<iTravelCommandType>1</iTravelCommandType>
<iEconomyType>2</iEconomyType>
but for the non-playable "Native" leaders these tags just aren't present. It's good that these tags are exposed to XML, I'm betting this is what allows moddability of things like access to normal Europe screen trading, automatic appearance of emigrants vs normal transport from Europe, and different kinds of endgame events on Revolution; but we'd need to find out about what specifically they do.

 

[Nightinggale]

Did you test the last commit? My game crashes in vanilla code whenever I try to start it. Going back one commit make it work just fine.

The crash is a strange one. It tries to allocate memory to store HillsYieldChanges for IMPROVEMENT_FARM and fails to do so. However RiverSideYieldChanges is allocated in the line before by calling the same function with the same arguments and that one is allocated just fine